Video processing method and apparatus, and device, computer-readable storage medium and computer program product

Abstract

Provided in the embodiments of the present disclosure are a video processing method and apparatus, and a device, a computer-readable storage medium and a product. The method comprises: acquiring a video to be processed, and on the basis of the current video frame and preceding video frame of the video to be processed, obtaining motion information corresponding to the current video frame; on the basis of the motion information, determining at least one motion particle to be added; on the basis of preset drawing parameters, drawing the at least one motion particle on a preset canvas, so as to obtain a displacement map, wherein the displacement map comprises pixel offsets corresponding to a plurality of pixels in the current video frame; and on the basis of the displacement map, performing offset operations on the pixels in the current video frame, so as to obtain target video frames, and on the basis of a plurality of target video frames corresponding to the video to be processed, obtaining a target video.

Description

Video processing method, device, equipment, computer-readable storage medium, and computer program product

[0001] This application claims priority to Chinese patent application No. 202311651245.6 filed on December 4, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby cited in their entirety as part of this application. Technical Field

[0002] The embodiments of the present disclosure relate to a video processing method, apparatus, device, computer-readable storage medium, and computer program product. Background Art

[0003] Users can edit videos to enhance visual quality and create richer visual effects. For example, users can specify a specific location in a video and use a pre-set image processing algorithm to add special effects to the video.

[0004] However, current special effects processing methods generally generate special effects based on user-specified locations. The generated effects are often relatively simple, dependent on user triggering operations, and have little relevance to the video, resulting in poor video quality. Summary of the Invention

[0005] The embodiments of the present disclosure provide a video processing method, apparatus, device, computer-readable storage medium, and computer program product, which are used to solve the technical problem that the existing ripple special effects have low correlation with videos.

[0006] In a first aspect, an embodiment of the present disclosure provides a video processing method, including:

[0007] Acquire a video to be processed, and obtain motion information corresponding to the current video frame based on a current video frame and a previous video frame of the video to be processed;

[0008] determining at least one moving particle to be added based on the motion information;

[0009] Drawing the at least one moving particle on a preset canvas according to preset drawing parameters to obtain a displacement map, wherein the displacement map includes pixel offsets corresponding to a plurality of pixels in the current video frame;

[0010] An offset operation is performed on pixels in the current video frame according to the displacement map to obtain a target video frame, and a target video is obtained according to multiple target video frames corresponding to the video to be processed.

[0011] In a second aspect, an embodiment of the present disclosure provides a video processing device, including:

[0012] An acquisition module, configured to acquire a video to be processed, and obtain motion information corresponding to the current video frame based on a current video frame and a previous video frame of the video to be processed;

[0013] a determining module, configured to determine at least one moving particle to be added based on the motion information;

[0014] a drawing module, configured to draw the at least one moving particle on a preset canvas according to preset drawing parameters to obtain a displacement map, wherein the displacement map includes pixel offsets corresponding to a plurality of pixels in a current video frame;

[0015] An offset module is configured to perform an offset operation on pixels in the current video frame according to the displacement map to obtain a target video frame, and obtain a target video according to multiple target video frames corresponding to the video to be processed. In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor and a memory;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video processing method described in the first aspect and various possible designs of the first aspect.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0019] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video processing method described in the first aspect and various possible designs of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] FIG1 is a flow chart of a video processing method according to an embodiment of the present disclosure;

[0022] FIG2 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0023] FIG3 is a flow chart of a video processing method according to another embodiment of the present disclosure;

[0024] FIG4 is a schematic diagram of a displacement diagram provided by an embodiment of the present disclosure;

[0025] FIG5 is a flow chart of a video processing method provided by another embodiment of the present disclosure;

[0026] FIG6 is a flow chart of a video processing method provided by yet another embodiment of the present disclosure;

[0027] FIG7 is a flow chart of a video processing method provided by yet another embodiment of the present disclosure;

[0028] FIG8 is a schematic structural diagram of a video processing device provided by an embodiment of the present disclosure;

[0029] FIG9 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0031] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0032] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0033] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0034] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0035] In order to solve the technical problem that the existing ripple special effects have low correlation with videos, the present disclosure provides a video processing method, device, equipment, computer-readable storage medium and product.

[0036] It should be noted that the video processing method, apparatus, device, computer-readable storage medium, and product provided by the present disclosure can be applied to any application scenario of performing special effects processing on a video.

[0037] Current special effects processing methods generally rely on obtaining the user's trigger location and then adding special effects at that location. However, because the addition of special effects often relies on user triggering, the resulting special effects often lack relevance to the video, resulting in unrealistic special effects and poor video processing quality.

[0038] In the process of solving the above technical problems, the inventors discovered through research that in order to improve the correlation between the added special effects content and the video to be processed, motion information can be determined based on two adjacent video frames in the video to be processed, wherein the motion information includes but is not limited to information such as optical flow maps that can describe the motion trend of the content body. Therefore, the motion trend of the content body in the video frame can be determined based on the motion information. Furthermore, at least one moving particle to be added can be determined based on the motion information, and a rendering operation can be performed on the at least one moving particle to obtain a displacement map including a pixel offset. The pixel offset in the displacement map is used to perform pixel offset processing on the video frame in the video to be processed, so that the processed target video can present a ripple effect that matches the motion trend of the content body. The video processing method, device, equipment, computer-readable storage medium and product provided in this embodiment can clearly determine the motion information of the picture body in the video to be processed by calculating the optical flow map corresponding to each video frame in the video to be processed.

[0039] The system architecture underlying this disclosure includes at least a terminal device and a server, wherein the terminal device and the server are in communication. A user can trigger a ripple effect processing request on the terminal device. In response, the server can obtain a video to be processed based on the ripple effect processing request. The server then calculates an optical flow map corresponding to the video to be processed and, based on the optical flow map, adds a ripple effect to the video to be processed that is associated with the motion trend of the main content.

[0040] FIG1 is a flow chart of a video processing method provided by an embodiment of the present disclosure. As shown in FIG1 , the method includes:

[0041] Step 101: Obtain a video to be processed, and obtain motion information corresponding to the current video frame based on the current video frame and the previous video frame of the video to be processed.

[0042] The execution subject of this embodiment is a video processing device. The video processing device can be coupled to a server that is communicatively connected to a terminal device. The server can calculate an optical flow map corresponding to the video to be processed based on a ripple special effect processing request triggered by a user on the terminal device, and add a ripple effect associated with the motion trend of the main content to the video to be processed based on the optical flow map.

[0043] Alternatively, the video processing device can be coupled to a terminal device, so that it can obtain the video to be processed in response to the user's trigger operation on the terminal device, calculate the optical flow map corresponding to the video to be processed, and add a ripple effect associated with the motion trend of the content body to the video to be processed based on the optical flow map.

[0044] In this embodiment, a user can trigger a ripple effect processing request on a terminal device. For example, the user can select a ripple effect in the video processing software pre-installed on the terminal device and trigger the ripple effect to generate a ripple effect processing request. The ripple effect includes, but is not limited to, a water ripple effect. For example, the ripple effect can be based on the motion of the content body, adding a water ripple effect to the moving part.

[0045] Accordingly, the video processing device can obtain a ripple special effect processing request and obtain a video to be processed. The video to be processed can be captured by the user in real time, uploaded by the user to a preset storage path, or selected by the user from multiple preset videos, which is not limited by the present disclosure.

[0046] Furthermore, in order to present a ripple effect on the video to be processed, for each video frame of the video to be processed, the video frame can be determined as the current video frame, and motion information corresponding to the current video frame is calculated based on the current video frame and its previous video frame.

[0047] The motion information includes, but is not limited to, information that can describe the motion trend of the content body, such as an optical flow map. Any optical flow calculation method can be used to calculate the optical flow map, and this disclosure does not limit this.

[0048] Step 102: Determine at least one moving particle to be added based on the motion information.

[0049] In this embodiment, the video to be processed may include moving content. For example, the video to be processed may be a video of a person. In a video of a person, the content may be the person, and the content may include physical movements, actions, and other motion behaviors. The motion information can accurately represent the motion trend of the content, including but not limited to movement direction and motion amplitude.

[0050] After obtaining motion information corresponding to the current video frame, at least one motion particle to be added can be determined based on the motion information. The motion particles can be used to describe the motion region of the content. The motion particles can move according to motion parameters, which can be determined based on the motion information.

[0051] Step 103: Draw the at least one moving particle on a preset canvas according to preset drawing parameters to obtain a displacement map, wherein the displacement map includes pixel offsets corresponding to a plurality of pixels in the current video frame.

[0052] In this embodiment, after determining at least one moving particle to be added, the at least one moving particle can be drawn on a preset canvas. The preset canvas can be a blank canvas, or a canvas set by the user according to actual needs, which is not limited in this disclosure.

[0053] Furthermore, drawing parameters may be preset, including but not limited to a first radius and a second radius of an offset region associated with drawing moving particles, and at least one assignment parameter in a preset assignment algorithm.

[0054] Therefore, at least one moving particle may be drawn on a preset canvas according to preset drawing parameters to obtain a displacement map, wherein the displacement map includes pixel offsets corresponding to a plurality of pixels in the current video frame.

[0055] Step 104 : performing an offset operation on pixels in the current video frame according to the displacement map to obtain a target video frame, and obtaining a target video according to multiple target video frames corresponding to the video to be processed.

[0056] In this embodiment, after constructing the displacement map based on the optical flow map, an offset operation may be performed on pixels in the current video frame based on the displacement map to obtain a target video frame.

[0057] Optionally, for each pixel in the displacement map, a target pixel in the current video frame that matches the pixel may be determined, and the pixel offset value may be superimposed on the target pixel to implement an offset operation on the pixel in the current video frame.

[0058] Furthermore, after obtaining the target video frame corresponding to each of the video frames to be processed, the multiple target video frames can be combined in chronological order to obtain a target video. In this target video, a ripple effect can be presented around the main content as it moves. This ripple can diffuse and disappear over time, making it more consistent with the movement trend of real ripples and improving the authenticity of the ripples in the target video.

[0059] FIG2 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure. As shown in FIG2 , in the previous video frame 21, the content subject 22 may be in a standing position. In the current video frame 23, the content subject 22 may switch its position to a hand-raising position. In response to the posture change of the content subject 22, a water ripple effect 24 may be added to the arm area where the change occurs. In addition, the refraction effect of the water surface may be simulated, so that the arm area of ​​the content subject 22 exhibits a wave effect. Thus, the addition of the water ripple effect can match the motion area of ​​the content subject, thereby improving the correlation between the ripple effect and the video to be processed.

[0060] The video processing method provided in this embodiment calculates the optical flow map corresponding to each video frame in the video to be processed, determines at least one moving particle based on the optical flow map, and performs a drawing operation on the at least one moving particle to generate a displacement map. The displacement map includes pixel offsets corresponding to multiple pixels, so that a pixel offset operation can be performed on the current video frame based on the displacement map. Since the optical flow map can accurately describe the motion trend of the content body in the current video frame, video processing based on the optical flow information can make the current video frame present a ripple effect that matches the motion trend of the content body, thereby improving the fit between the ripple effect and the video to be processed.

[0061] Further, based on any of the above embodiments, step 102 includes:

[0062] At least one target pixel point in the current video frame whose motion amplitude meets a preset condition is determined based on the motion information.

[0063] A moving particle is added at the location of the target pixel, wherein the moving direction of the moving particle is the same as the moving direction of the target pixel, the moving speed of the moving particle is proportional to the moving amplitude of the pixel, and a preset life cycle is set for the moving particle.

[0064] In this embodiment, after the motion information is acquired, since the motion information can characterize the motion trend of the pixels in the current video frame, at least one target pixel in the current video frame whose motion amplitude meets a preset condition can be determined based on the motion information.

[0065] For example, when an object moves in water, the greater the amplitude of the movement, the larger the water ripples generated. Therefore, after determining at least one target pixel whose motion amplitude meets a preset condition, a moving particle can be added to the target pixel's location to create a ripple effect based on the moving particle. The preset condition can be that the motion amplitude is greater than a preset amplitude threshold. Alternatively, the preset condition can be that the movement distance is greater than a preset distance threshold, etc., and this disclosure does not impose any restrictions on this.

[0066] Furthermore, motion parameters can be set for the moving particles based on the motion information, so that the moving particles move according to the motion parameters. The moving direction of the moving particles is the same as the moving direction of the target pixel, the speed of the moving particles is proportional to the amplitude of the pixel's motion, and a preset life cycle is set for the moving particles.

[0067] Taking the motion information as an optical flow map as an example, after obtaining the optical flow map corresponding to the current video frame, the optical flow map may include the optical flow direction and optical flow size corresponding to each pixel, which can represent the motion direction and motion amplitude of the content body.

[0068] Optionally, each pixel in the optical flow map can be traversed to calculate the sum of the vertical and horizontal optical flow values ​​of the pixel, and then detect whether the sum of the optical flow values ​​is greater than a preset optical flow threshold. The optical flow threshold can be 10 or a value set by the user according to actual needs, which is not limited by this disclosure.

[0069] Furthermore, if the sum of the optical flow values ​​corresponding to any pixel is detected to be greater than a preset optical flow threshold, a moving particle can be added to the pixel's location. The moving particle's direction of motion is the same as the pixel's optical flow direction, and its speed is proportional to the pixel's optical flow value. The larger the optical flow value, the faster the moving particle moves. A preset lifecycle is set for the moving particle. The longer this lifecycle, the longer the moving particle survives.

[0070] As an implementable method, the same life cycle can be set for all moving particles, or corresponding life cycles can be set for moving particles based on the sum of optical flow values ​​corresponding to pixel points. This disclosure does not impose any restrictions on this.

[0071] The video processing method provided in this embodiment, by presetting preset conditions, accurately adds moving particles based on these conditions to locations where the content has a large amplitude of motion. Furthermore, by rendering these moving particles, a ripple effect is generated. Adding a ripple effect to areas where the content has a large amplitude of motion allows the ripple effect to better match the motion trend of the content, thereby improving video quality.

[0072] Furthermore, based on any of the above embodiments, further, based on any of the above embodiments, after step 102, the method further includes:

[0073] The display position and life cycle of the moving particles are updated according to a preset time period.

[0074] In this embodiment, in order to simulate the effect of water waves expanding and gradually disappearing, the display position and life cycle of the moving particles may be updated according to a preset time period.

[0075] The preset time period may be to perform an update operation for each video frame. Alternatively, the user may set the time period according to actual needs, and the present disclosure does not impose any restrictions on this.

[0076] The video processing method provided in this embodiment can simulate the display effect of water waves gradually dissipating by updating the display position and life cycle of moving particles, thereby improving the authenticity of the ripple effect.

[0077] Furthermore, based on any of the above embodiments, the updating operation of the display position and life cycle of the moving particle according to a preset time period includes:

[0078] The display position of the moving particle is updated based on the moving speed and moving direction of the moving particle according to a preset time period.

[0079] The life cycle of the moving particle is decayed according to a preset decay speed until the life cycle of the moving particle decays to a preset life cycle threshold.

[0080] In this embodiment, in order to simulate the effect of water waves expanding and gradually disappearing, the display position and life cycle of the moving particles may be updated according to a preset time period.

[0081] Optionally, since the motion direction and motion speed corresponding to each moving particle are determined when the moving particles to be added are determined based on the optical flow map, the display position of the moving particles in the current frame can be determined based on the display position, motion direction, motion speed and the time period of the moving particles in the previous frame, thereby enabling the display position of the moving particles to be updated.

[0082] Furthermore, the life cycle of the moving particle may be decayed according to a preset time period and a preset decay speed until the life cycle of the moving particle decays to a preset life cycle threshold.

[0083] Optionally, the preset time interval may be used to update the position and life cycle of the moving particle for each video frame. The preset decay rate may be the life cycle of the moving particle minus 1 for each update. Alternatively, the user may adjust the preset time interval and decay rate based on actual needs, and this disclosure does not impose any limitations thereon.

[0084] Taking a practical application as an example, the life cycle corresponding to the previous video frame may be 10, and the life cycle of the moving particle in the current video frame may be -1, so the life cycle of the moving particle in the current video frame is 9.

[0085] The video processing method provided in this embodiment updates the displayed position of moving particles based on their speed and direction, thereby ensuring that the particle display position more closely matches the motion trend of the main content. Furthermore, by decaying the lifecycle of the moving particles at a preset decay rate, the method can simulate the effect of gradually dissipating water waves, enhancing the realism of the ripple effect.

[0086] Further, based on any of the above embodiments, step 104 includes:

[0087] A plurality of target moving particles having a life cycle greater than a life cycle threshold in a plurality of video frames corresponding to the video to be processed are determined.

[0088] The plurality of target moving particles are drawn on a preset canvas according to preset drawing parameters.

[0089] In this embodiment, during the process of drawing moving particles, since each video frame in the video to be processed corresponds to multiple moving particles, the life cycles of the moving particles corresponding to different video frames are different. Therefore, in order to simulate the effect of ripples gradually dissipating, only the target moving particles whose life cycle is greater than the preset life cycle threshold can be drawn.

[0090] Optionally, multiple target moving particles with a lifespan greater than a lifespan threshold can be determined in multiple video frames corresponding to the video to be processed. The lifespan threshold can be 0 or a value set by the user based on actual needs, which is not limited in this disclosure. The multiple target moving particles are drawn on a preset canvas according to preset drawing parameters.

[0091] The video processing method provided in this embodiment can make the ripple effect follow the moving content body and accurately simulate the visual effect of the ripple disappearing by rendering the target moving particles whose life cycle is greater than the life cycle threshold.

[0092] FIG3 is a flow chart of a video processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG3 , step 103 includes:

[0093] Step 301: For each moving particle, determine the position of the moving particle as the center position of the circle.

[0094] Step 302: Draw two concentric circles according to the center position of the circle and the preset first radius and second radius.

[0095] Step 303: Performing an assignment operation on the pixels in the region where the hollow ring formed by the two concentric circles is located according to a preset assignment algorithm to obtain pixel offsets corresponding to the pixels in the region where the hollow ring is located, thereby obtaining the displacement map.

[0096] In this embodiment, the preset drawing parameters include but are not limited to a first radius, a second radius, and parameters in an assignment algorithm.

[0097] After determining multiple moving particles to be added, for each moving particle, the moving particle can be used as the center of a circle. A first region can be drawn based on the center and a preset first radius, and a second region can be drawn based on the center and a preset second radius. The second radius is greater than the first radius. The first and second regions are concentric circles, and the portion of the first region is subtracted from the second region to form a hollow ring. A value assignment operation can be performed on the pixels within the hollow ring to obtain the pixel offset corresponding to the pixels within the region where the hollow ring is located, thereby obtaining a displacement map.

[0098] An offset algorithm may be pre-set, and a value assignment operation may be performed on pixels within the hollow circle based on the offset algorithm to obtain a displacement map.

[0099] FIG4 is a schematic diagram of a displacement map provided by an embodiment of the present disclosure. As shown in FIG4 , a first region 42 can be drawn with a moving particle 41 as the center of a circle according to a preset first radius, and a second region 43 can be drawn with a preset second radius. A circular region 44 formed by the first region 42 and the second region 43 is determined as the region to be assigned a value, and a value assignment operation can be performed on pixels 45 within the region. For example, through this assignment operation, the pixel 45 can be offset to the position of a sampling point 46.

[0100] Further, based on any of the above embodiments, step 303 includes:

[0101] The polar coordinates corresponding to the pixels in the area where the hollow ring is located are determined according to the Cartesian coordinates corresponding to the pixels in the area where the hollow ring is located.

[0102] The polar coordinates of the pixels within the area where the hollow ring is located are assigned based on a preset assignment algorithm.

[0103] In this embodiment, the pixel assignment operation can be implemented in polar coordinates.

[0104] Alternatively, the polar coordinates corresponding to the pixels within the region where the hollow ring is located can be determined based on the Cartesian coordinates corresponding to the pixels within the region where the hollow ring is located. After determining the plurality of pixels within the region of the hollow ring, a value can be assigned to each pixel based on the polar coordinates of the pixel and a preset value assignment algorithm. After the value assignment is completed, the polar coordinates can be converted to Cartesian coordinates.

[0105] Among them, the assignment algorithm may include parameters for adjusting the range of the hollow circle, parameters for adjusting the size of the hollow circle, and parameters for adjusting the position of the sampling point. Users can adjust the parameters in the assignment algorithm according to actual needs to present different ripple effects.

[0106] The video processing method provided in this embodiment determines the position of the moving particle as the center position of the circle, draws two concentric circles according to the center position of the circle and the preset first radius and second radius, and assigns values ​​to the pixels in the area of ​​the hollow ring formed by the two concentric circles according to a preset assignment algorithm, thereby being able to perform an offset operation on the pixels located in the hollow ring, and then performing pixel offset processing on the current video frame based on the pixel offset to present a ripple effect.

[0107] Further, based on any of the above embodiments, step 104 includes:

[0108] For a pixel in the displacement map, a target pixel in the current video frame that matches the pixel is determined.

[0109] The pixel offset corresponding to the pixel in the displacement map is superimposed on the target pixel to obtain the target video frame.

[0110] In this embodiment, after the displacement map is obtained, since the displacement map includes pixel offsets corresponding to a plurality of pixels, a pixel offset operation can be performed on the current video frame based on the displacement map.

[0111] Optionally, for each pixel in the displacement map, a target pixel in the current video frame that matches the pixel may be determined, and a pixel offset corresponding to the pixel in the displacement map may be superimposed on the target pixel to obtain the target video frame.

[0112] It should be noted that image frames typically have three RGB channels or four RGBW channels, while pixel offsets are typically two-dimensional data. Therefore, when generating a displacement map, only the R and G channels can be assigned values. This allows the target pixel to be offset based on the two-dimensional pixel offsets in the displacement map.

[0113] For example, the pixel offset can be an offset vector (1, 2), which can represent an X offset of one pixel to the right and a Y offset of two pixels downward. This allows accurate pixel offset operations to be performed on the current video frame based on the pixel offset.

[0114] The video processing method provided in this embodiment can shift the positions of the pixels in the current video frame by superimposing a pixel offset on the pixels in the current video frame, thereby presenting a water wave diffusion display effect.

[0115] FIG5 is a flowchart of a video processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG5 , after step 103, the method further includes:

[0116] Step 501: Perform a fuzzy operation on the displacement map using a preset fuzzy algorithm to obtain a blurred displacement map.

[0117] Step 104 includes:

[0118] Step 502: Perform an offset operation on pixels in the current video frame according to the blurred displacement map to obtain a target video frame.

[0119] In this embodiment, after obtaining the displacement map, directly performing a pixel shift operation on the current image frame based on the displacement map may result in a stiff, less realistic ripple effect. Therefore, after obtaining the displacement map, a blurring operation can be performed on the displacement map using a preset blurring algorithm to obtain a blurred displacement map. Any blurring algorithm can be used to implement the blurring operation on the target area, and this disclosure does not impose any restrictions thereto.

[0120] Optionally, the fuzzy algorithm may include preset fuzzy parameters, and the degree of fuzziness may be adjusted by adjusting the fuzzy parameters. An adjustment control for the fuzzy parameters may be displayed on the front end, allowing the user to adjust the fuzzy effect based on the adjustment control, which is not limited in this disclosure.

[0121] Furthermore, after obtaining the blurred displacement map, an offset operation may be performed on pixels in the current video frame based on the blurred displacement map to obtain a target video frame.

[0122] The video processing method provided in this embodiment performs a blurring operation on the displacement map and performs a pixel offset operation on the current video frame based on the blurred displacement map, thereby avoiding a harsh pixel offset effect, making the transition between the offset pixels and other positions in the current video frame more natural, and improving video quality.

[0123] FIG6 is a flow chart of a video processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG6 , step 104 includes:

[0124] Step 601: Perform recognition operation on the target area in the current video frame using a preset recognition algorithm.

[0125] Step 602: Perform an offset operation on pixels in an area other than the target area in the current video frame according to the displacement map to obtain a target video frame.

[0126] In this embodiment, when generating a ripple effect based on an optical flow map, the ripple effect can be generated across the entire image. However, when the video being processed includes people, animals, etc., generating a ripple effect across the entire image may affect the video quality. For example, it may cause occlusion of the person's face.

[0127] Therefore, in order to avoid occlusion of important areas while generating a ripple effect, a preset recognition algorithm can be used to identify the target area in the current video frame. This recognition algorithm includes, but is not limited to, facial recognition algorithms, preset object recognition algorithms, gesture recognition algorithms, and body recognition algorithms. Alternatively, the target area can be specified by the user based on actual needs, which is not limited in this disclosure.

[0128] Furthermore, after the target area is determined, an offset operation may be performed on pixels in an area other than the target area in the current video frame according to the displacement map to obtain a target video frame.

[0129] The video processing method provided in this embodiment performs an offset operation on pixels in an area other than a target area in the current video frame according to a displacement map, thereby preventing pixel offset from occurring in the display content in the target area and ensuring that the display content in the target area is not deformed.

[0130] Furthermore, based on any of the above embodiments, after step 601, the following steps may be further included:

[0131] A fuzzy operation is performed on the target area using a preset fuzzy algorithm to obtain a fuzzy target area.

[0132] Step 104 includes:

[0133] An offset operation is performed on pixels in an area other than the blurred target area in the current video frame according to the displacement map to obtain a target video frame.

[0134] In this embodiment, after the target area is obtained, a pixel shift operation may be performed on areas outside the target area, so that other areas present a ripple effect while the target area remains unchanged.

[0135] However, only performing pixel offset operations on areas outside the target area without processing the target area may result in a stiff connection between the target area and other areas, resulting in an unnatural ripple effect.

[0136] Therefore, a preset blur algorithm can be used to perform a blur operation on the target area to obtain a blurred target area. According to the displacement map, an offset operation is performed on pixels in an area other than the blurred target area in the current video frame to obtain a target video frame.

[0137] Among them, any fuzzy algorithm can be used to implement the fuzzy operation on the target area, and this disclosure does not impose any limitation on this.

[0138] Optionally, the fuzzy algorithm may include preset fuzzy parameters, and the degree of fuzziness may be adjusted by adjusting the fuzzy parameters. An adjustment control for the fuzzy parameters may be displayed on the front end, allowing the user to adjust the fuzzy effect based on the adjustment control, which is not limited in this disclosure.

[0139] The video processing method provided in this embodiment performs a blurring operation on the target area through a preset blurring algorithm to obtain a blurred target area, thereby enabling a smoother transition between the target area and other display areas and avoiding a harsh display effect.

[0140] FIG7 is a flowchart of a video processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG7 , after step 103, the method further includes:

[0141] Step 701: For a pixel in the displacement map, determine a target pixel in the current video frame that matches the pixel.

[0142] Step 702: Perform an offset operation on the color channel of the target pixel according to the pixel offset corresponding to the pixel in the displacement map.

[0143] In this embodiment, in order to improve the authenticity of video processing, while presenting the ripple effect, the color of the ripple area is adjusted to achieve a visual effect of color changes as the ripples flow.

[0144] Alternatively, the chromatic aberration effect is achieved by shifting different channels in an RGB image to varying degrees. Therefore, after obtaining a displacement map, for each pixel in the displacement map, the corresponding target pixel in the current video frame can be determined. The color channel of the target pixel is then shifted based on the pixel offset corresponding to that pixel.

[0145] Taking practical application as an example, after obtaining the pixel offset P, the R channel can be offset according to the offset P, the G channel can be offset according to 1.5 times the offset P, and the B channel can be offset according to 2 times the offset P, to achieve color adjustment based on the pixel offset, so that the color transformation is more in line with the movement trend of the content.

[0146] The video processing method provided in this embodiment performs an offset operation on the color channel of the target pixel based on the pixel offset, thereby being able to adjust the color of the ripple area while presenting the ripple effect, thereby improving the authenticity of the processed target video.

[0147] FIG8 is a schematic structural diagram of a video processing device provided by an embodiment of the present disclosure. As shown in FIG8 , the device includes: an acquisition module 81, a determination module 82, a drawing module 83, and an offset module 84. The acquisition module 81 is used to acquire a video to be processed, and obtain motion information corresponding to the current video frame based on the current video frame and the previous video frame of the video to be processed. The determination module 82 is used to determine at least one moving particle to be added based on the motion information. The drawing module 83 is used to draw the at least one moving particle on a preset canvas according to preset drawing parameters to obtain a displacement map, wherein the displacement map includes pixel offsets corresponding to multiple pixels in the current video frame. The offset module 84 is used to perform an offset operation on the pixels in the current video frame according to the displacement map to obtain a target video frame, and obtain a target video according to multiple target video frames corresponding to the video to be processed.

[0148] Furthermore, based on any of the above embodiments, the determination module is configured to: determine, based on the motion information, at least one target pixel in the current video frame whose motion amplitude satisfies a preset condition; add a moving particle at the location of the target pixel, wherein the moving particle has a motion direction that is the same as that of the target pixel, a motion speed that is proportional to the motion amplitude of the pixel, and a preset life cycle is set for the moving particle.

[0149] Furthermore, based on any of the above embodiments, the device further includes: an updating module, configured to update the display position and life cycle of the moving particles according to a preset time period.

[0150] Furthermore, based on any of the above embodiments, the updating module is configured to update the displayed position of the moving particle based on the moving speed and moving direction of the moving particle at a preset time period, and decay the life cycle of the moving particle at a preset decay rate until the life cycle of the moving particle decays to a preset life cycle threshold.

[0151] Furthermore, based on any of the above embodiments, the drawing module is configured to: determine a plurality of target moving particles having a life cycle greater than a life cycle threshold in a plurality of video frames corresponding to the video to be processed, and draw the plurality of target moving particles on a preset canvas according to preset drawing parameters.

[0152] Furthermore, based on any of the above embodiments, the drawing module is configured to: for each moving particle, determine the position of the moving particle as the center position of a circle; draw two concentric circles based on the center position and a preset first radius and a preset second radius; and assign values ​​to pixels within the hollow ring region formed by the two concentric circles according to a preset assignment algorithm to obtain pixel offsets corresponding to the pixels within the hollow ring region, thereby obtaining the displacement map.

[0153] Furthermore, based on any of the above embodiments, the drawing module is configured to: determine polar coordinates corresponding to pixels within the region where the hollow ring is located based on the Cartesian coordinates corresponding to the pixels within the region where the hollow ring is located, and assign values ​​to the polar coordinates of the pixels within the region where the hollow ring is located based on a preset assignment algorithm.

[0154] Furthermore, based on any of the above embodiments, the offset module is configured to: determine, for a pixel in the displacement map, a target pixel in the current video frame that matches the pixel, and superimpose a pixel offset corresponding to the pixel in the displacement map on the target pixel to obtain the target video frame.

[0155] Furthermore, based on any of the above embodiments, the apparatus further includes: a blurring module configured to blur the displacement map using a preset blurring algorithm to obtain a blurred displacement map; and an offset module configured to offset pixels in the current video frame according to the blurred displacement map to obtain a target video frame.

[0156] Furthermore, based on any of the above embodiments, the offset module is configured to: identify the target area in the current video frame using a preset recognition algorithm, and offset pixels in an area of ​​the current video frame other than the target area according to the displacement map to obtain a target video frame.

[0157] Furthermore, based on any of the above embodiments, the apparatus further includes: a blurring module configured to blur the target area using a preset blurring algorithm to obtain a blurred target area; and an offset module configured to offset pixels in an area of ​​the current video frame excluding the blurred target area according to the displacement map to obtain a target video frame.

[0158] Furthermore, based on any of the above embodiments, the apparatus further includes: a determination module configured to determine, for a pixel in the displacement map, a target pixel in the current video frame that matches the pixel; and an offset module configured to perform an offset operation on a color channel of the target pixel according to a pixel offset corresponding to the pixel in the displacement map.

[0159] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0160] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video processing method described in any of the above embodiments is implemented.

[0161] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer program product, including a computer program, which implements the video processing method as described in any of the above embodiments when executed by a processor.

[0162] In order to implement the above embodiment, the present disclosure further provides an electronic device, including: a processor and a memory;

[0163] The memory stores computer-executable instructions;

[0164] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method as described in any of the above embodiments.

[0165] FIG9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. The electronic device 900 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG9 is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0166] As shown in Figure 9, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0167] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although FIG9 shows an electronic device 900 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0168] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0169] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0170] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0171] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0172] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0173] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0174] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0175] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0176] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0177] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0178] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0179] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video processing method, comprising: Acquire a video to be processed, and obtain motion information corresponding to the current video frame based on a current video frame and a previous video frame of the video to be processed; determining at least one moving particle to be added based on the motion information; Drawing the at least one moving particle on a preset canvas according to preset drawing parameters to obtain a displacement map, wherein the displacement map includes pixel offsets corresponding to a plurality of pixels in the current video frame; An offset operation is performed on pixels in the current video frame according to the displacement map to obtain a target video frame, and a target video is obtained according to a plurality of target video frames corresponding to the video to be processed.

2. The method according to claim 1, wherein: The step of determining at least one moving particle to be added based on the movement information comprises: Determine at least one target pixel point in the current video frame whose motion amplitude meets a preset condition based on the motion information; A moving particle is added at the location of the target pixel, wherein the moving direction of the moving particle is the same as the moving direction of the target pixel, the moving speed of the moving particle is proportional to the moving amplitude of the pixel, and a preset life cycle is set for the moving particle.

3. The method according to claim 1, wherein: After determining at least one moving particle to be added based on the motion information, the method further includes: The display position and life cycle of the moving particles are updated according to a preset time period.

4. The method according to claim 3, wherein: The updating operation of the display position and life cycle of the moving particles according to a preset time period includes: updating the display position of the moving particle based on the moving speed and moving direction of the moving particle according to a preset time period; The life cycle of the moving particle is decayed according to a preset decay speed until the life cycle of the moving particle decays to a preset life cycle threshold.

5. The method according to claim 1, wherein: Drawing the at least one moving particle on a preset canvas according to preset drawing parameters to obtain a displacement map includes: Determine the position of the moving particle as the center of the circle; Draw two concentric circles according to the center position of the circle and the preset first radius and second radius; The pixels in the area where the hollow ring formed by the two concentric circles is located are assigned a value according to a preset assignment algorithm to obtain pixel offsets corresponding to the pixels in the area where the hollow ring is located, thereby obtaining the displacement map.

6. The method according to claim 5, wherein: The assigning operation on pixels within the region where the hollow ring formed by the two concentric circles is located according to a preset assignment algorithm includes: Determine the polar coordinates corresponding to the pixels in the area where the hollow circle is located according to the Cartesian coordinates corresponding to the pixels in the area where the hollow circle is located; The polar coordinates of the pixels in the area where the hollow ring is located are assigned based on a preset assignment algorithm.

7. The method according to claim 1, wherein: The performing an offset operation on pixels in the current video frame according to the displacement map to obtain a target video frame includes: For a pixel in the displacement map, determining a target pixel in the current video frame that matches the pixel; The pixel offset corresponding to the pixel in the displacement map is superimposed on the target pixel to obtain the target video frame.

8. The method according to any one of claims 1 to 7, wherein: After drawing the at least one moving particle on a preset canvas according to preset drawing parameters and obtaining a displacement map, the method further includes: Performing a fuzzy operation on the displacement map by using a preset fuzzy algorithm to obtain a blurred displacement map; The performing an offset operation on pixels in the current video frame according to the displacement map to obtain a target video frame includes: An offset operation is performed on pixels in the current video frame according to the blurred displacement map to obtain a target video frame.

9. The method according to any one of claims 1 to 7, wherein: The performing an offset operation on pixels in the current video frame according to the displacement map to obtain a target video frame includes: Performing an identification operation on the target area in the current video frame by using a preset identification algorithm; An offset operation is performed on pixels in an area other than the target area in the current video frame according to the displacement map to obtain a target video frame.

10. The method according to claim 9, wherein: After the target area in the current video frame is identified by a preset identification algorithm, the method further includes: Performing a fuzzy operation on the target area by using a preset fuzzy algorithm to obtain a fuzzy target area; The step of performing an offset operation on pixels in an area other than the target area in the current video frame according to the displacement map to obtain a target video frame includes: An offset operation is performed on pixels in an area other than the blurred target area in the current video frame according to the displacement map to obtain a target video frame.

11. The method according to any one of claims 1 to 7, wherein: After drawing the at least one moving particle on a preset canvas according to preset drawing parameters and obtaining a displacement map, the method further includes: For a pixel in the displacement map, determining a target pixel in the current video frame that matches the pixel; An offset operation is performed on the color channel of the target pixel according to the pixel offset corresponding to the pixel in the displacement map.

12. A video processing device, comprising: An acquisition module is configured to acquire a video to be processed, and obtain motion information corresponding to the current video frame based on a current video frame and a previous video frame of the video to be processed; a determination module, configured to determine at least one moving particle to be added based on the motion information; A drawing module, configured to draw the at least one moving particle on a preset canvas according to preset drawing parameters to obtain a displacement map, wherein the displacement map includes pixel offsets corresponding to a plurality of pixels in a current video frame; as well as The offset module is configured to perform an offset operation on pixels in the current video frame according to the displacement map to obtain a target video frame, and obtain a target video according to multiple target video frames corresponding to the video to be processed.

13. An electronic device, comprising: processor and memory; wherein, The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the video processing method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising: A computer program, characterized in that when the computer program is executed by a processor, it implements the video processing method according to any one of claims 1 to 11.

Images