Video processing method, device and equipment, computer readable storage medium and product

Abstract

The embodiment of the invention provides a video processing method, device and equipment, a computer readable storage medium and a product. The method comprises the following steps: acquiring a to-be-processed video; for a video frame in the to-be-processed video, a highlight area in the video frame is extracted, global motion information corresponding to the video frame is determined, and the global motion information comprises motion information corresponding to pixel points in the video frame; performing fuzzy processing on the highlight area based on the global motion information to obtain a light spot area and a smear area corresponding to the highlight area; and superposing the light spot region and the smear region on the highlight region in the video frame to obtain a target video. Therefore, the processed video frame can present metallic luster, colored glaze luster and other light rays to achieve a smooth object surface to form a luster effect. The smear area is constructed based on the global motion information and the highlight area, so that the light spot area can move along with the motion of the content main body to generate a smear effect, and the video processing effect is further improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the technical field of video processing, and in particular, to a video processing method, apparatus, device, computer-readable storage medium, and product. Background Art

[0002] With the booming development of photography technology, users can perform video acquisition operations based on video acquisition devices. In order to enrich the display effect of the acquired video, for example, to present display effects such as metallic luster and vitreous luster in at least some regions of the video, it is often necessary to adjust the camera position and lighting during shooting.

[0003] Therefore, the above method for video processing is often cumbersome and requires high professional skills of users, resulting in low video acquisition efficiency and unable to meet the actual needs of users. Summary of the Invention

[0004] Embodiments of the present disclosure provide a video processing method, apparatus, device, computer-readable storage medium, and product, which are used to solve the technical problem that the existing method for optimizing the video display effect is cumbersome to operate.

[0005] In a first aspect, embodiments of the present disclosure provide a video processing method, including:

[0006] Obtain a video to be processed;

[0007] For a video frame in the video to be processed, extract the highlight region in the video frame, and determine the global motion information corresponding to the video frame, where the global motion information includes the motion information corresponding to the pixel points in the video frame;

[0008] Blur the highlight region based on the global motion information to obtain a light spot region and a trailing shadow region corresponding to the highlight region;

[0009] Overlay the light spot region and the trailing shadow region on the highlight region in the video frame to obtain a target video.

[0010] In a second aspect, embodiments of the present disclosure provide a video processing apparatus, including:

[0011] An obtaining module, configured to obtain a video to be processed;

[0012] A determining module, configured to, for a video frame in the video to be processed, extract the highlight region in the video frame, and determine the global motion information corresponding to the video frame, where the global motion information includes the motion information corresponding to the pixel points in the video frame;

[0013] A processing module, configured to blur the highlight area based on the global motion information to obtain a spot area and a trailing area corresponding to the highlight area;

[0014] A synthesis module, configured to superimpose the spot area and the trailing area on the highlight area in the video frame to obtain a target video.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video processing method described in the first aspect and various possible designs of the first aspect above.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, the video processing method described in the first aspect and various possible designs of the first aspect above is implemented.

[0019] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the video processing method described in the first aspect and various possible designs of the first aspect above is implemented.

[0020] The video processing method, apparatus, device, computer-readable storage medium and product provided in this embodiment, after obtaining the video to be processed, identify the highlight area in the video frame corresponding to the video to be processed, and determine the global motion information corresponding to the video to be processed, so that the highlight area can be blurred based on the global motion information, and thus a spot area and a trailing area that match the global motion information can be obtained. Superimpose the spot area and the trailing area on the highlight area in the video frame, so that the processed video frame can present a luster effect formed by light such as metallic luster and glazed luster reaching the smooth object surface. In addition, by constructing a trailing area based on the global motion information and the highlight area, as the content main body in the video frame moves, the spot area can also move with the movement of the content main body to produce a trailing effect, further improving the video processing effect. Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 Schematic flowchart of the video processing method provided by an embodiment of the present disclosure;

[0023] Figure 2 Schematic flowchart of the video processing method provided by another embodiment of the present disclosure;

[0024] Figure 3 Schematic flowchart of the video processing method provided by another embodiment of the present disclosure;

[0025] Figure 4 Schematic flowchart of the video processing method provided by another embodiment of the present disclosure;

[0026] Figure 5 Schematic structural diagram of the video processing device provided by an embodiment of the present disclosure;

[0027] Figure 6 Schematic structural diagram of the electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present disclosure.

[0029] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained through appropriate means in accordance with relevant laws and regulations.

[0030] For example, when receiving the user's active request, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application program, server, or storage medium that executes the operations of the technical solutions of the present disclosure according to the prompt message.

[0031] As an optional but non-limiting implementation manner, in response to receiving an active request from a user, the manner of sending a prompt message to the user may be, for example, a pop-up window manner, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0032] It can be understood that the above notification and user authorization acquisition process is only illustrative and does not limit the implementation manner of the present disclosure. Other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0033] To solve the technical problem that the existing methods for optimizing video display effects are relatively cumbersome to operate, the present disclosure provides a video processing method, apparatus, device, computer-readable storage medium, and product.

[0034] It should be noted that the video processing method, apparatus, device, computer-readable storage medium, and product provided by the present disclosure can be applied to any video processing scenario in which at least part of the video presents gloss.

[0035] To enrich the display effect of the video, the high-light area or the specified area in the video can be controlled to present gloss effects such as metallic gloss and vitreous gloss formed by light irradiating the surface of a smooth object.

[0036] In the related art, in order to make at least part of the video present a gloss effect, it is necessary to manually adjust the camera position, lighting, etc. during the video acquisition process. The operation is often cumbersome and requires a high level of professional technology from the user.

[0037] In the process of solving the above technical problems, the inventor found through research that, in order to automatically implement video processing, after obtaining the video to be processed, for each video frame in the video to be processed, the high-light area in the video frame can be extracted. Subsequently, effects such as halos and trails can be added around the high-light area to present the gloss effect formed by light irradiating the surface of a smooth object.

[0038] Optionally, for each video frame, the global motion information corresponding to the video frame can be calculated based on the video frame and its adjacent video frames. The global motion information can characterize the motion direction and motion amplitude of the content in the video frame. Based on the global motion information, the high-light area is blurred to form a spot area and a trailing area corresponding to the high-light area. The spot area and the trailing area can match the motion trend of the content in the video frame. The spot area and the trailing area are superimposed on the high-light area in the video frame to obtain the target video.

[0039] Figure 1 It is a schematic flowchart of the video processing method provided by the embodiments of the present disclosure, asFigure 1 As shown, the method includes:

[0040] Step 101, obtain the video to be processed.

[0041] The execution subject of this embodiment is a video processing device. This video processing device can be coupled to a terminal device. Thus, it can obtain the video to be processed based on the trigger operation of the user on the terminal device, and superimpose a light spot area and a trailing shadow area on the highlight area in the video frame corresponding to the video to be processed to obtain a target video. Alternatively, this video processing device can also be coupled to a server that is communicatively connected to the terminal device. Thus, it can obtain a video processing request sent by the terminal device, obtain the video to be processed based on the video processing request, and superimpose a light spot area and a trailing shadow area on the highlight area in the video frame corresponding to the video to be processed to obtain a target video.

[0042] In this embodiment, in order to implement video processing operations, the video to be processed can be obtained. Among them, the video to be processed can be collected in real time by the user or uploaded by the user in a preset storage path. The present disclosure does not limit this.

[0043] Step 102, for the video frame in the video to be processed, extract the highlight area in the video frame, and determine the global motion information corresponding to the video frame, where the global motion information includes the motion information corresponding to the pixel points in the video frame.

[0044] In this embodiment, after obtaining the video to be processed, for each video frame in the video to be processed, the highlight area in the video frame can be extracted. Thus, subsequent image processing can be performed on the highlight area to make the highlight area present the luster effect formed by light irradiating the smooth surface of an object. Among them, the high-frequency area in the video frame can be identified, and the high-frequency part in the video frame can be filtered through box filtering to obtain a filtering result. Calculate the difference between the video frame and the filtering result to obtain the highlight area.

[0045] Further, since the content subject in the video to be processed moves continuously, in order to make the video processing result more conform to the motion trend of the content subject in the video to be processed, the global motion information corresponding to the video frame can also be determined, where the global motion information includes the motion information corresponding to the pixel points in the video frame. Among them, the motion information includes motion angle, motion amplitude, etc.

[0046] Step 103, perform blurring processing on the highlight area based on the global motion information to obtain a light spot area and a trailing shadow area corresponding to the highlight area.

[0047] In this embodiment, after obtaining the global motion information, the highlight area can be blurred based on the global motion information to obtain a light spot area and a trailing shadow area corresponding to the highlight area.

[0048] For example, Gaussian blur can be applied to the high - light area to obtain a light - spot area that matches the global motion information, and radial blur can be applied to the high - light area to obtain a trailing - shadow area that matches the global motion information. Among them, the global motion information can be the average horizontal motion and the average vertical motion of pixel points on the UV coordinate axes. Therefore, the sampling area in the blurring process can be adjusted based on the global motion information. Blurring operations are performed within the adjusted sampling area to present a dynamic blurring effect that matches the global motion information.

[0049] Step 104: Superimpose the light - spot area and the trailing - shadow area on the high - light area in the video frame to obtain a target video.

[0050] In this embodiment, after obtaining the light - spot area and the trailing - shadow area corresponding to the high - light area, the light - spot area and the trailing - shadow area can be superimposed on the high - light area in the video frame to obtain a processed video frame. The target video is obtained for all the processed video frames corresponding to the video to be processed. Thus, the target video can present the gloss effect formed by light irradiating the smooth surface of an object, and the gloss effect can match the motion of the content main body in the video to be processed.

[0051] Optionally, based on any of the above - mentioned embodiments, the method further includes:

[0052] Determine the noise map corresponding to the high - light area.

[0053] Superimpose the noise map on the high - light area to obtain the halo area corresponding to the high - light area.

[0054] In this embodiment, after extracting the high - light area, in order to make the high - light area present a flowing display effect, a noise that changes with time series can be applied to the high - light area.

[0055] Optionally, the noise map corresponding to the high - light area can be determined. Among them, the noise map can be a Perlin noise map. Or, it can also be any other noise map, and the present disclosure does not limit this.

[0056] Furthermore, the noise map can be superimposed on the high - light area to obtain the halo area corresponding to the high - light area. Since the high - light areas in each video frame are different, superimposing the noise map corresponding to the high - light area on each video frame can make each video frame present flowing high - light points.

[0057] The video processing method provided in this embodiment can extract the highlight regions in each video frame of the video to be processed after obtaining the video to be processed. Determine the global motion information corresponding to the video to be processed, and blur the highlight regions based on the global motion information, so as to obtain a spot region and a trailing region that match the global motion information. Superimpose the spot region and the trailing region on the highlight region in the video frame, so that the processed video frame can present a luster effect formed by light reaching the surface of a smooth object, such as metallic luster and vitreous luster. In addition, by constructing a trailing region based on the global motion information and the highlight region, as the content main body in the video frame moves, the spot region can also move with the movement of the content main body to produce a trailing effect, further enhancing the authenticity of the luster effect.

[0058] Figure 2 As shown in the flowchart of the video processing method provided in another embodiment of the present disclosure, on the basis of any of the above embodiments, Figure 2 as shown, step 102 includes:

[0059] Step 201, determine the adjacent video frame corresponding to the video frame.

[0060] Step 202, determine the optical flow map corresponding to the video frame based on the video frame and the adjacent video frame.

[0061] Step 203, determine the horizontal motion average value and the vertical motion average value corresponding to the optical flow map according to the pixels in the optical flow map.

[0062] Step 204, determine the horizontal motion average value and the vertical motion average value as the global motion information.

[0063] In this embodiment, in order to determine the global motion information corresponding to the video frame, the optical flow map corresponding to the video frame can be calculated based on the video frame and its adjacent video frame.

[0064] Optionally, the adjacent video frame corresponding to the video frame can be determined. Among them, the adjacent video frame can be the previous video frame or the next video frame of the current video frame, and the present disclosure does not limit this.

[0065] Determine the optical flow map corresponding to the video frame based on the video frame and the adjacent video frame. Among them, the optical flow calculation between adjacent sequence frames can be implemented by using the DIS optical flow method. Or, any other optical flow calculation method can also be used to determine the optical flow map corresponding to the video frame based on the video frame and the adjacent video frame, and the present disclosure does not limit this.

[0066] In the optical flow map, each pixel point represents the position degrees uv on the x-axis and y-axis. Therefore, after obtaining the optical flow map, the average horizontal motion u and the average vertical motion v of the optical flow map can be respectively calculated. The average horizontal motion and the average vertical motion are determined as the global motion information.

[0067] The video processing method provided in this embodiment determines the optical flow map corresponding to the video frame based on the video frame and the adjacent video frame. Since the optical flow map can characterize the motion trend of the pixel points in the video frame, the global motion information corresponding to the video frame can be accurately determined based on the optical flow map.

[0068] Further, based on any of the above embodiments, step 202 includes:

[0069] Perform a size adjustment operation on the video frame and the adjacent video frame according to the preset display parameters to obtain the adjusted video frame and the adjusted adjacent video frame of the video frame, where the display parameters include a preset resolution and a preset display ratio.

[0070] Determine the optical flow map corresponding to the video frame based on the adjusted video frame and the adjusted adjacent video frame of the video frame.

[0071] In this embodiment, in the process of calculating the optical flow map, the optical flow calculation between adjacent sequence frames can be implemented based on the DIS optical flow method. Since the overall motion information of the picture needs to be obtained during video processing, the optical flow information can be statistically calculated at a relatively low resolution to reduce the computational complexity in the process of processing the optical flow map.

[0072] To implement the adjustment operation on the video frame, the display parameters can be preset, where the display parameters include a preset resolution and a preset display ratio. The preset resolution can be preset to be less than the resolution of the video frame. By adjusting the preset display ratio, the horizontal or vertical motion information can be obtained in a targeted manner. For example, the preset display ratio can be 2:1, so that the horizontal motion information can be obtained more specifically.

[0073] Further, the size adjustment operation can be performed on the video frame and the adjacent video frame according to the preset display parameters to obtain the adjusted video frame and the adjusted adjacent video frame of the video frame. Determine the optical flow map corresponding to the video frame based on the adjusted video frame and the adjusted adjacent video frame of the video frame.

[0074] The video processing method provided in this embodiment can, by presetting display parameters, perform size adjustment operations on video frames and adjacent video frames based on these display parameters to obtain the adjusted video frames and the adjacent video frames of the adjusted video frames. Furthermore, when calculating the optical flow map based on the adjusted video frames and the adjacent video frames of the adjusted video frames, since the resolutions of the adjusted video frames and the adjacent video frames of the adjusted video frames are relatively low, computing power can be effectively saved during the optical flow map calculation process, improving the video processing efficiency. In addition, by setting a preset display ratio in the preset display parameters, horizontal or vertical motion information can be obtained more accurately based on this preset display ratio, making the light spot area and smear area generated based on the global motion information more in line with the personalized needs of users.

[0075] Optionally, based on any of the above embodiments, step 103 includes:

[0076] Perform Gaussian blur processing on the high-light area based on the global motion information to obtain the light spot area corresponding to the high-light area.

[0077] Perform radial blur processing on the high-light area based on the global motion information to obtain the smear area corresponding to the high-light area.

[0078] In this embodiment, in order to obtain the light spot area and the smear area based on the high-light area, two blur operations can be performed on the high-light area.

[0079] Optionally, Gaussian blur processing can be performed on the high-light area based on the global motion information to obtain the light spot area corresponding to the high-light area. Radial blur processing can be performed on the high-light area based on the global motion information to obtain the smear area corresponding to the high-light area.

[0080] The video processing method provided in this embodiment can, by performing Gaussian blur processing and radial blur processing on the high-light area respectively based on the global motion information, obtain the light spot area and the smear area corresponding to the high-light area. In addition, the display effects of the light spot area and the smear area can be made to match the global motion information corresponding to the video frame, improving the authenticity of the gloss effect.

[0081] Figure 3 As shown in the flowchart of the video processing method provided in another embodiment of the present disclosure, based on any of the above embodiments, Figure 3 as shown, the performing Gaussian blur processing on the high-light area based on the global motion information to obtain the light spot area corresponding to the high-light area includes:

[0082] Step 301: Adjust the sampling range corresponding to the Gaussian blur processing according to the global motion information to obtain an adjusted target sampling range.

[0083] Step 302: Perform Gaussian blur processing on the high-light area within the target sampling range to obtain the spot area corresponding to the high-light area.

[0084] In this embodiment, the global motion information may include the horizontal motion average value U and the vertical motion average value V. After determining the global motion information, the sampling range corresponding to the Gaussian blur processing can be adjusted based on the global motion information to obtain the adjusted target sampling range. For example, the X-axis corresponding to the Gaussian blur processing can be offset based on the horizontal motion average value U in the global motion information, and the Y-axis corresponding to the Gaussian blur processing can be offset based on the vertical motion average value V to obtain the adjusted target sampling range.

[0085] Further, after obtaining the target sampling range, perform Gaussian blur processing on the high-light area within the target sampling range to obtain the spot area corresponding to the high-light area. By adjusting the sampling range of the Gaussian blur based on the global motion information, the blurred area can be fused with the high-light point at a certain angle, improving the authenticity of the spot area.

[0086] The video processing method provided in this embodiment adjusts the sampling range corresponding to the Gaussian blur processing according to the global motion information, performs Gaussian blur processing on the high-light area within the target sampling range to obtain the spot area corresponding to the high-light area, so as to achieve dynamic-angle blur for the high-light map, make the blurred spot area fuse with the high-light point at a certain angle, create a stretching effect, and improve the authenticity of the spot.

[0087] Further, based on any of the above embodiments, after step 302, it further includes:

[0088] Stretch the color channels of the spot area according to the preset adjustment parameters.

[0089] In this embodiment, after obtaining the spot area based on the high-light area, in order to increase the authenticity of the spot area, a natural dispersion effect can be presented around the spot area.

[0090] Optionally, the adjustment parameters can be preset. For example, the adjustment parameters can be [2, 3, 4], or the adjustment parameters can be set by the user according to actual needs, and the present disclosure does not limit this. Stretch the RGB color channels of the spot area according to the preset adjustment parameters.

[0091] The video processing method provided in this embodiment stretches the color channels of the spot area according to the preset adjustment parameters, so as to stretch the color channels to different degrees, making the processed spot area present a natural dispersion effect.

[0092] Figure 4 A flowchart of a video processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as Figure 4 shown, the method of performing radial blur processing on the highlight area based on the global motion information to obtain a trailing area corresponding to the highlight area includes:

[0093] Step 401: Adjust the sampling range corresponding to the radial blur processing according to the global motion information to obtain an adjusted target sampling range.

[0094] Step 402: Determine the motion angle information and motion amplitude information of the trailing area based on the global motion information.

[0095] Step 403: Perform radial blur processing on the highlight area according to the target sampling range, the motion angle information, and the motion amplitude information to obtain the trailing area.

[0096] In this embodiment, the global motion information may include a horizontal motion average value U and a vertical motion average value V. After determining the global motion information, the sampling range corresponding to the radial blur processing can be adjusted based on the global motion information to obtain an adjusted target sampling range. For example, the X-axis corresponding to the radial blur processing can be offset based on the horizontal motion average value U in the global motion information, and the Y-axis corresponding to the radial blur processing can be offset based on the vertical motion average value V to obtain the adjusted target sampling range.

[0097] In addition, the motion angle information and motion amplitude information of the trailing area can also be determined based on the global motion information. Among them, the motion angle of the trailing area can be determined based on the ratio of the horizontal motion average value U and the vertical motion average value V on the UV coordinate axis. The motion amplitude information is determined based on the sum of the horizontal motion average value U and the vertical motion average value V.

[0098] After separately determining the target sampling range, the motion angle information, and the motion amplitude information, the highlight area can be subjected to radial blur processing based on the above parameters to obtain the trailing area.

[0099] The video processing method provided by this embodiment can adjust the sampling range corresponding to the radial blur processing according to the global motion information to obtain an adjusted target sampling range, and determine the motion angle information and motion amplitude information of the trailing area based on the global motion information, so that the highlight area can be subjected to radial blur processing based on the target sampling range, the motion angle information, and the motion amplitude information to accurately obtain a dynamic trailing area.

[0100] Further, based on any of the above embodiments, after step 402, the method further includes:

[0101] Determine the historical smear angle corresponding to the previous adjacent video frame associated with the video frame.

[0102] Based on the historical smear angle, preset parameters, the angle information, and a preset stability algorithm, determine the current smear angle corresponding to the video frame.

[0103] In this embodiment, during the process of generating an optical flow map based on the current video frame and the migrated adjacent video frame, errors may occur, resulting in discontinuous timing, and further may cause a jump in the angle of the smear area, affecting the display effect of the smear area. Therefore, in order to improve the stability of the smear area, a stability algorithm and preset parameters can be set in advance.

[0104] Optionally, the historical smear angle corresponding to the previous adjacent video frame associated with the video frame can be determined. Based on the historical smear angle, preset parameters, the angle information, and a preset stability algorithm, determine the current smear angle corresponding to the video frame.

[0105] Among them, the first product of the historical smear angle and the preset parameters, and the second product of the current smear and the difference between the preset value and the preset parameters can be determined, and the sum of the first product and the second product is used as the current smear angle.

[0106] The video processing method provided in this embodiment can, by presetting a stability algorithm and preset parameters, determine the current smear angle corresponding to the video frame based on the historical smear angle, preset parameters, the angle information, and the preset stability algorithm, effectively avoiding the problems of discontinuous timing and jump in the smear angle caused by errors during the generation of the optical flow map, and further improving the accuracy and stability of video processing.

[0107] Figure 5 FIG. is a schematic structural diagram of a video processing apparatus provided in an embodiment of the present disclosure. As Figure 5 shown, the apparatus includes: an acquisition module 51, a determination module 52, a processing module 53, and a synthesis module 54. The acquisition module 51 is configured to acquire a video to be processed. The determination module 52 is configured to extract a highlight area in the video frame for the video frame to be processed, and determine the global motion information corresponding to the video frame, where the global motion information includes the motion information corresponding to the pixel points in the video frame. The processing module 53 is configured to perform blurring processing on the highlight area based on the global motion information to obtain a light spot area and a smear area corresponding to the highlight area. The synthesis module 54 is configured to superimpose the light spot area and the smear area on the highlight area in the video frame to obtain a target video.

[0108] Further, based on any of the above embodiments, the determining module is configured to: determine adjacent video frames corresponding to the video frame; determine an optical flow map corresponding to the video frame based on the video frame and the adjacent video frames; determine a horizontal motion average value and a vertical motion average value corresponding to the optical flow map according to pixels in the optical flow map; and determine the horizontal motion average value and the vertical motion average value as the global motion information.

[0109] Further, based on any of the above embodiments, the determining module is configured to: perform a size adjustment operation on the video frame and the adjacent video frames according to preset display parameters to obtain an adjusted video frame and an adjusted adjacent video frame of the video frame, where the display parameters include a preset resolution and a preset display ratio; and determine an optical flow map corresponding to the video frame based on the adjusted video frame and the adjusted adjacent video frame of the video frame.

[0110] Further, based on any of the above embodiments, the processing module is configured to: perform Gaussian blur processing on the highlight area based on the global motion information to obtain a spot area corresponding to the highlight area; and perform radial blur processing on the highlight area based on the global motion information to obtain a trailing shadow area corresponding to the highlight area.

[0111] Further, based on any of the above embodiments, the processing module is configured to: adjust a sampling range corresponding to the Gaussian blur processing according to the global motion information to obtain an adjusted target sampling range; and perform Gaussian blur processing on the highlight area within the target sampling range to obtain a spot area corresponding to the highlight area.

[0112] Further, based on any of the above embodiments, the processing module is configured to: stretch a color channel of the spot area according to preset adjustment parameters.

[0113] Further, based on any of the above embodiments, the processing module is configured to: adjust a sampling range corresponding to the radial blur processing according to the global motion information to obtain an adjusted target sampling range; determine motion angle information and motion amplitude information of the trailing shadow area based on the global motion information; and perform radial blur processing on the highlight area according to the target sampling range, the motion angle information, and the motion amplitude information to obtain the trailing shadow area.

[0114] Further, based on any of the above embodiments, the processing module is configured to: determine a historical trailing shadow angle corresponding to a previous adjacent video frame associated with the video frame; and determine a current trailing shadow angle corresponding to the video frame based on the historical trailing shadow angle, preset parameters, the angle information, and a preset stabilization algorithm.

[0115] Further, based on any of the above embodiments, the apparatus further includes: a determination module, configured to determine a noise map corresponding to the highlight region; and a superimposition module, configured to superimpose the noise map on the highlight region to obtain a halo region corresponding to the highlight region.

[0116] The device provided in this embodiment can be used to execute the technical solutions of the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0117] To implement the above embodiments, the present disclosure also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the video processing method as described in any of the above embodiments.

[0118] To implement the above embodiments, the present disclosure also provides a computer program product including a computer program, which, when executed by a processor, implements the video processing method as described in any of the above embodiments.

[0119] To implement the above embodiments, the present disclosure also provides an electronic device, including: a processor and a memory;

[0120] The memory stores computer-executable instructions;

[0121] The processor executes the computer-executable instructions stored in the memory, such that the processor executes the video processing method as described in any of the above embodiments.

[0122] Figure 6 FIG. is a schematic structural diagram of the electronic device provided in the embodiments of the present disclosure. The electronic device 600 may be a terminal device or a server. Among them, the terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable media players (PMPs), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The illustrated electronic device is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0123] As Figure 6As shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0124] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 an electronic device 600 with various devices is shown, it should be understood that it is not required to implement or include all the shown devices. Instead, more or fewer devices may be implemented or included.

[0125] Specifically, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above functions defined in the method of the embodiment of the present disclosure are executed.

[0126] It should be noted that the computer-readable medium described above in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0127] The above computer-readable medium can be included in the above electronic device; or it can exist separately without being assembled into the electronic device.

[0128] The above computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device is caused to execute the method shown in the above embodiments.

[0129] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0131] The units described in the embodiments of the present disclosure may be implemented in software or in hardware. Among them, the name of the unit does not constitute a limitation to the unit itself in some cases. For example, the first acquisition unit may also be described as "the unit for acquiring at least two Internet protocol addresses".

[0132] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, by way of non-limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip (SOC), complex programmable logic devices (CPLD), and so on.

[0133] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0135] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0136] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video processing method, characterized in that, Including: Obtain the video to be processed; For the video frames in the video to be processed, extract the highlight regions in the video frames, and determine the global motion information corresponding to the video frames, where the global motion information includes the motion information corresponding to the pixel points in the video frames; Blur the highlight regions based on the global motion information to obtain the corresponding spot regions and smear regions of the highlight regions; Overlay the spot regions and smear regions on the highlight regions in the video frames to obtain the target video.

2. The method according to claim 1, characterized in that, The determining the global motion information corresponding to the video frames includes: Determine the adjacent video frames corresponding to the video frames; Determine the optical flow map corresponding to the video frames based on the video frames and the adjacent video frames; Determine the horizontal motion average value and the vertical motion average value corresponding to the optical flow map according to the pixels in the optical flow map; Determine the horizontal motion average value and the vertical motion average value as the global motion information.

3. The method according to claim 2, wherein The determining the optical flow map corresponding to the video frames based on the video frames and the adjacent video frames includes: Perform size adjustment operations on the video frames and the adjacent video frames according to preset display parameters to obtain the adjusted video frames and the adjacent video frames of the adjusted video frames, where the display parameters include a preset resolution and a preset display ratio; Determine the optical flow map corresponding to the video frames based on the adjusted video frames and the adjacent video frames of the adjusted video frames.

4. The method according to claim 1, wherein The blurring the highlight regions based on the global motion information to obtain the corresponding spot regions and smear regions of the highlight regions includes: Perform Gaussian blurring on the highlight regions based on the global motion information to obtain the corresponding spot regions of the highlight regions; Perform radial blurring on the highlight regions based on the global motion information to obtain the corresponding smear regions of the highlight regions.

5. The method according to claim 4, wherein The performing Gaussian blurring on the highlight regions based on the global motion information to obtain the corresponding spot regions of the highlight regions includes: Adjust the sampling range corresponding to the Gaussian blurring according to the global motion information to obtain the adjusted target sampling range; Perform Gaussian blurring on the highlight regions within the target sampling range to obtain the corresponding spot regions of the highlight regions.

6. The method according to claim 5, wherein After performing Gaussian blurring on the processed highlight regions within the target sampling range to obtain the corresponding spot regions of the highlight regions, it further includes: Stretch the color channels of the spot regions according to preset adjustment parameters.

7. The method according to claim 4, wherein The performing radial blurring on the highlight regions based on the global motion information to obtain the corresponding smear regions of the highlight regions includes: Adjust the sampling range corresponding to the radial blurring according to the global motion information to obtain the adjusted target sampling range; Determine the motion angle information and the motion amplitude information of the smear regions based on the global motion information; Perform radial blurring on the highlight regions according to the target sampling range, the motion angle information, and the motion amplitude information to obtain the smear regions.

8. The method according to claim 7, wherein After determining the angle information and amplitude information of the smear area based on the global motion information, the method further includes: Determining a historical smear angle corresponding to a previous adjacent video frame associated with the video frame; Determining a current smear angle corresponding to the video frame based on the historical smear angle, a preset parameter, the angle information, and a preset stabilization algorithm.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: Determining a noise map corresponding to the highlight area; Overlaying the noise map on the highlight area to obtain a halo area corresponding to the highlight area.

10. A video processing device, characterized in that, The method includes: An acquisition module, configured to acquire a video to be processed; A determination module, configured to, for a video frame in the video to be processed, extract a highlight area in the video frame and determine global motion information corresponding to the video frame, where the global motion information includes motion information corresponding to pixel points in the video frame; A processing module, configured to perform blurring processing on the highlight area based on the global motion information to obtain a spot area and a smear area corresponding to the highlight area; A synthesis module, configured to overlay the spot area and the smear area on the highlight area in the video frame to obtain a target video.

11. An electronic device, characterized in that, The device includes: A processor and a memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the video processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the processor executes the computer-executable instructions, the video processing method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the video processing method according to any one of claims 1 to 9 is implemented.

Images