A video enhancement method, device, apparatus and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MALANSHAN AUDIO & VIDEO LABORATORY
- Filing Date
- 2026-03-27
- Publication Date
- 2026-06-16
AI Technical Summary
Existing video enhancement algorithms cannot dynamically focus visual attention on key objects based on sound guidance, resulting in wasted computing resources and an inability to clearly define the degree of enhancement, and they lack cross-modal collaborative mechanisms.
Audio feature information is obtained through an audio stream separation model, and a target mask is generated by combining video object detection or semantic segmentation. The pixel-level enhancement value is determined using audio and spatial weight factors to perform differentiated video enhancement. Imperfections are eliminated through layered and smooth fusion, and finally integrated with the audio stream.
It achieves precise enhancement of the video stream, improves the image quality of core visual objects, ensures the integrity and naturalness of the video footage, and enhances the user experience.
Smart Images
Figure CN122227031A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a video enhancement method, apparatus, device, and storage medium. Background Technology
[0002] In current mainstream audio and video processing pipelines, audio enhancement and video enhancement are typically two independent and parallel technology stacks. Audio processing (such as object separation and spatial audio rendering) and video processing (such as super-resolution and SDR to HDR conversion) each pursue the optimization of their own technical indicators, but lack cross-modal collaborative mechanisms. This results in video enhancement algorithms being unable to "understand" the content of the picture and unable to dynamically focus visual attention (and corresponding enhancement resources) on key objects based on the guidance of sound. Existing video enhancement algorithms are mostly "blind enhancements," applying the same level of processing to people, objects, and background in the picture, failing to distinguish between the subject and the background, leading to a waste of computational resources, and potentially weakening the visual focus intended by the director; moreover, they rely entirely on information from the video modality itself, making it impossible to clearly define the degree of enhancement.
[0003] As can be seen from the above, how to achieve intelligent video enhancement and improve user experience is an urgent problem to be solved. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a video enhancement method, apparatus, device, and storage medium capable of intelligent video enhancement and improving user experience. The specific solution is as follows: In a first aspect, this application provides a video enhancement method, including: The original audio and video files are acquired, preprocessed, and extracted to obtain corresponding audio and video streams. The audio streams are then separated based on a pre-trained audio source separation model to obtain the target audio track, and the audio feature information corresponding to the target audio track is determined. The audio feature information includes different categories of audio objects and the confidence level of sound presence. The video stream is subjected to video object detection or semantic segmentation to obtain video feature information. Based on a preset mapping rule table, the audio objects in the audio feature information and the video objects in the video feature information of each frame are matched, and the target mask is determined using the obtained matching results. The video feature information includes different categories of video objects and the pixel-level positions corresponding to the video objects. The base pixel enhancement value corresponding to each pixel is determined based on the category of the video object in the target mask, the audio weight factor is determined using the sound presence confidence, the spatial weight factor is determined based on the spatial location features corresponding to the target mask, and the target enhancement value is determined using the audio weight factor, the spatial weight factor, and the base pixel enhancement value. The video stream is subjected to pixel-level differential enhancement processing using the target enhancement value and the target mask to obtain an enhanced video stream. The enhanced video stream is then layered and smoothly fused to obtain a target video stream. Finally, the target video stream is integrated with the audio stream to obtain a target audio and video file.
[0005] Optionally, the step of acquiring the original audio and video files, preprocessing and extracting the original audio and video files to obtain corresponding audio and video streams, separating the audio streams based on a pre-trained audio source separation model to obtain the target audio track, and determining the audio feature information corresponding to the target audio track includes: The original audio and video files are obtained, and preprocessing including timestamp alignment and normalization is performed on the original audio and video files to obtain processed audio and video files. The processed audio and video files are then extracted to obtain the corresponding audio stream and video stream. A pre-trained audio source separation model is determined based on a deep neural network, and the audio stream is separated using the audio source separation model to obtain the target audio track; Each of the target audio tracks is analyzed to obtain different categories of audio objects, sound start and end times, average loudness, and sound presence confidence; the sound presence confidence is a confidence level determined based on the instantaneous loudness of the audio object and using a clipping function and a preset gating function. If the audio stream is 3D audio, then the sound trajectory coordinates corresponding to the audio object are added to the audio feature information.
[0006] Optionally, the step of performing video object detection or semantic segmentation on the video stream to obtain video feature information, matching the audio objects in the audio feature information and the video objects in the video feature information of each frame based on a preset mapping rule table, and determining the target mask using the obtained matching results includes: The video stream is identified using an object detection model or a semantic segmentation model to obtain different categories of video objects and their corresponding pixel-level positions. A preset mapping rule table is constructed based on different categories of audio objects and corresponding video objects. Based on the audio objects in the audio feature information and the video objects in the video feature information of the same frame, the preset mapping rule table is used for matching, and the successfully matched video objects are determined as target video objects. If the number of target video objects is 1, then the pixel-level position corresponding to the target video object is converted into a binary mask to obtain the target mask; If the number of target video objects is greater than 1, then determine whether there are sound trajectory coordinates in the audio feature information of the audio object; If the sound trajectory coordinates exist, then the target mask is determined based on the pixel-level position of the video object corresponding to the sound trajectory coordinates; If the sound trajectory coordinates do not exist, a final video object is selected from the target video objects based on preset rules, and a target mask is determined based on the pixel-level position corresponding to the final video object.
[0007] Optionally, the step of determining the base pixel enhancement value corresponding to each pixel based on the category of the video object in the target mask, determining the audio weight factor using the sound presence confidence, determining the spatial weight factor based on the spatial location features corresponding to the target mask, and determining the target enhancement value using the audio weight factor, the spatial weight factor, and the base pixel enhancement value includes: The base pixel enhancement value of each pixel in each video frame of the video stream is determined based on the category of the video object in the target mask; different categories correspond to different base pixel enhancement values; The audio weighting factor is determined by the sound presence confidence and the preset audio adjustment coefficient, and the spatial weighting factor is determined based on the pixel coordinates corresponding to each pixel, the first pixel distance from the pixel to the geometric center of the target mask, and the second pixel distance from the geometric center of the target mask to the edge of the target mask. The target enhancement value is determined based on the audio weighting factor, the spatial weighting factor, and the base pixel enhancement value, using a clipping function.
[0008] Optionally, the step of performing pixel-level differential enhancement processing on the video stream using the target enhancement value and the target mask to obtain the enhanced video stream includes: The target fusion coefficient is determined based on the target enhancement value and the preset fusion coefficient, and the clear pixels corresponding to each pixel are generated using the super-resolution model. Super-resolution enhancement is performed on each video frame based on the pixel, the corresponding clear pixel, and the target fusion coefficient to obtain a first enhanced video frame, and the enhanced video stream is determined using the first enhanced video frame.
[0009] Optionally, determining the enhanced video stream using the first enhanced video frame includes: The stretching range is determined based on the target enhancement value, and the stretching range is used to perform local contrast stretching on each pixel in the first enhanced video frame to obtain the stretched video frame. The sharpening intensity is determined based on the target enhancement value, and the sharpening intensity is used to sharpen and enhance the contours of video objects in the stretched video frame to obtain a second enhanced video frame. The RGB color space of the second enhanced video frame is converted to the YCbCr color space, and the color change of the target pixel is restricted based on the YCbCr color space to obtain the third enhanced video frame. The enhanced video stream is determined using the third enhanced video frame. The target pixel is the pixel in the second enhanced video frame where the video object is a person.
[0010] Optionally, the step of performing layered and smooth fusion on the enhanced video stream to obtain the target video stream includes: A transition band of a preset width is generated based on the target boundary of the target mask, and a corresponding fusion weight is configured for the transition band; Based on the enhanced video stream, layers of different resolutions are obtained by using Laplacian pyramids or wavelet transforms for layering. The transition band is fused based on the fusion weight and the layer to obtain the fused transition band; The fused transition band and the enhanced video stream are smoothly integrated to obtain the integrated video stream; The global color statistics of each video frame in the integrated video stream are determined, and the global color of each video frame in the integrated video stream is corrected based on the global color statistics to obtain the target video stream; the global color statistics include the mean and variance of the color of each pixel in each video frame in the integrated video stream.
[0011] Secondly, this application provides a video enhancement device, comprising: The audio stream separation module is used to acquire the original audio and video files, preprocess and extract the original audio and video files to obtain the corresponding audio stream and video stream, separate the audio stream based on a pre-trained audio source separation model to obtain the target audio track, and determine the audio feature information corresponding to the target audio track; the audio feature information includes different categories of audio objects and sound presence confidence; The target mask determination module is used to perform video object detection or semantic segmentation on the video stream to obtain video feature information, match the audio objects in the audio feature information and the video objects in the video feature information of each frame based on a preset mapping rule table, and determine the target mask using the obtained matching results; the video feature information includes different categories of video objects and the pixel-level positions corresponding to the video objects; The enhancement value determination module is used to determine the basic pixel enhancement value corresponding to each pixel based on the category of the video object in the target mask, determine the audio weight factor using the sound presence confidence, determine the spatial weight factor based on the spatial location features corresponding to the target mask, and determine the target enhancement value using the audio weight factor, the spatial weight factor, and the basic pixel enhancement value. The video stream integration module is used to perform pixel-level differential enhancement processing on the video stream using the target enhancement value and the target mask to obtain an enhanced video stream, perform layering and smooth fusion on the enhanced video stream to obtain a target video stream, and integrate the target video stream with the audio stream to obtain a target audio and video file.
[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the aforementioned video enhancement method.
[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned video enhancement method.
[0014] This application acquires raw audio and video files, preprocesses and extracts the raw audio and video files to obtain corresponding audio and video streams, separates the audio stream based on a pre-trained audio source separation model to obtain a target audio track, and determines the audio feature information corresponding to the target audio track; the audio feature information includes different categories of audio objects and sound presence confidence; performs video object detection or semantic segmentation on the video stream to obtain video feature information, matches the audio objects in the audio feature information and the video objects in the video feature information of each frame based on a preset mapping rule table, and uses the obtained matching results to determine a target mask; the video feature information includes different categories of video objects. The image and the pixel-level position corresponding to the video object are determined; based on the category of the video object in the target mask, the basic pixel enhancement value corresponding to each pixel is determined; the audio weight factor is determined using the sound presence confidence; the spatial weight factor is determined based on the spatial position features corresponding to the target mask; the target enhancement value is determined using the audio weight factor, the spatial weight factor, and the basic pixel enhancement value; the video stream is subjected to pixel-level differential enhancement processing using the target enhancement value and the target mask to obtain an enhanced video stream; the enhanced video stream is layered and smoothly fused to obtain a target video stream; and the target video stream is integrated with the audio stream to obtain a target audio and video file.
[0015] As can be seen from the above, this application achieves standardization and independent parsing of audio and video streams through preprocessing and extraction of the original audio and video. A pre-trained audio source separation model is used to perform object-level splitting of the mixed audio to obtain audio feature information. Video object detection or semantic segmentation is then performed on the video stream to obtain video feature information including video object categories and pixel-level locations. Next, a preset mapping rule table is used to match audio and video objects, and a target mask is generated based on the matching results, giving the video enhancement a clear direction. Basic pixel enhancement values are configured based on the video object categories in the target mask, and audio weight factors are determined by combining sound presence confidence. Spatial weight factors are determined by the spatial location features of the mask, and these three factors are combined to obtain the target enhancement value. In this way, guided by the target mask and with the target enhancement value used for intensity control, pixel-level differentiated enhancement of the video stream is achieved. Layered and smooth fusion is then used to eliminate enhancement defects, and finally, the video stream is integrated with the audio stream to obtain the target audio and video file. This accurately improves the image quality of core visual objects while ensuring the integrity and naturalness of the video image, thus enhancing the user experience. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0017] Figure 1 This is a flowchart of a video enhancement method disclosed in this application; Figure 2 This is a schematic diagram of a basic pixel enhancement value disclosed in this application; Figure 3 This is a flowchart of a specific video enhancement method disclosed in this application; Figure 4 This is a schematic diagram of an audio analysis disclosed in this application; Figure 5 This is a schematic diagram of a video analysis and processing method disclosed in this application; Figure 6 This is a schematic diagram of the structure of a video enhancement device disclosed in this application; Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Currently, existing video enhancement algorithms apply the same level of processing to people, objects, and background in the image, failing to distinguish between the subject and background. This leads to wasted computational resources and may weaken the visual focus intended by the director. Furthermore, relying entirely on information from the video modality itself makes it impossible to clearly define the degree of enhancement. To address this, this application provides a video enhancement method that uses a target mask as a guide and the target enhancement value as intensity control to achieve pixel-level differentiated enhancement of the video stream. Layering and smoothing fusion are then used to eliminate enhancement defects, and finally, the method is integrated with the audio stream to obtain the target audio and video file. This accurately improves the image quality of the core visual object while ensuring the integrity and naturalness of the video image, thus enhancing the user experience.
[0020] See Figure 1 As shown, an embodiment of the present invention discloses a video enhancement method, including: Step S11: Obtain the original audio and video files, preprocess and extract the original audio and video files to obtain the corresponding audio stream and video stream, separate the audio stream based on the pre-trained audio source separation model to obtain the target audio track, and determine the audio feature information corresponding to the target audio track; the audio feature information includes different categories of audio objects and sound presence confidence.
[0021] In this embodiment, the original audio and video files undergo timestamp synchronization alignment, standardization preprocessing, and audio stream extraction, such as unifying volume and sampling frequency. A pre-trained audio source separation model is used to separate the audio stream to obtain target audio tracks corresponding to target categories. These target categories include human voices, airplane sounds, car sounds, and ambient sounds. Metadata is extracted from each target audio track to obtain the corresponding audio object, sound start and end times, average loudness, and sound presence confidence. The formula for the sound presence confidence is as follows: ; in, The confidence level of the sound at time t is given. This is a clipping function that restricts the calculated x value to between 0.0 and 1.0. The preset gating function determines whether time t is within the sound start and end time range. If it is within the sound start and end time range, the value is 1.0; otherwise, the value is 0.0. The confidence level base value ranges from [0,1]. Let be the instantaneous loudness at time t; Maximum instantaneous pixels; The loudness adjustment coefficient is typically set to 0.3~0.5, but can be set according to actual conditions. The sum of the loudness adjustment coefficient and the confidence baseline value is usually controlled within 1.5. It is worth mentioning that if the audio stream is 3D audio, the sound trajectory coordinates of each audio object are parsed and added to the audio feature information.
[0022] Specifically, the steps of acquiring the original audio and video files, preprocessing and extracting the original audio and video files to obtain corresponding audio and video streams, separating the audio streams based on a pre-trained audio source separation model to obtain target audio tracks, and determining the audio feature information corresponding to the target audio tracks include: acquiring the original audio and video files, performing preprocessing on the original audio and video files including timestamp alignment and standardization to obtain processed audio and video files, and extracting the processed audio and video files to obtain corresponding audio and video streams; determining a pre-trained audio source separation model based on a deep neural network, and using the audio source separation model to separate the audio streams to obtain target audio tracks; analyzing each target audio track to obtain different categories of audio objects, sound start and end times, average loudness, and sound presence confidence; the sound presence confidence is a confidence level determined based on the instantaneous loudness of the audio object and using a clipping function and a preset gating function; if the audio stream is 3D audio, then the sound trajectory coordinates corresponding to the audio object are added to the audio feature information.
[0023] Step S12: Perform video object detection or semantic segmentation on the video stream to obtain video feature information. Match the audio objects in the audio feature information and the video objects in the video feature information of each frame based on a preset mapping rule table, and use the obtained matching results to determine the target mask. The video feature information includes different categories of video objects and the pixel-level positions corresponding to the video objects.
[0024] In this embodiment, in a video stream synchronized with the audio stream, an object detection model or semantic segmentation model is used to identify the video stream to obtain different categories of video objects and their corresponding pixel-level positions; the pixel-level positions are the pixel-level contour position information of the video objects; a preset mapping rule table is established. In one specific implementation, the preset mapping rule table is: airplane sound - airplane; human voice - person; car sound - car; based on the preset mapping rule table, the active audio objects and video objects at the current moment are matched, and the video objects in the corresponding video frames that match the active audio objects are determined as the target video objects; if a target video object exists in the video frame... If a target video object is identified, its pixel-level position is directly converted into a binary mask to obtain the target mask. If multiple similar target video objects exist in the video frame, such as two cars matching the same car sound, it is determined whether sound trajectory coordinates exist in the audio feature information of the audio object. If they exist, the corresponding video object is determined according to the sound trajectory coordinates to generate the target mask. If they do not exist, a final video object is selected from the target video objects based on a preset rule. The preset rule can be the largest area or the closest to the center of the video frame, or it can be set according to the actual situation. The target mask is then determined based on the pixel-level position corresponding to the final video object.
[0025] Specifically, the step of performing video object detection or semantic segmentation on the video stream to obtain video feature information, matching audio objects in the audio feature information and video objects in the video feature information of each frame based on a preset mapping rule table, and determining a target mask using the obtained matching results includes: using an object detection model or semantic segmentation model to identify the video stream to obtain different categories of video objects and their corresponding pixel-level positions; constructing a preset mapping rule table based on different categories of audio objects and their corresponding video objects; and using the preset mapping rule table to determine the target mask based on the audio objects in the audio feature information and the video objects in the video feature information of the same frame. The rules table is used for matching, and the successfully matched video objects are identified as target video objects. If the number of target video objects is 1, the pixel-level position corresponding to the target video object is converted into a binary mask to obtain the target mask. If the number of target video objects is greater than 1, it is determined whether there are sound trajectory coordinates in the audio feature information of the audio object. If the sound trajectory coordinates exist, the target mask is determined based on the pixel-level position of the video object corresponding to the sound trajectory coordinates. If the sound trajectory coordinates do not exist, the final video object is selected from the target video objects according to preset rules, and the target mask is determined based on the pixel-level position corresponding to the final video object.
[0026] Step S13: Determine the basic pixel enhancement value corresponding to each pixel based on the category of the video object in the target mask, determine the audio weight factor using the sound presence confidence, determine the spatial weight factor based on the spatial location features corresponding to the target mask, and determine the target enhancement value using the audio weight factor, the spatial weight factor, and the basic pixel enhancement value.
[0027] In this embodiment, the base pixel enhancement value of each pixel in each video frame of the video stream is determined based on the category of the video object in the target mask. A larger value indicates a stronger base enhancement. In one specific implementation, the base pixel enhancement values corresponding to different video object categories are as follows: Figure 2 As shown, the base pixel enhancement value can also be adjusted according to the actual situation. The target enhancement value is jointly determined by the base pixel enhancement value, the audio weight factor, and the spatial weight factor. The audio weight factor maps the salience of the audio object to a dynamic adjustment amount of the enhancement intensity, and the corresponding formula is as follows: ; in, The audio weighting factor at time t; The confidence level of the sound at time t is given. The preset audio adjustment coefficient can be set to, for example, 0.4, or it can be adjusted according to the actual situation.
[0028] It is understood that the spatial weighting factor is used to achieve non-uniform enhancement within the target mask, typically resulting in the strongest enhancement at the object center and a natural transition at the edges. The corresponding formula is as follows: ; in, The spatial weighting factor; The base weight for the central region of the target mask is set, for example, 0.7, but it can also be adjusted according to the actual situation; The distance from the pixel to the geometric center of the target mask is the first pixel distance. The distance is the second pixel from the geometric center of the target mask to the farthest edge of the mask. The target enhancement value is determined based on the audio weighting factor, the spatial weighting factor, and the base pixel enhancement value, using a clipping function. The formula corresponding to the target enhancement value is as follows: ; in, The target enhancement value; This is the clipping function; The audio weighting factor at time t; The spatial weighting factor; The base pixel enhancement value corresponding to the type of the video object corresponding to the pixel.
[0029] Specifically, the process of determining the base pixel enhancement value corresponding to each pixel based on the category of the video object in the target mask, determining the audio weight factor using the sound presence confidence, determining the spatial weight factor based on the spatial location features corresponding to the target mask, and determining the target enhancement value using the audio weight factor, the spatial weight factor, and the base pixel enhancement value includes: determining the base pixel enhancement value of each pixel in each video frame of the video stream based on the category of the video object in the target mask; different categories correspond to different base pixel enhancement values; determining the audio weight factor using the sound presence confidence and a preset audio adjustment coefficient, and determining the spatial weight factor based on the pixel coordinates corresponding to each pixel, the first pixel distance from the pixel to the geometric center of the target mask, and the second pixel distance from the geometric center of the target mask to the edge of the target mask; and determining the target enhancement value based on the audio weight factor, the spatial weight factor, and the base pixel enhancement value using a clipping function.
[0030] Step S14: Perform pixel-level differential enhancement processing on the video stream using the target enhancement value and the target mask to obtain an enhanced video stream. Perform layering and smooth fusion on the enhanced video stream to obtain a target video stream. Integrate the target video stream with the audio stream to obtain a target audio and video file.
[0031] In this embodiment, after obtaining the target enhancement value, the target fusion coefficient is determined based on the target enhancement value and the preset fusion coefficient, and the corresponding formula is as follows: ; in, The target fusion coefficient; The target enhancement value; This is a preset fusion coefficient for the SR (Super Resolution) algorithm, such as 0.8, which can be adjusted according to the actual situation. After obtaining the target fusion coefficient, the pixels in each video frame are enhanced using super resolution based on the target fusion coefficient and the super resolution model to obtain the first enhanced video frame. The corresponding formula is as follows: ; in, The pixel value in the first enhanced video frame; The original pixel values before enhancement; The target fusion coefficient; The sharp pixel values generated for the super-resolution model.
[0032] Specifically, the step of performing pixel-level differential enhancement processing on the video stream using the target enhancement value and the target mask to obtain an enhanced video stream includes: determining a target fusion coefficient based on the target enhancement value and a preset fusion coefficient; generating sharp pixels corresponding to each pixel using a super-resolution model; performing super-resolution enhancement on each video frame based on the pixel, the corresponding sharp pixel, and the target fusion coefficient to obtain a first enhanced video frame; and determining the enhanced video stream using the first enhanced video frame.
[0033] It is understandable that after obtaining the first enhanced video frame, the stretching magnitude is determined based on the target enhancement value, and local contrast stretching is performed on each pixel in the first enhanced video frame to obtain the stretched video frame. The corresponding formula is as follows: ; in, The stretched video frame; These are the pixel values corresponding to the video frames before stretching; The darkest value around the pixel (x, y), i.e., the smallest pixel value; This represents the brightest value around the pixel (x, y), i.e., the maximum pixel value. The new brightness and darkness range width after contrast stretching is proportional to the target enhancement value; This is the upper limit of the new brightness range, i.e., the brightest value after stretching; This represents the lower limit of the new brightness range, i.e., the darkest value after stretching. The sharpening intensity is determined using the target enhancement value, and the corresponding formula is as follows: ; in, The sharpening intensity; The preset sharpening factor; The target enhancement value is set; then, the sharpening intensity is used to sharpen and enhance the contours of video objects in the stretched video frame to obtain a second enhanced video frame, and the corresponding formula is as follows: ; in, This is the sharpened video frame, i.e., the second enhanced video frame; These are the pixel values corresponding to the video frames before sharpening; The sharpening intensity; The Gaussian blur pixel value is denoted by .
[0034] Furthermore, after obtaining the second enhanced video frame, skin tone correction is performed on the human area to prevent skin tone from becoming too red, too pale, or too yellow due to over-enhancement. The RGB color space of the second enhanced video frame is converted to the YCbCr color space, and for the target skin tone pixels, the variation range of its Cb and Cr chromaticity channels is limited. The corresponding formula is as follows: ; in, The target color change amount after color change limitation; This represents the original color change amount before color change restriction, and the corresponding color change amount of the skin color pixel after enhancement. This is a skin color protection factor. Based on the target color change amount, color changes are restricted on the target skin color pixel set in the second enhanced video frame to obtain a third enhanced video frame, and the enhanced video stream is determined using the third enhanced video frame.
[0035] Specifically, determining the enhanced video stream using the first enhanced video frame includes: determining a stretching amplitude based on the target enhancement value, and performing local contrast stretching on each pixel in the first enhanced video frame using the stretching amplitude to obtain a stretched video frame; determining a sharpening intensity based on the target enhancement value, and performing sharpening enhancement on the contours of video objects in the stretched video frame using the sharpening intensity to obtain a second enhanced video frame; converting the RGB color space of the second enhanced video frame to the YCbCr color space, and limiting color changes of target pixels based on the YCbCr color space to obtain a third enhanced video frame, and determining the enhanced video stream using the third enhanced video frame; the target pixel is a pixel in the second enhanced video frame where the video object is a person.
[0036] In this embodiment, a transition band with a width of W is set at the boundary of the target mask. Within the transition band, the fusion weight changes linearly. Inside the target mask, the fusion weight is 1.0, and outside the target mask, the fusion weight is 0.0. Within the transition band, the fusion weight linearly decreases from 1.0 to 0.0. Based on the high-intensity enhancement regions and low-intensity enhancement regions in the enhanced video stream, the stream is split into layers of different sharpness using a Laplacian pyramid or wavelet transform. On each layer, the transition band is layered and fused based on the fusion weight after Gaussian blurring to obtain a fused transition band. The fused transition band is then smoothly integrated with each enhancement region in the enhanced video stream to obtain an integrated video stream. Finally, the global color statistics of each video frame in the integrated video stream are determined to fine-tune the fused region, making the overall picture tone uniform.
[0037] Specifically, the step of layering and smoothing the enhanced video stream to obtain the target video stream includes: generating a transition band of a preset width based on the target boundary of the target mask, and configuring corresponding fusion weights on the transition band; layering the enhanced video stream using Laplacian pyramid or wavelet transform to obtain layers of different sharpness; fusing the transition band based on the fusion weights and the layers to obtain a fused transition band; smoothly integrating the fused transition band and the enhanced video stream to obtain an integrated video stream; determining the global color statistics of each video frame in the integrated video stream, and performing global color correction on each video frame in the integrated video stream based on the global color statistics to obtain the target video stream; the global color statistics include the mean and variance of the color of each pixel in each video frame in the integrated video stream.
[0038] As can be seen from the above, this application achieves standardization and independent parsing of audio and video streams through preprocessing and extraction of the original audio and video. A pre-trained audio source separation model is used to perform object-level splitting of the mixed audio to obtain audio feature information. Video object detection or semantic segmentation is then performed on the video stream to obtain video feature information including video object categories and pixel-level locations. Next, a preset mapping rule table is used to match audio and video objects, and a target mask is generated based on the matching results, giving the video enhancement a clear direction. Basic pixel enhancement values are configured based on the video object categories in the target mask, and audio weight factors are determined by combining sound presence confidence. Spatial weight factors are determined by the spatial location features of the mask, and these three factors are combined to obtain the target enhancement value. In this way, guided by the target mask and with the target enhancement value used for intensity control, pixel-level differentiated enhancement of the video stream is achieved. Layered and smooth fusion is then used to eliminate enhancement defects, and finally, the video stream is integrated with the audio stream to obtain the target audio and video file. This accurately improves the image quality of core visual objects while ensuring the integrity and naturalness of the video image, thus enhancing the user experience.
[0039] As can be seen from the above embodiments, this application uses a target mask determined by an audio object to perform video enhancement processing. Therefore, the process of performing video enhancement processing using a target mask determined by an audio object is described.
[0040] See Figure 3 As shown, this embodiment of the invention discloses a specific video enhancement method, including: In this embodiment, the original audio and video files undergo synchronization alignment, standardization preprocessing, and extraction to obtain audio and video streams. Figure 4This is a schematic diagram of audio analysis. The audio stream is separated using a pre-trained sound source separation model to obtain the target audio track corresponding to the target category. The target audio track is then extracted to obtain audio feature information including audio object, sound start and end time, average loudness, and sound presence confidence. Figure 5 This is a schematic diagram of video analysis and processing. The video stream is preprocessed, and video object detection or image segmentation is performed on the video stream using an object detection model or semantic segmentation model to obtain video feature information including different categories of video objects and the pixel-level positions corresponding to the video objects. Based on a preset mapping rule table, the active audio objects and video objects at the current moment are matched, and the video objects of the corresponding video frames that match the active audio objects are determined as target video objects. The pixel-level positions corresponding to the target video objects are converted into binary masks to obtain the target mask.
[0041] Understandably, after obtaining the target mask, the base pixel enhancement value of each pixel in each video frame of the video stream is determined based on the category of the video object in the target mask. The target enhancement value is determined based on the base pixel enhancement value, audio weight factor, and spatial weight factor. The audio weight factor is determined based on the sound presence confidence and a preset audio adjustment coefficient. The spatial weight factor is determined using the pixel coordinates corresponding to each pixel, the first pixel distance from the pixel to the geometric center of the target mask, and the second pixel distance from the geometric center of the target mask to the edge of the target mask, thereby determining the target enhancement value.
[0042] Furthermore, a target fusion coefficient is determined based on the target enhancement value and a preset fusion coefficient. The target fusion coefficient and a super-resolution model are used to perform super-resolution enhancement on the pixels in each video frame to obtain a first enhanced video frame. A stretching amplitude is determined based on the target enhancement value, and the stretching amplitude is used to perform local contrast stretching on each pixel in the first enhanced video frame to obtain a stretched video frame. A sharpening intensity is determined based on the target enhancement value, and the sharpening intensity is used to sharpen and enhance the contours of video objects in the stretched video frame to obtain a second enhanced video frame. The color space of the second enhanced video frame is converted, and for the target skin tone pixels, the variation amplitude in the target chroma channel is limited to restrict the color variation of the target skin tone pixel set in the second enhanced video frame, resulting in a third enhanced video frame. The enhanced video stream is then determined using the third enhanced video frame.
[0043] In this embodiment, a transition band is set at the boundary of the target mask, and the fusion weights change linearly within the transition band. Based on the high-intensity enhancement regions and low-intensity enhancement regions in the enhanced video stream, the stream is split into layers of different sharpness using a Laplacian pyramid or wavelet transform. The transition band is then layer-by-layer fused on each layer based on the fusion weights after Gaussian blurring to obtain a fused transition band. This fused transition band is then smoothly integrated with each enhancement region in the enhanced video stream to obtain an integrated video stream. The global color statistics of each video frame in the integrated video stream are determined to correct the global color of the fused region, thus obtaining the target video stream.
[0044] As can be seen from the above, this application completes the matching of audio and video objects through a preset mapping rule table, generates a target mask based on the matching results, configures basic pixel enhancement values based on the video object category of the target mask, and determines the target enhancement value using audio weight factors and spatial weight factors. Guided by the target mask and with the target enhancement value controlled by intensity, pixel-level differential enhancement of the video stream is completed. Then, layering and smoothing fusion are used to eliminate enhancement defects, and global color is fine-tuned, so that the obtained target video stream can effectively improve the visual appeal of the main object and ensure the integrity and naturalness of the video image.
[0045] Accordingly, see Figure 6 As shown, this application also provides a video enhancement device, including: The audio stream separation module 11 is used to acquire the original audio and video files, preprocess and extract the original audio and video files to obtain the corresponding audio stream and video stream, separate the audio stream based on a pre-trained audio source separation model to obtain the target audio track, and determine the audio feature information corresponding to the target audio track; the audio feature information includes different categories of audio objects and sound presence confidence. The target mask determination module 12 is used to perform video object detection or semantic segmentation on the video stream to obtain video feature information, match the audio objects in the audio feature information and the video objects in the video feature information of each frame based on a preset mapping rule table, and determine the target mask using the obtained matching results; the video feature information includes different categories of video objects and the pixel-level positions corresponding to the video objects; The enhancement value determination module 13 is used to determine the basic pixel enhancement value corresponding to each pixel based on the category of the video object in the target mask, determine the audio weight factor using the sound presence confidence, determine the spatial weight factor based on the spatial location features corresponding to the target mask, and determine the target enhancement value using the audio weight factor, the spatial weight factor, and the basic pixel enhancement value. The video stream integration module 14 is used to perform pixel-level differential enhancement processing on the video stream using the target enhancement value and the target mask to obtain an enhanced video stream, perform layering and smooth fusion on the enhanced video stream to obtain a target video stream, and integrate the target video stream with the audio stream to obtain a target audio and video file.
[0046] In some specific embodiments, the audio stream separation module 11 may specifically include: The file extraction unit is used to acquire the original audio and video files, perform preprocessing on the original audio and video files including timestamp alignment and standardization to obtain processed audio and video files, and extract the processed audio and video files to obtain the corresponding audio stream and video stream. An audio stream separation unit is used to determine a pre-trained audio source separation model based on a deep neural network, and to separate the audio stream using the audio source separation model to obtain the target audio track; The audio track analysis unit is used to analyze each of the target audio tracks to obtain different categories of audio objects, sound start and end times, average loudness, and sound presence confidence; the sound presence confidence is a confidence level determined based on the instantaneous loudness of the audio object and using a clipping function and a preset gating function. The trajectory coordinate adding unit is used to add the sound trajectory coordinates corresponding to the audio object to the audio feature information if the audio stream is 3D audio.
[0047] In some specific embodiments, the target mask determination module 12 may specifically include: The video stream recognition unit is used to recognize the video stream using an object detection model or a semantic segmentation model to obtain different categories of video objects and the pixel-level positions corresponding to the video objects; The target object determination unit is used to construct a preset mapping rule table based on different categories of audio objects and corresponding video objects, match audio objects in the audio feature information and video objects in the video feature information of the same frame using the preset mapping rule table, and determine the successfully matched video objects as target video objects. A pixel-level position conversion unit is used to convert the pixel-level position corresponding to the target video object into a binary mask if the number of the target video objects is 1, so as to obtain the target mask. An audio information determination unit is used to determine whether sound trajectory coordinates exist in the audio feature information of the audio object if the number of the target video objects is greater than 1. The target mask determination unit is used to determine the target mask based on the pixel-level position of the video object corresponding to the sound trajectory coordinates if the sound trajectory coordinates exist. The final object selection unit is used to select a final video object from the target video objects based on a preset rule if the sound trajectory coordinates do not exist, and to determine the target mask based on the pixel-level position corresponding to the final video object.
[0048] In some specific embodiments, the enhancement value determination module 13 may specifically include: The base enhancement value determination unit is used to determine the base pixel enhancement value of each pixel in each video frame of the video stream based on the category of the video object in the target mask; different categories correspond to different base pixel enhancement values; The weighting factor determination unit is used to determine the audio weighting factor by using the sound presence confidence and the preset audio adjustment coefficient, and to determine the spatial weighting factor based on the pixel coordinates corresponding to each pixel, the first pixel distance from the pixel to the geometric center of the target mask, and the second pixel distance from the geometric center of the target mask to the edge of the target mask. The target enhancement value determination unit is used to determine the target enhancement value based on the audio weight factor, the spatial weight factor, and the basic pixel enhancement value, and using a clipping function.
[0049] In some specific embodiments, the video stream integration module 14 may specifically include: A clear pixel generation unit is used to determine the target fusion coefficient based on the target enhancement value and the preset fusion coefficient, and to generate clear pixels corresponding to each pixel using a super-resolution model. The video frame enhancement submodule is used to perform super-resolution enhancement on each video frame based on the pixel, the corresponding clear pixel and the target fusion coefficient to obtain a first enhanced video frame, and to determine the enhanced video stream using the first enhanced video frame.
[0050] In some specific embodiments, the video frame enhancement submodule may specifically include: A pixel stretching unit is used to determine the stretching range based on the target enhancement value, and to use the stretching range to perform local contrast stretching on each pixel in the first enhanced video frame to obtain a stretched video frame. A contour sharpening unit is used to determine the sharpening intensity based on the target enhancement value, and use the sharpening intensity to sharpen and enhance the contour of the video object in the stretched video frame to obtain a second enhanced video frame. A pixel color limiting unit is used to convert the RGB color space of the second enhanced video frame to the YCbCr color space, and to limit the color change of the target pixel based on the YCbCr color space to obtain a third enhanced video frame, and to determine the enhanced video stream using the third enhanced video; the target pixel is the pixel in the second enhanced video frame where the video object is a person object.
[0051] In some specific embodiments, the video stream integration module 14 may specifically include: The transition band generation unit is used to generate a transition band of a preset width based on the target boundary of the target mask, and to configure corresponding fusion weights for the transition band; Transition band layering unit, used to layer based on the enhanced video stream and using Laplacian pyramid or wavelet transform to obtain layers of different resolutions; A transition band fusion unit is used to fuse the transition band based on the fusion weight and the layer to obtain a fused transition band; The transition band integration unit is used to smoothly integrate the fused transition band and the enhanced video stream to obtain the integrated video stream; A color correction unit is used to determine the global color statistics of each video frame in the integrated video stream, and to perform global color correction on each video frame in the integrated video stream based on the global color statistics to obtain the target video stream; the global color statistics include the mean and variance of the color of each pixel in each video frame in the integrated video stream.
[0052] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the video enhancement method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0053] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0054] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0055] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the video enhancement method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.
[0056] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned video enhancement method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0057] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0058] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0059] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0060] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0061] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A video enhancement method, characterized in that, include: The original audio and video files are acquired, preprocessed, and extracted to obtain corresponding audio and video streams. The audio streams are then separated based on a pre-trained audio source separation model to obtain the target audio track, and the audio feature information corresponding to the target audio track is determined. The audio feature information includes different categories of audio objects and the confidence level of sound presence. The video stream is subjected to video object detection or semantic segmentation to obtain video feature information. Based on a preset mapping rule table, the audio objects in the audio feature information and the video objects in the video feature information of each frame are matched, and the target mask is determined using the obtained matching results. The video feature information includes different categories of video objects and the pixel-level positions corresponding to the video objects. The base pixel enhancement value corresponding to each pixel is determined based on the category of the video object in the target mask, the audio weight factor is determined using the sound presence confidence, the spatial weight factor is determined based on the spatial location features corresponding to the target mask, and the target enhancement value is determined using the audio weight factor, the spatial weight factor, and the base pixel enhancement value. The video stream is subjected to pixel-level differential enhancement processing using the target enhancement value and the target mask to obtain an enhanced video stream. The enhanced video stream is then layered and smoothly fused to obtain a target video stream. Finally, the target video stream is integrated with the audio stream to obtain a target audio and video file.
2. The video enhancement method according to claim 1, characterized in that, The process of acquiring the original audio and video files, preprocessing and extracting the original audio and video files to obtain corresponding audio and video streams, separating the audio streams based on a pre-trained audio source separation model to obtain the target audio track, and determining the audio feature information corresponding to the target audio track includes: The original audio and video files are obtained, and preprocessing including timestamp alignment and normalization is performed on the original audio and video files to obtain processed audio and video files. The processed audio and video files are then extracted to obtain the corresponding audio stream and video stream. A pre-trained audio source separation model is determined based on a deep neural network, and the audio stream is separated using the audio source separation model to obtain the target audio track; Each of the target audio tracks is analyzed to obtain different categories of audio objects, sound start and end times, average loudness, and sound presence confidence; the sound presence confidence is a confidence level determined based on the instantaneous loudness of the audio object and using a clipping function and a preset gating function. If the audio stream is 3D audio, then the sound trajectory coordinates corresponding to the audio object are added to the audio feature information.
3. The video enhancement method according to claim 2, characterized in that, The process of performing video object detection or semantic segmentation on the video stream to obtain video feature information, matching audio objects in the audio feature information and video objects in the video feature information of each frame based on a preset mapping rule table, and determining the target mask using the obtained matching results includes: The video stream is identified using an object detection model or a semantic segmentation model to obtain different categories of video objects and their corresponding pixel-level positions. A preset mapping rule table is constructed based on different categories of audio objects and corresponding video objects. Based on the audio objects in the audio feature information and the video objects in the video feature information of the same frame, the preset mapping rule table is used for matching, and the successfully matched video objects are determined as target video objects. If the number of target video objects is 1, then the pixel-level position corresponding to the target video object is converted into a binary mask to obtain the target mask; If the number of target video objects is greater than 1, then determine whether there are sound trajectory coordinates in the audio feature information of the audio object; If the sound trajectory coordinates exist, then the target mask is determined based on the pixel-level position of the video object corresponding to the sound trajectory coordinates; If the sound trajectory coordinates do not exist, a final video object is selected from the target video objects based on preset rules, and a target mask is determined based on the pixel-level position corresponding to the final video object.
4. The video enhancement method according to claim 1, characterized in that, The process of determining the base pixel enhancement value corresponding to each pixel based on the category of the video object in the target mask, determining the audio weight factor using the sound presence confidence, determining the spatial weight factor based on the spatial location features corresponding to the target mask, and determining the target enhancement value using the audio weight factor, the spatial weight factor, and the base pixel enhancement value includes: The base pixel enhancement value of each pixel in each video frame of the video stream is determined based on the category of the video object in the target mask; different categories correspond to different base pixel enhancement values; The audio weighting factor is determined by the sound presence confidence and the preset audio adjustment coefficient, and the spatial weighting factor is determined based on the pixel coordinates corresponding to each pixel, the first pixel distance from the pixel to the geometric center of the target mask, and the second pixel distance from the geometric center of the target mask to the edge of the target mask. The target enhancement value is determined based on the audio weighting factor, the spatial weighting factor, and the base pixel enhancement value, using a clipping function.
5. The video enhancement method according to claim 4, characterized in that, The step of performing pixel-level differential enhancement processing on the video stream using the target enhancement value and the target mask to obtain the enhanced video stream includes: The target fusion coefficient is determined based on the target enhancement value and the preset fusion coefficient, and the clear pixels corresponding to each pixel are generated using the super-resolution model. Super-resolution enhancement is performed on each video frame based on the pixel, the corresponding clear pixel, and the target fusion coefficient to obtain a first enhanced video frame, and the enhanced video stream is determined using the first enhanced video frame.
6. The video enhancement method according to claim 5, characterized in that, The step of determining the enhanced video stream using the first enhanced video frame includes: The stretching range is determined based on the target enhancement value, and the stretching range is used to perform local contrast stretching on each pixel in the first enhanced video frame to obtain the stretched video frame. The sharpening intensity is determined based on the target enhancement value, and the sharpening intensity is used to sharpen and enhance the contours of video objects in the stretched video frame to obtain a second enhanced video frame. The RGB color space of the second enhanced video frame is converted to the YCbCr color space, and the color change of the target pixel is restricted based on the YCbCr color space to obtain the third enhanced video frame. The enhanced video stream is determined using the third enhanced video frame. The target pixel is the pixel in the second enhanced video frame where the video object is a person.
7. The video enhancement method according to any one of claims 1 to 6, characterized in that, The step of performing layered and smooth fusion of the enhanced video stream to obtain the target video stream includes: A transition band of a preset width is generated based on the target boundary of the target mask, and a corresponding fusion weight is configured for the transition band; Based on the enhanced video stream, layers of different resolutions are obtained by using Laplacian pyramids or wavelet transforms for layering. The transition band is fused based on the fusion weight and the layer to obtain the fused transition band; The fused transition band and the enhanced video stream are smoothly integrated to obtain the integrated video stream; The global color statistics of each video frame in the integrated video stream are determined, and the global color of each video frame in the integrated video stream is corrected based on the global color statistics to obtain the target video stream; the global color statistics include the mean and variance of the color of each pixel in each video frame in the integrated video stream.
8. A video enhancement device, characterized in that, include: The audio stream separation module is used to acquire the original audio and video files, preprocess and extract the original audio and video files to obtain the corresponding audio stream and video stream, separate the audio stream based on a pre-trained audio source separation model to obtain the target audio track, and determine the audio feature information corresponding to the target audio track; the audio feature information includes different categories of audio objects and sound presence confidence; The target mask determination module is used to perform video object detection or semantic segmentation on the video stream to obtain video feature information, match the audio objects in the audio feature information and the video objects in the video feature information of each frame based on a preset mapping rule table, and determine the target mask using the obtained matching results; the video feature information includes different categories of video objects and the pixel-level positions corresponding to the video objects; The enhancement value determination module is used to determine the basic pixel enhancement value corresponding to each pixel based on the category of the video object in the target mask, determine the audio weight factor using the sound presence confidence, determine the spatial weight factor based on the spatial location features corresponding to the target mask, and determine the target enhancement value using the audio weight factor, the spatial weight factor, and the basic pixel enhancement value. The video stream integration module is used to perform pixel-level differential enhancement processing on the video stream using the target enhancement value and the target mask to obtain an enhanced video stream, perform layering and smooth fusion on the enhanced video stream to obtain a target video stream, and integrate the target video stream with the audio stream to obtain a target audio and video file.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the video enhancement method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the video enhancement method as described in any one of claims 1 to 7.