A method for intelligent enhancement and feature processing of film and television images

Abstract

The application provides a kind of intelligent enhancement and feature processing method for film and television image, it is related to image processing field, its steps include: by constructing lightweight adaptive convolution module, the cross-scale feature of single frame film and television image is decomposed, and multi-resolution pyramid level is formed, and the temporal correlation between adjacent frames is combined, multi-scale features are reconstructed and unified feature space fusion Feature flow, obtain film and television image feature representation;Introduce the feature level adaptive enhancement mechanism of time sequence perception, based on multi-scale structural saliency information and interframe temporal consistency constraint, the dynamic selective strengthening and redundancy suppression of film and television image feature representation are carried out, and the enhanced film and television image feature representation is obtained;Construct feature-driven adaptive reconstruction unit, according to the enhanced film and television image feature representation, the content-aware pixel-level reconstruction and detail compensation of single frame film and television image are carried out, and enhanced film and television image is generated.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and specifically relates to an intelligent enhancement and feature processing method for film and television images. Background Technology

[0002] With the rapid development of the film and television industry, the demand for the production and dissemination of high-quality film and television content is constantly growing. During the actual filming process, due to the complex lighting conditions, differences in shooting equipment, and dynamic changes in scenes, film and television images often suffer from quality problems such as underexposure, noise interference, low contrast, color distortion, and blurred details. These problems not only directly affect the visual presentation of film and television works and the audience's viewing experience, but also bring significant challenges to the post-production stage. For example, tasks such as image restoration, color correction, special effects compositing, and automated video analysis all require high-quality image data as a foundation. Inferior images will seriously reduce the efficiency and accuracy of post-production. Therefore, it is urgent to develop more robust and generalizable image enhancement processing methods to achieve high-quality brightness adjustment, detail enhancement, and structural optimization of film and television images, thereby improving visual effects and providing a reliable foundation for subsequent production and intelligent analysis.

[0003] Traditional image enhancement methods for film and television, including histogram equalization, Retinex algorithm, frequency domain filtering, and adaptive brightness adjustment, can improve the visual effect of images to a certain extent and enhance local contrast and detail by directly processing the brightness, contrast, or frequency domain features of the image. They have achieved certain application results, especially in simple scenes with underexposure or low noise. With the rise of deep learning technology and the rapid development of models such as convolutional neural networks, generative adversarial networks, and transformers, end-to-end learning frameworks have been gradually introduced into the field of film and television image enhancement. These methods can automatically learn the brightness distribution, texture features, and structural information of images and are widely used in specific tasks such as low-light enhancement, image denoising, dehazing, super-resolution reconstruction, and style transfer. Compared with traditional methods, they have certain advantages in handling complex scenes.

[0004] While existing methods have made some progress in film and television image enhancement and feature extraction, they still have some limitations. First, traditional algorithms are less adaptable to complex lighting changes, dynamic exposure, and strong noise interference, and are prone to problems such as loss of image details, structural distortion, or over-enhancement after processing. Second, existing deep learning methods usually rely on large-scale natural image datasets for training, which makes it difficult to fully adapt to the diverse shooting scenes, dynamic content characteristics, and professional color space requirements in film and television images, resulting in insufficient generalization ability. Third, most methods lack targeted and independent control mechanisms for high-frequency details and low-frequency structures in film and television images, and cannot achieve scene-adaptive intelligent enhancement based on the content characteristics of different frames, making it difficult to balance structural integrity and detail richness. Summary of the Invention

[0005] This invention proposes an intelligent enhancement and feature processing method for film and television images. Addressing issues such as image detail loss, structural blurring, and temporal instability caused by drastic lighting changes, large motion amplitudes, significant scale differences, and strong temporal correlations in complex film and television scenes, this invention proposes a multi-stage processing mechanism integrating cross-scale feature decomposition fusion, temporally aware feature-level adaptive enhancement, and feature-driven adaptive reconstruction. The method constructs a lightweight adaptive convolution module to perform cross-scale feature decomposition on a single frame of film and television image, forming a multi-resolution pyramid hierarchy. Combining the temporal correlation between adjacent frames, it reconstructs feature flows from multiple scales and fuses them in a unified feature space to obtain a temporally coherent film and television image feature representation. A temporally aware feature-level adaptive enhancement mechanism is introduced, dynamically selectively enhancing and suppressing redundancy in the film and television image feature representation based on multi-scale structural saliency information and inter-frame temporal consistency constraints, resulting in an enhanced film and television image feature representation. Finally, a feature-driven adaptive reconstruction unit is constructed to perform content-aware pixel-level reconstruction and detail compensation on a single frame of film and television image based on the enhanced film and television image feature representation, generating the enhanced film and television image.

[0006] A method for intelligent enhancement and feature processing of film and television images, the specific method is as follows: S1. Acquire film and television video data, parse the video data into a continuous sequence of single-frame film and television images, perform standardization and noise reduction preprocessing on the single-frame film and television images, and construct a film and television image dataset. S2. Construct a lightweight adaptive convolution module to perform cross-scale feature decomposition on single-frame film and television images in the film and television image dataset, construct a multi-resolution pyramid hierarchy, and perform feature flow reconstruction and unified feature space fusion on the multi-resolution pyramid hierarchy based on the temporal correlation between adjacent frames to obtain the feature representation of film and television images. S3. Based on the film and television image feature representation, a time-aware feature-level adaptive enhancement mechanism is constructed. The mechanism dynamically and selectively enhances and suppresses redundancy in the feature response of the film and television image feature representation according to the multi-scale structural saliency information and the inter-frame temporal consistency constraint, so as to obtain the enhanced film and television image feature representation. S4. Based on the enhanced film and television image feature representation, a feature-driven adaptive reconstruction unit is constructed to perform content-aware pixel-level reconstruction and detail compensation on the enhanced film and television image feature representation to generate the enhanced film and television image. S5. The processing steps S2 to S4 are integrated into an intelligent enhancement and feature processing model for film and television images. The model integrates cross-scale feature decomposition and fusion, time-aware feature-level adaptive enhancement, and feature-driven adaptive reconstruction functions to achieve enhancement processing of film and television images.

[0007] Preferably, for the construction of the film and television image dataset, film and television video data are acquired using multiple shooting devices under different scenes and lighting conditions to cover the visual differences of film and television images under different shooting environments, enrich the diversity of film and television image data, and the video data is parsed into a continuous single-frame film and television image sequence through video decoding technology to convert the time-series video data into frame-level image data that can be processed independently. The obtained single-frame video images are standardized sequentially, including unifying image resolution, pixel format and color space to eliminate differences caused by different shooting devices and encoding methods. After standardization, the single-frame video images are subjected to noise suppression processing. An adaptive median filtering algorithm is used to suppress salt-and-pepper noise in the single-frame video images to remove impulse noise while maintaining the edge information of the video images. Gaussian filtering is combined to smooth Gaussian noise in the single-frame video images to reduce random noise and improve the overall smoothness of the video images. The preprocessed single-frame video image sequence is filtered by inter-frame temporal verification to remove abnormally distorted frames. The inter-frame correlation information is used to identify and remove abnormally distorted frames caused by shooting failures, transmission errors, etc., to avoid poor data affecting the subsequent processing effect, so as to obtain a single-frame video image sequence with complete frame order and construct a video image dataset.

[0008] Preferably, in step S2, a lightweight adaptive convolution module is constructed. The module consists of two fully connected layers and an activation function. A single frame of a video image from the video image dataset is input into the lightweight adaptive convolution module, and a dynamic convolution kernel is output. The dynamic convolution kernel is used to perform sliding window convolution on the single frame of the video image to obtain a low-resolution feature map. Through multiple iterations of the above process, a multi-resolution pyramid hierarchy is constructed. The Lucas-Kanade optical flow algorithm is used to calculate the basic optical flow fields of the current frame and the previous frame to obtain pixel-level motion offset information. The feature maps of each pyramid level of the previous frame are spatially aligned with the feature maps of the corresponding pyramid level of the current frame. For each level, the aligned features of the previous frame and the features of the current frame are added element by element to obtain a single-level feature flow. By mapping the feature streams of each level to the same channel dimension through convolution, and concatenating all the feature streams of each level by channel, a multi-scale fused feature is obtained. The multi-scale fused feature is then subjected to convolutional dimensionality reduction and normalization to obtain a temporally coherent film and television image feature representation.

[0009] Furthermore, by constructing a lightweight adaptive convolution module and generating dynamic convolution kernels in step S2, the convolution parameters can be adaptively adjusted according to the content features of different single-frame video images. This effectively reduces the model parameter size and computational complexity while ensuring feature representation capabilities, thus balancing feature extraction accuracy and operational efficiency. By constructing a multi-resolution pyramid hierarchy, hierarchical modeling of structural information at different scales in video images is achieved, which helps enhance the model's comprehensive representation capabilities of detailed features and global semantic information. The optical flow algorithm is used to model pixel-level motion information between adjacent frames and spatially align feature mappings at different scales, ensuring that features between consecutive frames maintain consistency in the temporal dimension and effectively alleviating feature misalignment problems caused by motion changes or viewpoint changes. Through layer-by-layer fusion of feature flows and channel-level integration of multi-scale features, temporal and multi-scale information can be fully utilized to enhance the temporal coherence and robustness of features.

[0010] Preferably, in step S3, based on the multi-resolution pyramid hierarchy constructed in step S2, the feature response map of each level is obtained, the L2 norm and spatial gradient value of the feature response map of each level are calculated, and a multi-scale structural saliency heatmap within a single frame is generated by weighting. Based on the feature correspondence between adjacent frames established in step S2, the cosine similarity between the feature response maps of the current frame and adjacent frames is calculated in a unified feature space to obtain an inter-frame temporal consistency weight map. The inter-frame temporal consistency weight map is then multiplied element-wise with the multi-scale structural saliency heatmap to obtain a spatiotemporal joint saliency weight map. Based on the spatiotemporal joint saliency weight map, feature enhancement gating weights are generated. The feature response map is then gated and modulated according to the gating weights. Feature residual enhancement is introduced into feature regions with high gating weights, while feature regions with low gating weights are suppressed through a channel attention mechanism, resulting in an enhanced feature response map. A temporal smoothing constraint is introduced into the enhanced feature response map, and the gating weights of the current frame and the previous frame are smoothed by an exponential moving average. The dimension of the enhanced feature response map after the temporal smoothing constraint is calibrated by channel mapping, and the enhanced film and television image feature representation is output.

[0011] Furthermore, by constructing a time-aware feature-level adaptive enhancement mechanism in step S3, key regions with significant structural information and stable temporal characteristics in film and television images can be highlighted at the feature level, effectively suppressing background redundancy and noise interference. By comprehensively measuring the amplitude and spatial gradient of the multi-resolution pyramid-level feature responses, a multi-scale structural saliency heatmap within a single frame is constructed, which helps enhance the model's ability to perceive edges, textures, and structurally changing regions. By combining the temporal consistency weights between features of adjacent frames, joint modeling of spatial saliency and temporal stability is achieved, ensuring that salient regions are not only discriminative in space but also maintain continuity and consistency in the temporal dimension. Through a gated modulation and residual enhancement strategy based on spatiotemporal joint saliency weights, high-value feature regions can be strengthened, while feature channels with low contribution or instability are adaptively suppressed, thereby improving the discriminability and robustness of feature representation. By introducing temporal smoothing constraints and channel dimension calibration mechanisms, the drastic fluctuations of gate weights between frames are reduced, ensuring the smoothness and stability of enhanced features in the temporal series, ultimately outputting a compact, temporally coherent, and expressive feature representation of film and television images.

[0012] Preferably, in step S4, a feature-driven adaptive reconstruction unit is constructed based on the enhanced film and television image feature representation. The unit includes a feature mapping module, a structure reconstruction module, and a detail compensation module. The feature mapping module performs channel dimension transformation and spatial scale mapping on the enhanced video image feature representation to obtain an intermediate reconstructed feature map. The feature mapping module includes a channel transformation unit and a scale mapping unit. The structure reconstruction module restores the basic structural information of a single frame of video image based on the intermediate reconstruction feature map. Through multi-level spatial reconstruction and feature fusion operations, it gradually reconstructs the overall outline and spatial layout of the video image to generate an initial reconstructed video image. The detail compensation module performs detail enhancement processing on the initial reconstructed video image. Based on the local response information in the feature representation of the enhanced video image, it adaptively compensates for edges, textures, and high-frequency details to generate the enhanced video image.

[0013] Furthermore, by constructing a feature-driven adaptive reconstruction unit in step S4, the enhanced feature representation of the film and television image is deeply coupled with the image reconstruction process. This allows the reconstruction process to be guided by both high-level semantic features and structural information, thereby avoiding structural distortion problems caused by relying solely on pixel-level interpolation or fixed reconstruction strategies. The feature mapping module performs unified mapping of feature channel dimensions and spatial scales, achieving effective alignment between feature representation and reconstruction space, providing a stable intermediate representation basis for subsequent structural restoration. The structural reconstruction module, through multi-level spatial reconstruction and feature fusion, can gradually restore the overall contour and spatial layout of the film and television image, ensuring consistency and integrity of the reconstruction results at the macroscopic structural level. The detail compensation module, based on the local response information in the enhanced features, adaptively compensates for edges, textures, and high-frequency details, effectively compensating for the lack of detail information during structural reconstruction, thereby improving the clarity and detail of the film and television image while maintaining overall structural accuracy.

[0014] Preferably, in step S5, step S2 represents a cross-scale feature decomposition and fusion unit, which performs cross-scale feature decomposition and temporal feature fusion on the film and television image to generate a feature representation of the film and television image. Step S3 is represented as a time-aware feature-level adaptive enhancement unit, which performs dynamic selective enhancement and redundancy suppression on the feature representation of film and television images to generate an enhanced feature representation of film and television images. Step S4 refers to a feature-driven adaptive reconstruction unit, which performs film and television image reconstruction and detail compensation based on the enhanced film and television image feature representation to generate an enhanced film and television image. Steps S2 to S4 construct an intelligent enhancement and feature processing model for generating film and television images, thereby achieving enhancement processing of film and television images.

[0015] Furthermore, based on the constructed intelligent enhancement and feature processing model for film and television images, cross-scale feature decomposition and temporal feature fusion processing are performed on the input film and television images. Multi-resolution feature modeling suppresses interference from complex backgrounds and highlights key structural information, improving the discriminativeness and stability of feature representation. A temporally aware feature-level adaptive enhancement mechanism is introduced to dynamically and selectively enhance and suppress redundancy in feature responses at different scales and temporal dimensions, achieving focused enhancement of significant structural regions and temporally consistent regions, ensuring the coherence and accuracy of feature representation in spatial and temporal dimensions. Through feature-driven adaptive reconstruction and detail compensation units, structural reconstruction and detail enhancement of film and television images are performed under the guidance of high-level semantic features, effectively improving the overall structural integrity and local detail representation capabilities of film and television images. The method of this invention has significant advantages in cross-scale feature modeling, temporal consistency enhancement, and feature-guided reconstruction, enabling high-quality intelligent enhancement processing of film and television images under complex scene conditions.

[0016] Compared with the prior art, the present invention has the following technical effects: This invention constructs an intelligent enhancement and feature processing model that combines cross-scale feature decomposition and fusion, temporally aware feature-level adaptive enhancement, and feature-driven adaptive reconstruction. This model achieves collaborative modeling of film and television images across spatial and temporal dimensions, effectively overcoming the instability of enhancement effects caused by relying solely on single-scale features or neglecting temporal consistency in existing methods. By introducing a multi-resolution pyramid structure and dynamic convolution mechanism, the invention improves the ability to express structural information at different scales while reducing computational complexity while maintaining feature discriminative power. By combining multi-scale structural saliency with inter-frame temporal consistency constraints, selective enhancement and redundancy suppression of key feature regions are achieved, significantly improving the robustness and temporal coherence of feature representation. Through feature-driven adaptive reconstruction and detail compensation, the film and television image reconstruction process is guided by high-level features, effectively enhancing edge, texture, and high-frequency detail representation while ensuring overall structural consistency. In summary, this invention enables high-quality intelligent enhancement processing of film and television images under complex scene conditions. Attached Figure Description

[0017] Figure 1 This is a flowchart of an intelligent enhancement and feature processing method for film and television images provided by the present invention.

[0018] Figure 2 This is a structural diagram of the cross-scale feature decomposition and fusion unit provided by the present invention.

[0019] Figure 3 This is a structural diagram of the time-aware feature-level adaptive enhancement unit provided by the present invention.

[0020] Figure 4 This is a structural diagram of the feature-driven adaptive reconstruction unit provided by the present invention.

[0021] Figure 5 This is a pre-enhancement image of the video image provided by the present invention.

[0022] Figure 6 This is an image showing the enhanced video image provided by the present invention. Detailed Implementation

[0023] This invention proposes an intelligent enhancement and feature processing method for film and television images. Addressing issues such as image detail loss, structural blurring, and temporal instability caused by drastic lighting changes, large motion amplitudes, significant scale differences, and strong temporal correlations in complex film and television scenes, this invention proposes a multi-stage processing mechanism integrating cross-scale feature decomposition fusion, temporally aware feature-level adaptive enhancement, and feature-driven adaptive reconstruction. The method constructs a lightweight adaptive convolution module to perform cross-scale feature decomposition on a single frame of film and television image, forming a multi-resolution pyramid hierarchy. Combining the temporal correlation between adjacent frames, it reconstructs feature flows from multiple scales and fuses them in a unified feature space to obtain a temporally coherent film and television image feature representation. A temporally aware feature-level adaptive enhancement mechanism is introduced, dynamically selectively enhancing and suppressing redundancy in the film and television image feature representation based on multi-scale structural saliency information and inter-frame temporal consistency constraints, resulting in an enhanced film and television image feature representation. Finally, a feature-driven adaptive reconstruction unit is constructed to perform content-aware pixel-level reconstruction and detail compensation on a single frame of film and television image based on the enhanced film and television image feature representation, generating the enhanced film and television image.

[0024] Please see Figure 1 As shown in the embodiment of this application, there is an intelligent enhancement and feature processing method for film and television images.

[0025] S1. Acquire film and television video data, parse the video data into a continuous sequence of single-frame film and television images, perform standardization and noise reduction preprocessing on the single-frame film and television images, and construct a film and television image dataset.

[0026] Furthermore, under different scenarios and lighting conditions, various shooting devices are used to acquire film and television video data. The shooting devices include different models of cameras or video acquisition terminals. During the acquisition process, video resolution, frame rate, and lighting condition information are recorded. The video frame rate is set to 25fps, and the video resolution remains unchanged according to the original device output. The FFmpeg video decoding tool is used to decode the film and television video data frame by frame, parsing the video data into a continuous sequence of single-frame film and television images arranged in chronological order, and assigning a corresponding frame number and timestamp information to each frame of film and television images. The obtained single-frame video images are standardized sequentially. First, the single-frame video images are uniformly adjusted to a preset resolution of 640×360, and the size is scaled using bilinear interpolation. Then, the pixel format of the video images is unified by converting the original pixel format to RGB24 format and the color space of the video images from RGB to YCbCr color space. After the standardization process is completed, the single-frame video images are subjected to noise suppression processing. An adaptive median filtering algorithm is used to process salt-and-pepper noise. The initial window size of the adaptive median filter is set to 3×3, and the maximum window size is set to 7×7. After the median filtering is completed, Gaussian filtering is used to smooth the video images, and the Gaussian filter kernel size is set to 5×5. The preprocessed single-frame video image sequence is filtered by inter-frame temporal verification. The inter-frame temporal verification is based on the video image difference information between adjacent frames. The average pixel difference between two adjacent frames is calculated and compared with a preset threshold. The threshold is set by statistical method as the mean difference between adjacent frames plus twice the standard deviation. When the difference between adjacent frames exceeds the threshold, the corresponding video image frame is judged as an abnormal frame and is removed. Finally, the single-frame video image sequence with continuous frame order and no obvious distortion is retained. The filtered video image frames are stored uniformly to construct a video image dataset.

[0027] S2. Construct a lightweight adaptive convolution module to perform cross-scale feature decomposition on single-frame film and television images in the film and television image dataset, construct a multi-resolution pyramid hierarchy, and perform feature flow reconstruction and unified feature space fusion on the multi-resolution pyramid hierarchy based on the temporal correlation between adjacent frames to obtain the feature representation of film and television images.

[0028] Furthermore, in step S2, the feature representation of the film and television image is obtained, and the process is as follows: Figure 2 As shown, the specific steps are as follows: In this embodiment, the first image in the video image dataset constructed in step S1 is... Frame image denoted as A lightweight adaptive convolutional module is constructed, which consists of two fully connected layers and a non-linear activation function. The lightweight adaptive convolution module first performs global average pooling on the input image to obtain channel-level feature vectors. ,in It is a layer in a multi-resolution pyramid. The channel-level feature vectors are sequentially input into the first fully connected layer and the second fully connected layer, and dynamic weight coefficients are generated through an activation function. Based on the dynamic weight coefficients, a dynamic convolution kernel is generated by linearly combining the preset M basic convolution kernels, and the calculation formula is as follows: ; in, It is the first Frame number The dynamic convolutional kernels corresponding to the layer pyramid levels, where M is the number of basic convolutional kernels. In practice, M is set to 4. It is the first Frame number The weight coefficients corresponding to the m-th basic convolutional kernel in layer m. It is the first At the layer scale, the m-th basic convolutional kernel; Utilize the generated dynamic convolution kernels to analyze the features of the current layer. Perform a sliding window convolution operation with a kernel size of 3×3 and a stride of 2 to obtain the next layer of feature maps. The calculation formula is: ; in, It is the ReLU activation function. It is a convolution operation; Through multiple iterations of the above dynamic convolution and downsampling process, a multi-resolution pyramid hierarchy is constructed step by step, with the number of pyramid levels set to 4.

[0029] In this embodiment, the Lucas-Kanade optical flow algorithm is used to calculate the input video image of the current frame in the pixel domain. Input video image from the previous frame Based on the fundamental optical flow field between them, pixel-level motion offset information is obtained; using this information, the feature maps of each pyramid level in the previous frame are spatially aligned with the feature maps of the corresponding pyramid level in the current frame, calculated using the following formula: ; in, It is to put the previous frame in the 1st position The feature maps at each scale are spatially resampled based on the information, and the resulting feature maps are aligned with the coordinates of the current frame. It is the first Frame relative to the first In the optical flow of a frame, position Horizontal displacement component, It is the first Frame relative to the first In the optical flow of a frame, position The vertical displacement component; After spatial alignment is completed, for each pyramid level, the current frame feature map is... Aligned feature mapping with the previous frame Element-wise addition is performed to obtain the feature flow representation of the corresponding level; The feature streams at different levels are mapped to a unified channel dimension through a 1×1 convolution operation, with the number of unified channels set to 64. The feature streams at each level are upsampled to ensure consistent spatial resolution, and then concatenated according to the channel dimension to obtain multi-scale fused features. The multi-scale fused features are then subjected to 3×3 convolution for dimensionality reduction, and batch normalization is combined to complete feature normalization, outputting a temporally coherent representation of film and television image features.

[0030] S3. Based on the aforementioned film and television image feature representation, a time-aware feature-level adaptive enhancement mechanism is constructed. The mechanism dynamically and selectively enhances and suppresses redundancy in the feature response of the film and television image feature representation according to multi-scale structural saliency information and inter-frame temporal consistency constraints, thereby obtaining the enhanced film and television image feature representation.

[0031] Furthermore, in step S3, the enhanced feature representation of the film and television image is obtained, and the process is as follows: Figure 3 As shown, the specific steps are as follows: In this embodiment, based on the multi-resolution pyramid hierarchy constructed in step S2, for each pyramid level... Obtain the feature response map, and calculate the L2 norm of the feature response map at each level. The calculation formula is as follows: ; in, It is the first Frame number Layer, in L2 norm value of the location, It is the first The total number of channels in the layer feature response map. It is the first Frame number The spatial location of the c-th channel in the layer feature response map Eigenvalues ​​at; The gradient information of the feature response amplitude map is calculated in the spatial dimension by calculating the first-order difference in the horizontal and vertical directions respectively to obtain the spatial gradient value. The L2 norm of the feature response map and the spatial gradient value are weighted and fused according to a preset weight to obtain the first... Structural saliency response diagram of layers The calculation formula is: ; in, and These are weighting coefficients; during implementation, Set to 0.5. Set to 0.5; After normalizing and upsampling the structural saliency response maps at each level to a uniform spatial resolution, element-wise weighted fusion is performed to generate a multi-scale structural saliency heatmap within a single frame. .

[0032] In this embodiment, after obtaining the multi-scale structural saliency heatmap, based on the feature correspondence between adjacent frames established in step S2, the cosine similarity between the feature response maps of the current frame and the previous frame is calculated in a unified feature space. After normalizing and scale-aligning the similarity at each level, an inter-frame temporal consistency weight map is obtained. The inter-frame temporal consistency weight map is multiplied element-wise with the multi-scale structural saliency heatmap to obtain the spatiotemporal joint saliency weight map. ; Based on the spatiotemporal joint saliency weight map, feature-enhanced gating weights are generated, and these gating weights are mapped to the feature channel dimensions at each level. For the , Layer, in spatial location At this point, the feature response map is modulated according to the gating weights. When the gating weights are large, feature residual enhancement is introduced. The calculation formula is as follows: ; in, It is the first Frame number The spatial location of the c-th channel in the layer feature response map Eigenvalues ​​after residual enhancement; When the gating weights are small, the feature response map is suppressed using channel attention. First, the first... Layer channel attention weight The feature response map is suppressed based on the channel attention weights, and the calculation formula is as follows: ; in, No. Frame number The spatial location of the c-th channel in the layer feature response map Eigenvalues ​​after channel attention suppression; Based on the features after residual enhancement and the features after channel attention suppression, the enhanced feature response map is obtained; The gating weights are subject to an exponential moving average smoothing constraint in the time dimension, resulting in smoothed gating weights for the current frame. The calculation formula is: ; in, It is a smoothing coefficient, which is used during implementation. Set to 0.8, yes Frame smoothing gate weights; Based on the smoothed gating weights, the enhanced feature response map is re-modulated, and channel mapping and dimension calibration are performed through 1×1 convolution to output the enhanced film and television image feature representation.

[0033] S4. Based on the enhanced film and television image feature representation, construct a feature-driven adaptive reconstruction unit to perform content-aware pixel-level reconstruction and detail compensation on the enhanced film and television image feature representation to generate the enhanced film and television image.

[0034] Furthermore, in step S4, the enhanced video image is generated, and the process is as follows: Figure 4 As shown, the specific steps are as follows: In this embodiment, a feature-driven adaptive reconstruction unit is constructed based on the enhanced film and television image feature representation. The unit includes a feature mapping module, a structure reconstruction module, and a detail compensation module. The feature mapping module uses 1×1 convolution to perform channel mapping on the enhanced video image feature representation, adjusting the input channel number C to the preset reconstruction channel number. The calculation formula is: ; in, It is the first Frame output feature map at position The feature value of the c-th channel, where k is the input channel index. , These are the weighting coefficients used when mapping input channel k to output channel c. It is the first Frame input image at position The eigenvalue of the k-th channel, It is the constant bias corresponding to output channel c. ; Bilinear interpolation is used to upsample the feature map after channel dimension transformation, mapping the spatial resolution of the feature map to the same resolution as the target reconstructed image, thus obtaining the intermediate reconstructed feature map. .

[0035] In this embodiment, the structure reconstruction module employs a multi-level spatial reconstruction approach, processing the intermediate reconstructed feature maps step-by-step through multiple convolutional operations. Let the output of the k-th level structure reconstruction be... The calculation process is as follows: ; in, It is the ReLU activation function. It is the k-th level structure reconstruction convolution kernel, with the kernel size set to 3×3. It's a convolution operation. It is a bias term; Through progressive convolutional reconstruction and feature fusion operations, the overall outline, regional structure, and spatial layout of the film and television image are gradually restored, and the initial reconstructed film and television image is finally obtained.

[0036] In this embodiment, the detail compensation module uses the enhanced video image feature representation output in step S3 as a guiding input, models the local response information in the feature representation through multi-layer convolution, and retains high-frequency response components during the convolution process, thereby generating a detail compensation term consistent with the spatial resolution of the target reconstructed image; the detail compensation term is then superimposed element-wise with the initial reconstructed video image to obtain the enhanced video image.

[0037] S5. The processing steps S2 to S4 are integrated into an intelligent enhancement and feature processing model for film and television images. The model integrates cross-scale feature decomposition and fusion, time-aware feature-level adaptive enhancement, and feature-driven adaptive reconstruction functions to achieve enhancement processing of film and television images.

[0038] Furthermore, in step S5, the specific steps for constructing the intelligent enhancement and feature processing model for film and television images are as follows: In this embodiment, step S2 represents a cross-scale feature decomposition and fusion unit, which takes the preprocessed single-frame video image as input. The effect image before video image enhancement is shown below. Figure 5 As shown, by using a multi-resolution pyramid structure and a temporal feature alignment mechanism, feature decomposition of film and television images at different spatial scales is achieved, and temporal information between adjacent frames is fused to output a film and television image feature representation in a unified feature space. Step S3 represents the temporal-aware feature-level adaptive enhancement unit, which generates feature-level gating weights based on multi-scale structural saliency information and inter-frame temporal consistency information, performs selective enhancement and redundancy suppression on the feature representation of film and television images, and outputs the enhanced feature representation of film and television images. Step S4 represents the feature-driven adaptive reconstruction unit. It sequentially passes through three sub-modules: feature mapping, structural reconstruction, and detail compensation. The enhanced features are mapped to the target film / video image space, and the overall structure of the film / video image is reconstructed step-by-step. Furthermore, the local high-frequency response information from the enhanced features is used to compensate for details in the reconstruction results. Finally, the enhanced film / video image is output. The enhanced film / video image is shown in the image below. Figure 6 As shown; Steps S2 to S4 construct an intelligent enhancement and feature processing model for generating film and television images, thereby achieving enhancement processing of film and television images.

[0039] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these modifications and improvements all fall within the protection scope of the present invention.

Claims

1. A method for intelligent enhancement and feature processing of film and television images, characterized in that, Includes the following steps: S1. Acquire film and television video data, parse the video data into a continuous sequence of single-frame film and television images, perform standardization and noise reduction preprocessing on the single-frame film and television images, and construct a film and television image dataset. S2. Construct a lightweight adaptive convolution module to perform cross-scale feature decomposition on single-frame film and television images in the film and television image dataset, construct a multi-resolution pyramid hierarchy, and perform feature flow reconstruction and unified feature space fusion on the multi-resolution pyramid hierarchy based on the temporal correlation between adjacent frames to obtain the feature representation of film and television images. S3. Based on the film and television image feature representation, a time-aware feature-level adaptive enhancement mechanism is constructed. The mechanism dynamically and selectively enhances and suppresses redundancy in the feature response of the film and television image feature representation according to the multi-scale structural saliency information and the inter-frame temporal consistency constraint, so as to obtain the enhanced film and television image feature representation. S4. Based on the enhanced film and television image feature representation, a feature-driven adaptive reconstruction unit is constructed to perform content-aware pixel-level reconstruction and detail compensation on the enhanced film and television image feature representation to generate the enhanced film and television image. S5. The processing steps S2 to S4 are integrated into an intelligent enhancement and feature processing model for film and television images. The model integrates cross-scale feature decomposition and fusion, time-aware feature-level adaptive enhancement, and feature-driven adaptive reconstruction functions to achieve enhancement processing of film and television images.

2. The intelligent enhancement and feature processing method for film and television images according to claim 1, characterized in that, Under different scenes and lighting conditions, various shooting devices are used to acquire film and television video data, and the video data is parsed into a continuous sequence of single-frame film and television images using video decoding technology; The obtained single-frame video images are standardized sequentially, including unifying image resolution, pixel format and color space. After standardization, noise suppression is performed on the single-frame video images. An adaptive median filtering algorithm is used to suppress salt-and-pepper noise in the single-frame video images, and Gaussian filtering is combined to smooth Gaussian noise in the single-frame video images. The preprocessed single-frame video image sequence is filtered by inter-frame temporal verification to remove abnormal and distorted frames, resulting in a single-frame video image sequence with complete frame order, and thus constructing a video image dataset.

3. The intelligent enhancement and feature processing method for film and television images according to claim 2, characterized in that, A lightweight adaptive convolution module is constructed, which consists of two fully connected layers and an activation function. A single frame of a video image from the video image dataset is input into the lightweight adaptive convolution module, and a dynamic convolution kernel is output. The dynamic convolution kernel is used to perform sliding window convolution on the single frame of the video image to obtain a low-resolution feature map. Through multiple iterations of the above process, a multi-resolution pyramid hierarchy is constructed. The Lucas-Kanade optical flow algorithm is used to calculate the basic optical flow fields of the current frame and the previous frame to obtain pixel-level motion offset information. The feature maps of each pyramid level of the previous frame are spatially aligned with the feature maps of the corresponding pyramid level of the current frame using the information. For each level, the aligned features of the previous frame are added element-wise to the features of the current frame to obtain a single-level feature stream; By mapping the feature streams of each level to the same channel dimension through convolution, and concatenating all the feature streams of each level by channel, a multi-scale fused feature is obtained. The multi-scale fused feature is then subjected to convolutional dimensionality reduction and normalization to obtain a temporally coherent film and television image feature representation.

4. The intelligent enhancement and feature processing method for film and television images according to claim 3, characterized in that, Based on the multi-resolution pyramid hierarchy constructed in step S2, the feature response maps of each level are obtained, the L2 norm and spatial gradient values ​​of the feature response maps of each level are calculated, and a multi-scale structural saliency heatmap within a single frame is generated by weighting. Based on the feature correspondence between adjacent frames established in step S2, the cosine similarity between the feature response maps of the current frame and adjacent frames is calculated in a unified feature space to obtain an inter-frame temporal consistency weight map. The inter-frame temporal consistency weight map is then multiplied element-wise with the multi-scale structural saliency heatmap to obtain a spatiotemporal joint saliency weight map. Based on the spatiotemporal joint saliency weight map, feature enhancement gating weights are generated. The feature response map is then gated and modulated according to the gating weights. Feature residual enhancement is introduced into feature regions with high gating weights, while feature regions with low gating weights are suppressed through a channel attention mechanism, resulting in an enhanced feature response map. A temporal smoothing constraint is introduced into the enhanced feature response map, and the gating weights of the current frame and the previous frame are smoothed by an exponential moving average. The dimension of the enhanced feature response map after the temporal smoothing constraint is calibrated by channel mapping, and the enhanced film and television image feature representation is output.

5. The intelligent enhancement and feature processing method for film and television images according to claim 4, characterized in that, Based on the enhanced film and television image feature representation, a feature-driven adaptive reconstruction unit is constructed, which includes a feature mapping module, a structure reconstruction module, and a detail compensation module. The feature mapping module performs channel dimension transformation and spatial scale mapping on the enhanced video image feature representation to obtain an intermediate reconstructed feature map. The feature mapping module includes a channel transformation unit and a scale mapping unit. The structure reconstruction module restores the basic structural information of a single frame of video image based on the intermediate reconstruction feature map. Through multi-level spatial reconstruction and feature fusion operations, it gradually reconstructs the overall outline and spatial layout of the video image to generate an initial reconstructed video image. The detail compensation module performs detail enhancement processing on the initial reconstructed video image. Based on the local response information in the feature representation of the enhanced video image, it adaptively compensates for edges, textures, and high-frequency details to generate the enhanced video image.

6. The intelligent enhancement and feature processing method for film and television images according to claim 5, characterized in that, Step S2 is represented as a cross-scale feature decomposition and fusion unit, which performs cross-scale feature decomposition and temporal feature fusion on the film and television image to generate a feature representation of the film and television image. Step S3 is represented as a time-aware feature-level adaptive enhancement unit, which performs dynamic selective enhancement and redundancy suppression on the feature representation of film and television images to generate an enhanced feature representation of film and television images. Step S4 refers to a feature-driven adaptive reconstruction unit, which performs film and television image reconstruction and detail compensation based on the enhanced film and television image feature representation to generate an enhanced film and television image. Steps S2 to S4 construct an intelligent enhancement and feature processing model for generating film and television images, thereby achieving enhancement processing of film and television images.

Images