An image sequence flicker elimination method and system based on clustering and multi-scale histogram matching

Abstract

The application discloses a kind of based on clustering and multi-scale histogram matching image sequence flicker elimination method and system, it is related to image processing technical field, method includes: using non-local mean denoising algorithm to execute denoising processing to image and obtain denoised image sequence, then calculate histogram feature, using K-Means clustering algorithm to group image in denoised image sequence, to determine the class label of each frame image;For denoised image sequence, image with class label as standard class is regarded as reference, multi-scale histogram matching is executed to image with class label as abnormal class, and all scales under optimization image are weighted fusion by Gaussian kernel function to obtain optimized image sequence;For optimized image sequence, cross-fade technology is used to carry out time sequence smooth transition processing at the junction of image of different class labels, to obtain final image sequence.The application can effectively eliminate the flicker of image sequence, and improve visual experience.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for eliminating flicker in image sequences based on clustering and multi-scale histogram matching. Background Technology

[0002] In digital image sequence processing, such as video and motion image capture, it is common to encounter situations where some images in an image sequence differ significantly in brightness, contrast, or color distribution from other images in the sequence due to changes in shooting conditions, content switching, or other reasons. When such image sequences are played continuously, these differences manifest as a visual "flickering" phenomenon, where the image brightness or hue abruptly changes between different frames, severely impacting the viewing experience.

[0003] In existing technologies, commonly used flicker suppression methods include simple inter-frame interpolation and global histogram equalization, but these methods have the following drawbacks: 1) They do not consider the structural differences in image content: global adjustment ignores the feature differences in local areas of the image, which may lead to distortion of details; 2) They lack grouping processing strategies: the image sequences are not reasonably grouped, making it difficult to address the distribution differences between images in different groups; 3) The transition effect is abrupt: simple interpolation or single-scale matching cannot achieve a natural and smooth transition between groups, and visual discontinuities still exist. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for eliminating flicker in image sequences based on clustering and multi-scale histogram matching, which can effectively eliminate flicker in image sequences and improve the visual experience.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] In a first aspect, this application provides an image sequence flicker elimination method based on clustering and multi-scale histogram matching, including:

[0007] A nonlocal mean denoising algorithm is used to perform denoising processing on each frame of the image sequence to obtain a denoised image sequence.

[0008] Histogram features are calculated for each frame in the denoised image sequence. Then, based on the histogram features, the K-Means clustering algorithm is used to group the images in the denoised image sequence to determine the category label of each frame; wherein, the category label is either the standard class or the outlier class.

[0009] For the denoised image sequence, the images with the standard class label are used as the baseline, and multi-scale histogram matching is performed on the images with the abnormal class label. The optimized images at all scales are then weighted and fused using a Gaussian kernel function to obtain the optimized image sequence.

[0010] For the optimized image sequence, a cross-gradient technique is used at the boundary between images of different category labels to perform temporal smoothing transition processing to obtain the final image sequence.

[0011] Secondly, this application provides an image sequence flicker elimination system based on clustering and multi-scale histogram matching, which applies an image sequence flicker elimination method based on clustering and multi-scale histogram matching. The system includes:

[0012] The denoising module is used to perform denoising processing on each frame of the image sequence using a non-local mean denoising algorithm to obtain a denoised image sequence.

[0013] The grouping module is used to calculate histogram features for each frame of the denoised image sequence, and then use the K-Means clustering algorithm to group the images in the denoised image sequence based on the histogram features to determine the category label of each frame; wherein the category label is a standard class or an anomaly class;

[0014] The histogram matching module is used to perform multi-scale histogram matching on images with the standard class as a reference and images with the abnormal class as a reference for the denoised image sequence, and to perform weighted fusion of the optimized images at all scales through the Gaussian kernel function to obtain the optimized image sequence.

[0015] The smooth transition module is used to perform temporal smooth transition processing at the boundaries of images with different category labels for the optimized image sequence, using cross-gradient technology, to obtain the final image sequence.

[0016] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application employs a nonlocal mean denoising algorithm to perform denoising processing on each frame of the image sequence, which can reduce noise in the image and achieve robust preprocessing. Secondly, histogram features are calculated for each frame of the denoised image sequence. Based on this, the intrinsic shape feature vector of the image in the color histogram space can be obtained. Then, according to the distribution of histogram features in the manifold space, the K-Means clustering algorithm is used for adaptive grouping to determine the category label of each frame image. This step is key to solving the periodic flickering problem because it can automatically identify frames with normal visual features that can be used as standards (category label: standard class) and flickering groups with abnormal visual characteristics that need adjustment (category label: anomalous class). For images with category labels of standard class and images with category labels of anomalous class, multi-scale histogram matching is performed, and the optimized images at all scales are weighted and fused using a Gaussian kernel function to obtain the optimized image sequence. This processing effectively eliminates flickering in the image sequence, ensures information complementarity at different scales, and effectively avoids block artifacts. Finally, for the optimized image sequence, a cross-gradient technique is used at the boundary between images of different category labels to perform temporal smoothing processing to obtain the final image sequence; the cross-gradient technique used can ensure the visual continuity of the image sequence and improve the visual experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating an image sequence flicker elimination method based on clustering and multi-scale histogram matching in one embodiment of this application.

[0019] Figure 2 This is a flowchart illustrating an image sequence flicker elimination method based on clustering and multi-scale histogram matching in another embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] This application improves upon the limitations of traditional global histogram matching to the multi-scale local block level, and through a sophisticated weighted fusion strategy, effectively preserves the inherent local texture and edge details of the image while eliminating global luminosity inconsistencies, thereby effectively eliminating flicker in image sequences and enhancing the visual experience.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] In one exemplary embodiment, such as Figure 1 and Figure 2 As shown, an image sequence flicker elimination method based on clustering and multi-scale histogram matching is provided. This method is executed by a computer device, specifically by a computer device such as a terminal or server alone, or by a terminal and server together. In the embodiments of this application, it includes the following steps 101 to 104.

[0025] Step 101: The nonlocal mean denoising algorithm is used to perform denoising processing on each frame of the image sequence to obtain the denoised image sequence.

[0026] To ensure the accuracy of subsequent feature extraction and cluster analysis, the received image sequence first needs to undergo denoising preprocessing. This application employs the Non-Local Means (NLM) denoising algorithm, which is highly favored in the field of image denoising due to its excellent detail preservation capabilities and allows for adjustment of denoising intensity as needed. The core idea of ​​NLM is to utilize the self-similarity in the image, determining the new value of the pixel by weighted averaging of image patches similar to the neighborhood of the pixel to be processed. Specifically, when performing denoising processing on a frame of an image using the Non-Local Means denoising algorithm, the following formula is used:

[0027]

[0028] Among them, I denoise (x) represents the intensity value of pixel x in the denoised image, determined by its weighted average neighborhood value; C(x) is the weight normalization factor, which must ensure that the sum of the normalized weights is 1 to avoid image brightness distortion; Ω xI(y) is the search neighborhood of pixel x in the denoised image, and I(y) is the intensity value of pixel y in the image; ω(x,y) is the weight, which is used to reflect the similarity between image patches centered on pixel x and pixel y, and is usually measured by Gaussian weighted Euclidean distance; Patch(x) is the image patch centered on pixel x, Patch(y) is the image patch centered on pixel y, and h is the filtering parameter.

[0029] The above denoising process can effectively improve the signal-to-noise ratio of the image, providing a cleaner data foundation for subsequent feature extraction. In a specific application, the parameters of the NLM algorithm are set as follows: intensity parameter h∈[3,10], search window size is 21×21, and neighborhood window size is 7×7.

[0030] Step 102: Calculate histogram features for each frame of the denoised image sequence, and then use the K-Means clustering algorithm to group the images in the denoised image sequence based on the histogram features to determine the category label of each frame; wherein, the category label corresponds to the group category, which is either the standard class or the abnormal class.

[0031] Specifically, the category labels are binary labels generated through K-Means clustering (based on the histogram features of the images), divided into standard and anomalous classes. The standard class corresponds to image groups in the sequence with normal histogram features (reflecting characteristics such as image brightness, contrast, and color distribution), serving as the standard group and the correction benchmark. The anomalous class corresponds to image groups in the sequence whose histogram features differ significantly from those of the standard class image groups. The purpose of this classification is to achieve targeted flicker elimination by "correcting the anomalous group with the standard group." This grouping provides the data foundation for subsequent processing.

[0032] To effectively group image sequences, it is necessary to extract features that reflect the global visual characteristics of the images. Color histograms describe the distribution of pixel intensity values ​​in an image and have excellent characterization capabilities for brightness, contrast, and color. Therefore, this application selects a multi-channel normalized color histogram as the intrinsic shape descriptor of the image's color space distribution. The histogram is essentially a discrete approximation of the probability density function (PDF) of the pixel intensity distribution. For each frame in the denoised image sequence... Calculate its histogram features, which are the normalized color histogram features h of each color channel (e.g., R, G, B channels). c The combination of these steps specifically includes the following:

[0033] (21) For each frame of the image, the normalized color histogram feature of any color channel is calculated using the following formula:

[0034]

[0035] Among them, h c (i) represents the normalized color histogram feature of gray level i in the c-th color channel (with values ​​ranging from [0,1]), H c (i) is the pixel count (non-negative integer) of gray level i in the c-th color channel; ∈ is a preset minimal positive number, specifically a minimal positive number to avoid division by zero error.

[0036] (22) Concatenate the normalized color histogram features of all color channels to obtain the histogram features of the image. That is, concatenate the normalized color histogram features of the three color channels into a histogram feature: This vector represents the overall luminous properties of the image and can be viewed as the image's coordinates in the color histogram space.

[0037] Assume that the frames in the denoised image sequence form a manifold composed of multiple clusters in the histogram feature space. Some clusters correspond to frames with normal visual characteristics, labeled as the standard class; while other clusters correspond to frames with visual abnormalities caused by periodic or non-periodic changes (such as exposure flicker), labeled as the anomalous class. To identify these clusters in an unsupervised manner, this application employs the K-Means clustering algorithm.

[0038] In one application, when grouping images in a denoised image sequence using the K-Means clustering algorithm based on histogram features, the K-Means clustering algorithm is used to initialize centroids. Then, the data is partitioned by minimizing the sum of the squared Euclidean distances from samples within a cluster to their corresponding centroids. The corresponding function formula is as follows:

[0039]

[0040] Where E is the sum of squared Euclidean distances from the sample to the corresponding centroid, f m It is the histogram feature of the m-th frame image, referring to the sample; C j It is the j-th cluster, μ j Let K = 2 be the centroid of the j-th cluster, and K = 2 be the number of clusters. Choosing K = 2 effectively divides the image sequence into two main categories: "standard class" and "outlier class". Furthermore, the initialization of the centroids in the clustering algorithm is implemented using the K-means++ algorithm, which improves convergence speed and clustering performance.

[0041] Step 103: For the denoised image sequence, using the images labeled as standard class as a baseline, and for the images labeled as anomaly class, perform multi-scale histogram matching, and then use a Gaussian kernel function to weightedly fuse the optimized images at all scales to obtain the optimized image sequence. In a specific application, step 103 includes the following steps:

[0042] (31) Traverse the denoised image sequence, determine multiple image segments with consecutive identical category labels based on the category label of each frame, and record the segment category label of each image segment; in practical applications, in addition to recording the segment category label, the start index and end index are also recorded, and a unique image segment is determined by the start index, segment category label, and end index. Through this segmentation process, the global processing problem of the image sequence can be transformed into targeted processing of discrete image segments.

[0043] (32) For any abnormal image segment, select the corresponding standard image segment that meets the preset conditions; wherein, the abnormal image segment is an image segment with the segment category label as abnormal, and the standard image segment is an image segment with the segment category label as standard.

[0044] In a specific application, the immediately preceding standard class image segment in the entire image sequence can be selected as the corresponding standard image segment; wherein, the abnormal image segment is the image segment whose category label is abnormal after being grouped based on the K-Means clustering algorithm, and the standard image segment is the image segment whose category label is standard after being grouped based on the K-Means clustering algorithm.

[0045] (33) Based on the standard image segments, determine the reference image; in one application, for an abnormal image segment that requires histogram adjustment, the immediately preceding standard image segment can be selected, and then the last frame from this standard image can be selected as the reference image I. ref This strategy provides the most relevant and stable visual benchmark while ensuring temporal continuity. Furthermore, the benchmark image can be denoised as needed before subsequent processing.

[0046] In another application, for abnormal image segments that require histogram adjustment, one or more preset standard image segments can be selected; in terms of selecting the reference image, the average / representative image of the last few frames of the standard image segment that are immediately adjacent to the abnormal image segment can also be selected as the reference image.

[0047] (34) Based on the reference image, perform multi-scale histogram matching on each abnormal image in the abnormal image segment, and perform weighted fusion of the optimized images at all scales through the Gaussian kernel function to obtain the optimized image; all the optimized images constitute the optimized image segment; all the optimized image segments and all the standard image segments constitute the optimized image sequence.

[0048] Histogram matching, also known as histogram specification, is a technique that maps the histogram distribution of a source image to an anomalous histogram distribution. Its core principle utilizes the monotonicity of the cumulative distribution function (CDF), which provides the cumulative probability that a pixel value is less than or equal to a specific value. The goal of histogram matching is to find a mapping function T such that the CDF of the mapped pixel value v = T(u) of the source image is as close as possible to the CDF of the anomalous image. To preserve image details while correcting the histogram distribution, this application employs a multi-scale histogram matching strategy.

[0049] Based on the above principle, in step (34) of this application, multi-scale histogram matching is performed on each abnormal image in the abnormal image segment according to the reference image, including the following steps:

[0050] 1) The reference image is divided into overlapping blocks at multiple preset scales to obtain a set of standard image blocks corresponding to different scales; for any set of standard image blocks at any scale, the cumulative distribution function of each standard image block is calculated.

[0051] 2) For any abnormal image in the abnormal image segment, overlap and divide it into blocks at multiple preset scales to obtain a set of abnormal image blocks corresponding to different scales; for any set of abnormal image blocks at any scale, calculate the cumulative distribution function of each abnormal image block.

[0052] This application extends global histogram matching to the multi-scale block level through the aforementioned block segmentation steps. This strategy excels in preserving local image details, and the overlapping strategy employed during block segmentation further ensures smoother transitions between blocks. Specifically, for the reference image I... ref Or abnormal image I tar The segmentation method used is the same, which involves multiple preset scales S = s1, s2, ..., s n Overlapping blocks are performed on (for example, 128, 64, 32 pixels). Furthermore, when performing overlapping blocks, a step size t = s × t(1-θ) is used, where θ ∈ (0,1) is the overlap rate, which can be set to 0.5 to ensure smooth transitions between blocks.

[0053] After dividing the image into blocks, the normalized histogram of each color channel is calculated for each block, and then the cumulative distribution function (CDF) of each color channel is calculated and stored as a standard CDF map. If an edge block exceeds the image boundary, the starting coordinates are adjusted to ensure that the block is always within the image. Specifically, the calculation formula for the cumulative distribution function of the standard image block and the cumulative distribution function of the abnormal image block is the same, as shown below: Among them, F c (k) is the cumulative distribution function of the c-th color channel, satisfying F c (255)≈1(because h) c (i) already normalized), h c (i) represents the normalized color histogram feature of gray level i in the c-th color channel.

[0054] 3) For any given block at different scales, the cumulative distribution functions (CDFs) of multiple anomalous image blocks are matched with the cumulative distribution functions (CDFs) of multiple standard image blocks to generate a lookup table. Specifically, the lookup table is generated by matching the CDF of the anomalous image block with the standard CDF of the corresponding (or most similar) standard image block. This lookup table maps the pixel values ​​of the anomalous image block to new values, making its CDF approach the CDF of the corresponding standard image block. The mapping process uses the nearest neighbor method, expressed as:

[0055]

[0056] Among them, LUT s (p) is a lookup table at scale s, CDF ref,c (j) is the cumulative distribution function (CDF) of the standard image patch j. tar,c (p) is the cumulative distribution function of the abnormal image patch p.

[0057] 4) According to the lookup table, the pixel values ​​of each abnormal image block in the abnormal image are adjusted to obtain optimized images at different scales.

[0058] Based on the above principle, in step (34) of this application, the optimized images at all scales are weighted and fused using a Gaussian kernel function to obtain the optimized image. That is, the matching result at each scale will generate a corrected image, i.e., the scale-optimized image. To fuse these results into a final image, the following steps are included:

[0059] 1) A Gaussian kernel function based on the center position of image patches is used to calculate the weight of each image patch in the optimized image at each scale. This application uses Gaussian weights based on the center position of image patches for weighted fusion to ensure the smoothness of the contribution. This weight allocation strategy gives higher weights to the center region of the patch, while the weights of the edge regions are smoothly decayed, thereby effectively avoiding the block effect. The corresponding Gaussian weight ω(x,y) is calculated as follows:

[0060]

[0061] Among them, (c x ,c y ) represents the center coordinates of the image patch, while Let (x, y) be the normalized radius, and (x, y) be the coordinates of any pixel within the current image patch.

[0062] 2) Normalize the weights of the optimized images at all scales to obtain the corresponding normalized weights, ensuring that the sum of the weights is 1 when the blocks at each scale are fused, thus avoiding brightness distortion of the fused image;

[0063] 3) The normalized weights of the optimized images at all scales are weighted and superimposed to generate a single frame of optimized image. The final fused pixel intensity I result It is the superposition of weighted matching results at all scales, which is the result of the optimized image.

[0064] Step 104: For the optimized image sequence, a cross-gradient technique is used at the intersection of images with different category labels to perform temporal smoothing transition processing to obtain the final image sequence.

[0065] To avoid visual abrupt changes during image group switching, this application applies cross-fading technology at the boundary (i.e., when transitioning from one image segment to another image segment from a different group), thereby avoiding visual abrupt changes and performing smooth transition processing. The cross-fading technology, within a preset 2N frame transition interval, transforms the last image I of the previous standard image segment... prev Image I after matching the current anomalous image segment curr A weighted average is used to achieve a smooth visual transition. Specifically, this includes the following steps:

[0066] (41) At the intersection of images with different category labels, determine the transition interval with a preset frame number of 2N.

[0067] (42) Within the transition interval of a preset number of frames, a weighted average is calculated by combining multiple images of the previous category label with multiple images of the next category label. The calculation formula is as follows: I blend =(1-α) γ )·I prev +αγ ·I curr ; where I blend is the image after weighted average fusion, I prev is the image of the previous class label, I curr is the image of the subsequent class label; the fusion coefficient α γ =(γ + 1) / N (γ is the number of fusions, 0 ≤ γ < N), and it gradually changes from 0 to 1 with the number of frames.

[0068] During the weighted average process, specifically, the images of two different class labels at symmetric positions are weighted averaged, and the fusion coefficient is determined according to their positions, and then fed back to the corresponding positions. In a specific application, the preset number of frames N takes values from 2 to 10 frames. Taking N = 4 as an example, the last 4 frames of the standard class paragraph are [N1, N2, N3, N4], and the first 4 frames of the abnormal class paragraph are [T1, T2, T3, T4]. The fusion rules are N4 and T1 (α0 = 0.25), N3 and T2 (α1 = 0.5), N2 and T3 (α2 = 0.75), N1 and T4 (α3 = 1.0); the fusion coefficient α γ increases from 0.25 to 1.0 to ensure a natural transition.

[0069] Finally, the image frames processed through the above steps (including image denoising, grouping, matching, and transition smoothing) are arranged in the original time sequence and output as a stable image sequence without obvious flicker. This application is applicable to various "flickering" video processing, has the advantages of strong grouping pertinence, multi-scale detail retention, natural transition, etc., and can effectively eliminate the flicker of the image sequence and improve the visual experience.

[0070] This application also provides an application scenario. Running the above method in the Python 3.x environment, the main libraries relied on include OpenCV, Numpy, Scikit-learn, etc. In actual applications, the following parameters can be set and adjusted according to specific requirements:

[0071] (1) The initial centroids of the K-Means clustering algorithm: The K-means algorithm can be used to select the initial centroids to improve the clustering effect.

[0072] (2) The scale set for multi-scale histogram matching: The appropriate scale set can be selected according to the image resolution and detail features, such as {128, 64, 32} pixels.

[0073] (3) The overlap rate of the blocks: When performing block processing on standard images and abnormal images, the overlap rate can be set, such as 0.5, to ensure smooth transition between blocks.

[0074] (4) Number of transition frames: When smoothing the transition between groups, the number of transition frames can be set, such as 2-10 frames. Choose the appropriate number of transition frames according to actual needs to achieve a natural transition effect.

[0075] (5) Parameters of the nonlocal mean denoising algorithm: including intensity parameter h, color intensity parameter hcolor, search window size and neighborhood window size, which can be adjusted according to the image noise characteristics.

[0076] Based on the same inventive concept, this application also provides a system for implementing the methods described above. The solution provided by this system is similar to the solution described in the methods above; therefore, specific limitations in one or more system embodiments provided below can be found in the limitations of the methods described above, and will not be repeated here.

[0077] In one exemplary embodiment, this application provides an image sequence flicker elimination system based on clustering and multi-scale histogram matching, comprising:

[0078] The denoising module is used to perform denoising processing on each frame of the image sequence using a non-local mean denoising algorithm to obtain a denoised image sequence.

[0079] The grouping module is used to calculate histogram features for each frame of the denoised image sequence, and then use the K-Means clustering algorithm to group the images in the denoised image sequence based on the histogram features to determine the category label of each frame; wherein the category label is a standard class or an anomaly class.

[0080] The histogram matching module is used to perform multi-scale histogram matching on images labeled as standard class as a baseline and images labeled as abnormal class for the denoised image sequence, and to perform weighted fusion of the optimized images at all scales through a Gaussian kernel function to obtain the optimized image sequence.

[0081] The smooth transition module is used to perform temporal smooth transition processing at the boundaries of images with different category labels for the optimized image sequence, using cross-gradient technology, to obtain the final image sequence.

[0082] In practical applications, the system of this application also includes an initialization and image loading module, and a storage and control module. The initialization and image loading module is used to specify the input image folder path and the output folder path, and also to name image files in numerical order (e.g., 001.jpg, 002.jpg, etc.), define the scale parameters for multi-scale matching (e.g., [128, 64, 32]), the number of frames for smooth transition (e.g., 10 frames or 2 frames in the code), and the denoising intensity (e.g., 3.0). The storage and control module is used to store intermediate data and final results, and to control the entire processing flow.

[0083] Compared with the prior art, this application has the following advantages:

[0084] (1) Adaptive Grouping Processing: First, a nonlocal means denoising algorithm is used to preprocess the image sequence to reduce noise interference. Then, K-Means clustering is used to automatically identify image groups with different histogram features, without needing to know the flickering pattern in advance or manually specify the groups. In this application, the K-Means clustering algorithm is used to automatically group the image sequence, with a cluster size of 2, and Euclidean distance is used as the similarity measure between samples. This algorithm can divide the image sequence into two image groups with different histogram distribution features based on the color histogram features of the image, providing a foundation for subsequent multi-scale histogram matching and transition smoothing processing.

[0085] (2) Significant flicker elimination effect: Multi-scale histogram matching can effectively adjust the visual characteristics (brightness, contrast, color) of different image groups to a consistent level, thereby eliminating flicker. This application uses multi-scale histogram matching to divide the image into blocks at multiple scales, calculates and matches the cumulative distribution function of the blocks, and realizes the mapping of pixel values ​​through a lookup table, so that the histogram distribution of abnormal image blocks approaches that of standard image blocks. Gaussian weighted fusion is used between the matching results at different scales, which effectively preserves the multi-scale detail features of the image, while realizing the consistent adjustment of the histogram distribution.

[0086] (3) Smooth and natural transition: Gaussian weighted fusion and cross-gradient frame interpolation techniques are used at the image group switching points to ensure the smoothness of the visual transition and avoid new abrupt changes. At the boundaries of different image groups, this application uses cross-gradient technology to achieve a smooth transition and avoid visual abrupt changes when switching image groups.

[0087] (4) Innovative Breakthrough in Multi-Scale Processing: Addressing the limitation of existing histogram matching techniques in simultaneously eliminating global flicker and preserving local details, this application innovatively employs a multi-scale block matching strategy. Multiple block scales with different granularities are preset. Large-scale blocks are used to correct global luminance distribution differences, eliminating cross-frame flicker at its source; medium- and small-scale blocks are used to accurately match the feature distribution of local regions (such as textures and edges), avoiding local detail distortion caused by global adjustments.

[0088] (5) Optimized Design of Gaussian Weighted Fusion: To address the block effect (visual discontinuity at block boundaries) and information fragmentation at different scales that easily occur after multi-scale block matching, this application innovatively introduces a weighted fusion strategy based on the Gaussian kernel function. Through weighted fusion, the advantages of global consistency at large scales are efficiently integrated with the advantages of local detail at medium and small scales, resulting in a final optimized image free from both global flicker and local discontinuities, significantly improving visual continuity and detail integrity. This fusion strategy further amplifies the technical advantages of multi-scale processing, providing key support for achieving high-quality flicker elimination.

[0089] (6) Robustness: This application can still effectively cope with changes in image characteristics with non-fixed periods.

[0090] This application can adaptively identify image groups with different features in an image sequence and adjust their visual characteristics accordingly, while smoothing the transition between groups to effectively eliminate flickering.

[0091] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements an image sequence flicker elimination method based on clustering and multi-scale histogram matching.

[0092] Those skilled in the art will understand that Figure 3The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0093] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0094] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0095] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0096] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0097] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0098] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for eliminating flicker in a sequence of images based on clustering and multiscale histogram matching, characterized in that, The method comprises: The non-local mean denoising algorithm is used to perform denoising processing on each frame of image in the image sequence to obtain a denoised image sequence; The histogram feature of each frame of image in the denoised image sequence is calculated, and then the K-Means clustering algorithm is used to group the images in the denoised image sequence according to the histogram feature to determine the class label of each frame of image; wherein the class label is a standard class or an abnormal class; For the denoised image sequence, the images with the class label of the standard class are used as the reference, and the images with the class label of the abnormal class are subjected to multi-scale histogram matching, and the optimized images under all scales are fused by the Gaussian kernel function to obtain an optimized image sequence; comprising: The denoised image sequence is traversed, the multiple image segments with the same class label in succession are determined according to the class label of each frame of image, and the segment class label of each image segment is recorded; for any abnormal image segment, the corresponding standard image segment meeting the preset condition is selected; wherein the abnormal image segment is an image segment with the segment class label of the abnormal class, and the standard image segment is an image segment with the segment class label of the standard class; the reference image is determined based on the standard image segment; multi-scale histogram matching is performed on each abnormal image in the abnormal image segment according to the reference image, and the optimized images under all scales are fused by the Gaussian kernel function to obtain an optimized image; all the optimized images constitute an optimized image segment; all the optimized image segments and all the standard image segments constitute an optimized image sequence; For the optimized image sequence, a cross-fading technology is used to perform time sequence smooth transition processing at the boundary of images with different class labels to obtain a final image sequence.

2. The image sequence flicker removal method based on clustering and multi-scale histogram matching according to claim 1, characterized in that, When the non-local mean denoising algorithm is used to perform denoising processing on a frame of image, the following formula is used: ； ； wherein is the intensity value of pixel x in the denoised image, is a weight normalization factor, is the search neighborhood of pixel x in the denoised image, is the intensity value of pixel x in the denoised image, is the intensity value of pixel x in the denoised image, is the intensity value of pixel x in the denoised image, is a weight reflecting the similarity between the image patches centered at pixel x and pixel y, is the image patch centered at pixel x, is the image patch centered at pixel x, is the image patch centered at pixel x, is the image patch centered at pixel x, is the image patch centered at pixel x, is the image patch centered at pixel x, is a filter parameter.

3. The image sequence flicker removal method based on clustering and multi-scale histogram matching of claim 1, wherein, The histogram feature of each frame of image in the denoised image sequence is calculated, comprising: For each frame of image, the normalized color histogram feature of any color channel is calculated by the following formula: ； in, The gray level of the c-th color channel i The characteristics of the normalized color histogram. For the first c grayscale levels in color channels i Pixel count, The minimum positive number is preset; The normalized color histogram features of all color channels are concatenated to obtain the histogram feature of the image.

4. The image sequence flicker removal method based on clustering and multi-scale histogram matching of claim 1, wherein, When the K-Means clustering algorithm is used to group the images in the denoised image sequence according to the histogram feature, the K-Means clustering algorithm is initialized as the center, and then the data is divided by minimizing the square Euclidean distance of the samples in the cluster to the corresponding center, and the corresponding function formula is as follows: ； wherein E is the sum of squared Euclidean distances of the samples to the corresponding centroids, is the i-th m histogram feature of the frame image, denoting the samples; is the i-th cluster, is the i-th centroid of the cluster, and K is the number of clusters.

5. The image sequence flicker removal method based on clustering and multi-scale histogram matching of claim 1, wherein, Multi-scale histogram matching is performed on each abnormal image in the abnormal image segment according to the reference image, comprising: The reference image is divided into standard image blocks in multiple preset scales to obtain a standard image block set corresponding to different scales; for any standard image block set of a scale, the cumulative distribution function of each standard image block is calculated; The reference image is divided into standard image blocks in multiple preset scales to obtain a standard image block set corresponding to different scales; for any standard image block set of a scale, the cumulative distribution function of each standard image block is calculated; The reference image is divided into standard image blocks in multiple preset scales to obtain a standard image block set corresponding to different scales; for any standard image block set of a scale, the cumulative distribution function of each standard image block is calculated; For different scales of any sub-block, the cumulative distribution functions of the plurality of abnormal image blocks are matched with the cumulative distribution functions of the plurality of standard image blocks to generate a lookup table; According to the lookup table, pixel value adjustment is performed on each abnormal image block in the abnormal image to obtain an optimized image under different scales.

6. The image sequence flicker removal method based on clustering and multi-scale histogram matching of claim 1, wherein, The optimized images under all scales are weighted and fused by a Gaussian kernel function to obtain an optimized image, including the following steps: A Gaussian kernel function based on the center position of an image block is used to calculate the weight of each image block in the optimized image under each scale. The weights of the optimized images under all scales are normalized to obtain corresponding normalized weights. The normalized weights of the optimized images under all scales are weighted and superimposed to generate a single-frame optimized image.

7. The image sequence flicker removal method based on clustering and multi-scale histogram matching of claim 5, wherein, The calculation formula of the cumulative distribution function of the standard image block and the cumulative distribution function of the abnormal image block is the same, as follows: ； wherein, is the c cumulative distribution function of the color channel, is the c gray level of the color channel i normalized color histogram feature.

8. The image sequence flicker removal method based on clustering and multi-scale histogram matching of claim 1, wherein, The step of using cross-fading technology for time sequence smooth transition processing at the boundary of images of different categories includes: At the boundary of images of different categories, a transition interval of a preset number of frames is determined. In the transition interval of the preset frame number, a plurality of images of the previous category label and a plurality of images of the next category label are weighted and averaged, and the calculation formula is as follows: ; wherein, is the fused image by weighted average, is the image of the previous class label, is the image of the next class label; is the fusion coefficient.

9. An image sequence flicker removal system based on clustering and multi-scale histogram matching, applying the image sequence flicker removal method based on clustering and multi-scale histogram matching of any of claims 1-8, characterized in that, The system comprises: A denoising module is configured to perform denoising processing on each frame of image in the image sequence by using a non-local mean denoising algorithm to obtain a denoised image sequence. A grouping module is configured to calculate a histogram feature of each frame of image in the denoised image sequence, and then perform grouping on the images in the denoised image sequence by using a K-Means clustering algorithm according to the histogram feature to determine the category label of each frame of image; wherein the category label is a standard category or an abnormal category. A histogram matching module is configured to, for the denoised image sequence, take the images with the category label of the standard category as a reference, perform multi-scale histogram matching on the images with the category label of the abnormal category, and perform weighted fusion on the optimized images under all scales by using a Gaussian kernel function to obtain an optimized image sequence. A smooth transition module is configured to, for the optimized image sequence, perform time sequence smooth transition processing at the boundary of images of different categories by using cross-fading technology to obtain a final image sequence.

Images