Video conference image quality optimization method and system based on AI and hybrid model

Abstract

The invention relates to the technical field of image enhancement, in particular to a video conference image quality optimization method and system based on AI and a hybrid model, and the method comprises the following steps: obtaining RGB channel data of a video conference frame, calculating the minimum value of a pixel point in an RGB channel, building an initial dark channel distribution matrix, and mapping a pixel coordinate to a preset face probability distribution diagram. According to the method, gradient amplitude modulation is introduced to calculate a boundary discontinuity metric value, and a structured weighted variance is constructed in combination with gradient distribution in a macro block, so that accurate distinguishing between a weak block effect of a flat region and a real texture edge is realized, and the problem that low-contrast artifacts are difficult to identify in a traditional method is solved. According to the quantized artifact degree, a self-adaptive bandwidth parameter is generated to control a one-dimensional low-pass filter, target boundary pixels are smoothed in a targeted mode, the blocking effect of video coding is eliminated, meanwhile, unnecessary blurring of a non-boundary area is avoided, and the visual quality of a video stream is improved.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to a method and system for optimizing video conferencing image quality based on AI and hybrid models. Background Technology

[0002] Image enhancement technology uses specific algorithms to improve the visual effect of images or transform them into a form that is more suitable for human observation and machine analysis.

[0003] Current technologies for improving video conferencing image quality often employ global histogram equalization, neglecting the spatial differences in lighting distribution within the scene. This can easily lead to highlight clipping or color distortion while enhancing dark areas, making it difficult to accurately reproduce the realism of the physical scene. Furthermore, traditional spatial filtering algorithms typically use fixed-size convolutional kernels for noise removal, inevitably erasing subtle textures and weak edge information while suppressing noise, resulting in overall image blurring. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a video conferencing image quality optimization method and system based on AI and hybrid models.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a video conferencing image quality optimization method based on AI and hybrid models, comprising the following steps: The RGB channel data of the video conference frame is acquired, the minimum value of the pixel in the RGB channel is calculated, an initial dark channel distribution matrix is ​​established, the pixel coordinates are mapped to a preset face probability distribution map, a lower limit threshold constraint is applied to the value of the initial dark channel distribution matrix in the face area, and the constrained matrix data is subjected to guided filtering calculation using the original grayscale data to generate a refined optical transmission map. Based on the refined optical transmission map, the pixel set in the video conference frame is selected to generate an atmospheric illumination parameter vector. The atmospheric illumination parameter vector and the refined optical transmission map are used to perform backscatter recovery operation on the pixel data to generate a color correction scene radiation matrix. The color correction scene radiation matrix is ​​divided into video coding macroblock grids, the boundary discontinuity metric is calculated, the structured weighted variance is calculated based on the color correction scene radiation matrix, and the grid positions are selected in combination with the boundary discontinuity metric to generate an artifact block boundary set. Based on the artifact block boundary set, the corresponding boundary discontinuity metric is read, an adaptive filtering bandwidth parameter is generated, a one-dimensional low-pass filter is invoked, and the target boundary pixels in the color correction scene radiation matrix are smoothed under the control of the adaptive filtering bandwidth parameter to generate optimized conference video stream data.

[0006] Preferably, the step of obtaining the refined optical transmission pattern is as follows: Read the red, green, and blue channel values ​​pixel by pixel from the video conferencing frame. For each pixel position, compare the minimum values ​​of the red, green, and blue channels and write the minimum value into the matrix cell corresponding to the pixel position. Traverse all pixel positions of the video conferencing frame according to the row and column indices to generate the initial dark channel distribution matrix. Based on the initial dark channel distribution matrix, the pixel coordinates are read and mapped to the coordinate axes of the face probability distribution map. The probability values ​​of the face probability distribution map are retrieved according to the mapped pixel coordinates. The set of pixel positions with probability values ​​not less than the face threshold is located as the face region. The values ​​of the initial dark channel distribution matrix in the face region that are less than the lower threshold are replaced with the lower threshold to generate the constrained dark channel distribution matrix. Based on the constrained dark channel distribution matrix, the original grayscale data is read and a neighborhood window guided by the original grayscale data is established at each pixel position. The local mean, local variance, and local gradient magnitude of the original grayscale data within the neighborhood window are calculated. The pixel neighborhood weight allocation is determined based on the local mean, local variance, and local gradient magnitude, and neighborhood weighted smoothing is performed on the constrained dark channel distribution matrix to generate a refined optical transmission map.

[0007] Preferably, the step of obtaining the atmospheric illumination parameter vector is as follows: Based on the refined optical transmission spectrum, RGB channel values ​​are read pixel by pixel from the video conference frame, the brightness value of each pixel is calculated and a brightness sequence is formed. The sequence is arranged in descending order of brightness value, and the coordinates and RGB channel values ​​of the top 0.1% of pixels are extracted to form a set of pixels with the top 0.1% brightness. Based on the set of pixels with the highest brightness (top 0.1%), the red, green, and blue channel values ​​are extracted by pixel. The average values ​​of the red, green, and blue channels are calculated to generate an atmospheric illumination parameter vector.

[0008] Preferably, the step of obtaining the color correction scene radiation matrix is ​​as follows: Based on the atmospheric illumination parameter vector, the refined optical transmission spectrum and video conference frame are read, backscattering recovery is performed to obtain the recovered RGB channel values, the full-frame mean of the recovered RGB channel values ​​is calculated and the gain coefficient relative to the global grayscale is calculated, gain correction is completed by channel multiplication, and a color correction scene radiation matrix is ​​generated.

[0009] Preferably, the step of obtaining the boundary discontinuity metric is as follows: Based on the color correction scene radiation matrix, the grayscale values ​​of adjacent pixels are read along the shared boundary of the video coding macroblock, and the boundary discontinuity metric is calculated.

[0010] Preferably, the step of obtaining the artifact block boundary set is as follows: Calculate the structured weighted variance value based on the color correction scene radiation matrix; Based on the structured weighted variance value, shared boundaries that overlap with the preset real edge mask are deleted, and macroblocks whose structured weighted variance values ​​of two shared macroblocks are both lower than the flatness threshold and whose corresponding boundary discontinuity metric values ​​are higher than the difference threshold are selected. Macroblock shared boundaries that meet the conditions are marked, and an artifact block boundary set is generated.

[0011] Preferably, the step of obtaining the adaptive filtering bandwidth parameter is as follows: Based on the artifact block boundary set, the boundary discontinuity metric values ​​are read one by one and a pairing table is established according to the boundary index. The minimum and maximum values ​​of the boundary discontinuity metric values ​​are calculated, interval linear normalization is performed, and truncation mapping is performed according to the preset bandwidth upper and lower limits. The mapped values ​​are written into the bandwidth field of the corresponding boundary to generate adaptive filtering bandwidth parameters.

[0012] Preferably, the step of optimizing the acquisition of conference video stream data is as follows: Based on the adaptive filtering bandwidth parameter, the target boundary pixel is located in the color correction scene radiation matrix by boundary index, the cross-boundary one-dimensional pixel sequence is extracted along the normal direction, the endpoint mirror extension is set, the one-dimensional low-pass filter is called and the cutoff frequency is controlled by the adaptive filtering bandwidth parameter, the target boundary pixel value is replaced, and the cross-boundary smoothed color correction scene radiation matrix is ​​generated. Based on the color-corrected scene radiation matrix after cross-boundary smoothing, the video conferencing frame buffer is written back by frame index, the pixel arrangement is reorganized according to the original video coding macroblock grid, and encapsulated into a continuous frame sequence to generate optimized conference video stream data.

[0013] The present invention also provides a system comprising: The dark channel and optical transmission calculation module is used to acquire RGB channel data of video conferencing frames, calculate the minimum value of pixel points in the RGB channel, establish an initial dark channel distribution matrix, map pixel coordinates to a preset face probability distribution map, apply a lower limit threshold constraint to the values ​​of the initial dark channel distribution matrix in the face region, and use the original grayscale data to perform guided filtering calculation on the constrained matrix data to generate a refined optical transmission map. The backscatter recovery and color correction module is used to filter the pixel set in the video conference frame according to the refined optical transmission map, generate an atmospheric illumination parameter vector, and use the atmospheric illumination parameter vector and the refined optical transmission map to perform backscatter recovery operation on the pixel data to generate a color correction scene radiation matrix. The artifact detection and boundary analysis module is used to divide the color correction scene radiation matrix into video coding macroblock grids, calculate boundary discontinuity metric values, calculate structured weighted variance values ​​based on the color correction scene radiation matrix, filter grid positions based on the boundary discontinuity metric values, and generate a set of artifact block boundaries. The adaptive filtering and video stream optimization module is used to read the corresponding boundary discontinuity metric value based on the artifact block boundary set, generate adaptive filtering bandwidth parameters, call a one-dimensional low-pass filter, and smooth the target boundary pixels in the color correction scene radiation matrix under the control of the adaptive filtering bandwidth parameters to generate optimized conference video stream data.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: This invention establishes an initial distribution by acquiring RGB channel data from video conferencing frames and calculating the minimum pixel value. Combined with a face probability distribution map, it applies a numerical lower bound constraint to specific regions, effectively avoiding excessive darkening of facial areas and skin color distortion in dark channel prior processing, thus maintaining the naturalness of facial features. Using raw grayscale data to guide filtering calculations, edge gradient information is preserved when generating the optical transmission map, ensuring that the transmittance estimation matches the image texture edges and avoiding halo artifacts. Atmospheric illumination parameters are calculated based on a high-brightness pixel set, and inverse reconstruction is performed using a physical scattering model. This removes haziness and corrects color cast at the imaging mechanism level, improving image clarity and color reproduction under complex lighting conditions. By introducing gradient amplitude modulation to calculate boundary discontinuity metrics and constructing a structured weighted variance based on the gradient distribution within macroblocks, it achieves accurate differentiation between weak block effects in flat areas and true texture edges, solving the problem of traditional methods struggling to identify low-contrast artifacts. An adaptive bandwidth parameter is generated based on the degree of artifacts after quantization to control a one-dimensional low-pass filter, which specifically smooths the target boundary pixels. This eliminates the block effect in video coding while avoiding unnecessary blurring in non-boundary areas, thus improving the visual quality of the video stream. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the steps of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] Please see Figure 1 This invention provides a technical solution: a video conferencing image quality optimization method based on AI and hybrid models, comprising the following steps: Acquire RGB channel data of video conference frames, calculate the minimum value of pixel points in RGB channels, establish an initial dark channel distribution matrix, map pixel coordinates to a preset face probability distribution map, apply a lower limit threshold constraint to the initial dark channel distribution matrix values ​​in the face region, use the original grayscale data to perform guided filtering calculation on the constrained matrix data, and generate a refined optical transmission map. Based on the refined optical transmission map, the pixel set in the video conference frame is selected to generate an atmospheric illumination parameter vector. The atmospheric illumination parameter vector and the refined optical transmission map are used to perform backscatter recovery operation on the pixel data to generate a color correction scene radiation matrix. The color correction scene radiation matrix is ​​divided into video coding macroblock grids, the boundary discontinuity metric is calculated, the structured weighted variance is calculated based on the color correction scene radiation matrix, and the grid positions are selected by combining the boundary discontinuity metric to generate a set of artifact block boundaries. Based on the set of artifact block boundaries, the corresponding boundary discontinuity metric is read, an adaptive filtering bandwidth parameter is generated, a one-dimensional low-pass filter is called, and the target boundary pixels in the color correction scene radiation matrix are smoothed under the control of the adaptive filtering bandwidth parameter to generate optimized conference video stream data.

[0018] The steps for obtaining a refined optical transmission pattern are as follows: Read the red, green, and blue channel values ​​pixel by pixel from the video conferencing frame. For each pixel position, compare the minimum values ​​of the red, green, and blue channels and write the minimum value into the matrix cell corresponding to the pixel position. Traverse all pixel positions of the video conferencing frame according to the row and column indices to generate the initial dark channel distribution matrix. Based on the initial dark channel distribution matrix, the pixel coordinates are read and mapped to the coordinate axes of the face probability distribution map. The probability values ​​of the face probability distribution map are retrieved according to the mapped pixel coordinates. The set of pixel positions with probability values ​​not less than the face threshold is located as the face region. The values ​​of the initial dark channel distribution matrix within the face region that are less than the lower threshold are replaced with the lower threshold to generate the constrained dark channel distribution matrix. Based on the constrained dark channel distribution matrix, the original grayscale data is read and a neighborhood window guided by the original grayscale data is established at each pixel position. The local mean, local variance, and local gradient magnitude of the original grayscale data within the neighborhood window are calculated. The pixel neighborhood weight allocation is determined based on the local mean, local variance, and local gradient magnitude, and neighborhood weighted smoothing is performed on the constrained dark channel distribution matrix to generate a refined optical transmission map.

[0019] Specifically, the red, green, and blue channel values ​​are read pixel by pixel from the video conference frame, and the resolution of the video conference frame is set to... ,in Represents the number of pixels in the image width. Representing the number of pixels in the image height, create a size of Using the zero matrix as a container, define row index variables. The value range is 0 to Define column index variables The value range is 0 to It iterates through each pixel using a double loop, and for each coordinate... For each pixel at that location, extract its red component. Green components And the blue component The values ​​of these three components are all between 0 and 255. A minimum value comparison operation is performed. The specific calculation logic is to first compare... and Take the smaller value from the larger value, and then combine that smaller value with... The comparisons are performed, and the final determined value is the dark channel value of that pixel. The calculation expression is: ,in Representing coordinates The value of the dark channel at that location. This represents a function that takes the minimum value, and the calculated maximum and minimum values ​​are directly filled into the corresponding zero matrix. The location is not subjected to additional filtering or noise reduction during the traversal process to maintain the extreme value characteristics of the original data. For the bright background or reflective areas of the display screen that are common in video conferencing scenarios, the minimum value calculation can accurately reflect the lowest reflectivity characteristics of non-luminous objects. After all pixels have been traversed, it is checked whether there are missing values ​​in the matrix. If there are, they are filled with the minimum value of the adjacent pixels. For each pixel position, the minimum value of the red channel value, green channel value, and blue channel value are compared, and the minimum value is written into the matrix cell consistent with the pixel position. All pixel positions of the video conferencing frame are traversed according to the row index and column index to generate the initial dark channel distribution matrix.

[0020] Based on the initial dark channel distribution matrix, the pixel coordinates are read and mapped to the coordinate axes of the face probability distribution map. First, the physical resolution of the face probability distribution map is obtained. Calculate the frame resolution of a video conference. Horizontal scaling ratio relative to the resolution of the face probability distribution map and vertical scaling ratio For each pixel coordinate in the initial dark channel distribution matrix By calculating the mapped coordinates To align the data spaces of the two, where This indicates a floor operation, reading the face probability distribution map on the coordinate system. The probability value at the location Set face recognition threshold The threshold was set at 0.75, a value derived from the confidence level cutoff determined after statistical analysis of a large number of frontal and side profile samples from video conferences. That is, when... If the current pixel is determined to belong to the facial skin region, a lower limit threshold for the dark channel is set. The setup process involves collecting the average minimum reflectance of people of different skin tones under standard studio lighting, and quantifying this intensity into a value of 35 within a 0-255 color gradation. Iterate through all pixels. If a pixel simultaneously satisfies the following conditions: it belongs to the face region and its initial dark channel value is... If the value is not found, it indicates that the area may be incorrectly estimated as excessive shadow. A numerical truncation operation is then performed, forcibly modifying the value at that location to 35. If the above conditions are not met, the original value is kept unchanged. This selective threshold constraint prevents color distortion or excessive blackening of the face region in subsequent dehazing or enhancement operations. The probability values ​​of the face probability distribution map are retrieved according to the mapped pixel coordinates. The set of pixel positions with probability values ​​not less than the face threshold is located as the face region. The initial dark channel distribution matrix values ​​within the face region are replaced with values ​​less than the lower threshold value to generate the constrained dark channel distribution matrix.

[0021] Based on the constrained dark channel distribution matrix, the original grayscale data is read, and a neighborhood window guided by the original grayscale data is established at each pixel location. The side length of the square neighborhood window is set to... For example, take the radius Make the window size For each center pixel position in the image traverse windows All pixels within, using the original grayscale data Calculate local mean and local variance Simultaneously, the Sobel operator is used to perform convolution operations in the horizontal and vertical directions to calculate the gradient magnitude of the center pixel within the window. The calculation formula is: ,in The gradient is in the horizontal direction. For the vertical gradient, an adaptive regularization parameter is constructed based on the local variance and gradient magnitude. Set the basic regularization parameters Calculate the adaptive weights ,in To prevent the use of a tiny constant with a denominator of zero, a value of 0.0001 was used to calculate the linear regression coefficients. The calculation formula is: ,in The total number of pixels within the window. For the first in the window grayscale guide value of each pixel The values ​​in the constrained dark channel distribution matrix are the corresponding values. The average value of the dark channel within the window is used, followed by the calculation of the bias coefficient. After completing the coefficient calculation for the entire image, for all covered pixels... The window calculated and The mean value is then applied to obtain the final smoothing coefficient. and Applying the linear transformation formula The smoothed output value is obtained. This process can effectively suppress noise in flat areas while preserving depth change information in edge areas. The local mean, local variance, and local gradient magnitude of the original grayscale data within the neighborhood window are calculated. The pixel neighborhood weight allocation is determined based on the local mean, local variance, and local gradient magnitude, and neighborhood weighted smoothing is performed on the constrained dark channel distribution matrix to generate a refined optical transmission map.

[0022] The steps for obtaining the atmospheric illumination parameter vector are as follows: Based on the refined optical transmission map, the RGB channel values ​​are read pixel by pixel from the video conference frame. The brightness value of each pixel is calculated and a brightness sequence is formed. The pixels are sorted in descending order of brightness value, and the coordinates and RGB channel values ​​of the top 0.1% of pixels are extracted to form a set of pixels with the top 0.1% brightness. Based on the set of pixels with the highest brightness (top 0.1%), extract the red, green, and blue channel values ​​by pixel. Calculate the mean values ​​of the red, green, and blue channels respectively to generate an atmospheric illumination parameter vector.

[0023] Specifically, based on the refined optical transmission map, the RGB channel values ​​are read pixel by pixel from the video conferencing frame, and the width of the video conferencing frame is defined as... The height is A temporary data storage list is constructed to store pixel features, and each coordinate point on the image plane is traversed. ,in The value range is 0 to , The value range is 0 to Extract the red channel values ​​at each coordinate position. Green channel values and the blue channel value Simultaneously, the corresponding transmittance values ​​in the refined optical transmission spectrum are read. To accurately locate the regions in an image most severely affected by atmospheric scattering and possessing ambient light characteristics, a comprehensive brightness evaluation index is constructed. This metric combines the pixel's original brightness with its depth information in the haze model, and the calculation formula is set as follows: ,in This indicates that the maximum value among the three channels is used to represent the brightness intensity at that point. This indicates the weight of fog concentration; the lower the transmittance, the greater the weight. A small positive constant is defined, with a value of This is used to prevent the weight from being zero, and to calculate the evaluation index. With the corresponding pixel coordinates And the original RGB values ​​are packaged and stored in a data list, which is then used to... The candidate pixels are sorted in descending order of size, and a quicksort algorithm is used to ensure processing speed. The number of candidate pixels to be cropped is then calculated. Set the threshold for the interception ratio This threshold is derived from statistical analysis of the proportion of light source area in a large number of video conferencing scenarios. It aims to eliminate localized specular noise while retaining sufficient atmospheric light sampling points. The calculation formula is as follows: ,in This is the floor function, which truncates the list from the top of the sorted list. Each data item directly extracts the original coordinate information and unprocessed RGB channel values ​​recorded in it. The brightness value of each pixel is calculated and a brightness sequence is formed. The pixels are sorted in descending order of brightness value, and the coordinates and RGB channel values ​​of the top 0.1% of the pixels are extracted to form a set of pixels with the top 0.1% brightness.

[0024] Initialize three double-precision floating-point variables based on the set of pixels with the highest brightness (top 0.1%). , and Use it as an accumulator and set it to zero to get the total number of pixels in the set. This value is determined by the cropping ratio in the previous step, and each pixel sample in the set is traversed. ,in From 1 to Read the red channel values ​​stored in the samples respectively. Green channel values and blue channel values The channel values ​​are accumulated using the following formulas: , , After completing the traversal, the average value of each channel is calculated by accumulating the values ​​to obtain an estimate of the atmospheric illumination parameters. The calculation formula is as follows: ,in Represents color channel identifier , This represents the mean atmospheric light component of the corresponding channel. To sum the corresponding channels, a minimum light intensity threshold is set to avoid overflow in subsequent division operations or color distortion caused by excessively low calculated lighting parameters due to an overall dark scene. This threshold references the noise floor level of a standard digital camera in low-light conditions, examining each calculated threshold. ,like Then force Revised to The final three channel averages are combined sequentially to construct a three-dimensional vector. This vector represents the global ambient light chromaticity and luminance reference in the video frame. The red channel value, green channel value, and blue channel value are extracted by pixel. The mean values ​​of the red channel value, green channel value, and blue channel value are calculated respectively to generate an atmospheric illumination parameter vector.

[0025] The steps to obtain the color correction scene radiation matrix are as follows: Based on the atmospheric illumination parameter vector, the refined optical transmission spectrum and video conference frames are read, backscattering recovery is performed to obtain the recovered RGB channel values, the full-frame mean of the recovered RGB channel values ​​is calculated and the gain coefficient relative to the global grayscale is calculated, gain correction is completed by channel multiplication, and a color-corrected scene radiation matrix is ​​generated.

[0026] Specifically, based on the atmospheric illumination parameter vector, a refined optical transmission spectrum and video conference frames are read, and the atmospheric illumination parameter vector is defined as follows: Iterate through each pixel position of the video frame. Read the RGB values ​​of the raw image data and the corresponding transmittance values ,in Scene radiative recovery is performed using the inverse transform formula of the atmospheric scattering model, with a lower limit threshold for transmittance set. This threshold setting is designed to limit the noise amplification effect that occurs when the transmittance approaches zero, and to calculate the recovered pixel values. The calculation formula is: ,in To maximize the function, ensure the denominator is not less than 0.1. To observe pixel values, For atmospheric light components, the calculation results need to be limited to between 0 and 255. After completing the restoration operation, the pixel average of the entire frame image across the three channels is calculated. The average brightness of the restored full frame was calculated using formulas based on the characteristics of human visual perception. Set the target brightness reference value This value corresponds to the intermediate brightness grayscale at 8-bit color depth, and the gain coefficient relative to the global grayscale is calculated. The calculation formula is: ,in To prevent extremely small values ​​where the denominator is zero, and also to prevent overexposure of the image, a maximum gain limit is set. Determine the final gain coefficient The coefficient is used to perform pixel-by-pixel multiplication correction on the recovered matrix data, i.e. The backscattering recovery operation is performed to obtain the recovered RGB channel values. The average value of the recovered RGB channel values ​​across the entire frame is calculated, and the gain coefficient relative to the global grayscale is calculated. Gain correction is performed by channel multiplication, and a color correction scene radiation matrix is ​​generated.

[0027] The steps for obtaining boundary discontinuity measures are as follows: Based on the color-corrected scene radiation matrix, the grayscale values ​​of adjacent pixels are read along the shared boundary of the video coding macroblock, and the boundary discontinuity metric is calculated using the following formula: ; in, For the first The boundary discontinuity metric for shared boundaries of macroblocks. For the first The set of pixels sharing a boundary in a macroblock. for Pixel position within, To and Adjacent cross-boundary pixel positions, pixel position grayscale value, pixel position grayscale value, pixel position gradient magnitude, pixel position gradient magnitude, This is a gradient sensitivity control constant used to adjust the degree of suppression between the smooth boundary region and the high gradient region.

[0028] Specifically, the boundary discontinuity metric calculation formula introduces a gradient-based exponential decay weight to accurately distinguish between real object edges and compression artifact boundaries in video conferencing images. Block artifacts generated in flat regions are typically accompanied by low local gradients, where the exponential term approaches 1, causing the boundary discontinuity metric to primarily reflect pixel grayscale differences. However, at real edges, the local gradient is larger, and the exponential term rapidly decays to near 0, significantly reducing the metric value at that location and preventing false smoothing of real details.

[0029] For the first A macroblock is a set of pixels sharing a boundary, and its size is typically determined by the video coding standard (such as H.264). (pixels), this set contains all pixel pairs arranged along a shared boundary.

[0030] for The pixel position within refers to the pixel coordinates on one side of the boundary; To and Adjacent cross-boundary pixel positions refer to the coordinates of pixels directly adjacent to each other on the other side of the boundary.

[0031] pixel position The grayscale value, ranging from 0 to 255, is directly read from the color correction scene radiation matrix.

[0032] pixel position The grayscale values, ranging from 0 to 255, are also read from the matrix.

[0033] pixel position The gradient magnitude reflects the degree of drastic change in gray level around that point, and is calculated by convolution in the horizontal and vertical directions using the Sobel operator. .

[0034] pixel position The gradient magnitude is calculated in the same way as above.

[0035] This is a gradient sensitivity control constant. This parameter adjusts the degree of suppression between smooth boundary regions and high gradient regions, and its value directly determines the algorithm's ability to protect true edges. If the value is too large, true edges will be misjudged as artifacts; if the value is too small, artifacts cannot be effectively detected. Specifically, it is obtained by selecting a high-definition video conferencing test sequence without compression distortion, extracting the pixel gradient values ​​of strong edge regions in the image, and calculating the average of these gradient values. To ensure effective differentiation between weak edges (artifacts) and strong edges (real object outlines), set... for 0.4 times. For example, the average edge gradient of statistically high-definition test sequences. If the value is 50, then calculate. This setting ensures that for true edges with large gradients, the exponential term approaches 0, thus significantly reducing... Values ​​should be adjusted to avoid erroneous smoothing.

[0036] Calculations based on parameters: A simplified boundary segment containing two pixel pairs is selected for calculation example.

[0037] Set gradient sensitivity control constant .

[0038] First pair of pixels: , The gradient at this location is small (possibly an artifact), and the measured gradient is... , .

[0039] Second pair of pixels: , The gradient at this point is also relatively small, and it was measured that... , .

[0040] Calculate the contribution of the first pair of pixels: Squared grayscale difference: ; Index weighting: ; Weight values: ; First result: ; Calculate the contribution of the second pair of pixels: Squared grayscale difference: ; Index weighting: ; Weight values: ; Second result: ; Calculate the total measurement: ; Calculated boundary discontinuity measure A high value indicates a significant gray-level jump and low local gradient at the boundary, consistent with the characteristics of block artifacts. In subsequent steps, this value will be compared with a difference threshold; if it exceeds the threshold, the boundary is determined to require deblocking filtering. If the location is a true edge (large gradient), the exponential term will be extremely small (e.g., ...). ),lead to The value is very small, thus avoiding misprocessing.

[0041] The steps to obtain the artifact block boundary set are as follows: Based on the color correction scene radiation matrix, the structured weighted variance is calculated using the following formula: ; in, For the first The structured weighted variance values ​​of each video coded macroblock For the first A set of pixel locations for a macroblock. This represents the number of pixels within a macroblock. pixel position in macroblock grayscale value, This represents the average grayscale value of the macroblock. pixel position gradient magnitude, This is the average gradient magnitude of all pixels within the macroblock. This is a structural influence factor used to adjust the strength of the gradient's influence on the weighted results. It is a small constant used to avoid the case where the denominator is zero; Based on the structured weighted variance value, shared boundaries that overlap with the preset real edge mask are deleted, and macroblocks whose structured weighted variance values ​​of two shared macroblocks are both below the flatness threshold and whose corresponding boundary discontinuity metric values ​​are above the difference threshold are selected. Macroblock shared boundaries that meet the conditions are marked, and a set of artifact block boundaries is generated.

[0042] Specifically, the structured weighted variance calculation formula introduces a gradient-dependent weighting factor to construct a structure-sensitive variance calculation method. Traditional variance calculation cannot distinguish between noise fluctuations in flat regions and detail changes in textured regions. This formula uses gradient information to weight gray-level deviations, resulting in macroblocks containing rich texture details (high gradients) generating larger variance values. This makes them easier to identify and exclude in subsequent screening, avoiding over-smoothing of texture details as flat regions.

[0043] The number of pixels within a macroblock, for the standard macroblocks .

[0044] For the first A set of pixel locations for a macroblock, containing all coordinate points covered by that macroblock.

[0045] pixel position in macroblock The grayscale value is directly read from the image data.

[0046] It is the macroblock grayscale mean, which is the average of the grayscale values ​​of all pixels within the macroblock.

[0047] pixel position The gradient magnitude represents the edge strength at that point.

[0048] This is the average gradient magnitude of all pixels within the macroblock, used to normalize the current gradient contribution.

[0049] This is a small constant used to avoid the denominator being zero; it is usually taken as a value of 0. .

[0050] The structural influence factor is used to adjust the strength of the gradient's influence on the weighted result, aiming to amplify the variance value of textured regions, making them easier to distinguish from flat regions. In flat regions, the gradient value is small, and the weighting term is close to 1; while in textured regions, the gradient value is large. The presence of this significantly increases the variance value. The steps to obtain it are as follows: construct a validation dataset containing various texture features (such as hair and fabric) and a solid-color background, and calculate the average variance of the solid-color background regions. and the average variance of the texture region Set target weighting ratio. By reverse calculation based on the formula An approximate value is typically set between 1.0 and 2.0. In the embodiment, through comparative testing, when Need to reach To ensure differentiation, set 3 times. This enhances the algorithm's sensitivity to texture details.

[0051] Calculations based on parameters: Choose one The calculation is performed on the tiny pixel blocks, that is .

[0052] The set of pixel grayscale values ​​is: .

[0053] Calculate the macroblock grayscale mean: .

[0054] For example, the region is relatively flat but has slight undulations, and the gradient amplitude at each point is 1, that is... .

[0055] Calculate the average gradient: .

[0056] Setting parameters: , .

[0057] Calculation of weighting coefficients: ; Weighting factor: ; Calculate the summation term: Pixel 1 (value 10): ; Pixel 2 (value 10): ; Pixel 3 (value 12): ; Pixel 4 (value 12): ; Summation result: ; Calculate the final variance: ; Calculated structured weighted variance values A low value indicates that the macroblock has a smooth grayscale variation and a simple structure, making it highly likely to be a flat region. In subsequent filtering steps, if this value is below the set flatness threshold, the macroblock will be marked as a flat block, and its boundaries are more likely to produce artifacts due to compression, thus being included in the candidates for deblocking filtering. If the calculated value is very large, it indicates that the macroblock contains complex textures and should not be subjected to strong smoothing.

[0058] Based on the structured weighted variance values, shared boundaries that overlap with the preset true edge mask are deleted. Strong edge features in the image are extracted using the Canny edge detection operator, with a high threshold of 100 and a low threshold of 50, generating a binarized true edge mask. Pixels marked as 1 in this mask represent the outline of a real object. The shared boundaries of the video coding macroblocks are scanned one by one, checking whether the pixel coordinates on each boundary fall within the marked area of ​​the true edge mask. If coordinate overlap exists, the boundary is determined to be a real image edge rather than a compression artifact, and it is directly removed from the processing list to prevent erroneous smoothing. Next, the flattening threshold needs to be determined. and difference threshold Flat threshold used for subsequent filtering The settings are based on the noise level of smooth regions in the statistical image. A set of macroblocks in the textureless background region of the image is selected, and the structured weighted variance mean of these macroblocks is calculated. For example, if the calculated variance mean of the background region is 20, then the settings are... To accommodate slight noise fluctuations, the difference threshold The setup is based on training with a manually labeled block effect sample library, calculating the discontinuity measure of obvious block effect boundaries in the samples, and taking 80% of the minimum value as the judgment criterion. For example, if the minimum discontinuity measure in the samples is 200, then the following calculation is performed. After setting the threshold, iterate through the remaining shared boundaries and read the structured weighted variance values ​​of the adjacent macroblocks on both sides of the boundary. and and the boundary discontinuity metric of that boundary. Determine whether the conditions are met simultaneously. , as well as Only when all three conditions are met simultaneously does it indicate that both sides of the boundary are flat regions and there is a significant unnatural jump at the boundary. The boundary is identified as a compression artifact. The boundary index that meets the conditions is added to the artifact block boundary set. Macroblocks whose structured weighted variance values ​​of the two shared macroblocks are both lower than the flatness threshold and whose corresponding boundary discontinuity measure values ​​are higher than the difference threshold are selected. The shared boundaries of macroblocks that meet the conditions are marked, and the artifact block boundary set is generated.

[0059] The steps for obtaining the adaptive filter bandwidth parameter are as follows: Based on the artifact block boundary set, the boundary discontinuity metric values ​​are read one by one and a pairing table is built according to the boundary index. The minimum and maximum values ​​of the boundary discontinuity metric values ​​are calculated, interval linear normalization is performed, and truncation mapping is performed according to the preset bandwidth upper and lower limits. The mapped values ​​are written into the bandwidth field of the corresponding boundary to generate adaptive filtering bandwidth parameters.

[0060] Specifically, based on the artifact block boundary set, the boundary discontinuity metric is read one by one and a pairing table is built according to the boundary index. An empty list is initialized, and each boundary record in the artifact block boundary set is traversed to read the boundary discontinuity metric associated with that record. and unique boundary index Combine the two into a key-value pair Store in a list, iterate through all elements in the list, and then process the results. Statistically analyze the values ​​and find the minimum value. and maximum value Perform linear normalization and calculate the normalization coefficients. ,in For floating-point numbers between 0 and 1, define the mapping range of the filter bandwidth, including the minimum bandwidth. and maximum bandwidth These two values ​​were derived from experiments on the human eye's perception threshold for different intensities of block effects. Specifically, a smaller bandwidth of 0.5 was used for minor artifacts to preserve details, while a larger bandwidth of 2.5 was used for severe artifacts for strong smoothing. The initial bandwidth value was calculated using a normalization coefficient. Considering the impact of calculation errors or extreme outliers, for Perform truncation if Then take ,like Then take The final adaptive bandwidth is obtained. ,Will Write the bandwidth field of the corresponding index in the pairing table to ensure that each boundary to be processed obtains a filtering parameter that is proportional to its artifact intensity. Calculate the minimum and maximum values ​​of the boundary discontinuity metric, perform interval linear normalization, and truncate the mapping according to the preset upper and lower bandwidth limits. Write the mapped value into the bandwidth field of the corresponding boundary to generate the adaptive filtering bandwidth parameter.

[0061] The steps to optimize the acquisition of meeting video stream data are as follows: Based on the adaptive filtering bandwidth parameter, the target boundary pixel is located in the color correction scene radiation matrix by boundary index, the cross-boundary one-dimensional pixel sequence is extracted along the normal direction, the endpoint mirror extension is set, the one-dimensional low-pass filter is called and the cutoff frequency is controlled by the adaptive filtering bandwidth parameter, the target boundary pixel value is replaced, and the cross-boundary smoothed color correction scene radiation matrix is ​​generated. Based on the color-corrected scene radiation matrix after cross-boundary smoothing, the video conferencing frame buffer is written back by frame index, the pixel arrangement is reorganized according to the original video coding macroblock grid, and encapsulated into a continuous frame sequence to generate optimized conferencing video stream data.

[0062] Specifically, based on the adaptive filtering bandwidth parameter, target boundary pixels are located in the color correction scene radiation matrix by boundary index. For each recorded boundary index, its pixel coordinate range in the matrix is ​​determined, and the direction of the boundary is identified as horizontal or vertical. If it is a vertical boundary, pixels crossing the boundary are selected along the horizontal normal direction. Each pixel, set Extract the length of One-dimensional pixel sequence To address the potential data overflow risk at both ends of the sequence, a mirror extension method is used to extend the sequence by extending the first element. Copying from the outside , tail element Copying from the outside Construct an extended sequence and read the adaptive filtering bandwidth parameters corresponding to the boundary. A Gaussian low-pass filter is selected as the smoothing kernel, and its standard deviation parameter is set to... Generate discretized Gaussian kernel templates Its elements are derived from the formula The calculation yielded, where Using the relative position coordinates within the kernel, this template is used to perform convolution operations on the boundary pixels of the extended sequence center. The calculation formula is as follows: ,in The kernel radius is used to ensure the convolution window covers the target pixel, and the calculated smoothing value is used. Replace the corresponding boundary pixel values ​​in the original matrix, repeat this process for all pixels on the boundary, extract the cross-boundary one-dimensional pixel sequence along the normal direction, set the endpoint mirror extension, call the one-dimensional low-pass filter and control the cutoff frequency with the adaptive filter bandwidth parameter, replace the target boundary pixel values, and generate the cross-boundary smoothed color correction scene radiation matrix.

[0063] Based on the color-corrected scene radiation matrix after cross-boundary smoothing, the video conferencing frame buffer is written back by frame index to obtain the timestamp or sequence number of the current processing frame. The corresponding video memory area or memory buffer is locked, and the matrix data after color correction and deblocking filtering is converted into a standard 8-bit unsigned integer format. During processing, values ​​exceeding the range of 0-255 need to be saturated and truncated. According to the macroblock size defined by the original video coding standard (such as H.264 or HEVC) (e.g., 16x16 or 64x64), the entire image is re-divided into macroblock grids according to the raster scan order, i.e., from left to right and from top to bottom. Next, the pixel data is filled back into the corresponding macroblock positions to reconstruct the complete image frame structure. After checking the data integrity, the frame is marked as "optimized" and pushed into the first-in-first-out (FIFO) queue to be played or encoded. If the queue is full, it waits for the previous frame to be taken to maintain the temporal continuity of the video stream. Together with other processed frames, it forms a continuous video stream. Necessary streaming media header information (such as SPS and PPS parameter sets) is added, and it is encapsulated into data packets that conform to the RTP or RTMP protocol. The pixel arrangement is reorganized according to the original video coding macroblock grid, and it is encapsulated into a continuous frame sequence to generate optimized conference video stream data.

[0064] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for video conference image quality optimization based on AI and hybrid model, characterized in that, Includes the following steps: The RGB channel data of the video conference frame is acquired, the minimum value of the pixel in the RGB channel is calculated, an initial dark channel distribution matrix is ​​established, the pixel coordinates are mapped to a preset face probability distribution map, a lower limit threshold constraint is applied to the value of the initial dark channel distribution matrix in the face area, and the constrained matrix data is subjected to guided filtering calculation using the original grayscale data to generate a refined optical transmission map. Based on the refined optical transmission map, the pixel set in the video conference frame is selected to generate an atmospheric illumination parameter vector. The atmospheric illumination parameter vector and the refined optical transmission map are used to perform backscatter recovery operation on the pixel data to generate a color correction scene radiation matrix. The color correction scene radiation matrix is ​​divided into video coding macroblock grids, the boundary discontinuity metric is calculated, the structured weighted variance is calculated based on the color correction scene radiation matrix, and the grid positions are selected in combination with the boundary discontinuity metric to generate an artifact block boundary set. Based on the artifact block boundary set, the corresponding boundary discontinuity metric is read, an adaptive filtering bandwidth parameter is generated, a one-dimensional low-pass filter is invoked, and the target boundary pixels in the color correction scene radiation matrix are smoothed under the control of the adaptive filtering bandwidth parameter to generate optimized conference video stream data.

2. The AI and hybrid model-based video conference image quality optimization method according to claim 1, characterized in that, The steps for obtaining the refined optical transmission pattern are as follows: Read the red, green, and blue channel values ​​pixel by pixel from the video conferencing frame. For each pixel position, compare the minimum values ​​of the red, green, and blue channels and write the minimum value into the matrix cell corresponding to the pixel position. Traverse all pixel positions of the video conferencing frame according to the row and column indices to generate the initial dark channel distribution matrix. Based on the initial dark channel distribution matrix, the pixel coordinates are read and mapped to the coordinate axes of the face probability distribution map. The probability values ​​of the face probability distribution map are retrieved according to the mapped pixel coordinates. The set of pixel positions with probability values ​​not less than the face threshold is located as the face region. The values ​​of the initial dark channel distribution matrix in the face region that are less than the lower threshold are replaced with the lower threshold to generate the constrained dark channel distribution matrix. Based on the constrained dark channel distribution matrix, the original grayscale data is read and a neighborhood window guided by the original grayscale data is established at each pixel position. The local mean, local variance, and local gradient magnitude of the original grayscale data within the neighborhood window are calculated. The pixel neighborhood weight allocation is determined based on the local mean, local variance, and local gradient magnitude, and neighborhood weighted smoothing is performed on the constrained dark channel distribution matrix to generate a refined optical transmission map.

3. The video conferencing image quality optimization method based on AI and hybrid models according to claim 1, characterized in that, The steps for obtaining the atmospheric illumination parameter vector are as follows: Based on the refined optical transmission spectrum, RGB channel values ​​are read pixel by pixel from the video conference frame, the brightness value of each pixel is calculated and a brightness sequence is formed. The sequence is arranged in descending order of brightness value, and the coordinates and RGB channel values ​​of the top 0.1% of pixels are extracted to form a set of pixels with the top 0.1% brightness. Based on the set of pixels with the highest brightness (top 0.1%), the red, green, and blue channel values ​​are extracted by pixel. The average values ​​of the red, green, and blue channels are calculated to generate an atmospheric illumination parameter vector.

4. The video conferencing image quality optimization method based on AI and hybrid models according to claim 1, characterized in that, The steps for obtaining the color correction scene radiation matrix are as follows: Based on the atmospheric illumination parameter vector, the refined optical transmission spectrum and video conference frame are read, backscattering recovery is performed to obtain the recovered RGB channel values, the full-frame mean of the recovered RGB channel values ​​is calculated and the gain coefficient relative to the global grayscale is calculated, gain correction is completed by channel multiplication, and a color correction scene radiation matrix is ​​generated.

5. The video conferencing image quality optimization method based on AI and hybrid models according to claim 1, characterized in that, The steps for obtaining the boundary discontinuity metric are as follows: Based on the color correction scene radiation matrix, the grayscale values ​​of adjacent pixels are read along the shared boundary of the video coding macroblock, and the boundary discontinuity metric is calculated.

6. The video conferencing image quality optimization method based on AI and hybrid models according to claim 1, characterized in that, The steps for obtaining the artifact block boundary set are as follows: Calculate the structured weighted variance value based on the color correction scene radiation matrix; Based on the structured weighted variance value, shared boundaries that overlap with the preset real edge mask are deleted, and macroblocks whose structured weighted variance values ​​of two shared macroblocks are both lower than the flatness threshold and whose corresponding boundary discontinuity metric values ​​are higher than the difference threshold are selected. Macroblock shared boundaries that meet the conditions are marked, and an artifact block boundary set is generated.

7. The video conferencing image quality optimization method based on AI and hybrid models according to claim 1, characterized in that, The steps for obtaining the adaptive filtering bandwidth parameter are as follows: Based on the artifact block boundary set, the boundary discontinuity metric values ​​are read one by one and a pairing table is established according to the boundary index. The minimum and maximum values ​​of the boundary discontinuity metric values ​​are calculated, interval linear normalization is performed, and truncation mapping is performed according to the preset bandwidth upper and lower limits. The mapped values ​​are written into the bandwidth field of the corresponding boundary to generate adaptive filtering bandwidth parameters.

8. The video conferencing image quality optimization method based on AI and hybrid models according to claim 1, characterized in that, The steps for obtaining optimized conference video stream data are as follows: Based on the adaptive filtering bandwidth parameter, the target boundary pixel is located in the color correction scene radiation matrix by boundary index, the cross-boundary one-dimensional pixel sequence is extracted along the normal direction, the endpoint mirror extension is set, the one-dimensional low-pass filter is called and the cutoff frequency is controlled by the adaptive filtering bandwidth parameter, the target boundary pixel value is replaced, and the cross-boundary smoothed color correction scene radiation matrix is ​​generated. Based on the color-corrected scene radiation matrix after cross-boundary smoothing, the video conferencing frame buffer is written back by frame index, the pixel arrangement is reorganized according to the original video coding macroblock grid, and encapsulated into a continuous frame sequence to generate optimized conference video stream data.

9. The system for optimizing video conferencing image quality based on AI and hybrid models according to any one of claims 1-8, characterized in that, include: The dark channel and optical transmission calculation module is used to acquire RGB channel data of video conferencing frames, calculate the minimum value of pixel points in the RGB channel, establish an initial dark channel distribution matrix, map pixel coordinates to a preset face probability distribution map, apply a lower limit threshold constraint to the values ​​of the initial dark channel distribution matrix in the face region, and use the original grayscale data to perform guided filtering calculation on the constrained matrix data to generate a refined optical transmission map. The backscatter recovery and color correction module is used to filter the pixel set in the video conference frame according to the refined optical transmission map, generate an atmospheric illumination parameter vector, and use the atmospheric illumination parameter vector and the refined optical transmission map to perform backscatter recovery operation on the pixel data to generate a color correction scene radiation matrix. The artifact detection and boundary analysis module is used to divide the color correction scene radiation matrix into video coding macroblock grids, calculate boundary discontinuity metric values, calculate structured weighted variance values ​​based on the color correction scene radiation matrix, filter grid positions based on the boundary discontinuity metric values, and generate a set of artifact block boundaries. The adaptive filtering and video stream optimization module is used to read the corresponding boundary discontinuity metric value based on the artifact block boundary set, generate adaptive filtering bandwidth parameters, call a one-dimensional low-pass filter, and smooth the target boundary pixels in the color correction scene radiation matrix under the control of the adaptive filtering bandwidth parameters to generate optimized conference video stream data.

Images