Video processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide a video processing method and device, electronic equipment and storage medium, by obtaining target pixel data corresponding to a skin region in a to-be-processed image, the target pixel data representing a brightness value of a target pixel point located in the skin region; based on the target pixel data, obtaining brightness difference data, the brightness difference data representing a difference amount of the brightness value of the target pixel point and a corresponding surrounding pixel point, the brightness difference data being character type data; through preset weight mapping information, obtaining weight data corresponding to the brightness difference data, and performing guided mean filtering on the target pixel data based on the weight data to obtain a target image, solving the problem of large running overhead and slow calculation speed of a central processing unit in the image skin smoothing process, improving the running speed of the image skin smoothing process, shortening the running time, and improving the smoothness of the equipment.

Description

TECHNICAL FIELD

[0001] The embodiment of the present disclosure relates to the technical field of image processing, in particular to a video processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] At present, in video communication applications (APPs) such as video live broadcast and video conference, real-time image skin smoothing function, i.e. “beautifying” function, is provided for users to meet their needs for skin blemish concealment.

[0003] In the prior art, real-time image skin smoothing processing needs to consume corresponding computing resources, and terminal devices usually call graphics processing units (GPUs) that are better at processing image data to perform the processing, so as to achieve better image skin smoothing effect.

[0004] However, for terminal devices without a dedicated image processor, image skin smoothing processing can only be performed through a central processing unit (CPU), which results in large running overhead and slow computing speed, and further affects the real-time performance of image skin smoothing, causing problems such as device lag. SUMMARY

[0005] The embodiment of the present disclosure provides a video processing method and device, electronic equipment and storage medium to overcome the problems of large running overhead and slow computing speed when image skin smoothing processing is performed through a CPU.

[0006] In a first aspect, the embodiment of the present disclosure provides a video processing method, comprising:

[0007] obtaining target pixel data corresponding to a skin region in a to-be-processed image, the target pixel data representing luminance values of target pixel points located in the skin region; obtaining luminance difference data based on the target pixel data, the luminance difference data representing difference amounts between the luminance values of the target pixel points and luminance values of corresponding surrounding pixel points; obtaining weight data corresponding to the luminance difference data through preset weight mapping information, and performing guided mean filtering on the target pixel data based on the weight data to obtain a target image, wherein the weight data represents the intensity of mean filtering on each target pixel point; and generating a video data stream based on the target image.

[0008] In a second aspect, the embodiment of the present disclosure provides a video processing device, comprising:

[0009] an obtaining module configured to obtain target pixel data corresponding to a skin region in a to-be-processed image, the target pixel data representing luminance values of target pixel points located in the skin region;

[0010] a comparison module configured to obtain luminance difference data based on the target pixel data, the luminance difference data representing a difference between the luminance value of the target pixel and the luminance value of the corresponding surrounding pixel;

[0011] a filtering module configured to obtain weight data corresponding to the luminance difference data by using preset weight mapping information, and perform guided mean filtering on the target pixel data based on the weight data to obtain a target image, wherein the weight data represents the intensity of mean filtering on each target pixel;

[0012] a communication module configured to generate a video data stream based on the target image.

[0013] In a third aspect, an electronic device is provided, and the electronic device comprises:

[0014] a processor, and a memory connected to the processor in communication;

[0015] the memory stores computer-executable instructions;

[0016] the processor executes the computer-executable instructions stored in the memory to implement the video processing method according to the first aspect and various possible designs of the first aspect.

[0017] In a fourth aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the video processing method according to the first aspect and various possible designs of the first aspect is implemented.

[0018] In a fifth aspect, a computer program product is provided, and the computer program product comprises a computer program. When a processor executes the computer program, the video processing method according to the first aspect and various possible designs of the first aspect is implemented.

[0019] The video processing method, apparatus, electronic device, and storage medium provided in this embodiment acquire target pixel data corresponding to skin regions in an image to be processed. The target pixel data represents the brightness value of target pixels located within the skin region. Based on the target pixel data, brightness difference data is obtained, representing the difference between the brightness value of the target pixel and the brightness values ​​of its surrounding pixels. Weight data corresponding to the brightness difference data is obtained through preset weight mapping information, and guided mean filtering is performed on the target pixel data based on the weight data to obtain a target image. The weight data represents the intensity of the mean filtering applied to each target pixel. A video data stream is generated based on the target image. Because guided mean filtering is performed using preset weight mapping information and a lookup table to obtain the corresponding weight data, it avoids the limitations of floating-point operations, which are not well-suited for central processing units (CPUs). Therefore, it improves the CPU's operating efficiency during image smoothing, solving the problems of high overhead and slow calculation speed during image smoothing, increasing the running speed of the image smoothing process, shortening the running time, and improving device smoothness. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is an application scenario diagram of the video processing method provided in the embodiments of this disclosure;

[0022] Figure 2 Flowchart of the video processing method provided in the embodiments of this disclosure Figure 1 ;

[0023] Figure 3 for Figure 2 The flowchart of the specific implementation method of step S102 in the embodiment shown is as follows;

[0024] Figure 4 This is a schematic diagram illustrating a process for obtaining brightness difference data according to an embodiment of the present disclosure;

[0025] Figure 5 for Figure 3 The flowchart of the specific implementation method of step S1022 in the embodiment shown is as follows;

[0026] Figure 6Flowchart of the video processing method provided in the embodiments of this disclosure Figure 2 ;

[0027] Figure 7 for Figure 6 The flowchart of the specific implementation method of step S203 in the embodiment shown is as follows;

[0028] Figure 8 This is a schematic diagram of a sub-region in an image to be processed, provided by an embodiment of the present disclosure;

[0029] Figure 9 A schematic diagram of a skin color binary image provided in an embodiment of this disclosure;

[0030] Figure 10 This is a schematic diagram illustrating a process for determining a skin region, provided in an embodiment of the present disclosure.

[0031] Figure 11 This is a structural block diagram of a video processing apparatus provided in an embodiment of the present disclosure;

[0032] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0033] Figure 13 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0035] The application scenarios of the embodiments of this disclosure are explained below:

[0036] Figure 1 This diagram illustrates an application scenario of the video processing method provided in this disclosure. The video processing method can be applied to video communication scenarios such as video conferencing and live video streaming. More specifically, it can be applied, for example, to video communication scenarios involving video conferencing and video chat using a personal computer (PC) without a dedicated graphics processing unit (GPU). The method provided in this disclosure can be applied to terminal devices, such as personal computers and smartphones. Figure 1As shown, in this embodiment, a personal computer (shown as the target terminal device in the figure) is used as the execution subject of this method embodiment. The personal computer runs a video communication client, uses a camera unit (such as a camera built into or connected to the terminal device) to collect user video, and interacts with the video communication server (shown as the server in the figure) to send the real-time collected user video to the server for distribution, thereby realizing video communication with other terminal devices running the video communication client.

[0037] In existing technologies, video communication applications provide users with real-time image smoothing, or "beauty" functions, to meet users' needs for concealing skin imperfections. Real-time image smoothing processing consumes corresponding computing resources, and the terminal device (e.g., Figure 1 Laptops typically utilize GPUs (Graphics Processing Units), which are better at processing image data, to achieve superior skin smoothing effects. However, in some cases, terminal devices without dedicated GPUs must rely on CPUs for skin smoothing. Due to differences in their structural characteristics and design purposes, CPUs excel at logic control, while GPUs are better suited for large-scale concurrent computation. In existing skin smoothing solutions, the need for extensive floating-point operations (addition, subtraction, multiplication, and division) allows GPUs to achieve high processing efficiency, enabling real-time skin smoothing during video calls. On terminal devices without GPUs, using the CPU results in high overhead and slow computation speed, impacting the real-time performance of skin smoothing and causing device lag.

[0038] Therefore, there is an urgent need for a method that can efficiently process skin smoothing in images using a central processing unit. This disclosure provides a video processing method to solve the above-mentioned problem.

[0039] refer to Figure 2 , Figure 2 Flowchart of the video processing method provided in the embodiments of this disclosure Figure 1 The method of this embodiment can be applied in a terminal device, and the video processing method includes:

[0040] Step S101: Obtain the target pixel data corresponding to the skin region in the image to be processed. The target pixel data represents the brightness value of the target pixel point located in the skin region.

[0041] For example, refer to Figure 1The illustrated application scenario diagram shows that during video transmission, the terminal device captures images in real time through the camera unit. The image to be processed can be either the original image captured by the camera unit or a preprocessed image after filtering, downsampling, format conversion, and other preprocessing steps. The specific implementation method can be set as needed and is not limited here. After obtaining the image to be processed, image recognition is performed to determine the image region representing the user's skin, i.e., the skin region. The method for identifying specific targets in the image is existing technology known to those skilled in the art and will not be elaborated here. After determining the skin region, the pixels located within the skin region are the target pixels. Further, the brightness value corresponding to the target pixels in the skin region of the image to be processed is obtained, i.e., the target pixel data. In one possible implementation, the image to be processed is a YUV image. YUV is a color encoding method, where "Y" represents luminance (Luminance or Luma), i.e., grayscale value, and "U" and "V" represent chrominance (Chrominance or Chroma), which describes the color and saturation of the image and is used to specify the color of the pixels. After determining the skin region (i.e., the coordinates of the target pixels within the skin region), the "Y" channel data of the target pixels corresponding to the skin region is extracted to obtain the brightness values ​​representing the target pixels within the skin region, i.e., the target pixel data. Optionally, the target pixel data can be a grayscale image of the image to be processed.

[0042] Step S102: Based on the target pixel data, obtain the brightness difference data. The brightness difference data represents the difference between the brightness value of the target pixel and the brightness value of the corresponding surrounding pixels.

[0043] For example, the goal of image smoothing is to reduce blemishes on a user's skin in a captured video (such as dark spots or wrinkles that are noticeably different from the surrounding skin), thereby achieving the purpose of concealing imperfections. In this process, it is necessary to first locate the blemishes on the skin. In this embodiment, after obtaining the target pixel data, based on the luminance value represented by the target pixel data, the luminance value of each target pixel is compared with the luminance values ​​of the surrounding pixels to obtain the difference amount corresponding to each target pixel. If the difference amount is greater than a preset threshold, it indicates that the difference between the target pixel and the surrounding pixels is large, and it may be a blemish on the skin; conversely, if the difference amount is less than the preset threshold, it indicates that the difference between the target pixel and the surrounding pixels is small, and it may be normal skin. Then, based on the set of difference amounts corresponding to each target pixel, luminance difference data is generated.

[0044] In one possible implementation, the brightness difference data is character data (char), more specifically, unsigned character data (unsigned char). For example, character data is used to define character variables in C or C++, occupying only one byte. The value range of character data is -128 to 127, and the value range of unsigned character data is 0 to 255. Character data cannot be directly calculated, but it can be mapped using a lookup table to achieve fast guided mean filtering based on brightness difference data, thereby achieving skin smoothing of the image. Since floating-point calculations are not required, this method is more suitable for the central processing unit, thus improving overall processing efficiency. The specific process will be described in detail in subsequent implementation steps. For example, the Y-channel data of each pixel in the image to be processed is implemented based on character data. For instance, the YUV image acquired by the camera unit is saved as character data, so that the Y-channel data representing the brightness value of each pixel in the image to be processed is character data. Correspondingly, the target pixel data in the image to be processed is also character-type data, and consequently, the brightness difference data obtained based on the target pixel data is also character-type data.

[0045] In one possible implementation, such as Figure 3 The specific implementation steps of step S102 include:

[0046] Step S1021: Perform mean filtering on the target pixel data to obtain the mean brightness data.

[0047] Step S1022: Obtain brightness difference data based on the difference between the target pixel data and the average brightness data.

[0048] For example, in the field of image processing technology, mean filtering is a typical linear filtering algorithm. It obtains the brightness values ​​of multiple adjacent or neighboring pixels around the target pixel and calculates them by averaging. The average calculation result is used as the average brightness value corresponding to the target pixel (the set of brightness values ​​corresponding to each target pixel, i.e., the average brightness data). Then, the absolute value of the difference between the brightness value of each target pixel and the corresponding average brightness value is calculated to obtain the set of brightness value differences, i.e., brightness difference data. Figure 4 The figure illustrates a process for obtaining brightness difference data according to an embodiment of this disclosure. Taking a target pixel as an example, firstly, the eight neighboring pixels surrounding the target pixel are acquired. Then, the Y-channel data of these eight neighboring pixels, i.e., the brightness values ​​of the neighboring pixels, are extracted. An average calculation is then performed to obtain the average brightness data. Finally, based on the difference between the average brightness data and the target pixel data, the brightness difference data is obtained.

[0049] Furthermore, when the target pixel data is character-type data, for example, the target pixel data includes a first brightness value in unsigned character type corresponding to each target pixel, and the average brightness data includes a second brightness value in unsigned character type corresponding to each target pixel, such as... Figure 5 As shown, the specific implementation steps of step S1022 include:

[0050] Step S1022A: For each target pixel, calculate the difference between the first brightness value and the second brightness value to obtain the first difference value of the unsigned character type.

[0051] Step S1022B: For each target pixel, calculate the difference between the second brightness value and the first brightness value to obtain an unsigned character-type second difference value, wherein at least one of the first difference value or the second difference value is 0.

[0052] Step S1022C: Calculate the sum of the first difference and the second difference corresponding to each target pixel to obtain the brightness difference data.

[0053] The calculation of brightness difference data is shown in equations (1) and (2):

[0054] V=charCal(Y,M)+charCal(M,Y) (1)

[0055]

[0056] Where V represents the brightness difference data, Y represents the first brightness value of the unsigned character type in the target pixel data, M represents the second brightness value of the unsigned character type in the average brightness data, and charCal(x1,x2) is a function that calculates the difference between the unsigned character type data x1 and the unsigned character type data x2. Specifically, since the difference between the brightness value of the target pixel and the brightness value of its corresponding surrounding pixels may be greater than zero or less than zero, in order to improve the operating efficiency of the central processing unit, when calculating based on unsigned character type data (with a value range of 0 to 255), if the obtained difference is less than zero, it will lead to result overflow. Therefore, in this embodiment, the difference is quickly calculated by calculating the first difference and the second difference separately and utilizing the truncation characteristics of unsigned character type data. Specifically, for example, for a target pixel, the corresponding Y (first brightness value) = 100, and the corresponding M (second brightness value) = 85; in this case, Y is greater than M, that is, the brightness value of the target pixel is greater than the average brightness value, and visually the target pixel is a "bright spot". The corresponding first difference charCal(Y, M) = 15, the first difference charCal(M, Y) = 0, and thus, the brightness difference data V = 15 is obtained; as another example, for another target pixel, the corresponding Y (first brightness value) = 40, and the corresponding M (second brightness value) = 85; in this case, Y is less than M, that is, the brightness value of the target pixel is less than the average brightness value, and visually the target pixel is a "dark spot". The corresponding first difference charCal(Y, M) = 0, the first difference charCal(M, Y) = 45, and thus, the brightness difference data V = 45 is obtained.

[0057] As described above, since both the target pixel data and the average brightness data are unsigned character types, the brightness difference data calculated from them is also unsigned character type. In this embodiment, by optimizing the process of calculating the brightness difference data, the process of obtaining brightness difference data based on unsigned character type data is realized. Thus, in subsequent steps, the central processing unit can determine the weight data based on the brightness difference data of character type data and perform guided mean filtering, thereby improving the data processing efficiency of the central processing unit.

[0058] Step S103: Obtain the weight data corresponding to the brightness difference data through the preset weight mapping information, and perform guided mean filtering on the target pixel data based on the weight data to obtain the target image. The weight data represents the intensity of mean filtering on each target pixel.

[0059] For example, guided mean filtering is a technique that achieves directional filtering by setting different weight coefficients for different pixels. In this embodiment, during the image retouching process, a larger intensity (weight) is used to retouch the skin for more obvious blemishes (pixels) to improve the effect of removing blemishes; while a smaller intensity (weight) is used to retouch the skin for less obvious blemishes or normal skin areas (pixels) to restore the skin details in the image and improve the image quality.

[0060] Specifically, in this embodiment, after obtaining the brightness difference data, the difference between the brightness value of the target pixel and the brightness values ​​of its surrounding pixels, as represented by the brightness difference data, is used in conjunction with preset weight mapping information to obtain the weight value corresponding to each target pixel, i.e., weight data, by looking up a table. Then, based on the weight data, the target pixel data corresponding to the skin region in the image to be processed is processed to adjust the brightness of each target pixel by weighting, thereby correcting the brightness values ​​of pixels that deviate from the average brightness value (i.e., the brightness value corresponding to normal skin), ultimately achieving a skin smoothing effect. In this process, the weight mapping information is preset. After obtaining the brightness difference data, the central processing unit (CPU) can directly obtain the corresponding weight data based on the weight mapping information, leveraging its strength in logical operations. Weighted calculations are then performed based on the weight data to achieve guided mean filtering without performing floating-point addition, subtraction, multiplication, or division operations, significantly improving CPU efficiency and reducing computation time.

[0061] For example, after step S103, the following may also be included:

[0062] Step S104: Generate a video data stream based on the target image and send the video data stream to the video communication server.

[0063] For example, after obtaining the target image, the target image is encoded and a video data stream is generated, which is sent to the video communication server in real time. On the video communication server side, the video data stream is processed as needed and then distributed to other terminal devices connected to it (see [link to specific communication connection details]). Figure 1 (The illustrated application scenario diagram) enables other terminal devices to obtain and display the target image that has undergone skin smoothing in real time, achieving a beautification effect during video communication. Based on the above introduction, since the target image generation process does not require complex addition, subtraction, multiplication, and division operations using a central processing unit, the generation speed of the target image can be significantly improved, thereby ensuring the real-time performance of the video data stream. In application scenarios such as video conferencing and live video streaming, it can improve the synchronization of images displayed on both sides of the video communication while achieving a beautification effect, thus enhancing the display effect of video conferencing and live video streaming.

[0064] In this embodiment, target pixel data corresponding to the skin region in the image to be processed is acquired. Target pixel data represents the brightness value of the target pixel located within the skin region. Based on the target pixel data, brightness difference data is obtained, representing the difference between the brightness value of the target pixel and the brightness values ​​of its surrounding pixels. Through preset weight mapping information, weight data corresponding to the brightness difference data is obtained, and guided mean filtering is performed on the target pixel data based on the weight data to obtain the target image. The weight data represents the intensity of mean filtering on each target pixel. A video data stream is generated based on the target image and sent to a video communication server. Because guided mean filtering is performed by looking up the corresponding weight data using preset weight mapping information, it avoids floating-point operations, which are not well-suited for the central processing unit (CPU). Therefore, it can improve the CPU's operating efficiency during image smoothing, solve the problem of high operating overhead and slow calculation speed during image smoothing, improve the running speed of the image smoothing process, shorten the running time, and improve device smoothness.

[0065] refer to Figure 6 , Figure 6 Flowchart of the video processing method provided in the embodiments of this disclosure Figure 2 This embodiment is in Figure 2 Based on the illustrated embodiment, steps S102-S103 are further refined, and a step of determining the skin region is added. This video processing method includes:

[0066] Step S201: Downsample the image to be processed to obtain a downsampled image.

[0067] Step S202: Convert the downsampled image to obtain an RGB image.

[0068] For example, after obtaining the image to be processed, to further improve the image processing speed, the image to be processed can first be downsampled to obtain a downsampled image with a lower resolution. Specifically, for example, if the image to be processed is a 1024x1024 resolution image, it can be downsampled by a factor of 16 to obtain a downsampled image with a resolution of 64x64. Since the downsampled image obtained in this embodiment is used to locate the skin region, and in subsequent steps, combining it with guided mean filtering will not affect the quality of the final generated target image, but can significantly reduce the computational load of important processors and improve the processing efficiency of image skin smoothing.

[0069] Next, the downsampled image is converted to obtain the corresponding RGB image. For example, the image to be processed is in YUV format, and the corresponding downsampled image is also in YUV format. The specific implementation of converting a YUV image to an RGB image is existing technology and will not be elaborated here.

[0070] Step S203: Perform skin color detection on the RGB image to determine the skin region in the image to be processed.

[0071] RGB image processing is a technique that uses the combination of three color channels—R (red), G (green), and B (blue)—to represent the color of each pixel in an image. The pixels corresponding to skin tones have specific color characteristics. By detecting the pixel values ​​of each pixel in the RGB image and comparing them with preset color characteristics, the skin regions in the image to be processed can be obtained. One possible implementation is to use a binary image to represent the skin regions in the image to be processed.

[0072] For example, such as Figure 7 As shown, the specific implementation steps of step S203 include:

[0073] Step S2031: Based on preset segmentation parameters, the image to be processed is segmented into multiple sub-regions of the same size. The segmentation parameters represent the number or size of the sub-regions.

[0074] Step S2032: Perform skin color detection based on the RGB pixel values ​​of each pixel in the RGB image to obtain the target pixel, which is a pixel whose RGB pixel value has skin characteristics.

[0075] For example, Figure 8 This is a schematic diagram of a sub-region in an image to be processed, provided by an embodiment of this disclosure, such as... Figure 8 As shown, based on preset segmentation parameters, the image to be processed is divided into fixed-size square regions, i.e., sub-regions. Each sub-region includes at least one pixel from an RGB image (shown as RGB pixels in the figure). Optionally, the size of the RGB image is an integer multiple of the size of the sub-region; for example, the RGB image is generated based on a downsampled image obtained by downsampling the image to be processed by 16 times. Therefore, one RGB pixel corresponds to 16 pixels in the image to be processed, and the resolution of the RGB image is 64x64. A sub-region, for example, corresponds to 4 pixels in the RGB image. Figure 8As shown in the illustration, for example, in sub-region A of the image to be processed, there are 4 RGB pixels. After performing skin color detection on the RGB pixel values ​​of each pixel, it is determined that the RGB pixel values ​​of each pixel have skin features, indicating that the sub-region corresponds to a skin region. In sub-region B of the image to be processed, after performing skin color detection on the RGB pixel values ​​of each pixel, the RGB pixel values ​​of the four pixels do not have skin features. Therefore, the four pixels are not target pixels, indicating that the sub-region corresponds to a non-skin region. In sub-region C of the image to be processed, after performing skin color detection on the RGB pixel values ​​of each pixel, the RGB pixel values ​​of pixels p1 and p2 have skin features. Therefore, pixels p1 and p2 are target pixels, indicating that the sub-region corresponds to the edge of the non-skin region and the skin region.

[0076] For example, the specific implementation method of step S2032 includes:

[0077] The RGB pixel values ​​corresponding to each pixel in the RGB image are detected to obtain skin color pixels. The RGB pixel values ​​of the skin color pixels are located within a preset RGB threshold range. An opening operation is performed on each skin color pixel to obtain valid skin color pixels. Based on the valid skin color pixels, a skin color binary map is generated. The skin color binary map is used to represent the position of the target pixel in the RGB image.

[0078] For example, after detecting the RGB pixel values ​​of each pixel, the RGB pixel values ​​of each pixel are judged based on the preset RGB threshold region corresponding to the skin color, and the pixels that conform to the skin characteristics, i.e., skin color pixels, can be obtained. Then, an opening operation is performed on each skin color pixel to remove outliers, and a skin color binary map is obtained to represent the position of the target pixel in the RGB image. In this embodiment, by performing an opening operation on the skin color pixels to remove outliers far from the skin color pixel cluster, the detection accuracy of the skin region is improved, and the image quality of the final generated target image is improved.

[0079] Step S2033: Based on the target pixels, determine multiple target sub-regions, and define the set of target sub-regions as the skin region, wherein the target sub-region is a sub-region containing the target pixels.

[0080] For example, combined Figure 8In the illustrated embodiment, for sub-regions A, B, and C, sub-regions A and C containing the target pixel are determined as target sub-regions. Exemplarily, the above steps can be performed by sequentially traversing each sub-region to determine the target sub-region. Since it is not necessary to detect all pixels in a sub-region, only one target pixel is needed to determine it as a target sub-region. Therefore, fast traversal of each sub-region can be achieved. More specifically, for example, each sub-region corresponds to 4 RGB pixels. When traversing each sub-region sequentially, if the first RGB pixel in the first sub-region is identified as the target pixel, further detection is stopped, and the first sub-region is directly determined as the target sub-region. Then, the traversal continues to the second sub-region, and so on, until all sub-regions have been traversed. As can be seen, since it is not necessary to traverse all RGB pixels in a sub-region, the speed of traversing each sub-region is improved.

[0081] For example, the step of determining multiple target sub-regions based on target pixels includes: aligning a skin color binary map and the image to be processed, and marking each sub-region based on the skin color identifier in the skin color binary map to determine the target sub-region. Figure 9 This is a schematic diagram of a skin color binary image provided in an embodiment of the present disclosure, as shown below. Figure 9 As shown, a skin color binary image is used to represent the positional coordinates of skin color regions. For example, the skin color binary image is generated based on an RGB image; therefore, its size is the same as the RGB image. Interpolation can be used to process the skin color binary image to obtain a skin color binary image with the same size as the image to be processed, thus aligning the skin color binary image with the image to be processed. Then, for example, the skin color binary image includes skin color identifiers, such as the binary number "1" in the image (non-skin regions are represented by the binary number "0"). Regions with skin color identifiers represent skin regions. Sub-regions are marked according to the skin color identifiers (i.e., "1") in the skin color binary image to determine the target sub-region.

[0082] For example, in step S2033, the step of determining the set of target sub-regions as skin regions includes: detecting the adjacent distance of each target sub-region; merging target sub-regions with adjacent distances less than a preset distance into adjacent regions; and determining adjacent regions as skin regions based on segmentation parameters if the size of a target sub-region is less than a preset size threshold.

[0083] For example, after determining the target sub-regions, the final skin region is determined by combining the target sub-regions. Specifically, this involves detecting the adjacent distances of each target sub-region and fusing adjacent target sub-regions into a single adjacent region. In one possible implementation, the segmentation parameters corresponding to the sub-regions are used to determine the segmentation accuracy of the sub-regions. For the image to be processed, a larger sub-region size indicates lower segmentation accuracy, while a smaller sub-region size indicates higher segmentation accuracy. Based on the segmentation parameters, when the sub-region size is less than a preset size threshold, the accuracy of the sub-region is considered to have met the requirements. At this point, since human skin (e.g., exposed skin on the face, limbs, etc.) is usually a continuous region in the image, by controlling the segmentation parameters, the fused adjacent regions are determined as the skin region, while the individual target sub-regions that are not fused are considered as incorrectly detected matching regions. This optimizes the detection results of the skin region and improves the detection accuracy and correctness of the skin region. Figure 10 This is a schematic diagram illustrating a process for determining a skin region, as provided in an embodiment of this disclosure. Figure 10 As shown, after determining the target sub-regions A to E, based on the distance between them, target sub-regions A to D are merged into a single, adjacent region. Target sub-region E, however, is not merged and remains an independent region. Ultimately, the merged region corresponding to target sub-regions A to E is determined as the skin region, while the unmerged target sub-region E is not determined to be a skin region.

[0084] In this embodiment, skin color recognition is performed by extracting the RGB image of the image to be processed, and combined with image segmentation, this achieves fast and accurate localization of skin color regions.

[0085] Step S204: Obtain the target pixel data corresponding to the skin region in the image to be processed. The target pixel data represents the brightness value of the target pixel point located in the skin region.

[0086] Step S205: Obtain brightness difference data based on target pixel data.

[0087] Step S206: Obtain the weight data corresponding to the brightness difference data through the preset weight mapping information.

[0088] For example, steps S204 to S206 are steps of obtaining target pixel data corresponding to the skin region and calculating weight data based on the target pixel data. The above steps are... Figure 2 The illustrated embodiments have been described in detail; please refer to [link / reference]. Figure 2 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0089] Step S207: Based on the weight data, obtain the weight map. The weight map includes the weight coefficients corresponding to each target pixel. The weight coefficients represent the fusion ratio between the target pixel value and the corresponding average brightness value in the brightness channel of the target pixel. The average brightness value is the result of mean filtering on the target pixel value.

[0090] For example, after obtaining the weight data, it is necessary to map the weight coefficients of the weight data to each target pixel. This process can be achieved by generating a weight map. For example, the specific implementation steps of step S207 include:

[0091] Step S2071: Obtain the binary image of skin color corresponding to the skin region.

[0092] Step S2072: Obtain the weight map based on the skin color binary map and weight data.

[0093] For example, the skin color binary image is generated during the detection of skin regions in the image to be processed. It is used to represent the coordinates of the pixels corresponding to the service region. This skin color binary image can be generated based on the RGB image corresponding to the image to be processed. For details on obtaining the skin color binary image corresponding to the skin region, please refer to the relevant description in step S203, which will not be repeated here. Then, the skin color binary image and weight data are fused, and the data size is aligned to assign a corresponding weight coefficient to each target pixel indicated by the skin color binary image, thereby obtaining a weight map representing the weight coefficients of each target pixel. The weight coefficient represents the fusion ratio between the target pixel value of the brightness channel and the corresponding average brightness value of the target pixel. For example, the larger the weight coefficient, the greater the fusion weight of the target pixel value of the brightness channel of the target pixel. Therefore, the result obtained after fusion based on this weight coefficient is more influenced by the target pixel value of the brightness channel of the target pixel (i.e., the weaker the skin smoothing effect). Conversely, the smaller the weight coefficient, the smaller the fusion weight of the target pixel value of the brightness channel of the target pixel. Therefore, the result obtained after fusion based on this weight coefficient is more influenced by the average brightness value corresponding to the target pixel (i.e., the stronger the skin smoothing effect).

[0094] Step S208: Based on the weight map, fuse the target pixel data and the corresponding average brightness data to obtain the target image, wherein the average brightness data is the set of average brightness values ​​corresponding to each target pixel.

[0095] For example, subsequently, based on the fusion ratio of the target pixel value and the corresponding average brightness value of the brightness channel of each target pixel represented by the weight map, the two are fused to obtain the fused brightness corresponding to each target pixel, i.e., the structure of the skin smoothing process. The specific method for obtaining the average brightness value is discussed in... Figure 2 Step S102 of the illustrated embodiment has been described and will not be repeated here.

[0096] Step S209: Generate a video data stream based on the target image and send the video data stream to the video communication server.

[0097] The specific implementation method of step S209 in this embodiment is the same as Figure 2 The specific implementation of step S104 in the illustrated embodiment is the same, and will not be repeated here.

[0098] The video processing method provided in this disclosure is described below with a more detailed embodiment, in which the execution subject is a terminal device or the central processing unit of the terminal device:

[0099] Step 01: Obtain the original image using the camera. yuv (YUV format, size 720*1280), type is 3 channels, unsigned char type.

[0100] Step 02: Obtain the segmentation parameter w n and h n Divide the original image into a grid set of fixed size, denoted as G, containing (w n +1)*(h n +1) small grids.

[0101] Step 03: Process the original image I yuv Perform a significant downsampling (e.g., 16x downsampling), then convert it to an RGB image to obtain the downsampled RGB image I. ds ;

[0102] Step 04: For I ds Using skin color detection algorithms:

[0103] The skin color region is determined by (R>95&&G>40&&B>20&&R>G&&R>B&&max(R,G,B)-min(R,G,B)>15&&abs(RG)>15), and an opening operation is performed to reduce noise, resulting in the skin color in the downsampled space. Figure 2 Value Map

[0104] Step 05: Traversal For each pixel in G, calculate which small grids in G each pixel can be mapped to, and mark the small grids with skin color information.

[0105] Step 06: Merge the small grids marked in G according to their adjacency relationships to obtain one or more non-intersecting rectangular regions (Rect). roi .

[0106] Step 07: Obtain the Y channel information of the original image (I) y (Size is 720*1280).

[0107] Step 08: In I y Take the rectangle of each region of interest (ROI) containing skin color information. roi The image information is denoted as Data types and I y They are consistent, both being single-channel and of type unsigned char.

[0108] Step 09: [Regarding...] Perform a small downsampling (e.g., 4 times) to obtain Data types and I y They are consistent, both being single-channel and of type unsigned char.

[0109] Step 10: In Take each rectangular region of interest (ROI) containing skin color information (Rect) roi The corresponding skin tone information, and the size adjusted to match. Same, denoted as

[0110] Step 11: [Regarding...] Perform a mean filter once to obtain Data types and I y They are consistent, both being single-channel and of type unsigned char.

[0111] Step 12: Through calculation Obtain the difference map Data types and I y They are consistent, both being single-channel and of type unsigned char.

[0112] Step 13: Analyze the difference graph For each value in the table, a weighted map is obtained by looking up table M (pre-calculated, representing the mapping relationship between difference values ​​and weight values) to subsequently mix the original image and the mean image. Data types and I y They are consistent, both being single-channel and of type unsigned char.

[0113] Step 14: Use libyuv's weighted algorithm to mix and The final weight map used for mixing is obtained. Data types and I y They are consistent, both being single-channel and of type unsigned char.

[0114] Step 15: and Upsampling, size adjusted to match Same, denoted as M roi and WS roi All are single-channel and of type unsigned char.

[0115] Step 16: Utilize libyuv's weighted algorithm to calculate the weighted average of WS. roi For weights, mixed and M roi This yields the target image, i.e., the skin smoothing result.

[0116] Step 17: Generate a video data stream based on the target image and send the video data stream to the video communication server.

[0117] Corresponding to the video processing method in the above embodiments, Figure 11 This is a structural block diagram of a video processing apparatus provided in an embodiment of the present disclosure. For ease of explanation, only the parts relevant to the embodiments of the present disclosure are shown.

[0118] Reference Figure 11 The video processing device 3 includes:

[0119] The acquisition module 31 is used to acquire target pixel data corresponding to the skin region in the image to be processed. The target pixel data represents the brightness value of the target pixel point located in the skin region.

[0120] The comparison module 32 is used to obtain brightness difference data based on the target pixel data. The brightness difference data represents the amount of difference between the brightness value of the target pixel and the brightness value of the corresponding surrounding pixels.

[0121] The filtering module 33 is used to obtain the weight data corresponding to the brightness difference data through the preset weight mapping information, and to perform guided mean filtering on the target pixel data based on the weight data to obtain the target image. The weight data represents the intensity of mean filtering on each target pixel.

[0122] Communication module 34 is used to generate video data streams based on target images.

[0123] In one possible implementation, the comparison module 32 is specifically used to: perform mean filtering on the target pixel data to obtain average brightness data; and obtain brightness difference data based on the difference between the target pixel data and the average brightness data, wherein the brightness difference data is character data.

[0124] In one possible implementation, the target pixel data includes a first brightness value in unsigned character type corresponding to each target pixel, and the average brightness data includes a second brightness value in unsigned character type corresponding to each target pixel. When the comparison module 32 obtains the brightness difference data based on the difference between the target pixel data and the average brightness data, it specifically performs the following steps: for each target pixel, calculate the difference between the first brightness value and the second brightness value to obtain a first difference value in unsigned character type; for each target pixel, calculate the difference between the second brightness value and the first brightness value to obtain a second difference value in unsigned character type, wherein at least one of the first difference value or the second difference value is 0; calculate the sum of the first difference value and the second difference value corresponding to each target pixel to obtain the brightness difference data.

[0125] In one possible implementation, when the filtering module 33 performs guided mean filtering on the target pixel data based on the weight data to obtain the target image, it specifically performs the following steps: Based on the weight data, it obtains a weight map, which includes weight coefficients corresponding to each target pixel. The weight coefficients represent the fusion ratio between the target pixel value and the corresponding average brightness value in the brightness channel of the target pixel, and the average brightness value is the result of mean filtering on the target pixel value. Based on the weight map, it fuses the target pixel data and the corresponding average brightness data to obtain the target image, wherein the average brightness data is the set of average brightness values ​​corresponding to each target pixel.

[0126] In one possible implementation, when the filtering module 33 obtains the weight map based on the weight data, it is specifically used to: obtain the skin color binary map corresponding to the skin region; and obtain the weight map based on the skin color binary map and the weight data.

[0127] In one possible implementation, before acquiring the target pixel data corresponding to the skin region in the image to be processed, the acquisition module 31 is further configured to: acquire the RGB image corresponding to the image to be processed; perform skin color detection on the RGB image to determine the skin region in the image to be processed.

[0128] In one possible implementation, before acquiring the RGB image corresponding to the image to be processed, the acquisition module 31 is further configured to: downsample the image to be processed to obtain a downsampled image; when acquiring the RGB image corresponding to the image to be processed, the acquisition module 31 is specifically configured to: convert the downsampled image to obtain an RGB image.

[0129] In one possible implementation, when the acquisition module 31 performs skin color detection on an RGB image to determine the skin region in the image to be processed, it is specifically used to: divide the image to be processed into multiple sub-regions of the same size based on preset segmentation parameters, wherein the segmentation parameters represent the number or size of the sub-regions; perform skin color detection based on the RGB pixel values ​​of each pixel in the RGB image to obtain target pixels, wherein the target pixels are pixels whose RGB pixel values ​​have skin features; determine multiple target sub-regions based on the target pixels, and determine the set of target sub-regions as the skin region, wherein the target sub-region is a sub-region containing the target pixels.

[0130] In one possible implementation, when the acquisition module 31 performs skin color detection based on the RGB pixel values ​​of each pixel in the RGB image to obtain the target pixel, it specifically performs the following: detects the RGB pixel values ​​corresponding to each pixel in the RGB image to obtain skin color pixels, wherein the RGB pixel values ​​of the skin color pixels are located within a preset RGB threshold range; performs an opening operation on each skin color pixel to obtain valid skin color pixels; and generates a skin color binary map based on the valid skin color pixels, wherein the skin color binary map is used to represent the position of the target pixel in the RGB image.

[0131] When the acquisition module 31 determines multiple target sub-regions based on the target pixels, it is specifically used to: align the skin color binary image and the image to be processed, and mark each sub-region based on the skin color identifier in the skin color binary image to determine the target sub-region.

[0132] In one possible implementation, when determining the set of target sub-regions as skin regions, the acquisition module 31 is specifically used to: detect the adjacent distance of each target sub-region; merge target sub-regions with adjacent distances less than a preset distance into adjacent regions; and, based on the segmentation parameters, if the size of the target sub-region is less than a preset size threshold, determine the adjacent region as a skin region.

[0133] The acquisition module 31, comparison module 32, filtering module 33, and communication module 34 are connected sequentially. The video processing device 3 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effects are similar; therefore, it will not be described again here.

[0134] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 12 As shown, the electronic device 4 includes:

[0135] Processor 41, and memory 42 communicatively connected to processor 41;

[0136] Memory 42 stores instructions executed by the computer;

[0137] The processor 41 executes computer execution instructions stored in the memory 42 to achieve, for example, Figures 2-10 The video processing method in the illustrated embodiment.

[0138] Optionally, the processor 41 and the memory 42 are connected via a bus 43.

[0139] For relevant instructions, please refer to the corresponding text. Figures 2-10 The relevant descriptions and effects of the steps in the corresponding embodiments are understood, and will not be elaborated on here.

[0140] refer to Figure 13 The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 13 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0141] like Figure 13 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0142] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 13 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0143] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0144] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0145] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0146] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0147] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0149] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0150] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0151] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0152] In a first aspect, according to one or more embodiments of this disclosure, a video processing method is provided, comprising:

[0153] The process involves: acquiring target pixel data corresponding to the skin region in the image to be processed, whereby the target pixel data represents the brightness value of the target pixel located within the skin region; obtaining brightness difference data based on the target pixel data, whereby the brightness difference data represents the amount of difference between the brightness value of the target pixel and the brightness values ​​of its corresponding surrounding pixels; obtaining weight data corresponding to the brightness difference data through preset weight mapping information, and performing guided mean filtering on the target pixel data based on the weight data to obtain the target image, whereby the weight data represents the intensity of mean filtering on each target pixel; and generating a video data stream based on the target image.

[0154] According to one or more embodiments of this disclosure, obtaining brightness difference data based on target pixel data includes: performing mean filtering on the target pixel data to obtain average brightness data; and obtaining brightness difference data based on the difference between the target pixel data and the average brightness data, wherein the brightness difference data is character-type data.

[0155] According to one or more embodiments of this disclosure, the target pixel data includes a first brightness value in unsigned character type corresponding to each target pixel, and the average brightness data includes a second brightness value in unsigned character type corresponding to each target pixel. Brightness difference data is obtained based on the difference between the target pixel data and the average brightness data, including: for each target pixel, calculating the difference between the first brightness value and the second brightness value to obtain a first difference value in unsigned character type; for each target pixel, calculating the difference between the second brightness value and the first brightness value to obtain a second difference value in unsigned character type, wherein at least one of the first difference value or the second difference value is 0; and calculating the sum of the first difference value and the second difference value corresponding to each target pixel to obtain the brightness difference data.

[0156] According to one or more embodiments of this disclosure, a target image is obtained by performing guided mean filtering on target pixel data based on weight data, including: obtaining a weight map based on the weight data, the weight map including weight coefficients corresponding to each target pixel, the weight coefficients representing the fusion ratio between the target pixel value and the corresponding average brightness value of the brightness channel of the target pixel, the average brightness value being the result of mean filtering on the target pixel value; and fusing the target pixel data and the corresponding average brightness data based on the weight map to obtain the target image, wherein the average brightness data is the set of average brightness values ​​corresponding to each target pixel.

[0157] According to one or more embodiments of this disclosure, obtaining a weight map based on weight data includes: obtaining a skin color binary map corresponding to a skin region; and obtaining a weight map based on the skin color binary map and weight data.

[0158] According to one or more embodiments of this disclosure, before acquiring the target pixel data corresponding to the skin region in the image to be processed, the method further includes: acquiring an RGB image corresponding to the image to be processed; performing skin color detection on the RGB image to determine the skin region in the image to be processed.

[0159] According to one or more embodiments of this disclosure, before obtaining the RGB image corresponding to the image to be processed, the method further includes: downsampling the image to be processed to obtain a downsampled image; obtaining the RGB image corresponding to the image to be processed includes: converting the downsampled image to obtain an RGB image.

[0160] According to one or more embodiments of this disclosure, skin color detection is performed on an RGB image to determine the skin region in the image to be processed, including: dividing the image to be processed into multiple sub-regions of the same size based on preset segmentation parameters, wherein the segmentation parameters characterize the number or size of the sub-regions; performing skin color detection based on the RGB pixel values ​​of each pixel in the RGB image to obtain target pixels, wherein the target pixels are pixels whose RGB pixel values ​​have skin features; determining multiple target sub-regions based on the target pixels, and determining the set of target sub-regions as the skin region, wherein the target sub-region is a sub-region containing the target pixels.

[0161] According to one or more embodiments of this disclosure, skin color detection is performed based on the RGB pixel values ​​of each pixel in an RGB image to obtain target pixels, including: detecting the RGB pixel values ​​corresponding to each pixel in the RGB image to obtain skin color pixels, wherein the RGB pixel values ​​of the skin color pixels are located within a preset RGB threshold range; performing an opening operation on each skin color pixel to obtain valid skin color pixels; generating a skin color binary image based on the valid skin color pixels, wherein the skin color binary image is used to characterize the position of the target pixel in the RGB image; and determining multiple target sub-regions based on the target pixels, including: aligning the skin color binary image and the image to be processed, and marking each sub-region based on the skin color identifier in the skin color binary image to determine the target sub-regions.

[0162] According to one or more embodiments of this disclosure, determining a set of target sub-regions as skin regions includes: detecting the adjacent distances of each target sub-region; merging target sub-regions with adjacent distances less than a preset distance into adjacent regions; and determining adjacent regions as skin regions based on segmentation parameters if the size of a target sub-region is less than a preset size threshold.

[0163] Secondly, according to one or more embodiments of the present disclosure, a video processing apparatus is provided, comprising:

[0164] The acquisition module is used to acquire target pixel data corresponding to the skin region in the image to be processed. The target pixel data represents the brightness value of the target pixel point located in the skin region.

[0165] The comparison module is used to obtain brightness difference data based on the target pixel data. The brightness difference data represents the amount of difference between the brightness value of the target pixel and the brightness value of the corresponding surrounding pixels.

[0166] The filtering module is used to obtain the weight data corresponding to the brightness difference data through the preset weight mapping information, and to perform guided mean filtering on the target pixel data based on the weight data to obtain the target image. The weight data represents the intensity of mean filtering on each target pixel.

[0167] The communication module is used to generate a video data stream based on the target image and send the video data stream to the video communication server.

[0168] According to one or more embodiments of this disclosure, the comparison module is specifically used for: performing mean filtering on the target pixel data to obtain average brightness data; and obtaining brightness difference data based on the difference between the target pixel data and the average brightness data, wherein the brightness difference data is character-type data.

[0169] According to one or more embodiments of this disclosure, the target pixel data includes a first brightness value in unsigned character type corresponding to each target pixel, and the average brightness data includes a second brightness value in unsigned character type corresponding to each target pixel. When the comparison module obtains brightness difference data based on the difference between the target pixel data and the average brightness data, it is specifically used to: calculate the difference between the first brightness value and the second brightness value for each target pixel to obtain a first difference value in unsigned character type; calculate the difference between the second brightness value and the first brightness value for each target pixel to obtain a second difference value in unsigned character type, wherein at least one of the first difference value or the second difference value is 0; and calculate the sum of the first difference value and the second difference value corresponding to each target pixel to obtain the brightness difference data.

[0170] According to one or more embodiments of this disclosure, when the filtering module performs guided mean filtering on target pixel data based on weight data to obtain a target image, it is specifically used to: obtain a weight map based on the weight data, the weight map including weight coefficients corresponding to each target pixel, the weight coefficients representing the fusion ratio between the target pixel value and the corresponding average brightness value of the brightness channel of the target pixel, the average brightness value being the result of mean filtering on the target pixel value; and fuse the target pixel data and the corresponding average brightness data based on the weight map to obtain the target image, wherein the average brightness data is the set of average brightness values ​​corresponding to each target pixel.

[0171] According to one or more embodiments of this disclosure, when the filtering module obtains a weight map based on the weight data, it is specifically used to: obtain a skin color binary map corresponding to the skin region; and obtain a weight map based on the skin color binary map and the weight data.

[0172] According to one or more embodiments of this disclosure, before acquiring the target pixel data corresponding to the skin region in the image to be processed, the acquisition module is further configured to: acquire the RGB image corresponding to the image to be processed; perform skin color detection on the RGB image to determine the skin region in the image to be processed.

[0173] According to one or more embodiments of this disclosure, before acquiring the RGB image corresponding to the image to be processed, the acquisition module is further configured to: downsample the image to be processed to obtain a downsampled image; when acquiring the RGB image corresponding to the image to be processed, the acquisition module is specifically configured to: convert the downsampled image to obtain an RGB image.

[0174] According to one or more embodiments of this disclosure, when the acquisition module performs skin color detection on an RGB image to determine the skin region in the image to be processed, it is specifically used to: divide the image to be processed into multiple sub-regions of the same size based on preset segmentation parameters, wherein the segmentation parameters characterize the number or size of the sub-regions; perform skin color detection based on the RGB pixel values ​​of each pixel in the RGB image to obtain target pixels, wherein the target pixels are pixels whose RGB pixel values ​​have skin features; determine multiple target sub-regions based on the target pixels, and determine the set of target sub-regions as the skin region, wherein the target sub-region is a sub-region containing the target pixels.

[0175] According to one or more embodiments of this disclosure, when the acquisition module performs skin color detection based on the RGB pixel values ​​of each pixel in the RGB image to obtain target pixels, it is specifically used to: detect the RGB pixel values ​​corresponding to each pixel in the RGB image to obtain skin color pixels, wherein the RGB pixel values ​​of the skin color pixels are located within a preset RGB threshold range; perform an opening operation on each skin color pixel to obtain effective skin color pixels; and generate a skin color binary image based on the effective skin color pixels, wherein the skin color binary image is used to characterize the position of the target pixel in the RGB image; when the acquisition module 31 determines multiple target sub-regions based on the target pixels, it is specifically used to: align the skin color binary image and the image to be processed, and mark each sub-region based on the skin color identifier in the skin color binary image to determine the target sub-region.

[0176] According to one or more embodiments of this disclosure, when the acquisition module determines the set of target sub-regions as skin regions, it is specifically used to: detect the adjacent distance of each target sub-region; merge the target sub-regions with adjacent distances less than a preset distance into adjacent regions; and, based on the segmentation parameters, if the size of the target sub-region is less than a preset size threshold, determine the adjacent region as a skin region.

[0177] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, including: a processor, and a memory communicatively connected to the processor;

[0178] The memory stores the instructions that the computer executes;

[0179] The processor executes computer-executable instructions stored in memory to implement the video processing method described in the first aspect above and various possible designs of the first aspect.

[0180] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions that, when executed by a processor, implement the video processing method described in the first aspect and various possible designs of the first aspect.

[0181] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the video processing method as described in the first aspect and various possible designs of the first aspect.

[0182] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0183] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0184] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A video processing method, characterized in that, include: Acquire target pixel data corresponding to the skin region in the image to be processed, wherein the target pixel data represents the brightness value of the target pixel located within the skin region; Based on the target pixel data, brightness difference data is obtained, which represents the difference between the brightness value of the target pixel and the brightness value of the corresponding surrounding pixels. The weight data corresponding to the brightness difference data is obtained by using preset weight mapping information; Based on the weight data, a weight map is obtained; the weight map includes weight coefficients corresponding to each target pixel, the weight coefficients represent the fusion ratio between the target pixel value and the corresponding average brightness value of the brightness channel of the target pixel, and the average brightness value is the result of mean filtering on the target pixel value; Based on the weight map, the target pixel data and the corresponding average brightness data are fused to obtain the target image; The average brightness data is a set of average brightness values ​​corresponding to each of the target pixels; wherein, the weight data represents the intensity of mean filtering applied to each of the target pixels. A video data stream is generated based on the target image.

2. The method according to claim 1, characterized in that, The process of obtaining brightness difference data based on the target pixel data includes: The target pixel data is subjected to mean filtering to obtain the average brightness data; The brightness difference data is obtained based on the difference between the target pixel data and the average brightness data, wherein the brightness difference data is character data.

3. The method according to claim 2, characterized in that, The target pixel data includes a first brightness value in unsigned character type corresponding to each target pixel, and the average brightness data includes a second brightness value in unsigned character type corresponding to each target pixel. The brightness difference data is obtained based on the difference between the target pixel data and the average brightness data, including: For each target pixel, the difference between the first brightness value and the second brightness value is calculated to obtain the first difference value of the unsigned character type. For each target pixel, the difference between the second brightness value and the first brightness value is calculated to obtain an unsigned character-type second difference value, wherein at least one of the first difference value or the second difference value is 0; The sum of the first difference and the second difference corresponding to each target pixel is calculated to obtain the brightness difference data.

4. The method according to claim 1, characterized in that, The step of obtaining the weight graph based on the weight data includes: Obtain the binary image of the skin color corresponding to the skin region; The weight map is obtained based on the skin color binary map and the weight data.

5. The method according to any one of claims 1-4, characterized in that, Before acquiring the target pixel data corresponding to the skin region in the image to be processed, the method further includes: Obtain the RGB image corresponding to the image to be processed; Skin color detection is performed on the RGB image to determine the skin regions in the image to be processed.

6. The method according to claim 5, characterized in that, Before obtaining the RGB image corresponding to the image to be processed, the process also includes: The image to be processed is downsampled to obtain a downsampled image; The step of obtaining the RGB image corresponding to the image to be processed includes: The downsampled image is converted to obtain the RGB image.

7. The method according to claim 5, characterized in that, The step of performing skin color detection on the RGB image to determine the skin region in the image to be processed includes: Based on preset segmentation parameters, the image to be processed is divided into multiple sub-regions of the same size, wherein the segmentation parameters represent the number or size of the sub-regions; Skin color detection is performed based on the RGB pixel values ​​of each pixel in the RGB image to obtain target pixels, where the target pixels are pixels whose RGB pixel values ​​have skin features. Based on the target pixel, multiple target sub-regions are determined, and the set of the target sub-regions is determined as the skin region, wherein the target sub-region is a sub-region that contains the target pixel.

8. The method according to claim 7, characterized in that, The step of detecting skin color based on the RGB pixel values ​​of each pixel in the RGB image to obtain the target pixel includes: The RGB pixel values ​​corresponding to each pixel in the RGB image are detected to obtain skin color pixels, and the RGB pixel values ​​of the skin color pixels are located within a preset RGB threshold range; Perform an opening operation on each of the skin color pixels to obtain the effective skin color pixels; Based on the effective skin color pixels, a skin color binary map is generated, which is used to characterize the position of the target pixel in the RGB image; The step of determining multiple target sub-regions based on the target pixels includes: Align the skin color binary image and the image to be processed, and mark each of the sub-regions based on the skin color identifier in the skin color binary image to determine the target sub-region.

9. The method according to claim 7, characterized in that, The step of determining the set of target sub-regions as the skin region includes: Detect the adjacent distances of each of the target sub-regions; The target sub-regions whose adjacent distance is less than a preset distance are merged into adjacent regions; Based on the segmentation parameters, if the size of the target sub-region is smaller than a preset size threshold, then the adjacent region is determined as the skin region.

10. A video processing apparatus, characterized in that, include: The acquisition module is used to acquire target pixel data corresponding to the skin region in the image to be processed, wherein the target pixel data represents the brightness value of the target pixel point located in the skin region; The comparison module is used to obtain brightness difference data based on the target pixel data, wherein the brightness difference data represents the amount of difference between the brightness value of the target pixel and the brightness value of the corresponding surrounding pixels; The filtering module is used to obtain weight data corresponding to the brightness difference data through preset weight mapping information; and to obtain a weight map based on the weight data; the weight map includes weight coefficients corresponding to each target pixel, the weight coefficients representing the fusion ratio of the target pixel value of the brightness channel of the target pixel to the corresponding average brightness value, and the average brightness value being the result of mean filtering on the target pixel value; Based on the weight map, the target pixel data and the corresponding average brightness data are fused to obtain the target image; The average brightness data is a set of average brightness values ​​corresponding to each of the target pixels; wherein, the weight data represents the intensity of mean filtering applied to each of the target pixels. A communication module is used to generate a video data stream based on the target image.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the video processing method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the video processing method according to any one of claims 1 to 9.

Images