Image processing and live broadcast method, device, equipment and storage medium

Abstract

The present invention discloses an image processing and live broadcasting method, device, equipment and storage medium, the method comprising: determining original image data, the original image data having a plurality of pixels; for each pixel, adjusting the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value; determining the skin color area in the original image data; generating blur intensity according to the target brightness value and the skin color area; based on the original image data, blurring the skin color area according to the blur intensity and the target brightness value to obtain the target image data. Through this method, the effect of processing the skin color area in a targeted manner is achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to image processing technology, and more particularly to an image processing and live broadcasting method, apparatus, device and storage medium. Background Art

[0002] With the popularity of beauty enhancement, more and more users hope to use beauty enhancement tools to beautify the characters in pictures or videos.

[0003] Especially in the field of mobile phone photography and live broadcasting, the demand for beauty enhancement is becoming more and more prominent. Common beauty enhancements include face slimming and skin smoothing. Taking skin smoothing as an example, it is generally achieved by identifying the skin color, blurring the skin area, and then superimposing the skin area back into the original image.

[0004] However, skin color recognition in color space is very sensitive to light, which may result in the inability to fully recognize the skin area at night, and may cause the skin area to be expanded in strong light outdoors, which in turn leads to inaccurate determination of the blurred area, causing uneven skin color or blurred facial contours in photos or videos. Summary of the invention

[0005] The present invention provides an image processing and live broadcasting method, device, equipment and storage medium to solve the problem of uneven facial skin color caused by skin grinding operation due to inability to fully identify skin areas and the problem of blurred facial contour caused by skin grinding operation due to enlarging the range of skin areas.

[0006] In a first aspect, an embodiment of the present invention provides an image processing method, including:

[0007] Determine original image data, wherein the original image data has a plurality of pixels;

[0008] For each pixel, adjusting the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value;

[0009] Determining a skin color area in the original image data;

[0010] generating a blur intensity according to the target brightness value and the skin color area;

[0011] Based on the original image data, the skin color area is blurred according to the blur intensity and the target brightness value to obtain target image data.

[0012] On this basis, for each pixel point, adjusting the original brightness value of the pixel point on the color component based on the original brightness values ​​of other pixels surrounding the pixel point on the color component as the target brightness value includes:

[0013] Determine a first target point and at least two second target points from the pixel points, wherein the second target points are adjacent to texture coordinates of the first target point, and the texture coordinates have a first direction and a second direction;

[0014] Calculate an average of original brightness values ​​of the color component of the second target point in the first direction as a target brightness value of the color component of the first target point in the first direction;

[0015] An average value of original brightness values ​​of the color component of the second target point in the second direction is calculated as the target brightness value of the color component of the first target point in the second direction.

[0016] On this basis, determining the skin color area in the original image data includes:

[0017] Convert the color components of the pixel points from RGB color components to YCBCR color components;

[0018] The pixel points are input into a preset skin color discrimination model to output pixel points belonging to the skin color area.

[0019] On this basis, generating blur intensity according to the target brightness value and the skin color area includes:

[0020] Determine the pixel points representing the contour according to the target brightness value as the contour points;

[0021] Determine a first weight for the pixel point based on the contour point;

[0022] Determine the degree of dispersion of the pixel points in the skin color area on the color components;

[0023] Determining a second weight for the pixel point based on the discrete degree;

[0024] The first weight and the second weight are combined to generate a blur strength.

[0025] On this basis, the pixel points representing the contour are determined according to the target brightness value as contour points, including:

[0026] Calculate the difference between the original brightness value and the target brightness value of the pixel in the original image data;

[0027] When the difference does not belong to the preset edge range interval, the pixel point corresponding to the difference is determined as a contour point.

[0028] On this basis, determining a first weight for the pixel point based on the contour point includes:

[0029] When the pixel point is a contour point, determining a first weight of the pixel point to be a first value;

[0030] When the pixel point is not a contour point, the first weight of the pixel point is determined to be a second value, and the first value is smaller than the second value.

[0031] On this basis, determining the discrete degree of the pixel points in the skin color area on the color component includes:

[0032] Calculating an average value of color components of pixels in the skin color area;

[0033] Determine the maximum and minimum values ​​of the color components of the pixels in the skin color area;

[0034] For a certain pixel point, determine the quotient of the third value and the fourth value of the pixel point to represent the degree of discreteness, the third value is the difference between the color component of the pixel point and the intermediate value, and the fourth value is the difference between the maximum value and the minimum value.

[0035] On this basis, determining a second weight for the pixel point based on the discrete degree includes:

[0036] The discrete degree is brought into a preset monotonically decreasing exponential function to calculate the second weight.

[0037] On this basis, the combining the first weight and the second weight to generate the blur strength includes:

[0038] determining a product between the first weight and the second weight;

[0039] The product is set as the blur strength.

[0040] On this basis, the skin color area is blurred according to the blur intensity and the target brightness value in the original image data to obtain the target image data, including:

[0041] calculating a third weight based on the blur strength;

[0042] Calculate the product of the original brightness value of the color component and the third weight as a first blur result;

[0043] Calculating the product of the target brightness value of the color component and the blur intensity as a second blur result;

[0044] The first blur result and the second blur result are superimposed to obtain a color component of a target pixel constituting target image data.

[0045] In a second aspect, an embodiment of the present invention further provides a live broadcast method, comprising:

[0046] Collecting original video data, wherein the original video data includes original image data, and the original image data has a plurality of pixels;

[0047] For each pixel, adjusting the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value;

[0048] Determining a skin color area in the original image data;

[0049] generating a blur intensity according to the target brightness value and the skin color area;

[0050] Based on the original image data, blurring the skin color area according to the blur intensity and the target brightness value to obtain target image data;

[0051] The live video data including the target image data is released in the live broadcast room.

[0052] In a third aspect, an embodiment of the present invention further provides an image processing device, including:

[0053] An original image data determination module, used to determine original image data, wherein the original image data has a plurality of pixel points;

[0054] a first target brightness value determination module, configured to adjust, for each pixel, the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component, to take the original brightness value as the target brightness value;

[0055] A first skin color region determining module, used for determining the skin color region in the original image data;

[0056] A blur intensity first determination module, configured to generate blur intensity according to the target brightness value and the skin color area;

[0057] The first target image data obtaining module is used to perform blur processing on the skin color area based on the original image data according to the blur intensity and the target brightness value to obtain the target image data.

[0058] In a fourth aspect, an embodiment of the present invention further provides a live broadcast device, including:

[0059] An original video data acquisition module is used to acquire original video data, wherein the original video data includes original image data, and the original image data has a plurality of pixel points;

[0060] a second target brightness value determination module, configured to adjust, for each pixel, the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component, as the target brightness value;

[0061] A second skin color area determination module, used to determine the skin color area in the original image data;

[0062] A blur intensity second determination module, configured to generate blur intensity according to the target brightness value and the skin color area;

[0063] A second target image data acquisition module is used to perform blur processing on the skin color area according to the blur intensity and the target brightness value on the basis of the original image data to obtain target image data;

[0064] The live video data publishing module is used to publish the live video data containing the target image data in the live broadcast room.

[0065] In a fifth aspect, an embodiment of the present invention further provides an electronic device, including:

[0066] one or more processors;

[0067] A memory for storing one or more programs;

[0068] When the one or more programs are executed by the one or more processors, the one or more processors implement an image processing method as described in the first aspect, or a live broadcast method as described in the second aspect.

[0069] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an image processing method as described in the first aspect, or a live broadcast method as described in the second aspect.

[0070] The embodiment of the present invention determines the original image data, which has a plurality of pixels; for each pixel, adjusts the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value; determines the skin color area in the original image data; generates a blur intensity according to the target brightness value and the skin color area; based on the original image data, blurs the skin color area according to the blur intensity and the target brightness value to obtain the target image data. At this time, the skin color area has been accurately determined in the target image data, and the highest blur intensity is configured for the skin color area, and the skin color area is blurred according to the intensity, so as to achieve the effect of processing the skin color area in a targeted manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A flowchart of an image processing method provided in Embodiment 1 of the present invention;

[0072] Figure 2A A flowchart of an image processing method provided in Embodiment 2 of the present invention;

[0073] Figure 2B A schematic diagram of an image processing method provided in Embodiment 2 of the present invention;

[0074] Figure 2C A schematic diagram of a possible second target point provided in Embodiment 2 of the present invention;

[0075] Figure 2D A schematic diagram of pixel processing provided in the second embodiment of the present invention

[0076] Figure 3 A flowchart of a live broadcast method provided in Embodiment 3 of the present invention;

[0077] Figure 4 An image processing device provided for implementation of the fourth embodiment of the present invention;

[0078] Figure 5 A live broadcast device provided for implementation five of the present invention;

[0079] Figure 6 A schematic diagram of the structure of an electronic device provided in Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0080] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0081] Embodiment 1

[0082] Figure 1 This is a flowchart of an image processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the scenario of blurring the original image data when the skin color area is determined. The method can be executed by an image processing device, which can be implemented by software and / or hardware. The device can be configured in an electronic device, such as a mobile phone, a tablet computer, a wearable device (such as smart glasses, smart watches), etc. The electronic device is configured with a screen, a central processing unit (CPU) and a graphics processing unit (GPU). Reference Figure 1 , the method specifically comprises:

[0083] S101: Determine original image data.

[0084] The original image data refers to a collection of values ​​of each pixel represented by a numerical value. Determining the original image data means processing the received image data and modifying it into a unified format for subsequent processing. Generally, the original image data includes portraits. That is, the original image data can be divided into a human body part and a background part. The original image data that needs to be blurred and adjusted in brightness to achieve a "skin smoothing" effect generally includes facial images.

[0085] The original image data may be expressed in a variety of color models, such as RGB color mode, CMYK color mode, YUV color mode and Lab color mode. Since different color modes can be converted to each other, the color mode used by the original image data is not limited.

[0086] S102: For each pixel, adjust the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component, as a target brightness value.

[0087] Optionally, the color mode in the original image data is determined to be the RGB color mode. If the original image data is in another color model, it is converted into the RGB color mode.

[0088] Generally, the original image data is used to create a first texture through a rendering engine. The first texture includes four parameters, which can be used to store the red color component, the green color component and the blue color component of the RGB color component respectively, and the remaining parameter can be used to store the transparency component. At this time, the stored red color component, green color component and blue color component can be understood as the original brightness value of the pixel.

[0089] Texture coordinates have an X direction and a Y direction. For a pixel, determine other pixels around the pixel. Adjust the original brightness value of the pixel according to the original brightness values ​​of other pixels, and use the adjusted brightness value as the target brightness value of the pixel.

[0090] S103: Determine a skin color area in the original image data.

[0091] In a feasible implementation, the color components of each pixel in the original image data are first used as a basis, and the image to be processed is identified through a target recognition model to obtain the skin color area in the original image data.

[0092] Optionally, the color mode in the original image data is determined to be a YCBCR color mode, and the color components are the Y color component, the CB color component, and the CR color component. If the original image data is another color model, it is converted into the YCBCR color mode.

[0093] S104: Generate blur intensity according to the target brightness value and the skin color area.

[0094] The blur intensity can be understood as the degree to which an area is blurred. For each pixel, it can be understood as the degree to which the brightness of a pixel is increased. In the embodiment, the blur intensity is affected by two conditions, one is the target brightness, and the other is the skin color area.

[0095] In this embodiment, the target brightness can be used to determine the contour points in the original image data. The contour points are equipped with a lower blur intensity, and the skin area can be blurred and the brightness can be increased while protecting the contour, which can improve the three-dimensional effect of the object in the original image data. Taking the original image data including a face as an example, the pixel points between the face and the background can be determined as contour points, and the contour points are blurred with a lower intensity, thereby ensuring that the edge of the face will not be confused with the background during the blurring process, and ensuring the three-dimensional effect of the face.

[0096] Skin color area is an important object for blur processing. Applying a higher blur intensity to the skin color area can achieve a better beautification effect.

[0097] At the same time, the blur intensity of the pixel is determined according to the target brightness value and the skin color area, which can well distinguish the background area, contour points and skin color area in the original image data.

[0098] S105. Based on the original image data, blur the skin color area according to the blur intensity and the target brightness value to obtain target image data.

[0099] In a feasible implementation, the pixel points in the original image data are processed. In order to distinguish it from the following text, in this embodiment, the pixel points in the original image are called first pixel points, and the processed pixel points constituting the target image data are called second pixel points.

[0100] A first pixel includes an original brightness value, which is stored in the texture coordinates; the original brightness of the first pixel is adjusted to obtain the target brightness value of the first pixel. The first pixel is composed of color components, which include red color components, green color components, and blue color components. The three color components need to be adjusted separately for blurring. Take the red color component as an example for description:

[0101] The original brightness value of the red color component of the first pixel is adjusted up or down according to the blur intensity, and the original brightness value of the red color component of the first pixel after processing is saved as the first intermediate data. The target brightness value of the red color component of the first pixel is adjusted up or down according to the blur intensity, and the target brightness value of the red color component of the first pixel after processing is saved as the second intermediate data. The first intermediate data and the second intermediate data are superimposed as the brightness value of the red color component of the second pixel.

[0102] According to the above method, the brightness value of the green color component of the second pixel and the brightness value of the blue color component of the second pixel are obtained.

[0103] All second pixel points are combined to form target image data.

[0104] The embodiment of the present invention determines the original image data, which has a plurality of pixels; for each pixel, adjusts the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value; determines the skin color area in the original image data; generates a blur intensity according to the target brightness value and the skin color area; based on the original image data, blurs the skin color area according to the blur intensity and the target brightness value to obtain the target image data. At this time, the skin color area has been accurately determined in the target image data, and the highest blur intensity is configured for the skin color area, and the skin color area is blurred according to the intensity, so as to achieve the effect of processing the skin color area in a targeted manner.

[0105] Embodiment 2

[0106] Figure 2A The present invention provides a flowchart of an image processing method according to a second embodiment of the present invention. Figure 2BThis is a schematic diagram of an image processing method provided by Embodiment 2 of the present invention. This embodiment is a refinement of Embodiment 1, and describes in detail the specific process of adjusting the original brightness value of the pixel point on the color component based on the original brightness values ​​of other pixels surrounding the pixel point on the color component as the target brightness value for each pixel point. Figure 2A , the method comprising:

[0107] S201: Determine original image data.

[0108] The method of acquiring image data may be to capture the live broadcast screen uploaded by the anchor user through a camera device, or to receive the image input by the user. Generally, the original image data includes a person. That is, the original image data can be divided into a human body part and a background part.

[0109] Generally, the received original image data is in RGB color mode. Each pixel in the original image data has texture coordinates, and the texture coordinates include four parameters, which are respectively used to store the red color component, green color component, blue color component and transparent component in the RGB color mode. Among them, the red color component represents the value of the red channel of the pixel, the green color component represents the value of the green channel of the pixel, the blue color component represents the value of the blue channel of the pixel, and the transparent component represents the value of the transparency of the pixel.

[0110] S202: Determine a first target point and at least two second target points from the pixel points.

[0111] A first target point is determined in the original image data, and the first target point is a pixel point whose brightness needs to be adjusted currently. The first target point is determined by sequential traversal, generally from top to bottom, from left to right, or in the reverse direction.

[0112] The second target point is a pixel point associated with the first target point, and the association means that the second target point is adjacent to the first target point in the texture coordinates. When the number of second target points is an odd number, it can be preset to take one more pixel point in a certain direction. When the number of second target points is an even number, the second target points can be determined symmetrically on both sides with the first target point as the center.

[0113] Figure 2C A schematic diagram of a possible second target point provided in the second embodiment of the present invention. Figure 2C , when the first target point 10 is determined, if the number of the second target points is determined to be 4. A circular area 11 is determined with the first target point 10 as the center and two pixels as the radius, and all pixels in the circular area 11 can be considered as pixels associated with the first target point, that is, the second target point.

[0114] The usual convention is to use U and V as the axes in the texture space, with U corresponding to the X axis of the 2D coordinate system and V corresponding to the Y axis. In OpenGL, the values ​​on the UV axis are processed as follows: increasing from left to right on the U axis and increasing from bottom to top on the V axis (with the origin in the lower left corner).

[0115] Generally, the first direction may be determined along the X-axis direction, along the Y-axis direction, or along the angle bisector of the X-axis and the Y-axis direction.

[0116] In a feasible implementation, the direction along the X-axis is taken as the first direction. Figure 2D A schematic diagram of pixel processing provided in Embodiment 2 of the present invention. Pixel point (1,1) can be determined as the first target point. After adjusting the brightness value of pixel point (1,1), pixel point (1,2) can be determined as the first target point. After adjusting the brightness values ​​of the pixels in the first row, the first target point is determined in the second row. When determining the first target point in the second row, pixel point (2,1) can be determined as the first target point from left to right, or pixel point (2,7) can be determined as the first target point in a serpentine manner.

[0117] S203: Calculate an average value of original brightness values ​​of the color component of the second target point in the first direction as a target brightness value of the color component of the first target point in the first direction.

[0118] A first target range is determined in the original image data with the coordinates of the first target point in the first direction as the center. Figure 2D , the pixel point (4, 4) is taken as the first target point, and the first direction is described along the X-axis direction. The first target range is the range area of ​​the second target point. Generally, the first target range can be determined as 2 pixel points, that is, the first target range 12 is determined with the coordinates of the first target point in the first direction as the center and 2 pixel points as the radius.

[0119] If the coordinates of the pixel point in the first direction are within the first target range, the pixel point is set as the second target point. The pixel points located in the first target range 12 of the pixel point (4, 4) in the first direction as the first target point are determined, that is, the pixel point (4, 2), the pixel point (4, 3), the pixel point (4, 5) and the pixel point (4, 6) are used as the second target point.

[0120] A first weight of the second target point relative to the first target point is determined.

[0121] The first weight is used to reflect the influence of the brightness of the second target point on the brightness of the first target point. The same weight can be set for the second target point, or the same weight can be set for the second target point. The brightness of the first target point can be added to the weight setting, or it can be excluded. This embodiment does not limit this.

[0122] In a feasible implementation, the weight of the second target point can be determined according to the distance between the second target point and the first target point. The closer the distance between the second target point and the first target point is, the closer the relationship between the second target point and the first target point is, and the greater the impact on the first target point should be. Determine the distance between the second target point and the first target point; determine the weight of the second target point, and the weight is positively correlated with the distance. For ease of calculation, the sum of all weights is set to 1. Reference Figure 2D , the weight of the pixel (4, 4) as the first target point can be set to 0.4, the weight of the pixel (4, 2) in the second target point can be set to 0.1, the weight of the pixel (4, 3) can be set to 0.2, the weight of the pixel (4, 5) can be set to 0.2, and the weight of the pixel (4, 6) can be set to 0.1.

[0123] Specifically, this step includes the following methods:

[0124] The direction of the X axis is determined as the first direction. After the second target point is determined, the sum of the brightness component Y of a color component of the second target point is determined, and the sum is divided by the number of the second target points to obtain the average value of the brightness component Y of a color component of the second target point in the first direction, and the average value is used as the first data.

[0125] After determining the second target point, determine the weight of the second target point, and determine the brightness component Y of a color component of the second target point. The product of the brightness component Y of a color component of the second target point and the weight is used as the intermediate weighted brightness of the second target point. Determine the sum of multiple weighted brightnesses, divide the sum by the number of second target points, so as to obtain the average value of the weighted sum of the brightness component Y of a color component of the second target point in the first direction, and use the average value as the second data.

[0126] Optionally, the second target point is configured with different weights according to different distances from the first target point.

[0127] After determining the second target points, determine the square value of the brightness component Y of a color component of each second target point, divide the square value by the number of second target points to obtain the average of the squares of the brightness component Y of the second target points in the first direction, and use the average as the third data.

[0128] The original brightness statistical data includes the mean of the brightness component Y of the second target point in the first direction, i.e. the first data, the mean of the weighted sum of the brightness component Y of the second target point in the first direction, i.e. the second data, and the mean of the square of the brightness component Y of the second target point in the first direction, i.e. the third data.

[0129] S204: Calculate an average value of the original brightness values ​​of the color component of the second target point in the second direction as the target brightness value of the color component of the first target point in the second direction.

[0130] Since the previous step only calculates the average brightness value of the color component of the second target point in the first direction, the relationship between the brightness components Y of the color components in the second direction is not considered. By adjusting the original brightness statistical data through the brightness component Y of the color component of the second target point in the second direction, more accurate statistical data can be obtained, providing high-quality materials for accurate detection of contour points in the later stage.

[0131] Specifically, this step includes the following methods:

[0132] The direction of the Y axis is determined as the second direction. After the second target point is determined, the sum of the brightness components Y of the second target points is determined, and the sum is divided by the number of the second target points to obtain the average of the brightness components Y of the second target points in the second direction, and the average is used as the fourth average.

[0133] Optionally, the average of the fourth mean and the first mean is used as the fourth data.

[0134] After determining the second target point, determine the second weight of the second target point, and determine the brightness component Y of the second target point. The product of the brightness component Y of the second target point and the second weight is used as the intermediate weighted brightness of the second target point. Determine the sum of multiple weighted brightnesses, divide the sum by the number of second target points, so as to obtain the average of the weighted sum of the brightness component Y of the second target point in the second direction, and use the average as the fifth average.

[0135] Optionally, the second target point is configured with different second weights according to different distances from the first target point.

[0136] Optionally, the average of the fifth mean and the second mean is used as the fifth data.

[0137] After determining the second target point, determine the square value of the brightness component Y of each second target point, divide the square value by the number of second target points to obtain the mean of the squares of the brightness component Y of the second target point in the second direction, and use the mean as the sixth mean.

[0138] Optionally, the average of the sixth mean and the three means is used as the sixth data.

[0139] The target brightness statistical data includes the mean of the brightness component Y of the second target point in the first direction and the second direction, that is, the fourth data; the mean of the weighted sum of the brightness component Y of the second target point in the first direction and the second direction, that is, the fifth data, and the mean of the squares of the brightness component Y of the second target point in the first direction and the second direction.

[0140] Finally, the target brightness values ​​of the color component of the first target point in the first direction and the second direction are averaged to serve as the final target brightness value of the first target point.

[0141] S205: Convert the color component of the pixel point from RGB color component to YCBCR color component.

[0142] Among them, the RCB component represents the color space, R, G, and B are the component values ​​of RGB, R represents the red component, G represents the green component, and B represents the blue component; the YCBCR color space component represents the color space, Y, CB, and CR are the specific component values ​​of YCBCR, specifically, Y refers to the brightness component, CB refers to the blue chromaticity component, and CR refers to the red chromaticity component. In this embodiment, the specific expression for converting the YUV component obtained for each pixel of the image to be identified into the RGB component is:

[0143] C RGB =C YUV 2C RGB ×C YUV (1)

[0144] In the formula, C RGB is the matrix of RGB components, C YUV is the matrix of YUV components, C YUV 2C RGB is a conversion matrix of RGB components and YUV components; the color gamut conversion of the RGB components to obtain the YCBCR color gamut space components of each pixel point includes:

[0145] C YCBCR =C RGB 2C YCBCR ×C RGB (2)

[0146] In the formula, C YCBCR is the matrix of YCBCR color space components, C RGB is the matrix of RCB components obtained by the previous formula, C RGB 2C YCBCR It is the conversion matrix between YCBCR color gamut space components and RGB components.

[0147] S206: Input the pixel points into a preset skin color discrimination model to output pixel points belonging to a skin color area.

[0148] In this embodiment, the color gamut range value of the target recognition model is obtained by statistically analyzing the data about the color gamut value under different targets and / or different scenes. The YCBCR color gamut space component of each pixel point of the acquired image to be processed is obtained by color gamut conversion. After comparative analysis, when the YCBCR color gamut space component meets the color gamut range value, the pixel point corresponding to the image to be processed is the pixel point of the target area. The expression of the recognition function I(x) about the target area obtained by the comparative analysis process is:

[0149]

[0150] Among them, x is the three spatial components of Y, CB, and CR in the YCBCR color gamut space components corresponding to the target pixel point in the target area; Y′, CB′, CR′ are the color gamut range values ​​of the target recognition model obtained through statistical data on color gamut values ​​under different targets and / or different scenes.

[0151] From expression (3), it can be known that the three spatial components of Y, CB, and CR in the YCBCR color gamut space components for each pixel point in the target area in the image to be processed must simultaneously satisfy the three spatial components of Y′, CB′, and CR′ in the color gamut range value of the target recognition model obtained through statistical data on color gamut values ​​under different targets and / or different scenes. Otherwise, it is not considered as a target pixel point in the target area, that is, the corresponding pixel point is not in the target area. The method compares the color gamut space components obtained from the pixel points of the image to be processed with the color gamut range value of the target recognition model. The amount of comparison statistics involved only increases linearly with the size of the image. There is no need to build a color gamut database for the target area, so it will not cause a large increase in the amount of cached data. Therefore, the image processing method can be applied to mobile terminals such as mobile phones.

[0152] In the above-mentioned related image recognition methods involving different usage scenarios or for different target areas, the present invention provides a skin color processing method in live broadcast, taking this as an example, the above-mentioned image recognition processing method is described in more detail. In the skin color processing method in live broadcast, the image to be processed is generated in the live video stream, specifically the image of each frame generated in the live video stream, and the image information of each pixel thereof is relative to the image information of each pixel in each frame of the image. Correspondingly, the target recognition model is a live skin color model, and the target area is a skin color area. The skin color model and skin color area are mainly aimed at the skin color of the anchor in the live video broadcast, and the range that can be included can involve the anchor's face, neck and other skin color areas where the skin color is revealed. The processing method improves the existing technology that only simply distinguishes the skin color area and the background, reduces the separation of different processing areas, and enables the image processing to have both key processing parts in the skin color area and a smooth transition with the background boundary, and finally provides an image processing effect.

[0153] S207 . Determine pixel points representing the contour according to the target brightness value as contour points.

[0154] The target brightness statistical data includes fourth data representing the mean of the brightness component Y, fifth data representing the mean of the weighted sum of the brightness component Y, and sixth data representing the mean of the square sum of the brightness component Y. Specifically, the following operations are included:

[0155] The fourth data is the average value of the brightness component Y of the second target point in the first direction and the second direction, denoted as Y m . The product Y of the fourth data and the fourth data m 2 As the seventh data.

[0156] The difference between the sixth data and the seventh data is

[0157] Specifically, the brightness components Y of several pixel points are 1, 1, 1, 5, 5, 5 respectively.

[0158] Among them, the fourth data Y of the pixel point m is 1, 1, 3, 3, 5, 5;

[0159] The sixth data of the pixel It is 1, 1, 13, 13, 25, 25.

[0160] The seventh data of the pixel It is 1, 1, 9, 9, 25, 25.

[0161] The eighth data is the difference between the sixth data and the seventh data of the pixel point. It is 0, 0, 4, 4, 0, 0. When the eighth data has a value, it indicates that the pixel point is a contour point.

[0162] Normalization is to limit the data to be processed (through a certain algorithm) within a certain range. First, normalization is for the convenience of subsequent data processing, and second, it ensures that the program converges faster when running. The specific role of normalization is to summarize the statistical distribution of unified samples. Normalization between 0-1 is a statistical probability distribution, and normalization in a certain interval is a statistical coordinate distribution.

[0163] Determine all the numerical values ​​in the eighth data, change the sum of the numerical values ​​to 1, and determine the processed value of each numerical value in the eighth data.

[0164] This step can be understood as the following two steps:

[0165] S2071. Calculate the difference between the original brightness value and the target brightness value of the pixel in the original image data.

[0166] The original brightness value of the pixel in the original image data can be obtained by extraction, which is recorded as RGB. The target brightness value of the pixel in the original image data has been obtained by calculation, which is recorded as R m G m B m The difference is recorded as d. The difference d is calculated as follows:

[0167] R d G d B d =RGB-R m G m B m (4);

[0168] d=(R d +G d +B d )÷3 (5).

[0169] Further processing of d to obtain D, Where E is the preset edge range interval. Of course, D can also be averaged in the first direction and the second direction to obtain the mean value D m .

[0170] S2072: When the difference does not belong to a preset edge range, determine the pixel point corresponding to the difference as a contour point.

[0171] The edge range is a pre-entered range. When a value falls within the edge range, it can be considered that the difference between the pixels associated with the value is relatively small, which may be relatively close pixels rather than the edge between objects. When a value exceeds the edge range, it can be considered that the difference between the pixels associated with the value is relatively large, which may be the pixels representing the edge between objects.

[0172] Therefore, the difference between the original brightness value and the target brightness value of the pixel in the original image data is calculated to determine whether the difference falls within the edge range. When the difference does not fall within the preset edge range, the pixel corresponding to the difference is determined as a contour point.

[0173] S208: Determine a first weight for the pixel point based on the contour point.

[0174] The first weight is configured according to whether a pixel represents the contour and edge of an object. The more likely a pixel is a contour point, the greater the first weight of the pixel, and the first weight is recorded as DIF.

[0175] Specifically, this step includes the following specific implementation methods:

[0176] S2081. When the pixel point is a contour point, determine that a first weight of the pixel point is a first value.

[0177] When a pixel point is determined to be a contour point, the first weight of the pixel point is assigned to 0 (or a number close to 0), and this 0 (or a number close to 0) is the first value.

[0178] S2082: When the pixel point is not a contour point, determine that the first weight of the pixel point is a second value, and the first value is smaller than the second value.

[0179] When a pixel point is determined to be not a contour point, a second value of the pixel point is determined as a first weight according to the difference d. Generally, the second value is a number between 0 and 1, and the second value is greater than the first value.

[0180] S209: Determine the degree of discreteness of the color components of the pixels in the skin color area.

[0181] The degree of discreteness refers to the degree of difference between a pixel and other pixels in the skin color area. The formula for expressing the degree of discreteness is: Wherein, I(x) is the color gamut recognition function of skin color, which is used to represent the YCBCR color gamut space component of the corresponding pixel point in the skin color area; M is the average value calculated for the color component of the pixel point in the skin color area; Max is the maximum value of the color component of the pixel point in the skin color area, and Min is the minimum value. I(x)-M is used as the third value, Max-Min is used as the fourth value, and for a certain pixel point, the quotient of the third value and the fourth value of the pixel point is determined to represent the degree of discreteness.

[0182] S210: Determine a second weight for the pixel point based on the discrete degree.

[0183] The specific formula for calculating the second weight is: Wherein, F(x) is a monotonically decreasing exponential function, and W(x) represents the second weight.

[0184] When I(x) is equal to the middle value M, that is, At this time, the corresponding pixel point Max-Min and the middle position of the skin color area, if the skin color area is a face area, the middle position is exactly the position of the facial features, that is, the position that needs more emphasis on processing, the higher the corresponding processing weight.

[0185] According to the curve change characteristics of the exponential function, as the distance between the pixel point corresponding to the skin color area and the position of the middle value changes, the obtained weight value gradually changes in a gentle manner. That is, in the skin color area, the processing weight of each pixel point according to the image processing algorithm also changes in a gentle manner. Taking the facial skin color area as an example, the image processing weight of the pixel point where the facial features are located is the highest. As it changes along the middle value area to the edge area, the corresponding image processing weight gradually decreases until it reaches the boundary position between the skin color area and the background, so as to achieve a smooth transition of the processing effect of the skin color area, and the image display effect of the skin color area and the background is naturally connected, avoiding image faults and making the display effect natural.

[0186] The means for determining the color gamut range value of the live skin color model can be to divide the data modules corresponding to the color gamut range values ​​of different live objects and / or scenes from the skin color database in the historical live video stream through comparison, testing, training, etc., and automatically switch different data modules according to different application situations, such as different IP addresses and preliminary judgment of the live scene based on the acquired image, or the user can switch different data modules according to actual conditions. Or, according to the needs of special live effects, the color gamut range values ​​of the live skin color models for different live themes are set. Or, according to the personalized needs of the user, the color gamut range value of the live skin color model set by the user is obtained, etc. to confirm the color gamut range value of the live skin color model.

[0187] S211. Generate blur strength by combining the first weight and the second weight.

[0188] The blur strength is the basis for the degree of blur processing for a pixel point. The product of the first weight and the second weight is set as the blur strength, which is recorded as Strength.

[0189] S212: Based on the original image data, blur the skin color area according to the blur intensity and the target brightness value to obtain target image data.

[0190] The color components include a red color component, a green color component, and a blue color component. Taking the red color component as an example, firstly, the third weight is calculated based on the blur strength, and the third weight is the difference between 1 and the blur strength, which is recorded as (1-Strength). The original brightness value of the red color component is calculated, which is recorded as R, and the product between R and the third weight is taken as the first blur result, which is recorded as (1-Strength)*R. The target brightness value of the red color component is calculated, which is recorded as R m , R m The product of the fuzzy strength is the second fuzzy result, denoted as Strength*R m The first fuzzy result and the second fuzzy result are superimposed to obtain the red color component of the target pixel constituting the target image data, which is recorded as (1-Strength)*R+Strength*R m The same operation is performed on the green color component and the blue color component to obtain the target image data.

[0191] The greater the weight of the skin color area, the stronger the beauty algorithm processing effect of the corresponding area. Take the face area as an example. From the above, it can be seen that the position of the middle value of the face is located at the position of the facial features. According to the expression (1-Strength)*R+Strength*R m The image processing weight of the pixels in the corresponding area is the highest, and its weight is 1, that is, the corresponding area needs to be processed by the highest degree of beautification algorithm to ensure the beautification effect of the host. As it changes from the middle value area to the edge area, the corresponding beautification algorithm processing weight gradually decreases until it reaches the boundary between the skin color area and the background, so as to achieve a smooth transition of the processing effect of the skin color area, and the image display effect of the skin color area and the background is naturally connected, avoiding the fault between the host's face area and the background, making the display effect natural.

[0192] Embodiment 3

[0193] Figure 3This is a flowchart of a live broadcast method provided in Embodiment 3 of the present invention. This embodiment is applicable to the scenario of processing video data uploaded by the anchor client to the live broadcast room. The method can be executed by a live broadcast device, especially a mobile phone, computer, etc. used by the anchor client. The device can be implemented by software and / or hardware. Figure 3 , the method specifically comprises:

[0194] S301, collecting original video data.

[0195] The original video data includes original image data, and the original image data has a plurality of pixels. It should be understood that since this embodiment is applicable to the scene where the anchor performs live broadcast, the audio data should be collected at the same time as the original live video data is collected.

[0196] In a feasible implementation, when the anchor user starts the live broadcast function of the live broadcast room, the live broadcast device collects the live broadcast screen of the anchor user through a camera device to collect original live broadcast video data and obtain original image data therefrom.

[0197] S302: For each pixel point, adjust the original brightness value of the pixel point on the color component based on the original brightness values ​​of other pixels surrounding the pixel point on the color component, and use the original brightness value as the target brightness value.

[0198] On this basis, a first target point and at least two second target points are determined from the pixel points, wherein the second target points are adjacent to texture coordinates of the first target point, and the texture coordinates have a first direction and a second direction;

[0199] Calculate the average value of the brightness of the color component of the second target point in the first direction as the brightness value of the color component of the first target point in the first direction;

[0200] An average value of the brightness values ​​of the color component of the second target point in the second direction is calculated as the brightness value of the color component of the first target point in the second direction.

[0201] S303: Determine the skin color area in the original image data.

[0202] On this basis, the color components of the pixel points are converted from RGB color components to YCBCR color components;

[0203] The pixel points are input into a preset skin color discrimination model to output the pixel points belonging to the skin color area.

[0204] S304: Generate blur intensity according to the target brightness value and the skin color area.

[0205] On this basis, the pixel points representing the contour are determined according to the target brightness value as contour points;

[0206] Determine a first weight for the pixel point based on the contour point;

[0207] Determine the degree of dispersion of the color components of the pixels in the skin color area;

[0208] Determining a second weight for the pixel based on the degree of discreteness;

[0209] The blur strength is generated by combining the first weight and the second weight.

[0210] The pixel points representing the contour are determined according to the target brightness value as contour points, including:

[0211] Calculate the difference between the original brightness value and the target brightness value of the pixel in the original image data;

[0212] When the difference does not belong to the preset edge range interval, the pixel point corresponding to the difference is determined as a contour point.

[0213] Wherein, determining the first weight of the pixel point based on the contour point includes:

[0214] When the pixel point is a contour point, determining a first weight of the pixel point to be a first value;

[0215] When the pixel point is not a contour point, the first weight of the pixel point is determined to be a second value, and the first value is smaller than the second value.

[0216] Wherein, the average value is calculated for the color components of the pixels in the skin color area;

[0217] Determine the maximum and minimum values ​​of the color components of the pixels in the skin color area;

[0218] For a certain pixel, the quotient of the third value and the fourth value of the pixel is determined to represent the degree of discreteness, the third value is the difference between the color component of the pixel and the middle value, and the fourth value is the difference between the maximum value and the minimum value.

[0219] Wherein, determining the second weight for the pixel point based on the discrete degree includes:

[0220] The discrete degree is subjected to a preset monotonically decreasing exponential function to calculate the second weight.

[0221] The fuzzy strength is generated by combining the first weight and the second weight, including:

[0222] determining a product between the first weight and the second weight;

[0223] Set the product as blur strength.

[0224] S305 , based on the original image data, blurring the skin color area according to the blur intensity and the target brightness value to obtain target image data.

[0225] The color components include red color component, green color component and blue color component. Taking the red color component as an example, this step can be understood as:

[0226] Based on the original image data, the skin color area is blurred according to the blur intensity and the target brightness value to obtain the target image data, including:

[0227] calculating a third weight based on the blur strength;

[0228] Calculate the product between the original brightness value of the red color component and the third weight as the first blur result;

[0229] Calculate the product between the target brightness value of the red color component and the blur intensity as the second blur result;

[0230] The first fuzzy result and the second fuzzy result are superimposed to obtain a red color component of a target pixel constituting target image data.

[0231] For steps S302 - S304 , reference may be made to the description of the relevant steps in Embodiment 1 and Embodiment 2, and will not be repeated here.

[0232] S306: Publish live video data including the target image data in the live broadcast room.

[0233] The target image data is spliced ​​and rendered into target live video data, and the live video data is sent to the live server to be played in the live room created by the anchor client.

[0234] Among them, there are audience users logged in to the live broadcast room, and the live broadcast server distributes the live video data to the audience users logged in to the live broadcast room.

[0235] The live video data includes a data stream for local display and a data stream for transmission to the viewer user client. At the same time, the audio and video are packaged into a video file and uploaded to the live server in a streaming manner, which can be provided to the audience.

[0236] The embodiment of the present invention determines the original image data, which has a plurality of pixels; for each pixel, adjusts the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value; determines the skin color area in the original image data; generates a blur intensity according to the target brightness value and the skin color area; based on the original image data, blurs the skin color area according to the blur intensity and the target brightness value to obtain the target image data. At this time, the skin color area has been accurately determined in the target image data, and the highest blur intensity is configured for the skin color area, and the skin color area is blurred according to the intensity, so as to achieve the effect of processing the skin color area in a targeted manner.

[0237] Embodiment 4

[0238] Figure 4 An image processing device provided for implementation four of the present invention is characterized in that it includes: an original image data determination module 41, a target brightness value first determination module 42, a skin color area first determination module 43, a blur intensity first determination module 44 and a target image data first acquisition module 45.

[0239] The original image data determination module 41 is used to determine the original image data, wherein the original image data has a plurality of pixels;

[0240] a first target brightness value determination module 42, configured to adjust, for each pixel, the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component, as the target brightness value;

[0241] A first skin color region determining module 43, configured to determine a skin color region in the original image data;

[0242] A blur strength first determination module 44, configured to generate a blur strength according to the target brightness value and the skin color area;

[0243] The target image data first obtaining module 45 is used to perform blur processing on the skin color area according to the blur intensity and the target brightness value on the basis of the original image data to obtain the target image data.

[0244] The embodiment of the present invention determines the original image data, which has a plurality of pixels; for each pixel, adjusts the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value; determines the skin color area in the original image data; generates a blur intensity according to the target brightness value and the skin color area; based on the original image data, blurs the skin color area according to the blur intensity and the target brightness value to obtain the target image data. At this time, the skin color area has been accurately determined in the target image data, and the highest blur intensity is configured for the skin color area, and the skin color area is blurred according to the intensity, so as to achieve the effect of processing the skin color area in a targeted manner.

[0245] On this basis, the target brightness value first determination module 42 includes:

[0246] A target point determination submodule, used to determine a first target point and at least two second target points from the pixel points, wherein the second target points are adjacent to texture coordinates of the first target point, and the texture coordinates have a first direction and a second direction;

[0247] A first target brightness value determination submodule, used to calculate an average of original brightness values ​​of the color component of the second target point in the first direction as a target brightness value of the color component of the first target point in the first direction;

[0248] The second target brightness value determination submodule is used to calculate the average value of the original brightness values ​​of the color component of the second target point in the second direction as the target brightness value of the color component of the first target point in the second direction.

[0249] On this basis, the skin color area first determination module 43 includes:

[0250] A color component conversion submodule, used to convert the color component of the pixel point from RGB color component to YCBCR color component;

[0251] The skin color area determination submodule is used to input the pixel points into a preset skin color discrimination model to output the pixel points belonging to the skin color area.

[0252] On this basis, the blur strength first determination module 44 is used to:

[0253] A contour point determination submodule is used to determine the pixel points representing the contour according to the target brightness value as contour points;

[0254] A first weight determination submodule, configured to determine a first weight for the pixel point based on the contour point;

[0255] A discrete degree determination submodule, used to determine the discrete degree of the pixel points in the skin color area on the color component;

[0256] A second weight determination submodule, configured to determine a second weight for the pixel point based on the discrete degree;

[0257] The blur strength generating submodule is used to generate blur strength by combining the first weight and the second weight.

[0258] On this basis, the contour point determination submodule includes:

[0259] A difference determination unit, used to calculate the difference between the original brightness value and the target brightness value of the pixel point in the original image data;

[0260] The contour point determination unit is used to determine the pixel point corresponding to the difference as a contour point when the difference does not belong to a preset edge range interval.

[0261] On this basis, the first weight determination submodule includes:

[0262] A first value determining unit, configured to determine, when the pixel point is a contour point, a first weight of the pixel point as a first value;

[0263] The second value determination unit is used to determine, when the pixel point is not a contour point, that the first weight of the pixel point is a second value, and the first value is smaller than the second value.

[0264] On this basis, the discrete degree determination submodule includes:

[0265] an average value calculation unit, used for calculating an average value of color components of pixel points in the skin color area;

[0266] A maximum value determining unit, used to determine the maximum value and the minimum value of the color component of the pixel point in the skin color area;

[0267] The discrete degree determination unit is used to determine, for a certain pixel point, the quotient of a third value and a fourth value of the pixel point to represent the discrete degree, wherein the third value is the difference between the color component of the pixel point and the intermediate value, and the fourth value is the difference between the maximum value and the minimum value.

[0268] On this basis, the second weight determination submodule includes:

[0269] The second weight calculation unit is used to bring the discrete degree into a preset monotonically decreasing exponential function to calculate a second weight.

[0270] On this basis, the fuzzy intensity generation submodule includes:

[0271] a product determining unit, configured to determine a product between the first weight and the second weight;

[0272] The blur strength determination unit is used to set the product as the blur strength.

[0273] On this basis, the target image data first obtaining module 45 includes:

[0274] A third weight calculation unit, configured to calculate a third weight based on the blur strength;

[0275] A first fuzzy result calculation unit, used for calculating the product between the original brightness value of the color component and the third weight as a first fuzzy result;

[0276] A second fuzzy result calculation unit, used for calculating the product between the target brightness value of the color component and the fuzzy intensity as a second fuzzy result;

[0277] The superposition processing unit is used to superimpose the first blur result and the second blur result to obtain the color component of the target pixel point constituting the target image data.

[0278] An image processing device provided in this embodiment can be used to execute an image processing method provided in Embodiment 1 and Embodiment 2, and has corresponding functions and beneficial effects.

[0279] Embodiment 5

[0280] Figure 5 A live broadcast device provided for implementation five of the present invention is characterized in that it includes: an original video data acquisition module 51, a second target brightness value determination module 52, a skin color area second determination module 53, a blur intensity second determination module 54, a target image data second acquisition module 55 and a live video data publishing module 56.

[0281] The original video data acquisition module 51 is used to acquire original video data, wherein the original video data includes original image data, and the original image data has a plurality of pixels;

[0282] A second target brightness value determination module 52 is used to adjust, for each pixel, the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component, as the target brightness value;

[0283] A second skin color area determination module 53, used to determine the skin color area in the original image data;

[0284] A blur intensity second determination module 54, configured to generate blur intensity according to the target brightness value and the skin color area;

[0285] A second target image data obtaining module 55 is used for performing blur processing on the skin color area according to the blur intensity and the target brightness value based on the original image data to obtain target image data;

[0286] The live video data publishing module 56 is used to publish the live video data containing the target image data in the live broadcast room.

[0287] The embodiment of the present invention determines the original image data, which has a plurality of pixels; for each pixel, adjusts the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value; determines the skin color area in the original image data; generates a blur intensity according to the target brightness value and the skin color area; based on the original image data, blurs the skin color area according to the blur intensity and the target brightness value to obtain the target image data. At this time, the skin color area has been accurately determined in the target image data, and the highest blur intensity is configured for the skin color area, and the skin color area is blurred according to the intensity, so as to achieve the effect of processing the skin color area in a targeted manner.

[0288] A live broadcast device provided in this embodiment can be used to execute a live broadcast method provided in Example 3, and has corresponding functions and beneficial effects.

[0289] Embodiment 6

[0290] Figure 6 This is a schematic diagram of the structure of an electronic device provided by Embodiment 6 of the present invention. Figure 6 As shown, the electronic device includes a processor 60, a memory 61, a communication module 62, an input device 63 and an output device 64; the number of the processor 60 in the electronic device can be one or more. Figure 6 A processor 60 is taken as an example; the processor 60, the memory 61, the communication module 62, the input device 63 and the output device 64 in the electronic device can be connected by a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0291] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as the modules corresponding to an image processing method in this embodiment (for example, an original image data determination module 41, a target brightness value first determination module 42, a skin color area first determination module 43, a blur intensity first determination module 44 and a target image data first acquisition module 45 in an image processing device). Such as the modules corresponding to a live broadcast method in this embodiment (for example, an original video data acquisition module 51, a target brightness value second determination module 52, a skin color area second determination module 53, a blur intensity second determination module 54, a target image data second acquisition module 55 and a live video data publishing module 56 in a live broadcast device). The processor 60 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 61, that is, realizing the above-mentioned image processing method and live broadcast method.

[0292] The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include a memory remotely arranged relative to the processor 60, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0293] The communication module 62 is used to establish a connection with the display screen and implement data interaction with the display screen. The input device 63 can be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device.

[0294] An electronic device provided in this embodiment can execute an image processing method and a live broadcast method provided in any embodiment of the present invention, with specific corresponding functions and beneficial effects.

[0295] Embodiment 7

[0296] Embodiment 7 of the present invention further provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute an image processing method when executed by a computer processor, the method comprising:

[0297] Determine original image data, wherein the original image data has a plurality of pixels;

[0298] For each pixel, adjusting the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value;

[0299] Determining a skin color area in the original image data;

[0300] generating a blur intensity according to the target brightness value and the skin color area;

[0301] Based on the original image data, the skin color area is blurred according to the blur intensity and the target brightness value to obtain target image data.

[0302] or,

[0303] Used to perform a live broadcast method, the method comprising:

[0304] Collecting original video data, wherein the original video data includes original image data, and the original image data has a plurality of pixels;

[0305] For each pixel, adjusting the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value;

[0306] Determining a skin color area in the original image data;

[0307] generating a blur intensity according to the target brightness value and the skin color area;

[0308] Based on the original image data, blurring the skin color area according to the blur intensity and the target brightness value to obtain target image data;

[0309] The live video data including the target image data is released in the live broadcast room.

[0310] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided by an embodiment of the present invention are not limited to the method operations described above, and can also execute related operations in an image processing method and a live broadcast method provided by any embodiment of the present invention.

[0311] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer electronic device (which can be a personal computer, a server, or a network electronic device, etc.) to execute the methods described in each embodiment of the present invention.

[0312] It is worth noting that in the embodiments of the above-mentioned image processing device and the live broadcast device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0313] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An image processing method, characterized in that: include: Determine original image data, wherein the original image data has a plurality of pixels; For each pixel, adjusting the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value; Determining a skin color area in the original image data; generating a blur intensity according to the target brightness value and the skin color area; Based on the original image data, blurring the skin color area according to the blur intensity and the target brightness value to obtain target image data; The generating the blur intensity according to the target brightness value and the skin color area comprises: Determine the pixel points representing the contour according to the target brightness value as the contour points; Determine a first weight for the pixel point based on the contour point; Determine the degree of dispersion of the pixel points in the skin color area on the color components; Determining a second weight for the pixel point based on the discrete degree; combining the first weight and the second weight to generate a blur strength; The step of determining the pixel points representing the contour according to the target brightness value as contour points includes: Calculate the difference between the original brightness value and the target brightness value of the pixel in the original image data; When the difference does not belong to the preset edge range interval, the pixel point corresponding to the difference is determined as a contour point; Wherein, determining a first weight for the pixel point based on the contour point includes: When the pixel point is a contour point, determining a first weight of the pixel point to be a first value; When the pixel point is not a contour point, the first weight of the pixel point is determined to be a second value, and the first value is smaller than the second value.

2. The method according to claim 1, characterized in that The adjusting, for each pixel, the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value includes: Determine a first target point and at least two second target points from the pixel points, wherein the second target points are adjacent to texture coordinates of the first target point, and the texture coordinates have a first direction and a second direction; Calculate an average of original brightness values ​​of the color component of the second target point in the first direction as a target brightness value of the color component of the first target point in the first direction; An average value of original brightness values ​​of the color component of the second target point in the second direction is calculated as the target brightness value of the color component of the first target point in the second direction.

3. The method according to claim 1, characterized in that The determining of the skin color area in the original image data comprises: Convert the color components of the pixel points from RGB color components to YCBCR color components; The pixel points are input into a preset skin color discrimination model to output pixel points belonging to the skin color area.

4. The method according to claim 1, characterized in that: The determining the discrete degree of the pixel points in the skin color area on the color components includes: Calculating an average value of color components of pixels in the skin color area; Determine the maximum and minimum values ​​of the color components of the pixels in the skin color area; For a certain pixel point, determine the quotient of the third value and the fourth value of the pixel point to represent the degree of discreteness, the third value is the difference between the color component of the pixel point and the average value, and the fourth value is the difference between the maximum value and the minimum value.

5. The method according to claim 4, characterized in that The determining a second weight for the pixel point based on the discrete degree includes: The discrete degree is brought into a preset monotonically decreasing exponential function to calculate the second weight.

6. The method according to any one of claims 1 to 5, characterized in that: The combining the first weight and the second weight to generate the blur strength comprises: determining a product between the first weight and the second weight; The product is set as the blur strength.

7. The method according to any one of claims 1 to 5, characterized in that: The step of performing blur processing on the skin color area based on the original image data according to the blur intensity and the target brightness value to obtain the target image data comprises: calculating a third weight based on the blur strength; Calculate the product of the original brightness value of the color component and the third weight as a first blur result; Calculating the product of the target brightness value of the color component and the blur intensity as a second blur result; The first blur result and the second blur result are superimposed to obtain a color component of a target pixel constituting target image data.

8. A live broadcast method, characterized in that: include: Collecting original video data, wherein the original video data includes original image data, and the original image data has a plurality of pixels; For each pixel, adjusting the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component as the target brightness value; Determining a skin color area in the original image data; generating a blur intensity according to the target brightness value and the skin color area; Based on the original image data, blurring the skin color area according to the blur intensity and the target brightness value to obtain target image data; Publishing live video data including the target image data in a live broadcast room; The generating the blur intensity according to the target brightness value and the skin color area comprises: Determine the pixel points representing the contour according to the target brightness value as the contour points; Determine a first weight for the pixel point based on the contour point; Determine the degree of dispersion of the pixel points in the skin color area on the color components; Determining a second weight for the pixel point based on the discrete degree; combining the first weight and the second weight to generate a blur strength; The step of determining the pixel points representing the contour according to the target brightness value as contour points includes: Calculate the difference between the original brightness value and the target brightness value of the pixel in the original image data; When the difference does not belong to the preset edge range interval, the pixel point corresponding to the difference is determined as a contour point; Wherein, determining a first weight for the pixel point based on the contour point includes: When the pixel point is a contour point, determining a first weight of the pixel point to be a first value; When the pixel point is not a contour point, the first weight of the pixel point is determined to be a second value, and the first value is smaller than the second value.

9. An image processing device, characterized in that: include: An original image data determination module, used to determine original image data, wherein the original image data has a plurality of pixel points; a first target brightness value determination module, configured to adjust, for each pixel, the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component, to take the original brightness value as the target brightness value; A first skin color region determining module, used for determining the skin color region in the original image data; A blur intensity first determination module, configured to generate blur intensity according to the target brightness value and the skin color area; A first target image data obtaining module, configured to perform blur processing on the skin color area according to the blur intensity and the target brightness value based on the original image data to obtain target image data; The blur strength first determination module comprises: A contour point determination submodule is used to determine the pixel points representing the contour according to the target brightness value as contour points; A first weight determination submodule, configured to determine a first weight for the pixel point based on the contour point; A discrete degree determination submodule, used to determine the discrete degree of the pixel points in the skin color area on the color component; A second weight determination submodule, configured to determine a second weight for the pixel point based on the discrete degree; A fuzzy strength generating submodule, configured to generate a fuzzy strength by combining the first weight and the second weight; The contour point determination submodule comprises: A difference determination unit, used to calculate the difference between the original brightness value and the target brightness value of the pixel point in the original image data; A contour point determination unit, configured to determine a pixel point corresponding to the difference as a contour point when the difference does not belong to a preset edge range interval; The first weight determination submodule includes: A first value determining unit, configured to determine, when the pixel point is a contour point, a first weight of the pixel point as a first value; The second value determination unit is used to determine, when the pixel point is not a contour point, that the first weight of the pixel point is a second value, and the first value is smaller than the second value.

10. A live broadcast device, characterized in that: include: An original video data acquisition module is used to acquire original video data, wherein the original video data includes original image data, and the original image data has a plurality of pixel points; a second target brightness value determination module, configured to adjust, for each pixel, the original brightness value of the pixel on the color component based on the original brightness values ​​of other pixels surrounding the pixel on the color component, as the target brightness value; A second skin color area determination module, used to determine the skin color area in the original image data; A blur intensity second determination module, configured to generate blur intensity according to the target brightness value and the skin color area; A second target image data acquisition module is used to perform blur processing on the skin color area according to the blur intensity and the target brightness value on the basis of the original image data to obtain target image data; A live video data publishing module, used for publishing live video data containing the target image data in a live broadcast room; The blur strength second determination module comprises: A contour point determination submodule is used to determine the pixel points representing the contour according to the target brightness value as contour points; A first weight determination submodule, configured to determine a first weight for the pixel point based on the contour point; A discrete degree determination submodule, used to determine the discrete degree of the pixel points in the skin color area on the color component; A second weight determination submodule, configured to determine a second weight for the pixel point based on the discrete degree; A fuzzy strength generating submodule, configured to generate a fuzzy strength by combining the first weight and the second weight; The contour point determination submodule comprises: A difference determination unit, used to calculate the difference between the original brightness value and the target brightness value of the pixel point in the original image data; A contour point determination unit, configured to determine a pixel point corresponding to the difference as a contour point when the difference does not belong to a preset edge range interval; The first weight determination submodule includes: A first value determining unit, configured to determine, when the pixel point is a contour point, a first weight of the pixel point as a first value; The second value determination unit is used to determine, when the pixel point is not a contour point, that the first weight of the pixel point is a second value, and the first value is smaller than the second value.

11. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement an image processing method as described in any one of claims 1 to 7, or a live broadcast method as described in claim 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements an image processing method as described in any one of claims 1 to 7, or a live broadcast method as described in claim 8.

Images