Live broadcast processing method and device, electronic equipment and storage medium

Abstract

The application of the invention is a divisional application of 202110518478.3. The invention provides a live broadcast processing method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: receiving a video of a live broadcasting room; playing a video of a live broadcast room in a human-computer interaction interface, and displaying a countdown control and a color cast correction closing button in the human-computer interaction interface; and when the color cast correction closing button is still not triggered before the countdown in the countdown control ends, automatically carrying out color cast correction processing on the video, and playing the video after the color cast correction processing.

Description

[0001] This application is a divisional application of application number 202110518478.3, application date May 12, 2021, and invention name “Live broadcast processing method, device, electronic device and storage medium”. Technical Field

[0002] The present application relates to Internet technology, and in particular to a live broadcast processing method, device, electronic device and computer-readable storage medium. Background Art

[0003] Live streaming is an important way of disseminating information on the Internet. Based on cloud technology, big data processing can collect live content from massive anchors and distribute it to the audience participating in the anchors in real time.

[0004] Livestreamers can perform various recommendations in their livestream studios. For example, they can present items in a comprehensive manner. However, due to limitations in camera equipment, ambient lighting, and viewer devices, the colors of items displayed on the viewer's device often differ from the actual colors of the items. This color difference can affect viewers' viewing and purchasing experience, leading to ineffective recommendations and unnecessary waste of computing and communication resources in the livestream system. There is currently no effective solution to this problem. Summary of the Invention

[0005] The embodiments of the present application provide a live broadcast processing method, device, electronic device and computer-readable storage medium, which can ensure the accuracy of video display color during live broadcast, thereby improving the visual perception effect of the video.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The present invention provides a method for processing live broadcast, including:

[0008] Receive live broadcast video;

[0009] Playing the live broadcast room video in the human-computer interaction interface, and displaying the color cast correction entrance in the human-computer interaction interface;

[0010] In response to a triggering operation on the color cast correction entry, color cast correction is performed on the video, and the video after the color cast correction is played.

[0011] In the above solution, determining the color cast detection result of the video according to the color cast factor of each image frame includes:

[0012] Among the multiple image frames, determining an image frame whose color cast factor is greater than a color cast factor threshold as a color cast image frame;

[0013] When the ratio between the number of the color cast image frames and the number of the plurality of image frames is greater than a color cast number threshold, determining that the video has color cast;

[0014] When the ratio between the number of the color cast image frames and the number of the plurality of image frames is not greater than a color cast number threshold, it is determined that the video does not have color cast.

[0015] In the above solution, determining the average chromaticity and chromaticity center distance of the image frame includes:

[0016] Determining the color value of each pixel in the image frame in a red, green, and blue color space;

[0017] Converting the color value of each pixel in the red, green and blue color space into a first color-opponent dimension and a second color-opponent dimension in a color-opponent space;

[0018] performing a summing process on the first color-opponent dimension of each pixel, and determining a ratio between the summing result and the number of pixels in the image frame as a first chrominance component;

[0019] performing a summing process on the second color-opponent dimension of each pixel, and determining a ratio between the summing result and the number of pixels in the image frame as a second chroma component;

[0020] determining an average chroma of the image frame according to the first chroma component and the second chroma component;

[0021] Determine a first chromaticity center distance component and a second chromaticity center distance component according to the first color opposition dimension and the second color opposition dimension of each pixel point;

[0022] The chromaticity center distance of the image frame is determined according to the first chromaticity center distance component and the second chromaticity center distance component.

[0023] In the above solution, determining the average chroma of the image frame according to the first chroma component and the second chroma component includes:

[0024] performing a squaring process on the first chrominance component to obtain a first squaring result;

[0025] performing a squaring process on the second chrominance component to obtain a second squaring result;

[0026] The first square result and the second square result are summed, and the sum is squared to obtain the average chromaticity of the image frame.

[0027] In the above solution, determining the first chromaticity center distance component and the second chromaticity center distance component according to the first color opposition dimension and the second color opposition dimension of each pixel point includes:

[0028] For each pixel, the following processing is performed: subtracting a first color-opponent dimension of the pixel from the first chromaticity component, squaring the subtraction result to obtain a first chromaticity distance, subtracting a second color-opponent dimension of the pixel from the second chromaticity component, and squaring the subtraction result to obtain a second chromaticity distance;

[0029] performing summing processing on the first chromaticity distance of each pixel point, and determining a ratio between the summing result and the number of pixels in the image frame as the first chromaticity center distance component;

[0030] The second chromaticity distance of each pixel is summed up, and a ratio between the summed up result and the number of pixels in the image frame is determined as the second chromaticity center distance component.

[0031] In the above solution, determining the chromaticity center distance of the image frame according to the first chromaticity center distance component and the second chromaticity center distance component includes:

[0032] Squaring the first chromaticity center distance component to obtain a third square result;

[0033] Squaring the first chromaticity center distance component to obtain a fourth square result;

[0034] The third power result and the fourth power result are added together to obtain the chromaticity center distance of the image frame.

[0035] In the above solution, the color cast correction processing on the video includes:

[0036] The following processing is performed for each image frame in the video:

[0037] performing a first color cast correction process on the image frame according to the color value of each pixel in the image frame;

[0038] Acquire screen color cast information of the terminal, and perform a second color cast correction process on the image frame after the first color cast correction process according to the screen color cast information.

[0039] In the above solution, before obtaining the screen color cast information of the terminal, the method further includes:

[0040] When at least one of the following conditions is met, it is determined that the operation of obtaining the screen color cast information of the terminal will be performed:

[0041] Receiving a color cast correction operation submitted by a login account, wherein the login account is the account that logged into the live broadcast room;

[0042] The terminal does not have a privacy protection function enabled, wherein the privacy protection function is used to shield the device information of the terminal from being read.

[0043] In the above solution, performing a first color cast correction process on the image frame according to the color value of each pixel in the image frame includes:

[0044] Performing the following processing for each pixel in the image frame: determining a plurality of color values ​​corresponding one-to-one to a plurality of color channels in the pixel, performing equalization processing on the plurality of color values ​​to obtain a plurality of equalization data corresponding one-to-one to the plurality of color values;

[0045] determining a plurality of reference white pixels in the image frame according to a plurality of equalization data corresponding to each pixel in the image frame;

[0046] determining a first scale factor according to the brightness values ​​and color values ​​of the plurality of reference white pixels;

[0047] determining a second scale factor according to a color value of each pixel in the image frame;

[0048] A first color cast correction process is performed on the image frame according to the first scale factor and the second scale factor.

[0049] In the above solution, determining a plurality of reference white pixels in the image frame according to a plurality of equalization data corresponding to each pixel in the image frame includes:

[0050] Performing the following processing for each pixel in the image frame: determining, from a plurality of equalization data corresponding to the pixel, first equalization data corresponding to a first color channel, second equalization data corresponding to a second color channel, and third equalization data corresponding to a third color channel;

[0051] Selecting a pixel point that satisfies the following conditions simultaneously as a white pixel point in the image frame: a first equalization data of the pixel point is greater than or equal to a first preset value, a second equalization data of the pixel point is greater than or equal to a second preset value, and a third equalization data of the pixel point is less than or equal to a third preset value;

[0052] Selecting a pixel with the largest brightness value from the plurality of white pixel points as a brightness white pixel point;

[0053] A plurality of reference white pixel points are determined in the image frame according to the first equalization data, the second equalization data, and the third equalization data corresponding to the brightness white pixel point.

[0054] In the above solution, determining a plurality of reference white pixels in the image frame according to the first equalization data, the second equalization data, and the third equalization data corresponding to the brightness white pixel includes:

[0055] Determining the first equalization data corresponding to the luminance white pixel as first luminance equalization data, determining the second equalization data corresponding to the luminance white pixel as second luminance equalization data, and determining the third equalization data corresponding to the luminance white pixel as third luminance equalization data;

[0056] performing summing processing on the first equalized data corresponding to each of the white pixels, and determining a ratio between the summed result and the number of the white pixels as first equalized average data;

[0057] performing summing processing on the second equalized data corresponding to each of the white pixels, and determining a ratio between the summed result and the number of the white pixels as second equalized average data;

[0058] performing summing processing on the third equalized data corresponding to each of the white pixels, and determining a ratio between the summed result and the number of the white pixels as third equalized average data;

[0059] A pixel point that satisfies the following conditions simultaneously is selected in the image frame as the reference white pixel point: the first equalization data of the pixel point is between the first brightness equalization data and the first equalization average data, the second equalization data of the pixel point is between the second brightness equalization data and the second equalization average data, and the third equalization data of the pixel point is between the third brightness equalization data and the third equalization average data.

[0060] In the above solution, determining the first scale factor according to the brightness values ​​and color values ​​of the plurality of reference white pixels includes:

[0061] summing the brightness values ​​of each of the reference white pixels, and determining the ratio between the summed value and the number of the reference white pixels as the average brightness value;

[0062] Determining a first color value corresponding to a first color channel, a second color value corresponding to a second color channel, and a third color value corresponding to a third color channel for each of the reference white pixels;

[0063] summing the first color values ​​corresponding to each of the reference white pixels, and determining a ratio between the summed value and the number of the reference white pixels as a first color average value;

[0064] summing the second color values ​​corresponding to each of the reference white pixels, and determining a ratio between the summed value and the number of the reference white pixels as a second color average value;

[0065] summing the third color values ​​corresponding to each of the reference white pixels, and determining a ratio between the summed value and the number of the reference white pixels as a third color average value;

[0066] A first component, a second component, and a third component are determined according to the brightness average, the first color average, the second color average, and the third color average, and the first component, the second component, and the third component are combined into the first scale factor.

[0067] In the above solution, determining the first component, the second component, and the third component according to the brightness average, the first color average, the second color average, and the third color average includes:

[0068] determining a ratio between the brightness average value and the first color average value as the first component;

[0069] determining a ratio between the brightness average value and the second color average value as the second component;

[0070] A ratio between the brightness average value and the third color average value is determined as the third component.

[0071] In the above solution, determining the second scale factor according to the color value of each pixel in the image frame includes:

[0072] summing the first color value of the first color channel corresponding to each pixel in the image frame, and determining a ratio between the summed value and the number of pixels in the image frame as a fourth color average value;

[0073] performing summing processing on the second color value of the second color channel corresponding to each pixel in the image frame, and determining a ratio between the summing result and the number of pixels in the image frame as a fifth color average value;

[0074] summing the third color value of the third color channel corresponding to each pixel in the image frame, and determining a ratio between the summed value and the number of pixels in the image frame as a sixth color average value;

[0075] Determine an average value of the fourth color average, the fifth color average, and the sixth color average as a total color average;

[0076] A fourth component, a fifth component, and a sixth component are determined according to the total color average, the fourth color average, the fifth color average, and the sixth color average, and the fourth component, the fifth component, and the sixth component are combined into the second scale factor.

[0077] In the above solution, determining the fourth component, the fifth component, and the sixth component according to the total color average, the fourth color average, the fifth color average, and the sixth color average includes:

[0078] determining a ratio between the total color average and the fourth color average as the fourth component;

[0079] determining a ratio between the total color average and the fifth color average as the fifth component;

[0080] A ratio between the total color average value and the sixth color average value is determined as the sixth component.

[0081] In the above solution, performing a first color cast correction process on the image frame according to the first scale factor and the second scale factor includes:

[0082] determining a color to which the image frame is biased based on a first chrominance component and a second chrominance component of the image frame;

[0083] When the image frame is biased toward red, performing gain processing on the color value of each pixel in the image frame by using the fourth component of the second scale factor, the second component of the first scale factor, and the third component of the first scale factor as gains;

[0084] When the image frame tends to be blue, performing gain processing on the color value of each pixel in the image frame by using the first component of the first scale factor, the second component of the first scale factor, and the sixth component of the second scale factor as gains;

[0085] When the image frame leans toward green, performing gain processing on the color value of each pixel in the image frame by using the first component of the first scale factor, the fifth component of the second scale factor, and the third component of the first scale factor as gains;

[0086] When the image frame leans toward yellow, gain processing is performed on the color value of each pixel in the image frame by using the fourth component of the second scale factor, the fifth component of the second scale factor, and the third component of the first scale factor as gains.

[0087] In the above solution, determining the biased color of the image frame according to the first chrominance component and the second chrominance component of the image frame includes:

[0088] When the first chrominance component is greater than a fourth preset value, determining that the image frame is biased towards red;

[0089] When the first chrominance component is not greater than a fourth preset value, determining that the image frame is biased towards green;

[0090] When the second chromaticity component is greater than a fifth preset value, determining that the image frame is biased toward yellow;

[0091] When the second chromaticity component is not greater than a fifth preset value, it is determined that the image frame is biased towards blue.

[0092] In the above solution, performing a second color cast correction process on the image frame after the first color cast correction process according to the screen color cast information includes:

[0093] Querying a third scale factor corresponding to the screen color cast information;

[0094] The third scale factor is used as a gain to perform gain processing on the color value of each pixel in the image frame after the first color cast correction processing.

[0095] In the above solution, the color cast correction processing on the video includes:

[0096] Obtaining screen color cast information of the terminal, and querying a fourth scale factor corresponding to the screen color cast information;

[0097] The following processing is performed on each image frame in the video: using the fourth scale factor as a gain, performing gain processing on the color value of each pixel in the image frame.

[0098] The present invention provides a live broadcast processing device, including:

[0099] A receiving module, used to receive the video from the live broadcast room;

[0100] A display module is used to play the video of the live broadcast room in the human-computer interaction interface and display the color cast correction entrance in the human-computer interaction interface;

[0101] The correction module is used to perform color cast correction processing on the video in response to a trigger operation on the color cast correction entrance, and play the video after the color cast correction processing.

[0102] In the above scheme, the correction module is also used to play the video after color cast correction processing in the first area of ​​the human-computer interaction interface; play the video before color cast correction processing in the second area of ​​the human-computer interaction interface; and exchange the videos played in the first area and the second area in response to the area switching operation.

[0103] In the above solution, the correction module is further used to display a closing entrance of the second area in the human-computer interaction interface; in response to a triggering operation on the closing entrance, automatically stop the video playing in the second area and hide the second area.

[0104] In the above solution, the correction module is further configured to automatically stop playing the video in the second area and hide the second area when the area switching operation is not received within a preset waiting time.

[0105] In the above solution, the correction module is further used to automatically replace the video of the live broadcast room before the color cast correction processing with the video after the color cast correction processing.

[0106] In the above scheme, the correction module is also used to display a color cast correction parameter setting page; in response to the color cast correction parameter configuration operation received on the color cast correction parameter setting page, obtain the color cast correction parameters configured by the login account, wherein the login account is the account for logging into the live broadcast room; and perform color cast correction processing on the video according to the color cast correction parameters configured by the login account.

[0107] In the above solution, the display module is also used to obtain the color cast tolerance of the login account, wherein the login account is the account for logging into the live broadcast room; when the color cast degree of the video exceeds the color cast tolerance, the color cast correction entrance is automatically triggered.

[0108] In the above scheme, the correction module is also used to perform the following processing on each image frame in the video: performing a first color cast correction processing on the image frame according to the color value of each pixel in the image frame; obtaining the screen color cast information of the terminal, and performing a second color cast correction processing on the image frame after the first color cast correction processing according to the screen color cast information.

[0109] In the above scheme, the correction module is also used to determine that the operation of obtaining the screen color cast information of the terminal will be executed when at least one of the following conditions is met: receiving a color cast correction operation submitted by a login account, wherein the login account is the account for logging into the live broadcast room; the terminal does not have a privacy protection function enabled, wherein the privacy protection function is used to shield the device information of the reading terminal.

[0110] In the above scheme, the correction module is further configured to perform the following processing for each pixel in the image frame: determining multiple color values ​​corresponding one-to-one to multiple color channels in the pixel, performing equalization processing on the multiple color values ​​to obtain multiple equalization data corresponding one-to-one to the multiple color values; determining multiple reference white pixels in the image frame based on the multiple equalization data corresponding to each pixel in the image frame; determining a first scale factor based on the brightness values ​​and color values ​​of the multiple reference white pixels; determining a second scale factor based on the color value of each pixel in the image frame; and performing a first color cast correction processing on the image frame based on the first scale factor and the second scale factor.

[0111] In the above scheme, the correction module is further used to perform the following processing for each pixel point in the image frame: determine the first equalization data corresponding to the first color channel, the second equalization data corresponding to the second color channel, and the third equalization data corresponding to the third color channel from the multiple equalization data corresponding to the pixel point; select the pixel point in the image frame that satisfies the following conditions at the same time as the white pixel point: the first equalization data of the pixel point is greater than or equal to the first preset value, the second equalization data of the pixel point is greater than or equal to the second preset value, and the third equalization data of the pixel point is less than or equal to the third preset value; select the pixel point with the largest brightness value from the multiple white pixel points as the brightness white pixel point; and determine multiple reference white pixel points in the image frame based on the first equalization data, the second equalization data, and the third equalization data corresponding to the brightness white pixel point.

[0112] In the above scheme, the correction module is further used to determine the first equalization data corresponding to the brightness white pixel as the first brightness equalization data, determine the second equalization data corresponding to the brightness white pixel as the second brightness equalization data, and determine the third equalization data corresponding to the brightness white pixel as the third brightness equalization data; sum the first equalization data corresponding to each of the white pixels, and determine the ratio between the summation result and the number of the white pixels as the first equalization average data; sum the second equalization data corresponding to each of the white pixels, and determine the ratio between the summation result and the number of the white pixels as second equalized average data; summing the third equalized data corresponding to each white pixel, and determining the ratio between the summed data and the number of white pixels as the third equalized average data; selecting a pixel point in the image frame that simultaneously meets the following conditions as the reference white pixel point: the first equalized data of the pixel point is between the first brightness equalized data and the first equalized average data, the second equalized data of the pixel point is between the second brightness equalized data and the second equalized average data, and the third equalized data of the pixel point is between the third brightness equalized data and the third equalized average data.

[0113] In the above scheme, the correction module is further used to sum the brightness values ​​of each of the reference white pixels and determine the ratio between the summation result and the number of the reference white pixels as the brightness average; determine the first color value of the first color channel corresponding to each of the reference white pixels, the second color value of the second color channel corresponding to the second color channel, and the third color value of the third color channel corresponding to the third color channel; sum the first color values ​​corresponding to each of the reference white pixels and determine the ratio between the summation result and the number of the reference white pixels as the first color average; sum the second color values ​​corresponding to each of the reference white pixels and determine the ratio between the summation result and the number of the reference white pixels as the second color average; sum the third color values ​​corresponding to each of the reference white pixels and determine the ratio between the summation result and the number of the reference white pixels as the third color average; determine the first component, the second component and the third component based on the brightness average, the first color average, the second color average and the third color average, and combine the first component, the second component and the third component into the first scale factor.

[0114] In the above scheme, the correction module is further used to determine the ratio between the brightness average value and the first color average value as the first component; determine the ratio between the brightness average value and the second color average value as the second component; and determine the ratio between the brightness average value and the third color average value as the third component.

[0115] In the above scheme, the correction module is further used to sum the first color values ​​of the first color channel corresponding to each pixel in the image frame, and determine the ratio of the summation result to the number of pixels in the image frame as a fourth color average; sum the second color values ​​of the second color channel corresponding to each pixel in the image frame, and determine the ratio of the summation result to the number of pixels in the image frame as a fifth color average; sum the third color values ​​of the third color channel corresponding to each pixel in the image frame, and determine the ratio of the summation result to the number of pixels in the image frame as a sixth color average; determine the average of the fourth color average, the fifth color average, and the sixth color average as a total color average; determine a fourth component, a fifth component, and a sixth component based on the total color average, the fourth color average, the fifth color average, and the sixth color average, and combine the fourth component, the fifth component, and the sixth component into the second scale factor.

[0116] In the above scheme, the correction module is also used to determine the ratio between the total color average value and the fourth color average value as the fourth component; determine the ratio between the total color average value and the fifth color average value as the fifth component; and determine the ratio between the total color average value and the sixth color average value as the sixth component.

[0117] In the above solution, the correction module is further configured to determine the color to which the image frame is biased based on the first chromaticity component and the second chromaticity component of the image frame; when the image frame is biased toward red, using the fourth component of the second scale factor, the second component of the first scale factor, and the third component of the first scale factor as gains to perform gain processing on the color value of each pixel in the image frame; when the image frame is biased toward blue, using the first component of the first scale factor, the second component of the first scale factor, and the sixth component of the second scale factor as gains to perform gain processing on the color value of each pixel in the image frame; when the image frame is biased toward green, using the first component of the first scale factor, the fifth component of the second scale factor, and the third component of the first scale factor as gains to perform gain processing on the color value of each pixel in the image frame; and when the image frame is biased toward yellow, using the fourth component of the second scale factor, the fifth component of the second scale factor, and the third component of the first scale factor as gains to perform gain processing on the color value of each pixel in the image frame.

[0118] In the above scheme, the correction module is also used to determine that the image frame is biased towards red when the first chromaticity component is greater than a fourth preset value; determine that the image frame is biased towards green when the first chromaticity component is not greater than the fourth preset value; determine that the image frame is biased towards yellow when the second chromaticity component is greater than the fifth preset value; and determine that the image frame is biased towards blue when the second chromaticity component is not greater than the fifth preset value.

[0119] In the above solution, the correction module is further used to query a third scale factor corresponding to the screen color cast information; and use the third scale factor as a gain to perform gain processing on the color value of each pixel in the image frame after the first color cast correction processing.

[0120] In the above scheme, the correction module is further used to obtain screen color cast information of the terminal and query a fourth scale factor corresponding to the screen color cast information; and perform the following processing on each image frame in the video: using the fourth scale factor as a gain, performing gain processing on the color value of each pixel in the image frame.

[0121] In the above scheme, the display module is also used to perform color cast detection processing on the video in the live broadcast room to obtain a color cast detection result; when the color cast detection result indicates that the video has color cast, information for prompting that the video has color cast is displayed in the human-computer interaction interface, and it is determined that an operation of displaying a color cast correction entrance in the human-computer interaction interface will be executed.

[0122] In the above scheme, the display module is also used to extract multiple image frames from the video; perform the following processing for each of the image frames: determine the average chromaticity and chromaticity center distance of the image frame, and determine the ratio between the average chromaticity and the chromaticity center distance as the color cast factor of the image frame; and determine the color cast detection result of the video based on the color cast factor of each of the image frames.

[0123] In the above scheme, the display module is further used to determine, among the multiple image frames, the image frames whose color cast factors are greater than the color cast factor threshold as color cast image frames; when the ratio between the number of the color cast image frames and the number of the multiple image frames is greater than the color cast number threshold, determine that the video has color cast; when the ratio between the number of the color cast image frames and the number of the multiple image frames is not greater than the color cast number threshold, determine that the video has no color cast.

[0124] In the above scheme, the display module is further configured to determine the color value of each pixel in the image frame in a red, green, and blue color space; convert the color value of each pixel in the red, green, and blue color space into a first color-opponent dimension and a second color-opponent dimension in a color-opponent space; sum the first color-opponent dimension of each pixel, and determine the ratio between the summation result and the number of pixels in the image frame as a first chromaticity component; sum the second color-opponent dimension of each pixel, and determine the ratio between the summation result and the number of pixels in the image frame as a second chromaticity component; determine the average chromaticity of the image frame based on the first chromaticity component and the second chromaticity component; determine a first chromaticity center distance component and a second chromaticity center distance component based on the first color-opponent dimension and the second color-opponent dimension of each pixel; and determine the chromaticity center distance of the image frame based on the first chromaticity center distance component and the second chromaticity center distance component.

[0125] In the above scheme, the display module is also used to perform a square process on the first chromaticity component to obtain a first square result; perform a square process on the second chromaticity component to obtain a second square result; perform a sum process on the first square result and the second square result, and perform a square root process on the sum result to obtain the average chromaticity of the image frame.

[0126] In the above scheme, the display module is further used to perform the following processing on each pixel: subtracting the first color-opposite dimension of the pixel from the first chromaticity component, and squaring the subtraction result to obtain a first chromaticity distance; subtracting the second color-opposite dimension of the pixel from the second chromaticity component, and squaring the subtraction result to obtain a second chromaticity distance; summing the first chromaticity distance of each pixel, and determining the ratio between the summation result and the number of pixels in the image frame as the first chromaticity center distance component; summing the second chromaticity distance of each pixel, and determining the ratio between the summation result and the number of pixels in the image frame as the second chromaticity center distance component.

[0127] In the above scheme, the display module is also used to square the first chromaticity center distance component to obtain a third square result; square the first chromaticity center distance component to obtain a fourth square result; and add the third square result and the fourth square result to obtain the chromaticity center distance of the image frame.

[0128] In the above scheme, the display module is also used to perform scene detection processing on the video of the live broadcast room. When a color display sensitive scene appears in the video of the live broadcast room, it is determined to perform a color cast detection processing operation on the video of the live broadcast room; wherein, the types of color display sensitive scenes include: recommended scenes for multiple items of different colors, recommended scenes for at least one item of a specific color, and recommended scenes for items that meet the preferences of the login account, and the login account is the account that logs in to the live broadcast room.

[0129] In the above solution, the display module is also used to display a color cast detection entrance in the human-computer interaction interface; in response to a trigger operation on the color cast detection entrance, it is determined to perform a color cast detection operation on the video in the live broadcast room.

[0130] In the above scheme, the display module is also used to perform scene detection processing on the video of the live broadcast room played in the human-computer interaction interface; when a color display sensitive scene appears in the video of the live broadcast room, it is determined to automatically switch to the operation of displaying the color cast correction entrance in the human-computer interaction interface; wherein, the types of color display sensitive scenes include: recommended scenes for multiple items of different colors, recommended scenes for at least one specific color item, and recommended scenes for items that meet the preferences of the login account; the login account is the account for logging into the live broadcast room.

[0131] An embodiment of the present application provides an electronic device, including:

[0132] a memory for storing computer-executable instructions;

[0133] The processor is used to implement the live broadcast processing method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0134] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing the live broadcast processing method provided in the embodiment of the present application when executed by a processor.

[0135] An embodiment of the present application provides a computer program product, which includes computer-executable instructions for implementing the live broadcast processing method provided in the embodiment of the present application when executed by a processor.

[0136] The embodiments of the present application have the following beneficial effects:

[0137] After the audience triggers the color cast correction entrance, the video in the live broadcast room is color-corrected and played. This can ensure the accuracy of the video display color during the live broadcast, improve the visual perception of the video, thereby improving the audience's live viewing experience and saving computing resources and communication resources used for live broadcast. BRIEF DESCRIPTION OF THE DRAWINGS

[0138] Figure 1 This is a schematic diagram of the architecture of the live broadcast processing system provided in an embodiment of the present application;

[0139] Figure 2 is a schematic structural diagram of a viewer terminal provided in an embodiment of the present application;

[0140] Figure 3 This is a flowchart of a live broadcast processing method provided by an embodiment of the present application;

[0141] Figure 4 This is a flowchart of a live broadcast processing method provided by an embodiment of the present application;

[0142] Figure 5 This is a flowchart of a live broadcast processing method provided by an embodiment of the present application;

[0143] Figure 6 This is a flowchart of a live broadcast processing method provided by an embodiment of the present application;

[0144] Figure 7 This is a flowchart of a live broadcast processing method provided by an embodiment of the present application;

[0145] Figure 8 This is a flowchart of a live broadcast processing method provided by an embodiment of the present application;

[0146] Figure 9This is a schematic diagram of an application scenario of the live broadcast processing method provided in an embodiment of the present application;

[0147] Figure 10 This is a schematic diagram of the principle of the live broadcast processing method provided in an embodiment of the present application;

[0148] Figure 11 This is a schematic diagram of an application scenario of the live broadcast processing method provided in an embodiment of the present application;

[0149] Figure 12 This is a schematic diagram of an application scenario of the live broadcast processing method provided in an embodiment of the present application;

[0150] Figure 13 This is a schematic diagram of an application scenario of the live broadcast processing method provided in an embodiment of the present application;

[0151] Figure 14 This is a schematic diagram of an application scenario of the live broadcast processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0152] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0153] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0154] In the following description, the terms "first / second" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0155] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0156] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0157] 1) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed can be real-time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0158] 2) Live streaming: This involves producing and distributing information on-site as events unfold and progress, enabling a two-way flow of information across networks. Specifically, this involves collecting data from the broadcaster through a device, processing it through a series of processes, such as video encoding and compression, to create a viewable and transmittable video stream, which is then output to the viewing user.

[0159] 3) Color space, also known as color space or color coordinate system, is a method of abstractly representing and describing color. The Red, Green, Blue (RGB) color space is one of the most commonly used color models, and images and videos output by digital imaging devices are often in RGB format. However, the biggest limitation of the RGB color space is that when the Euclidean distance is used to characterize the difference between two colors, the calculated distance between the two colors cannot accurately represent the actual difference between the two colors that people actually perceive. The distance between colors calculated using the Lab color space (also known as the color opposition space or Lab space) is basically consistent with the difference in people's actual perception.

[0160] 4) Color cast. The human visual system has color constancy, which can, to a certain extent, eliminate the influence of factors such as lighting conditions on color and accurately perceive the color of objects. However, imaging devices do not have this "adjustment" function. Therefore, there is a certain degree of error between the color of the image captured by the imaging device and the actual color of the object surface, which is called color cast.

[0161] 5) Color cast correction, also known as color restoration, is to eliminate color cast. Specifically, it refers to removing the influence of factors such as the color temperature of the light source in the shooting environment, thereby reproducing the true color of the object.

[0162] The embodiments of the present application provide a live broadcast processing method that can ensure the accuracy of the video display color during the live broadcast process, thereby improving the visual perception effect of the video. The following describes an exemplary application of the live broadcast processing method provided by the embodiments of the present application. The live broadcast processing method provided by the embodiments of the present application can be implemented by various electronic devices, for example, it can be applied to various types of user terminals (hereinafter referred to as terminals), such as smartphones, tablet computers, vehicle-mounted terminals, and smart wearable devices.

[0163] Next, taking an electronic device as a terminal as an example, an exemplary application system architecture of a terminal implementing the live broadcast processing method provided in the embodiment of the present application is described. Figure 1 , Figure 11 is a schematic diagram of the architecture of the live broadcast processing system 100 provided in an embodiment of the present application. The live broadcast processing system 100 includes: a server 200, a network 300, and a viewer terminal 400, which will be described separately.

[0164] The server 200 is the backend server of the client 410 and is used to send the video of the live broadcast room to the client 410 .

[0165] The network 300 is used as a medium for communication between the server 200 and the viewer terminal 400, and can be a wide area network or a local area network, or a combination of the two.

[0166] Viewer terminal 400, belonging to the viewer, is used to run client 410, which is a client with live broadcasting capabilities. Client 410 is used to receive live broadcast room video from server 200 and play the live broadcast room video on human-computer interaction interface 411. Client 410 is also used to display a color cast correction entry on human-computer interaction interface 411 and, in response to a viewer triggering the color cast correction entry, perform color cast correction on the video and play the corrected video on human-computer interaction interface 411.

[0167] In some embodiments, the audience terminal 400 implements the live broadcast processing method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a live broadcast APP; it can also be a mini-program, that is, a program that can be run by simply downloading it into a browser environment, such as a live broadcast mini-program; it can also be a live broadcast mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in.

[0168] The embodiments of the present application can be implemented with the help of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network within a wide area network or a local area network to realize data calculation, storage, processing, and sharing.

[0169] Cloud technology is a general term for network, information, integration, management platform, and application technologies used in the cloud computing business model. It can form a resource pool that can be used flexibly and conveniently on demand. Cloud computing technology will become a key support. The backend services of technical network systems require a large amount of computing and storage resources.

[0170] As an example, server 200 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The audience terminal 400 and server 200 can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0171] Next, let’s explain Figure 1 The structure of the viewer terminal 400 in FIG. Figure 2 , Figure 2 is a schematic diagram of the structure of the viewer terminal 400 provided in an embodiment of the present application. Figure 2 The viewer terminal 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the viewer terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0172] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0173] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0174] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0175] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0176] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0177] The operating system 451 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic businesses and process hardware-based tasks.

[0178] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include Bluetooth, Wireless LAN (WiFi), and Universal Serial Bus (USB).

[0179] The presentation module 453 is configured to enable presentation of information (eg, a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (eg, a display screen, a speaker, etc.) associated with the user interface 430 .

[0180] The input processing module 454 is configured to detect one or more user inputs or interactions from one of the one or more input devices 432 and to translate the detected inputs or interactions.

[0181] In some embodiments, the live broadcast processing device provided in the embodiments of the present application can be implemented in software. Figure 2 The live broadcast processing device 455 stored in the memory 450 is shown. This can be software in the form of a program or plug-in, and includes the following software modules: a receiving module 4551, a display module 4552, and a correction module 4553. These modules are logical and can be arbitrarily combined or further separated according to the functions implemented. The functions of each module will be described below.

[0182] Below, by Figure 1 The live broadcast processing method provided in the embodiment of the present application is described as an example in which the audience terminal 400 alone executes the live broadcast processing method provided in the embodiment of the present application. Figure 3 , Figure 3This is a flow chart of the live broadcast processing method provided by the embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0183] It should be noted that Figure 3 The method shown can be executed by various forms of computer programs running on the audience terminal 400, and is not limited to the above-mentioned client 410, such as the above-mentioned operating system 451, software modules, scripts and applets. Therefore, the example of the client below should not be regarded as a limitation on the embodiments of the present application.

[0184] It should also be noted that the viewer account mentioned below belongs to the viewer, and for the sake of convenience, no specific distinction is made between the viewer account and the viewer. For example, operations on the viewer account specifically refer to operations performed by the viewer holding the viewer account.

[0185] In step S101, a video of a live broadcast room is received.

[0186] In some embodiments, the anchor terminal collects the video and audio of the anchor's performance in the live broadcast room, and sends the video and audio to the server in the form of a live stream. The server pushes the live stream of the live broadcast room to the viewer terminal.

[0187] In step S102, the video of the live broadcast room is played in the human-computer interaction interface.

[0188] In some embodiments, the video images (i.e., image frames) of the live broadcast room are played in the human-computer interaction interface according to the video in the live broadcast stream, and when playing the video images of the live broadcast room, the corresponding audio is played synchronously according to the audio in the live broadcast stream.

[0189] In step S103, a color cast correction entrance is displayed in the human-computer interaction interface.

[0190] As an example, Figure 9 In the live broadcast, a color cast correction entry 902 is displayed in the live broadcast screen. When the viewer triggers the color cast correction entry 902, the color cast correction function can be enabled. The color cast correction entry 902 can be displayed continuously during the live broadcast, or only when color cast occurs in the live broadcast screen, or only when the color cast of the video exceeds a color cast threshold (which can be a default value or a value set by the host, viewer, client, or server), thereby saving terminal display resources.

[0191] In some embodiments, executing step S103 may include: obtaining the color cast tolerance of the login account, where the login account is the account that logs into the live broadcast room; when the color cast of the video exceeds the color cast tolerance, the color cast correction entrance is automatically triggered.

[0192] As an example, the color cast tolerance level can be a value set by the login account, or it can be determined based on the login account's historical viewing records, that is, it is determined based on the color cast levels of videos that have not been corrected for color cast that have been viewed by the login account. For example, the average, maximum, or minimum value of the color cast levels of all videos that have not been corrected for color cast that have been viewed by the login account can be used as the color cast tolerance level.

[0193] As an example, before automatically triggering the color cast correction entrance, a countdown control and a color cast correction close button can also be displayed. When no trigger operation for the color cast correction close button is received before the countdown in the countdown control ends, it is determined that the operation of automatically triggering the color cast correction entrance will be executed.

[0194] For example, Figure 11 In the example, when the color cast of a video exceeds the color cast tolerance, a countdown control 112 and a color cast correction off button 111 are displayed. If the viewer does not trigger the color cast correction off button 111 before the countdown in the countdown control 112 ends, the video is automatically color-corrected. This automatically triggers the color cast correction entry based on the color cast tolerance of the logged-in account, reducing the number of operations required by the viewer and conserving terminal operating resources.

[0195] In some embodiments, before step S103, it may also include: performing scene detection processing on the video of the live broadcast room played in the human-computer interaction interface; when a color display sensitive scene appears in the video of the live broadcast room, determining to automatically enter the operation of displaying the color cast correction entrance in the human-computer interaction interface; wherein, the types of color display sensitive scenes include: recommended scenes for multiple items of different colors, recommended scenes for at least one specific color item, and recommended scenes for items that meet the preferences of the login account.

[0196] As an example, when a color display sensitive scene appears in the video of the live broadcast room, a prompt message indicating that the scene has entered a color display sensitive scene can be displayed in the human-computer interaction interface.

[0197] The embodiment of the present application can automatically display a color cast correction entrance according to the scene appearing in the video, that is, directly display the color cast correction entrance without performing color cast detection processing, so that when a viewer who is sensitive to color finds that the video has color cast, or when the viewer knows that he or she has entered a color display sensitive scene according to the prompt information, the color cast correction processing can be directly triggered according to the color cast correction entrance, thereby avoiding unnecessary color cast detection and saving color cast detection resources.

[0198] In step S104 , in response to a trigger operation on a color cast correction entry, a color cast correction process is performed on the video.

[0199] In some embodiments, as an alternative to step S104, when the color cast of the video exceeds a color cast threshold, the video is automatically color-corrected. This eliminates the need for the viewer to trigger the color cast correction entry, thereby reducing the number of operations required by the viewer and conserving terminal operating resources.

[0200] In some embodiments, in response to a trigger operation for a color cast correction entrance, it can include: displaying a color cast correction parameter setting page; in response to a color cast correction parameter configuration operation received on the color cast correction parameter setting page, obtaining the color cast correction parameters configured by the login account; in this way, performing color cast correction processing on the video can include: performing color cast correction processing on the video according to the color cast correction parameters configured by the login account.

[0201] For example, Figure 12 When the viewer triggers the color cast correction entry, a color cast correction parameter setting page 121 is displayed, wherein the color cast correction parameter setting page 121 includes a color cast correction parameter setting entry 122, and the viewer can configure the color cast correction parameters in the color cast correction parameter setting entry 122.

[0202] As an example, the color cast correction parameter setting page may include default color cast correction parameters, where the types of default color cast correction parameters include: color cast correction parameters determined by the server or viewer terminal based on the color cast degree of the video; historical color cast correction parameters; color cast correction parameters used by the same live broadcast room of the anchor. Because the same anchor uses the same video acquisition equipment, the degree of color cast caused may be the same. Therefore, the color cast correction parameters used by the same live broadcast room of the anchor can be used, thereby saving computing resources. In this way, the color cast correction parameter configuration operation can be an operation for selecting the default color cast correction parameters, and the selected default color cast correction parameters are the color cast correction parameters configured for the login account.

[0203] As an example, the color cast correction parameters configured by the login account can be subsequently reused to perform color cast correction processing on videos in other live broadcast rooms. For example, they can be reused to perform color cast correction processing on videos in the same live broadcast room as the anchor, thereby saving resources for calculating color cast correction parameters.

[0204] In some embodiments, the client can call a corresponding service (e.g., a color cast correction service) on the viewer's terminal to perform color cast correction on the video through the viewer's terminal. The client can also call a corresponding service (e.g., a color cast correction service) on the server to perform color cast correction on the video through the server.

[0205] As an example, when the client calls the corresponding service of the server (for example, the color cast correction service) to perform color cast correction on the video, the replacement step of step S104 is: the viewer terminal responds to the trigger operation for the color cast correction entrance, sends a color cast correction instruction to the server, the server performs color cast correction on the video according to the color cast correction instruction, and sends the video after color cast correction to the viewer terminal.

[0206] The following describes an example of a client invoking a corresponding service (e.g., a color cast correction service) on a viewer terminal to perform color cast correction on a video. It should be noted that the process of the client invoking a corresponding service (e.g., a color cast correction service) on a server to perform color cast correction on a video is similar to the following and will not be further described.

[0207] In some embodiments, performing color cast correction on a video may be performing color cast correction on each image frame in the video. Figure 4 , Figure 4 This is a flow chart of the live broadcast processing method provided by the embodiment of the present application, which will be combined with Figure 4 The specific implementation method of color cast correction processing for each image frame in the video is described.

[0208] In step S401 , a first color cast correction process is performed on the image frame according to the color value of each pixel in the image frame.

[0209] In some embodiments, see Figure 5 , Figure 5 This is a flow chart of the live broadcast processing method provided by the embodiment of the present application, based on Figure 4 , step S401 may include steps S4011 to S4015.

[0210] In step S4011, the following processing is performed for each pixel point in the image frame: multiple color values ​​corresponding one-to-one to multiple color channels are determined in the pixel point, and the multiple color values ​​are equalized to obtain multiple equalization data corresponding one-to-one to the multiple color values.

[0211] In some embodiments, the multiple color values ​​(R org , G org , B org ) corresponds to multiple color channels, for example, the color value R org Corresponding to the red channel (or the first color channel), the color value G org Corresponding to the green channel (or second color channel), color value B org Corresponding to the blue channel (or the third color channel); for the color value R of each pixel org , color value G organd color value B org Perform histogram equalization to obtain the corresponding color value R org Equalized data Y Hist , corresponding color value G org Equalization data Cr Hist , and the corresponding color value B org Equalized data Cb Hist .

[0212] For example, histogram equalization uses a histogram to adjust the contrast of each pixel in an image frame, which can be used to increase the global contrast of the image frame. Histogram equalization can better distribute the brightness of the image frame along the histogram, thereby enhancing local contrast without affecting overall contrast.

[0213] In step S4012, a plurality of reference white pixels are determined in the image frame according to a plurality of equalization data corresponding to each pixel in the image frame.

[0214] In some embodiments, the following processing is performed for each pixel in the image frame: determining the first equalization data (Y) corresponding to the first color channel from the multiple equalization data corresponding to the pixel. Hist ), the second equalization data corresponding to the second color channel (Cr Hist ), and the third equalization data corresponding to the third color channel (Cb Hist ); selecting a pixel point that simultaneously satisfies the following conditions in the image frame as a white pixel point: the first equalization data of the pixel point is greater than or equal to a first preset value, the second equalization data of the pixel point is greater than or equal to a second preset value, and the third equalization data of the pixel point is less than or equal to a third preset value; selecting a pixel point with the largest brightness value among multiple white pixel points as a brightness white pixel point; and determining multiple reference white pixel points in the image frame based on the first equalization data, the second equalization data, and the third equalization data corresponding to the brightness white pixel point.

[0215] Taking the first preset value as 210, the second preset value as -3, and the third preset value as 3 as an example, pixels that simultaneously satisfy formula (1) are selected in the image frame as white pixels:

[0216]

[0217] Among the white pixels that satisfy formula (1), the one with the largest Y Hist The value and the Cr closest to zero Hist , Cb Hist Find the pixel with the largest brightness value (Y Hist bright , Cr Histbright , Cb Hist bright ) as the brightness white pixel.

[0218] As an example, determining a plurality of reference white pixel points in an image frame according to the first equalization data, the second equalization data, and the third equalization data corresponding to the brightness white pixel point may include: determining the first equalization data corresponding to the brightness white pixel point as the first brightness equalization data (Y Hist bright ), determining the second equalization data corresponding to the brightness white pixel as the second brightness equalization data (Cr Hist bright ), and determining the third equalization data corresponding to the brightness white pixel as the third brightness equalization data (Cb Hist bright ); summing the first equalized data corresponding to each white pixel, and determining the ratio between the summed result and the number of white pixels as the first equalized average data (Y Hist avg ); summing the second equalized data corresponding to each white pixel, and determining the ratio between the summed result and the number of white pixels as the second equalized average data (Cr Hist avg ); summing the third equalized data corresponding to each white pixel, and determining the ratio between the summed result and the number of white pixels as the third equalized average data (Cb Hist avg ); selecting pixels that simultaneously meet the following conditions in the image frame as reference white pixels: the first equalized data of the pixel is between the first brightness equalized data and the first equalized average data, the second equalized data of the pixel is between the second brightness equalized data and the second equalized average data, and the third equalized data of the pixel is between the third brightness equalized data and the third equalized average data.

[0219] For example, all pixels satisfying formula (2) are selected in the image frame as reference white pixels.

[0220]

[0221] Among them, Y L and Y H are respectively selected from the first brightness equalization data Y Hist bright and the first equalized average data Y Hist avg The minimum and maximum values ​​between Cr L Cr H are respectively selected from the second brightness equalization data CrHist bright and the second equalized average data Cr Hist avg The minimum and maximum values ​​between Cb L and Cb H are respectively selected from the third brightness equalization data Cb Hist bright and the third equalized average data Cb Hist avg The minimum and maximum values ​​between .

[0222] In step S4013 , a first scale factor is determined according to the brightness values ​​and color values ​​of a plurality of reference white pixels.

[0223] In some embodiments, the brightness value of each reference white pixel is summed, and the ratio between the summed value and the number of reference white pixels is determined as the average brightness value (Y w ); determining a first color value of the first color channel, a second color value of the second color channel, and a third color value of the third color channel corresponding to each reference white pixel; summing the first color values ​​corresponding to each reference white pixel, and determining the ratio between the summed value and the number of reference white pixels as the first color average value (R w ); summing up the second color values ​​corresponding to each reference white pixel, and determining the ratio between the summed result and the number of reference white pixels as the second color average value (G w ); summing the third color values ​​corresponding to each reference white pixel, and determining the ratio between the summed result and the number of reference white pixels as the third color average value (B w ); Determine the first component (R according to the brightness average, the first color average, the second color average and the third color average scale ), the second component (G scale ) and the third component (B scale ), and combining the first component, the second component, and the third component into a first scale factor.

[0224] As an example, determining the first component, the second component, and the third component based on the brightness average, the first color average, the second color average, and the third color average may include: determining the ratio between the brightness average and the first color average as the first component; determining the ratio between the brightness average and the second color average as the second component; and determining the ratio between the brightness average and the third color average as the third component.

[0225] For example, the average color value of all reference white pixels (R w , G w , Bw ). Calculate the first scale factor (R) according to formula (3) scale , G scale , B scale ).

[0226]

[0227] Among them, Y w is the average brightness value of the reference white pixels, that is, the value obtained by averaging the brightness values ​​of all reference white pixels.

[0228] In step S4014, a second scale factor is determined according to the color value of each pixel in the image frame.

[0229] In some embodiments, the first color value of the first color channel corresponding to each pixel in the image frame is summed, and the ratio between the summed result and the number of pixels in the image frame is determined as the fourth color average value (R aver ); summing the second color value of the second color channel corresponding to each pixel in the image frame, and determining the ratio between the summation result and the number of pixels in the image frame as the fifth color average value (G aver ); summing the third color value of the third color channel corresponding to each pixel in the image frame, and determining the ratio between the summation result and the number of pixels in the image frame as the sixth color average value (B aver ); the average value of the fourth color average, the fifth color average and the sixth color average is determined as the total color average (Gray); the fourth component (R GWA ), the fifth component (G GWA ) and the sixth component (B GWA ), and combining the fourth, fifth, and sixth components into a second scale factor.

[0230] As an example, determining the fourth component, the fifth component and the sixth component based on the total color average, the fourth color average, the fifth color average and the sixth color average may include: determining the ratio between the total color average and the fourth color average as the fourth component; determining the ratio between the total color average and the fifth color average as the fifth component; and determining the ratio between the total color average and the sixth color average as the sixth component.

[0231] For example, the second scale factor (R GWA , G GWA , B GWA ).

[0232]

[0233] Gray=(R aver +G aver +B aver ) / 3, and (R aver , G aver , B aver ) is the original data of each pixel in the image in the RGB color space (R org , G org , B org ), namely the fourth color average, the fifth color average and the sixth color average mentioned above.

[0234] In step S4015 , a first color cast correction process is performed on the image frame according to the first scale factor and the second scale factor.

[0235] In some embodiments, according to the first chrominance component (d a ) and the second chrominance component (d b ), determine the color to which the image frame leans; when the image frame leans toward red, use the fourth component of the second scale factor, the second component of the first scale factor, and the third component of the first scale factor as gains to perform gain processing on the color value of each pixel in the image frame; when the image frame leans toward blue, use the first component of the first scale factor, the second component of the first scale factor, and the sixth component of the second scale factor as gains to perform gain processing on the color value of each pixel in the image frame; when the image frame leans toward green, use the first component of the first scale factor, the fifth component of the second scale factor, and the third component of the first scale factor as gains to perform gain processing on the color value of each pixel in the image frame; when the image frame leans toward yellow, use the fourth component of the second scale factor, the fifth component of the second scale factor, and the third component of the first scale factor as gains to perform gain processing on the color value of each pixel in the image frame.

[0236] For example, when the image is reddish, (R GWA , G scale , B scale ) is used as the gain for gain calculation. When the image is blue, (R scale , G scale , B GWA ) is used as the gain for gain calculation. When the image is greenish, (R scale , G GWA , B scale ) is used as the gain for gain calculation. When the image is yellowish, (R GWA , G GWA , B scale ) is used as the gain for gain calculation.

[0237] For example, when the image is reddish, the color value (R org , G org , B o rg ), with (R GWA , G scale , B scale ) is used as the gain to calculate the gain. After calculation, the color value of each pixel is (R GWA ×R org , G scale ×G org , B scale ×B org ), and so on.

[0238] As an example, determining the color toward which the image frame is biased based on the first chromaticity component and the second chromaticity component of the image frame may include: when the first chromaticity component is greater than a fourth preset value, determining that the image frame is biased toward red; when the first chromaticity component is not greater than the fourth preset value, determining that the image frame is biased toward green; when the second chromaticity component is greater than the fifth preset value, determining that the image frame is biased toward yellow; and when the second chromaticity component is not greater than the fifth preset value, determining that the image frame is biased toward blue.

[0239] Taking the fourth preset value of 0 and the fifth preset value of 0 as an example, when the first chromaticity component is greater than 0, the image frame is determined to be biased towards red; when the first chromaticity component is not greater than 0, the image frame is determined to be biased towards green; when the second chromaticity component is greater than 0, the image frame is determined to be biased towards yellow; when the second chromaticity component is not greater than 0, the image frame is determined to be biased towards blue.

[0240] The embodiment of the present application takes into account that the models of video acquisition devices used by the anchor side are different and the color cast of the captured videos are also different. Therefore, the video is subjected to a first color cast correction process based on the color value of each pixel in the video. The original video sent by the anchor side can be corrected to a video with a unified color cast standard. This not only reduces the computational complexity of the subsequent second color cast correction process, but also improves the accuracy of the color cast correction.

[0241] In step S402, screen color cast information of the viewer terminal is obtained, and a second color cast correction process is performed on the image frame after the first color cast correction process according to the screen color cast information.

[0242] In some embodiments, a third scale factor corresponding to the screen color cast information is queried; and gain processing is performed on the color value of each pixel in the image frame after the first color cast correction processing using the third scale factor as gain.

[0243] As an example, multiple candidate screen color cast information and corresponding candidate third scale factors are collected, and a first mapping table is created based on the multiple candidate screen color cast information and the corresponding candidate third scale factors. The third scale factor corresponding to the screen color cast information is queried in the first mapping table. The third scale factor is used as a gain to perform gain processing on the color value of each pixel in the image frame after the first color cast correction processing.

[0244] For example, when the third scale factor corresponding to the screen color cast information is (R a , G a , B a ), for the color value (R org , G org , B org ), with (R a , G a , B a ) is used as the gain to calculate the gain. After calculation, the color value of each pixel is (R a ×R org , G a ×G org , B a ×B org ).

[0245] The embodiment of the present application takes into account that the models of terminals used on the audience side are different and the color cast of the videos played are also different. Therefore, according to the screen color cast information of the audience terminal, the video with a unified color cast standard after the first color cast correction processing is subjected to a second color cast correction processing, which can improve the accuracy of the color cast correction.

[0246] In some embodiments, before step S402, it may also include: when at least one of the following conditions is met, determining that an operation of obtaining the screen color cast information of the viewer terminal will be executed: receiving a color cast correction operation submitted by a login account; the terminal does not have a privacy protection function turned on, wherein the privacy protection function is used to shield the device information of the reading terminal.

[0247] For example, a color cast correction operation submitted by a logged-in account can trigger a second color cast correction entry. This allows the second color cast correction to be performed on the video based on the viewer's instructions, reducing unnecessary computational overhead. When the viewer's terminal has privacy protection enabled, the client is prohibited from accessing the terminal's device information. This prevents the client from obtaining the terminal's screen color cast information, thereby achieving privacy protection.

[0248] In some embodiments, as an alternative to step S401 and step S402, the method may also include: obtaining screen color cast information of the viewer terminal, and querying a fourth scale factor corresponding to the screen color cast information; performing the following processing on each image frame in the video: using the fourth scale factor as a gain, performing gain processing on the color value of each pixel in the image frame.

[0249] As an example, multiple candidate screen color cast information and corresponding candidate fourth scale factors are collected, and a second mapping table is created based on the multiple candidate screen color cast information and the corresponding candidate fourth scale factors. The fourth scale factor corresponding to the screen color cast information is searched in the second mapping table. The following processing is performed on each image frame in the video: the color value of each pixel in the image frame is gain-processed using the fourth scale factor as a gain.

[0250] For example, when the fourth scale factor corresponding to the screen color cast information is (R b , G b , B b ), for the color value (R org , G org , B org ), with (R b , G b , B b ) is used as the gain to calculate the gain. After calculation, the color value of each pixel is (R b ×R org , G b ×G org , B b ×B org ).

[0251] In step S105 , the video after the color cast correction process is played.

[0252] In some embodiments, step S105 may include: playing the video after color cast correction processing in the first area of ​​the human-computer interaction interface; when executing step S105, it may include: playing the video before color cast correction processing in the second area of ​​the human-computer interaction interface; in response to the area switching operation, swapping the videos played in the first area and the second area, that is, switching the video after color cast correction processing played in the first area to playing the video before color cast correction processing, and switching the video before color cast correction processing played in the second area to playing the video after color cast correction processing.

[0253] For example, the first area and the second area can be in a side-by-side mode or a picture-in-picture mode. For example, the second area is located within the first area, and the area of ​​the first area is larger than that of the second area. Figure 9In the live broadcast page, the image 905 after color cast correction and the image 904 before color cast correction are simultaneously displayed, wherein the image 904 before color cast correction is located within the image 905 after color cast correction. The area switching operation can be an operation of triggering the second area. That is, when the viewer triggers the image 904 before color cast correction, the videos played in the first area and the second area can be swapped.

[0254] As an example, when playing a video before color cast correction processing in the second area of ​​the human-computer interaction interface, it can include: displaying a closing entrance of the second area in the human-computer interaction interface; in response to a trigger operation for the closing entrance, automatically stopping the video playing in the second area, and hiding (for example, closing or minimizing) the second area.

[0255] For example, Figure 13 When the viewer triggers the closing entrance 131, the video played in the second area 132 is automatically stopped and the second area 132 is closed, thereby saving the display resources of the terminal.

[0256] As another example, when playing a video before color cast correction processing in the second area of ​​the human-computer interaction interface, it can include: when no area switching operation is received within a preset waiting time, automatically stopping the video playing in the second area and hiding (for example, closing or minimizing) the second area.

[0257] For example, the preset waiting time may be a default value, or a value set by the viewer, the client, or the server.

[0258] The embodiment of the present application simultaneously displays the video before and after color cast correction on the live broadcast page, and supports viewers to switch displays, which can meet the diverse needs of viewers, reduce the audience's operation path, and thus save terminal operation resources.

[0259] In some embodiments, step S105 may include automatically replacing the live broadcast video before the color cast correction process with the video after the color cast correction process. In this way, when the video after the color cast correction process is played, the video before the color cast correction process is automatically turned off, thereby saving display resources of the terminal and communication resources between the server and the terminal.

[0260] In some embodiments, see Figure 6 , Figure 6 This is a flow chart of the live broadcast processing method provided by the embodiment of the present application, based on Figure 3 , step S106 and step S107 may also be included before step S103.

[0261] In step S106, color cast detection processing is performed on the video in the live broadcast room to obtain a color cast detection result.

[0262] In some embodiments, the client can call a corresponding service (e.g., a color cast detection service) on the viewer's terminal to perform color cast detection on the video. The client can also call a corresponding service (e.g., a color cast detection service) on the server to perform color cast detection on the video.

[0263] As an example, when the client calls the corresponding service of the server (for example, the color cast detection service) to perform color cast detection on the video, the replacement step of step S106 is: the server performs color cast detection on the video in the live broadcast room, obtains the color cast detection result, and sends the color cast detection result to the viewer terminal.

[0264] The following describes an example of a client invoking a corresponding service (e.g., a color cast detection service) on a viewer's terminal to perform color cast detection on a video. It should be noted that the process of the client invoking a corresponding service (e.g., a color cast detection service) on a server to perform color cast detection on a video is similar to the following and will not be further described.

[0265] In some embodiments, see Figure 7 , Figure 7 This is a flow chart of the live broadcast processing method provided by the embodiment of the present application, based on Figure 6 , step S106 may include steps S1061 to S1063.

[0266] In step S1061 , a plurality of image frames are extracted from the video.

[0267] In some embodiments, each image frame may be extracted from the video, or a plurality of video frames may be extracted at intervals of a fixed number of frames in the video.

[0268] In step S1062 , the following processing is performed for each image frame: the average chromaticity (D) and chromaticity center distance (H) of the image frame are determined, and the ratio between the average chromaticity and the chromaticity center distance is determined as the color cast factor (K) of the image frame.

[0269] In some embodiments, determining the average chromaticity and chromaticity center distance of an image frame may include: determining the color value of each pixel in the image frame in a red, green, and blue color space (e.g., an RGB color space); converting the color value of each pixel in the red, green, and blue color space into a first color-opponent dimension (a) and a second color-opponent dimension (b) in a color-opponent space (e.g., a Lab space); summing the first color-opponent dimension of each pixel, and determining the ratio between the summed value and the number of pixels in the image frame as the first chromaticity component (d a); performing summation on the second color opposite dimension of each pixel, and determining the ratio between the summation result and the number of pixels in the image frame as the second chromaticity component (d b ); determining the average chroma of the image frame according to the first chroma component and the second chroma component; determining the first chroma center distance component (M) according to the first color opposition dimension and the second color opposition dimension of each pixel point a ) and the second chromaticity center distance component (M b ); determining the chromaticity center distance of the image frame according to the first chromaticity center distance component and the second chromaticity center distance component.

[0270] As an example, determining the average chroma of an image frame based on a first chroma component and a second chroma component may include: squaring the first chroma component to obtain a first chroma result; squaring the second chroma component to obtain a second chroma result; adding the first chroma result and the second chroma result, and taking the square root of the sum to obtain the average chroma of the image frame.

[0271] Taking the power of quadratic as an example, the first chromaticity component d is determined according to formula (5) and formula (6): a and the second chrominance component d b :

[0272]

[0273] Where M and N are the width and height of the image frame, respectively, in pixels. a and b are the two color channels of the pixel points of the image frame in Lab space, namely the first color opposition dimension and the second color opposition dimension mentioned above. The average chromaticity D of the image frame is determined according to formula (7):

[0274]

[0275] As an example, determining the first chromaticity center distance component and the second chromaticity center distance component according to the first color opposition dimension and the second color opposition dimension of each pixel point may include: performing the following processing for each pixel point: converting the first color opposition dimension (a) and the first chromaticity component (d a ) is subtracted, and the subtraction result is squared to obtain the first chromaticity distance, and the second color opposition dimension (b) and the second chromaticity component (d b ) is subtracted, and the subtraction result is squared to obtain the second chromaticity distance; the first chromaticity distance of each pixel is summed, and the ratio between the summed result and the number of pixels in the image frame is determined as the first chromaticity center distance component (M a); summing the second chromaticity distance of each pixel, and determining the ratio between the summation result and the number of pixels in the image frame as the second chromaticity center distance component (M b ).

[0276] As an example, determining the chromaticity center distance of an image frame based on the first chromaticity center distance component and the second chromaticity center distance component may include: squaring the first chromaticity center distance component to obtain a third chromaticity center distance result; squaring the first chromaticity center distance component to obtain a fourth chromaticity center distance result; and adding the third chromaticity center distance result and the fourth chromaticity center distance result to obtain the chromaticity center distance of the image frame.

[0277] Taking the quadratic power as an example, the first chromaticity center distance component M is determined according to formula (8) and formula (9): a and the second chromaticity center distance component M b :

[0278]

[0279]

[0280] Determine the chromaticity center distance H of the image frame according to formula (10):

[0281]

[0282] The color cast factor K of the image frame is determined according to formula (11):

[0283]

[0284] In step S1063 , a color cast detection result is determined according to the color cast factor of each image frame.

[0285] In some embodiments, among multiple image frames, image frames whose color cast factors are greater than a color cast factor threshold are determined to be color cast image frames; when the ratio between the number of color cast image frames and the number of multiple image frames is greater than a color cast number threshold, it is determined that the video has color cast; when the ratio between the number of color cast image frames and the number of multiple image frames is not greater than the color cast number threshold, it is determined that the video has no color cast.

[0286] As an example, the color cast factor threshold and the color cast amount threshold can be default values ​​or values ​​set by the viewer, client, or server. The larger the value of the color cast factor, the greater the degree of color cast of the image frame. When the color cast factor is greater than the color cast factor threshold, it can be determined that the image frame has color cast. In this way, it can be determined that the number of image frames with color cast among the E image frames extracted from the video is e. When e / E is greater than the color cast amount threshold, it is determined that the original video captured by the host has color cast.

[0287] In some embodiments, before step S106, the following may also be included: performing scene detection processing on the video of the live broadcast room, and when a color display sensitive scene appears in the video of the live broadcast room, determining to perform color cast detection processing on the video of the live broadcast room.

[0288] As an alternative to this embodiment, the video in the live broadcast room can be continuously subjected to color cast detection during the live broadcast, or the video in the live broadcast room can be subjected to color cast detection only after the viewer turns on the color cast detection function. For example, when the viewer triggers the color cast detection entrance in the human-computer interaction interface, the video in the live broadcast room can be subjected to color cast detection.

[0289] As an example, the types of color display sensitive scenarios include: recommendation scenarios for multiple items of different colors, recommendation scenarios for at least one item of a specific color, and recommendation scenarios for items that meet the preferences of the logged-in account.

[0290] For example, the live broadcast schedule can be used to determine whether color-sensitive scenes will appear in the video. For example, the live broadcast schedule will preview recommended items and the recommended time of the corresponding items, so as to determine the time of appearance of color-sensitive scenes. The audio content of the live broadcast room can also be recognized and processed to determine whether color-sensitive scenes will appear in the video of the live broadcast room. Items that meet the preferences of the logged-in account can be items whose similarity with the items in the logged-in account's historical purchase, historical attention, or historical collection is higher than the similarity threshold (which can be a default value or a value set by the viewer, client, or server).

[0291] Taking live streaming as an example, when the host recommends a dress, the dress has high requirements for the displayed color, because the color of the dress directly determines whether the audience will buy it. In this case, the live broadcast screen can be detected for color cast. However, when the host recommends a mobile hard drive, the color of the mobile hard drive does not affect the audience's purchasing experience. In this case, there is no need to detect the color cast of the live broadcast screen, thus saving computing resources.

[0292] In some embodiments, before step S106, it may also include: displaying a color cast detection entrance in the human-computer interaction interface; and in response to a trigger operation on the color cast detection entrance, determining to perform a color cast detection operation on the video in the live broadcast room.

[0293] As an example, Figure 14 In the live broadcast page, a color cast detection entry 141 is displayed. When a viewer triggers the color cast detection entry 141, a color cast detection is performed on the video in the live broadcast room. When a color cast is detected in the video, a prompt message is displayed. In this way, the color cast detection function is only performed on the video in the live broadcast room when the viewer turns on the color cast detection function, which can save color cast detection resources.

[0294] In step S107 , when the color cast detection result indicates that the video has color cast, information indicating that the video has color cast is displayed in the human-computer interaction interface, and it is determined that an operation of displaying a color cast correction entry in the human-computer interaction interface will be executed.

[0295] In some embodiments, before displaying information in the human-computer interaction interface to prompt that the video has color cast, it may include: when the color cast detection result indicates that the color cast degree of the video exceeds a color cast degree threshold (which can be a default value or a value set by the viewer, anchor, client or server), determining to execute an operation of displaying information in the human-computer interaction interface to prompt that the video has color cast.

[0296] As an example, the degree of color cast in a video can be determined based on the color cast factors of the image frames in the above embodiment. Specifically, the color cast factor of each image frame extracted from the video is determined, the color cast factors of each image frame are summed, and the ratio of the summed result to the number of extracted image frames is used to determine the degree of color cast in the video. In this way, a message indicating the presence of color cast in the video is displayed only when the degree of color cast in the video is high, which can save terminal display resources.

[0297] Below, the live broadcast processing method provided by the embodiment of the present application is described by taking the collaborative execution of the viewer terminal, the anchor terminal and the server as an example. Figure 8 , Figure 8 This is a flow chart of the live broadcast processing method provided by the embodiment of the present application, which will be combined with Figure 8 The steps shown are explained.

[0298] In step S801, the anchor terminal sends the video of the live broadcast room to the server.

[0299] In step S802, the viewer terminal receives the live broadcast room video sent by the server and plays the live broadcast room video in the human-computer interaction interface.

[0300] In step S803, the server performs color cast detection on the video stream of the live broadcast room to obtain a color cast detection result.

[0301] In step S804, when the color cast detection result indicates that the video has color cast, information indicating that the video has color cast is sent to the viewer terminal, and a color cast correction entry is displayed on the viewer terminal.

[0302] In step S805 , the viewer terminal sends a color cast correction instruction to the server in response to the triggering operation on the color cast correction entry.

[0303] In step S806 , the server performs a first color cast correction process on the video according to the color cast correction instruction.

[0304] In step S807, the viewer terminal sends the screen color cast information to the server.

[0305] In step S808 , the server performs a second color cast correction on the video after the first color cast correction according to the screen color cast information, and sends the video after the second color cast correction to the viewer terminal.

[0306] In step S809 , the viewer terminal plays the video after the second color cast correction process.

[0307] It should be noted that the specific implementation of steps S801 to S809 is similar to that of the above embodiment and will not be described in detail here.

[0308] In the embodiment of the present application, after the viewer triggers the color cast correction entrance, the video in the live broadcast room is subjected to color cast correction processing, and the video after color cast correction processing is played. This can ensure the accuracy of the video display color during the live broadcast process, improve the visual perception effect of the video, thereby improving the viewer's live broadcast viewing experience, and further saving computing resources and communication resources used for the live broadcast.

[0309] Below, a specific application scenario is used as an example to illustrate the live broadcast processing method provided in the embodiment of the present application.

[0310] In an embodiment of the present application, when the live broadcast system detects that the live broadcast source video of the anchor has color cast, it will actively remind the audience of the color cast and provide an automatic color correction and restoration function. When the viewer end turns on the color correction (i.e., the above-mentioned color cast correction) function, the live broadcast platform performs a first color correction process on the live broadcast source video of the anchor (i.e., the above-mentioned first color cast correction process), and at the same time, the viewer end reports the device information used to play the live broadcast (i.e., the above-mentioned terminal screen color cast information). The live broadcast platform performs a second color correction process on the live broadcast video based on the device information (i.e., the above-mentioned second color cast correction process), and transmits the video image after the second color correction process to the viewer end.

[0311] The following describes how the live broadcast processing method provided in the embodiment of the present application is performed in the terminal.

[0312] The embodiment of the present application provides a button for the color correction and restoration function (i.e., the color cast correction entrance mentioned above). At the same time, when the live broadcast platform detects that the live broadcast source video of the anchor has color cast, it will actively prompt the audience end that the anchor video has color cast, and support the audience to choose to turn on the color correction and restoration function. When the audience clicks the button for the color correction and restoration function, the color correction and restoration function is turned on, and the background server performs the first color cast correction on the live broadcast source video of the anchor. At the same time, the background server obtains the device information uploaded by the audience end, and performs a second color correction on the live broadcast video after the first color correction according to the device information. The background server pushes the picture after the second color correction and the original picture with color cast to the audience end at the same time, and displays them for comparison on the audience end. The audience can manually switch between the "original picture" and the "picture after color restoration".

[0313] For example, see Figure 9 , Figure 9 This is a schematic diagram of an application scenario of the live broadcast processing method provided in an embodiment of the present application. Figure 9 In the process of watching the live broadcast, when the live broadcast system detects that the live broadcast source video screen of the anchor has color cast, the first prompt information 901 and the color cast correction entrance 902 are displayed to prompt the audience to turn on the color correction and restoration function; when the audience triggers the color cast correction entrance 902, the color correction and restoration function can be turned on, the second prompt information 903 is displayed, and the color cast correction processing is performed on the background server; when the color cast correction processing is completed, the picture after the color cast correction processing 905 and the picture before the color cast correction processing 904 can be displayed on the live broadcast page, and the picture before the color cast correction processing 904 can be displayed in the small window in the upper left corner of the live broadcast page. The audience can manually switch between the picture after the color cast correction processing 905 and the picture before the color cast correction processing 904.

[0314] The following will be combined Figure 10 The specific implementation of the live broadcast processing method provided in the embodiment of the present application is described. Figure 10 It is a schematic diagram of the principle of the live broadcast processing method provided in an embodiment of the present application.

[0315] In step S110, the anchor terminal collects live video and sends the live video to the backend server.

[0316] In step S120, the backend server performs color cast detection on the live video, and sends a prompt message to the viewer terminal when color cast exists in the video.

[0317] In some embodiments, the backend server performs frame extraction detection (the number of extracted frames is E) on the video captured by the camera on the anchor side at regular intervals (the duration can be set, because live broadcasts generally do not change the background and light frequently, so under the condition of limited computing power, the duration can be set), and uses the equivalent circle method to perform color cast detection on each image (or image frame) obtained by the frame extraction. Specifically, the image is first converted from RGB space to Lab space, for example, the image is first converted from RGB space to XYZ space, and then converted from XYZ space to Lab space; then the ratio between the average chromaticity D of the image and the chromaticity center distance H, that is, the color cast factor K is used to measure the degree of color cast of the image.

[0318] As an example, the calculation method of the color cast factor K is shown in formulas (12)-(18).

[0319]

[0320]

[0321] Where M and N are the width and height of the image, in pixels. a and b are the two color channels of the image pixel in Lab space. On the ab chromaticity plane, the center coordinates of the equivalent circle are (d a , d b ), radius H = M a 2 +M b 2 , the distance from the center of the equivalent circle to the origin of the neutral axis of the ab chromaticity plane (a=0, b=0) The color cast of the entire image is determined by the specific position ab of the equivalent circle on the ab chromaticity plane. a When d > 0, the image is reddish, otherwise the image is greenish. b When K is greater than 0, the image has a yellowish cast; otherwise, the image has a bluish cast. The larger the value of the color cast factor, K, the greater the degree of color cast. A color cast factor threshold is pre-set (typically set to 1). When the calculated K is greater than the color cast factor threshold, the image is considered to have a color cast. In this way, we can determine that the number of images with color cast in E frames extracted from the video is e. When e / E is greater than the set value, it is determined that the original video captured by the host has a color cast.

[0322] In step S130 , when the viewer terminal turns on the color correction and restoration function, the backend server performs a first color correction process on the video with color cast.

[0323] In some embodiments, when a viewer clicks the color correction and restoration function button, the color correction and restoration function is enabled, and the backend server corrects the color cast of the host's original video. Specifically, each frame of the video is processed as follows:

[0324] In the RGB color space, the original data (R org , G org , B org ), perform histogram equalization on the data of the three channels R, G, and B respectively, and then convert the image from RGB color space to YCrCb color space to obtain the histogram equalization data of each pixel in the image (Y Hist , Cr Hist , Cb Hist ), and finally find all white pixels that satisfy formula (19) from all pixels of the image:

[0325]

[0326] Among the white pixels that satisfy formula (19), the one with the largest Y Hist The value and the Cr closest to zero Hist , Cb Hist Find the pixel with the highest brightness (Y Hist bright , Cr Hist bright , Cb Hist bright ). At the same time, calculate the average value (Y Hist avg , Cr Hist avg , Cb Hist avg ). Find all pixels that satisfy formula (20) from all pixels of the image:

[0327]

[0328] Among them, Y L and Y H Selected from Y Hist bright and Y Hist avg The minimum and maximum values ​​between Cr L Cr H They are selected from Cr Hist bright and Cr Hist avg The minimum and maximum values ​​between Cb L and Cb H Selected from Cb Hist bright and Cb Hist avgSelect the pixels that satisfy formula (20) from the image as reference white pixels, and calculate the average value of all reference white pixels (R w , G w , B w ).

[0329] According to formula (21), the first scale factor (R scale , G scale , B scale ).

[0330]

[0331] Among them, Y w is the average brightness value of the reference white pixels, that is, the value obtained by averaging the brightness values ​​of all reference white pixels.

[0332] According to formula (22), the second scale factor (R GWA , G GWA , B GWA ).

[0333]

[0334] Gray=(R aver +G aver +B aver ) / 3, and (R aver , G aver , B aver ) is the original data of each pixel in the image in the RGB color space (R org , G org , B org ) is the average value of .

[0335] When the image is reddish, (R GWA , G scale , B scale ) is used as the scale factor to calculate the gain. When the image is blue, (R scale , G scale , B GWA ) is used as the scale factor to calculate the gain. When the image is green, (R scale , G GWA , B scale ) is used as the scale factor to calculate the gain. When the image is yellowish, (R GWA , G GWA , B scale ) is the scale factor for gain calculation.

[0336] In step S140 , the backend server obtains device information of the viewer terminal, and performs a second color correction process on the video after the first color correction process according to the device information of the viewer terminal.

[0337] In some embodiments, the screen color cast and adjustment values ​​of mainstream mobile devices on the market can be collected in advance, and a second color correction process can be performed when streaming the video to the viewer's terminal to eliminate the impact of different mobile devices. For example, if the viewer's mobile phone model is found to have a reddish screen, the proportion of the R channel in the video stream can be reduced during streaming.

[0338] In step S150 , the video after the second color correction process is pushed to the viewer terminal.

[0339] Live streaming is the mainstream form of displaying items in online e-commerce. The embodiments of the present application restore the colors of items in the live streaming room, making it easier for viewers to quickly identify the true colors of items, and to achieve "seeing is believing" in the live streaming room, thereby reducing the losses caused by returns due to color differences.

[0340] The following combination Figure 2 The live broadcast processing device provided in the embodiment of the present application is described as an exemplary structure of a software module.

[0341] In some embodiments, as Figure 2 As shown, the software modules stored in the live broadcast processing device 455 of the memory 450 may include: a receiving module 4551, used to receive the video of the live broadcast room; a display module 4552, used to play the video of the live broadcast room in the human-computer interaction interface, and display the color cast correction entrance in the human-computer interaction interface; a correction module 4553, used to respond to the trigger operation of the color cast correction entrance, perform color cast correction on the video, and play the video after color cast correction.

[0342] In the above scheme, the correction module 4553 is also used to play the video after color cast correction processing in the first area of ​​the human-computer interaction interface; play the video before color cast correction processing in the second area of ​​the human-computer interaction interface; and exchange the videos played in the first area and the second area in response to the area switching operation.

[0343] In the above solution, the correction module 4553 is further used to display a closing entrance of the second area in the human-computer interaction interface; in response to a triggering operation for the closing entrance, the video played in the second area is automatically stopped, and the second area is hidden.

[0344] In the above solution, the correction module 4553 is further configured to automatically stop the video played in the second area and hide the second area when no area switching operation is received within a preset waiting time.

[0345] In the above solution, the correction module 4553 is also used to automatically replace the video in the live broadcast room before the color cast correction processing with the video after the color cast correction processing.

[0346] In the above scheme, the correction module 4553 is also used to display the color cast correction parameter setting page; in response to the color cast correction parameter configuration operation received on the color cast correction parameter setting page, the color cast correction parameters configured by the login account are obtained, wherein the login account is the account for logging into the live broadcast room; according to the color cast correction parameters configured by the login account, the video is subjected to color cast correction processing.

[0347] In the above scheme, the display module 4552 is also used to obtain the color cast tolerance of the login account, where the login account is the account that logs into the live broadcast room; when the color cast of the video exceeds the color cast tolerance, the color cast correction entrance is automatically triggered.

[0348] In the above scheme, the correction module 4553 is also used to perform the following processing on each image frame in the video: performing a first color cast correction processing on the image frame according to the color value of each pixel in the image frame; obtaining the screen color cast information of the terminal, and performing a second color cast correction processing on the image frame after the first color cast correction processing according to the screen color cast information.

[0349] In the above scheme, the correction module 4553 is also used to determine that an operation of obtaining the screen color cast information of the terminal will be executed when at least one of the following conditions is met: a color cast correction operation submitted by a login account is received, wherein the login account is an account that logs in to the live broadcast room; the terminal does not have a privacy protection function turned on, wherein the privacy protection function is used to shield the device information of the terminal from being read.

[0350] In the above scheme, the correction module 4553 is further used to perform the following processing on each pixel point in the image frame: determining multiple color values ​​corresponding one-to-one to multiple color channels in the pixel point, performing equalization processing on the multiple color values, and obtaining multiple equalization data corresponding one-to-one to the multiple color values; determining multiple reference white pixel points in the image frame based on the multiple equalization data corresponding to each pixel point in the image frame; determining a first scale factor based on the brightness values ​​and color values ​​of the multiple reference white pixel points; determining a second scale factor based on the color value of each pixel point in the image frame; and performing a first color cast correction processing on the image frame based on the first scale factor and the second scale factor.

[0351] In the above scheme, the correction module 4553 is also used to perform the following processing on each pixel point in the image frame: determine the first equalization data corresponding to the first color channel, the second equalization data corresponding to the second color channel, and the third equalization data corresponding to the third color channel among the multiple equalization data corresponding to the pixel point; select the pixel point that meets the following conditions at the same time in the image frame as the white pixel point: the first equalization data of the pixel point is greater than or equal to the first preset value, the second equalization data of the pixel point is greater than or equal to the second preset value, and the third equalization data of the pixel point is less than or equal to the third preset value; select the pixel point with the largest brightness value among the multiple white pixel points as the brightness white pixel point; and determine multiple reference white pixel points in the image frame according to the first equalization data, the second equalization data and the third equalization data corresponding to the brightness white pixel point.

[0352] In the above scheme, the correction module 4553 is further used to determine the first equalization data corresponding to the brightness white pixel as the first brightness equalization data, determine the second equalization data corresponding to the brightness white pixel as the second brightness equalization data, and determine the third equalization data corresponding to the brightness white pixel as the third brightness equalization data; sum the first equalization data corresponding to each white pixel, and determine the ratio between the summation result and the number of white pixels as the first equalization average data; sum the second equalization data corresponding to each white pixel, and determine the ratio between the summation result and the number of white pixels as the first equalization average data. The ratio between the first and second equalized average data is determined as the second equalized average data; the third equalized data corresponding to each white pixel is summed, and the ratio between the summed result and the number of white pixels is determined as the third equalized average data; and pixels that simultaneously meet the following conditions are selected in the image frame as reference white pixels: the first equalized data of the pixel is between the first brightness equalized data and the first equalized average data, the second equalized data of the pixel is between the second brightness equalized data and the second equalized average data, and the third equalized data of the pixel is between the third brightness equalized data and the third equalized average data.

[0353] In the above scheme, the correction module 4553 is also used to sum the brightness values ​​of each reference white pixel and determine the ratio between the summation result and the number of reference white pixels as the brightness average; determine the first color value of the first color channel corresponding to each reference white pixel, the second color value of the second color channel corresponding to the second color channel, and the third color value of the third color channel corresponding to the third color channel; sum the first color values ​​corresponding to each reference white pixel and determine the ratio between the summation result and the number of reference white pixels as the first color average; sum the second color values ​​corresponding to each reference white pixel and determine the ratio between the summation result and the number of reference white pixels as the second color average; sum the third color values ​​corresponding to each reference white pixel and determine the ratio between the summation result and the number of reference white pixels as the third color average; determine the first component, the second component and the third component based on the brightness average, the first color average, the second color average and the third color average, and combine the first component, the second component and the third component into a first scale factor.

[0354] In the above scheme, the correction module 4553 is further used to determine the ratio between the brightness average and the first color average as the first component; determine the ratio between the brightness average and the second color average as the second component; and determine the ratio between the brightness average and the third color average as the third component.

[0355] In the above scheme, the correction module 4553 is further used to sum the first color values ​​of the first color channel corresponding to each pixel in the image frame, and determine the ratio between the summation result and the number of pixels in the image frame as the fourth color average; sum the second color values ​​of the second color channel corresponding to each pixel in the image frame, and determine the ratio between the summation result and the number of pixels in the image frame as the fifth color average; sum the third color values ​​of the third color channel corresponding to each pixel in the image frame, and determine the ratio between the summation result and the number of pixels in the image frame as the sixth color average; determine the average value of the fourth color average, the fifth color average and the sixth color average as the total color average; determine the fourth component, the fifth component and the sixth component based on the total color average, the fourth color average, the fifth color average and the sixth color average, and combine the fourth component, the fifth component and the sixth component into a second scale factor.

[0356] In the above scheme, the correction module 4553 is further used to determine the ratio between the total color average value and the fourth color average value as the fourth component; determine the ratio between the total color average value and the fifth color average value as the fifth component; and determine the ratio between the total color average value and the sixth color average value as the sixth component.

[0357] In the above solution, the correction module 4553 is further configured to determine the color to which the image frame is biased based on the first chroma component and the second chroma component of the image frame; when the image frame is biased toward red, the fourth component of the second scale factor, the second component of the first scale factor, and the third component of the first scale factor are used as gains to perform gain processing on the color value of each pixel in the image frame; when the image frame is biased toward blue, the first component of the first scale factor, the second component of the first scale factor, and the sixth component of the second scale factor are used as gains to perform gain processing on the color value of each pixel in the image frame; when the image frame is biased toward green, the first component of the first scale factor, the fifth component of the second scale factor, and the third component of the first scale factor are used as gains to perform gain processing on the color value of each pixel in the image frame; and when the image frame is biased toward yellow, the fourth component of the second scale factor, the fifth component of the second scale factor, and the third component of the first scale factor are used as gains to perform gain processing on the color value of each pixel in the image frame.

[0358] In the above scheme, the correction module 4553 is also used to determine that the image frame is biased towards red when the first chromaticity component is greater than the fourth preset value; determine that the image frame is biased towards green when the first chromaticity component is not greater than the fourth preset value; determine that the image frame is biased towards yellow when the second chromaticity component is greater than the fifth preset value; and determine that the image frame is biased towards blue when the second chromaticity component is not greater than the fifth preset value.

[0359] In the above solution, the correction module 4553 is further configured to query a third scale factor corresponding to the screen color cast information; and perform gain processing on the color value of each pixel in the image frame after the first color cast correction processing using the third scale factor as gain.

[0360] In the above scheme, the correction module 4553 is further used to obtain the screen color cast information of the terminal and query the fourth scale factor corresponding to the screen color cast information; and perform the following processing on each image frame in the video: using the fourth scale factor as the gain, gain processing is performed on the color value of each pixel in the image frame.

[0361] In the above scheme, the display module 4552 is also used to perform color cast detection processing on the video in the live broadcast room to obtain a color cast detection result; when the color cast detection result indicates that the video has color cast, information for prompting that the video has color cast is displayed in the human-computer interaction interface, and it is determined that an operation of displaying a color cast correction entrance in the human-computer interaction interface will be executed.

[0362] In the above scheme, the display module 4552 is also used to extract multiple image frames from the video; perform the following processing on each image frame: determine the average chromaticity and chromaticity center distance of the image frame, and determine the ratio between the average chromaticity and the chromaticity center distance as the color cast factor of the image frame; determine the color cast detection result of the video based on the color cast factor of each image frame.

[0363] In the above scheme, the display module 4552 is further used to determine, among multiple image frames, an image frame whose color cast factor is greater than a color cast factor threshold value as a color cast image frame; when the ratio between the number of color cast image frames and the number of multiple image frames is greater than the color cast number threshold value, it is determined that the video has color cast; when the ratio between the number of color cast image frames and the number of multiple image frames is not greater than the color cast number threshold value, it is determined that the video has no color cast.

[0364] In the above scheme, the display module 4552 is further used to determine the color value of each pixel in the image frame in the red, green and blue color space; convert the color value of each pixel in the red, green and blue color space into a first color-opponent dimension and a second color-opponent dimension in the color-opponent space; sum the first color-opponent dimension of each pixel, and determine the ratio between the summation result and the number of pixels in the image frame as a first chromaticity component; sum the second color-opponent dimension of each pixel, and determine the ratio between the summation result and the number of pixels in the image frame as a second chromaticity component; determine the average chromaticity of the image frame based on the first chromaticity component and the second chromaticity component; determine the first chromaticity center distance component and the second chromaticity center distance component based on the first color-opponent dimension and the second color-opponent dimension of each pixel; and determine the chromaticity center distance of the image frame based on the first chromaticity center distance component and the second chromaticity center distance component.

[0365] In the above scheme, the display module 4552 is also used to perform a square process on the first chromaticity component to obtain a first square result; perform a square process on the second chromaticity component to obtain a second square result; perform a sum process on the first square result and the second square result, and perform a square root process on the sum result to obtain the average chromaticity of the image frame.

[0366] In the above scheme, the display module 4552 is also used to perform the following processing on each pixel: subtract the first color opposite dimension of the pixel and the first chromaticity component, and square the subtraction result to obtain a first chromaticity distance, subtract the second color opposite dimension of the pixel and the second chromaticity component, and square the subtraction result to obtain a second chromaticity distance; sum the first chromaticity distance of each pixel, and determine the ratio between the summation result and the number of pixels in the image frame as the first chromaticity center distance component; sum the second chromaticity distance of each pixel, and determine the ratio between the summation result and the number of pixels in the image frame as the second chromaticity center distance component.

[0367] In the above scheme, the display module 4552 is also used to square the first chromaticity center distance component to obtain a third square result; square the first chromaticity center distance component to obtain a fourth square result; and add the third square result and the fourth square result to obtain the chromaticity center distance of the image frame.

[0368] In the above scheme, the display module 4552 is also used to perform scene detection processing on the video of the live broadcast room. When a color display sensitive scene appears in the video of the live broadcast room, it is determined to perform the operation of color cast detection processing on the video of the live broadcast room; wherein, the types of color display sensitive scenes include: recommended scenes for multiple items of different colors, recommended scenes for at least one item of a specific color, and recommended scenes for items that meet the preferences of the login account, and the login account is the account that logs in to the live broadcast room.

[0369] In the above solution, the display module 4552 is also used to display the color cast detection entrance in the human-computer interaction interface; in response to the trigger operation on the color cast detection entrance, it is determined to perform the operation of color cast detection processing on the video in the live broadcast room.

[0370] In the above scheme, the display module 4552 is also used to perform scene detection processing on the video of the live broadcast room played in the human-computer interaction interface; when a color display sensitive scene appears in the video of the live broadcast room, it is determined to automatically switch to the operation of displaying the color cast correction entrance in the human-computer interaction interface; wherein, the types of color display sensitive scenes include: recommended scenes for multiple items of different colors, recommended scenes for at least one specific color item, and recommended scenes for items that meet the preferences of the login account; the login account is the account that logs in to the live broadcast room.

[0371] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the live broadcast processing method described above in the present invention.

[0372] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions. The computer-executable instructions are stored in the storage medium. When the computer-executable instructions are executed by a processor, the processor will execute the live broadcast processing method provided by the embodiment of the present application. For example, Figures 3 to 8 The live broadcast processing method shown is used by various electronic devices including computers, smart terminals and servers.

[0373] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0374] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0375] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a hypertext markup language document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0376] By way of example, computer-executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.

[0377] To sum up, after the audience triggers the color cast correction entrance, the embodiment of the present application performs color cast correction on the video in the live broadcast room and plays the video after color cast correction. This can ensure the accuracy of the video display color during the live broadcast, improve the visual perception effect of the video, thereby improving the audience's live viewing experience, and saving computing resources and communication resources used for live broadcast.

[0378] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A live broadcast processing method, characterized in that: The method comprises: Receive live broadcast video; Playing the live broadcast room video in the human-computer interaction interface, and displaying a countdown control and a color cast correction off button in the human-computer interaction interface; When the color cast correction off button is not triggered before the countdown in the countdown control ends, the video is automatically subjected to color cast correction, and the video after the color cast correction is played.

2. The method according to claim 1, characterized in that Playing the video after color cast correction processing includes: Playing the video after color cast correction in the first area of ​​the human-computer interaction interface; When playing the video after the color cast correction process, the method further includes: The video before the color cast correction process is played in the second area of ​​the human-computer interaction interface.

3. The method according to claim 2, characterized in that The method further comprises: In response to a zone switching operation, the videos played in the first zone and the second zone are swapped.

4. The method according to claim 2, characterized in that The method further comprises: Displaying a closing entrance of the second area in the human-computer interaction interface; In response to a triggering operation on the closing entrance, the video played in the second area is automatically stopped, and the second area is hidden.

5. The method according to claim 2, characterized in that The method further comprises: When the area switching operation is not received within a preset waiting time, the video played in the second area is automatically stopped and the second area is hidden.

6. The method according to claim 1, characterized in that Playing the video after color cast correction processing includes: The video being played before the color cast correction process is automatically replaced with the video after the color cast correction process.

7. The method according to claim 1, characterized in that The method further comprises: Display the color cast correction parameter setting page; In response to a color cast correction parameter configuration operation received on the color cast correction parameter setting page, obtaining the color cast correction parameters configured by the login account, wherein the login account is the account that logged into the live broadcast room; The performing color cast correction processing on the video includes: Perform color cast correction on the video according to the color cast correction parameters configured by the login account.

8. The method according to claim 1, characterized in that The method further comprises: Performing scene detection processing on the video of the live broadcast room played in the human-computer interaction interface; When a color display-sensitive scene appears in the video of the live broadcast room, determining to automatically switch to executing an operation of displaying a color cast correction entrance in the human-computer interaction interface; Among them, the types of color display sensitive scenes include: recommendation scenes for multiple items of different colors, recommendation scenes for at least one item of a specific color, and recommendation scenes for items that meet the preferences of the login account; the login account is the account that logs in to the live broadcast room.

9. The method according to claim 1, characterized in that The method further comprises: Performing color cast detection processing on the video of the live broadcast room to obtain a color cast detection result; When the color cast detection result indicates that the video has color cast, information indicating that the video has color cast is displayed in the human-computer interaction interface.

10. The method according to claim 9, characterized in that Before performing color cast detection processing on the video in the live broadcast room, the method further includes: Performing scene detection processing on the video of the live broadcast room, and when a color display-sensitive scene appears in the video of the live broadcast room, determining to perform a color cast detection processing operation on the video of the live broadcast room; Among them, the types of color display sensitive scenes include: recommendation scenes for multiple items of different colors, recommendation scenes for at least one item of a specific color, and recommendation scenes for items that meet the preferences of the login account; the login account is the account that logs in to the live broadcast room.

11. The method according to claim 9, characterized in that Before performing color cast detection processing on the video in the live broadcast room, the method further includes: Displaying a color cast detection entrance in the human-computer interaction interface; In response to a trigger operation on the color cast detection entry, it is determined that an operation of performing color cast detection processing on the video in the live broadcast room will be performed.

12. The method according to claim 9, characterized in that The performing color cast detection processing on the video of the live broadcast room to obtain a color cast detection result includes: extracting a plurality of image frames from the video; Performing the following processing for each of the image frames: determining the average chromaticity and the chromaticity center distance of the image frame, and determining the ratio between the average chromaticity and the chromaticity center distance as the color cast factor of the image frame; The color cast detection result of the video is determined according to the color cast factor of each image frame.

13. A live broadcast processing device, characterized in that: include: A receiving module, used to receive the video from the live broadcast room; A display module is used to play the video of the live broadcast room in the human-computer interaction interface, and to display a countdown control and a color cast correction off button in the human-computer interaction interface; The correction module is used to automatically perform color cast correction on the video and play the video after color cast correction when the color cast correction off button is not triggered before the countdown in the countdown control ends.

14. An electronic device, characterized in that: include: a memory for storing computer-executable instructions; The processor is configured to implement the live broadcast processing method according to any one of claims 1 to 12 when executing the computer executable instructions stored in the memory.

15. A computer-readable storage medium, characterized in that Computer-executable instructions are stored, and when the computer-executable instructions are executed, they are used to implement the live broadcast processing method described in any one of claims 1 to 12.

Images