Human face special effect live broadcast strategy analysis system

Abstract

The invention relates to a human face special effect live broadcast strategy analysis system, and the system comprises a cache processing mechanism which is used for receiving each frame of live broadcast image sent by a set live broadcast client; the special effect processing device is used for processing the human face when receiving the human face extraction success signal; and transferring the special effect playing position of the live broadcast picture which is newest received and cached by the cache processing mechanism to an image area which is away from the position of the human face of the live broadcast picture which is newest received and cached by the cache processing mechanism by a set pixel point number, and playing the special effect playing position of the live broadcast picture which is newest received and cached by the cache processing mechanism. According to the invention, a targeted detection mechanism is adopted to identify that the human face exists in the newly received and cached live broadcast picture; and the special effect playing position of the newly received and cached live broadcast picture is transferred to the image area which is away from the position of the human face of the newly received and cached live broadcast picture by the set pixel point number for playing, so that the special effect is prevented from obstructing the effect of the live broadcast picture.

Description

Technical Field

[0001] The present invention relates to the field of Internet live broadcasting, and in particular to a special effects live broadcasting strategy analysis system for human face. Background Art

[0002] Wonderful and good-looking special effects will give the audience a different experience, which can retain the audience, attract the audience's interest, and increase the fun of the live broadcast. The live broadcast special effects are tailor-made for anchors to enrich the live broadcast content. There are special effects such as dynamic text, embellishment animation, transition full-screen animation, timed reminders, etc. The live broadcast room all-round special effects display plug-in supports adding entry prompts, button triggers, fan barrage feedback, timed scene effects and other forms of wonderful effect packaging to the live broadcast content. While providing basic functions such as live broadcast streaming, it can also quickly integrate and launch special effects functions such as beauty filters, big eyes and thin face, and beauty makeup.

[0003] However, when the anchor is showing his talent in the live broadcast, the various special effects brought about by the audience's tips will be different. The playback area of ​​each special effect will be different depending on the amount of the tip. If the special effects brought by some of the tips are superimposed on the key areas of the live broadcast where the anchor shows his talent, such as the head, and hinder the audience's normal viewing, this feature will obviously greatly reduce the anchor's playback effect. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a special effects live broadcast strategy analysis system for human faces, which can use a targeted detection mechanism to identify the presence of human faces in the latest received and cached live broadcast picture, and move the special effects playback position of the latest received and cached live broadcast picture to an image area that is a set number of pixels away from the position of the human face in the latest received and cached live broadcast picture for playback, so as to avoid special effects interfering with the key playback area of ​​the human face in the latest received and cached live broadcast picture.

[0005] The system comprises: A cache processing mechanism is provided at the live broadcast server, and is used to receive each frame of live broadcast picture sent from a set live broadcast client, and cache each frame of live broadcast picture when each frame of live broadcast picture is distributed to other live broadcast clients; A contrast retention device, arranged at the live broadcast server and connected to the cache processing mechanism, is used to perform a process of retaining the junction of two parts with obvious color or light-dark contrast in the image on the live broadcast image that is most recently received and cached by the cache processing mechanism, so as to obtain and output a corresponding contrast-retained image; An image rendering device, connected to the contrast preserving device, for performing image rendering processing on the received contrast preserving image to obtain and output a corresponding current rendered image; A grayscale correction device, connected to the image rendering device, for performing grayscale non-uniformity correction processing on the received current rendered image to obtain and output a corresponding grayscale corrected image; A multi-layer detection device is arranged at a live broadcast server and connected to the grayscale correction device, comprising a pixel resolution device, a numerical analysis device and a signal output device, wherein the numerical analysis device is respectively connected to the pixel resolution device and the signal output device, and the multi-layer detection device is used to perform the following detection operation on a single pixel of the received grayscale correction image as a current pixel: when the red channel value, green channel value and blue channel value of the current pixel in the RGB color space fall within the red channel value range, green channel value range and blue channel range corresponding to the human face, respectively, the current pixel is regarded as a valid pixel; otherwise, the current pixel is regarded as an invalid pixel; and when the number of valid pixels in the grayscale correction image exceeds or equals to a set number threshold, a human face extraction success signal is issued; and when the number of valid pixels in the grayscale correction image is less than the set number threshold, a human face extraction failure signal is issued; The special effects processing device is connected to the multi-layer detection device and is used to move the special effects playback position of the live broadcast picture most recently received and cached by the cache processing mechanism to an image area with a set number of pixels away from the position of the human face of the live broadcast picture most recently received and cached by the cache processing mechanism for playback when a human face extraction success signal is received. DETAILED DESCRIPTION

[0006] The implementation scheme of the human face special effects live broadcast strategy analysis system of the present invention will be described in detail below.

[0007] AImplementation Plan According to embodiment A of the present invention, a special effects live broadcast strategy parsing system for human face is shown, and the system comprises: A cache processing mechanism is provided at the live broadcast server, and is used to receive each frame of live broadcast picture sent from a set live broadcast client, and cache each frame of live broadcast picture when each frame of live broadcast picture is distributed to other live broadcast clients; For example, the cache processing mechanism is set at the live broadcast server, and is used to receive each frame of live broadcast picture sent from the set live broadcast client, and when each frame of live broadcast picture is distributed to other live broadcast clients, cache each frame of live broadcast picture, including: an SD storage device or a static storage device can be selected to implement the cache processing mechanism; A contrast retention device, arranged at the live broadcast server and connected to the cache processing mechanism, is used to perform a process of retaining the junction of two parts with obvious color or light-dark contrast in the image on the live broadcast image that is most recently received and cached by the cache processing mechanism, so as to obtain and output a corresponding contrast-retained image; An image rendering device, connected to the contrast preserving device, for performing image rendering processing on the received contrast preserving image to obtain and output a corresponding current rendered image; A grayscale correction device, connected to the image rendering device, for performing grayscale non-uniformity correction processing on the received current rendered image to obtain and output a corresponding grayscale corrected image; A multi-layer detection device is arranged at a live broadcast server and connected to the grayscale correction device, comprising a pixel resolution device, a numerical analysis device and a signal output device, wherein the numerical analysis device is respectively connected to the pixel resolution device and the signal output device, and the multi-layer detection device is used to perform the following detection operation on a single pixel of the received grayscale correction image as a current pixel: when the red channel value, green channel value and blue channel value of the current pixel in the RGB color space fall within the red channel value range, green channel value range and blue channel range corresponding to the human face, respectively, the current pixel is regarded as a valid pixel; otherwise, the current pixel is regarded as an invalid pixel; and when the number of valid pixels in the grayscale correction image exceeds or equals to a set number threshold, a human face extraction success signal is issued; and when the number of valid pixels in the grayscale correction image is less than the set number threshold, a human face extraction failure signal is issued; For example, when the red channel value, green channel value and blue channel value of the current pixel in the RGB color space respectively fall within the red channel value range, green channel value range and blue channel range corresponding to the human face, taking the current pixel as a valid pixel includes: the value range of each of the red channel value, green channel value and blue channel value of the current pixel in the RGB color space is between 0 and 255; A special effect processing device is connected to the multi-layer detection device and is used to move the special effect playback position of the live picture most recently received and cached by the cache processing mechanism to an image area with a set number of pixels away from the position of the human face of the live picture most recently received and cached by the cache processing mechanism for playback when a human face extraction success signal is received; Among them, the special effects processing device is also used to maintain the special effects playback position of the live broadcast picture most recently received and cached by the cache processing mechanism at the default playback position of the live broadcast picture most recently received and cached by the cache processing mechanism when a human face extraction failure signal is received.

[0008] B Implementation Plan The human face special effects live broadcast strategy parsing system shown in Embodiment B of the present invention may include: A cache processing mechanism is provided at the live broadcast server, and is used to receive each frame of live broadcast picture sent from a set live broadcast client, and cache each frame of live broadcast picture when each frame of live broadcast picture is distributed to other live broadcast clients; A contrast retention device, arranged at the live broadcast server and connected to the cache processing mechanism, is used to perform a process of retaining the junction of two parts with obvious color or light-dark contrast in the image on the live broadcast image that is most recently received and cached by the cache processing mechanism, so as to obtain and output a corresponding contrast-retained image; An image rendering device, connected to the contrast preserving device, for performing image rendering processing on the received contrast preserving image to obtain and output a corresponding current rendered image; A grayscale correction device, connected to the image rendering device, for performing grayscale non-uniformity correction processing on the received current rendered image to obtain and output a corresponding grayscale corrected image; A multi-layer detection device is arranged at a live broadcast server and connected to the grayscale correction device, comprising a pixel resolution device, a numerical analysis device and a signal output device, wherein the numerical analysis device is respectively connected to the pixel resolution device and the signal output device, and the multi-layer detection device is used to perform the following detection operation on a single pixel of the received grayscale correction image as a current pixel: when the red channel value, green channel value and blue channel value of the current pixel in the RGB color space fall within the red channel value range, green channel value range and blue channel range corresponding to the human face, respectively, the current pixel is regarded as a valid pixel; otherwise, the current pixel is regarded as an invalid pixel; and when the number of valid pixels in the grayscale correction image exceeds or equals to a set number threshold, a human face extraction success signal is issued; and when the number of valid pixels in the grayscale correction image is less than the set number threshold, a human face extraction failure signal is issued; A special effect processing device is connected to the multi-layer detection device and is used to move the special effect playback position of the live picture most recently received and cached by the cache processing mechanism to an image area with a set number of pixels away from the position of the human face of the live picture most recently received and cached by the cache processing mechanism for playback when a human face extraction success signal is received; a signal sharpening mechanism, connected to the contrast preserving device, the image rendering device and the grayscale correction device, respectively, for performing signal sharpening on the respective output image signals of the contrast preserving device, the image rendering device and the grayscale correction device, so as to obtain respective signal sharpened images corresponding to the respective output image signals of the contrast preserving device, the image rendering device and the grayscale correction device; The signal sharpening is performed on the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively to obtain the signal-sharpened images corresponding to the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively, including: the first level signal sharpening is performed on the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively in the horizontal direction; The signal sharpening is performed on the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively to obtain the signal-sharpened images corresponding to the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively, and further comprises: the second level signal sharpening is performed on the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively in the vertical direction; The signal sharpening is performed on the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively to obtain the signal-sharpened images corresponding to the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively, and further comprises: the third level signal sharpening is performed on the output image signals of the contrast retaining device, the image rendering device and the grayscale correction device respectively by using the spatial differential method; Among them, signal sharpening is performed on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device to obtain respective signal-sharpened images corresponding to the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device, and also includes: the fourth level signal sharpening performs edge sharpening on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device.

[0009] C Implementation Plan According to the C embodiment of the present invention, a human face special effects live broadcast strategy parsing system is shown, which may include: A cache processing mechanism is provided at the live broadcast server, and is used to receive each frame of live broadcast picture sent from a set live broadcast client, and cache each frame of live broadcast picture when each frame of live broadcast picture is distributed to other live broadcast clients; A contrast retention device, arranged at the live broadcast server and connected to the cache processing mechanism, is used to perform a process of retaining the junction of two parts with obvious color or light-dark contrast in the image on the live broadcast image that is most recently received and cached by the cache processing mechanism, so as to obtain and output a corresponding contrast-retained image; An image rendering device, connected to the contrast preserving device, for performing image rendering processing on the received contrast preserving image to obtain and output a corresponding current rendered image; A grayscale correction device, connected to the image rendering device, for performing grayscale non-uniformity correction processing on the received current rendered image to obtain and output a corresponding grayscale corrected image; A multi-layer detection device is arranged at a live broadcast server and connected to the grayscale correction device, comprising a pixel resolution device, a numerical analysis device and a signal output device, wherein the numerical analysis device is respectively connected to the pixel resolution device and the signal output device, and the multi-layer detection device is used to perform the following detection operation on a single pixel of the received grayscale correction image as a current pixel: when the red channel value, green channel value and blue channel value of the current pixel in the RGB color space fall within the red channel value range, green channel value range and blue channel range corresponding to the human face, respectively, the current pixel is regarded as a valid pixel; otherwise, the current pixel is regarded as an invalid pixel; and when the number of valid pixels in the grayscale correction image exceeds or equals to a set number threshold, a human face extraction success signal is issued; and when the number of valid pixels in the grayscale correction image is less than the set number threshold, a human face extraction failure signal is issued; A special effect processing device is connected to the multi-layer detection device and is used to move the special effect playback position of the live picture most recently received and cached by the cache processing mechanism to an image area with a set number of pixels away from the position of the human face of the live picture most recently received and cached by the cache processing mechanism for playback when a human face extraction success signal is received; A timing service device, connected to the contrast retention device, the image rendering device and the grayscale correction device, respectively, for respectively providing the contrast retention device, the image rendering device and the grayscale correction device with the timing services required by each device; The timing service device is connected to the contrast retention device, the image rendering device and the grayscale correction device respectively, and is used to provide the contrast retention device, the image rendering device and the grayscale correction device with the timing services required by each device respectively, including: the timing service device has a built-in quartz oscillation unit, and is connected to the contrast retention device, the image rendering device and the grayscale correction device respectively, and is used to provide the contrast retention device, the image rendering device and the grayscale correction device with the timing services required by each device respectively; And wherein, the timing service device is respectively connected to the contrast retention device, the image rendering device and the grayscale correction device, and is used to respectively provide the contrast retention device, the image rendering device and the grayscale correction device with the respective required timing services, including: using a SOC device to implement the timing service device.

[0010] In addition, in the special effects live broadcast strategy analysis system for the human face, when a human face extraction success signal is received, the special effects playback position of the live broadcast picture most recently received and cached by the cache processing mechanism is moved to an image area that is a set number of pixels away from the position of the human face of the live broadcast picture most recently received and cached by the cache processing mechanism for playback, including: the pixel point at the centroid position of the image area that is a set number of pixels away from the position of the human face of the live broadcast picture most recently received and cached by the cache processing mechanism is a first pixel point, the pixel point at the centroid position of the image area where the human face of the live broadcast picture most recently received and cached by the cache processing mechanism is a second pixel point, and in the live broadcast picture most recently received and cached by the cache processing mechanism, the number of pixels traversed by the shortest path between the first pixel point and the second pixel point is equal to the set number of pixels.

[0011] The technical points of the implementation scheme of the present invention are: Firstly: when a targeted detection mechanism is used to identify the presence of a human face in the most recently received and cached live picture, the special effects playback position for the most recently received and cached live picture is moved to an image area that is a set number of pixels away from the position of the human face in the most recently received and cached live picture for playback, so as to avoid interference of the special effects on the key playback area of ​​the human face in the most recently received and cached live picture; Secondly: the effective pixel points constituting the human face are targetedly identified by using the red channel value range, green channel value range and blue channel range corresponding to the human face; Thirdly: based on the number of effective pixels in the grayscale corrected image, it is determined whether there is a set human face in the grayscale corrected image, thereby completing targeted and high-precision detection of the set human face.

[0012] The human face special effects live broadcast strategy analysis system of the present invention addresses the technical problem in the prior art that rewarding special effects will hinder the key areas of the host's talent display. By adopting a targeted detection mechanism to identify the presence of a human face in the most recently received and cached live broadcast picture, the special effects playback position for the most recently received and cached live broadcast picture is moved to an image area that is a set number of pixels away from the position of the human face in the most recently received and cached live broadcast picture for playback, thereby avoiding special effects from hindering the live broadcast picture effect.

[0013] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A human face special effects live broadcast strategy analysis system, characterized in that: The system comprises: A cache processing mechanism is provided at the live broadcast server, and is used to receive each frame of live broadcast picture sent from a set live broadcast client, and cache each frame of live broadcast picture when each frame of live broadcast picture is distributed to other live broadcast clients; A contrast retention device, arranged at the live broadcast server and connected to the cache processing mechanism, is used to perform a process of retaining the junction of two parts with obvious color or light-dark contrast in the image on the live broadcast image that is most recently received and cached by the cache processing mechanism, so as to obtain and output a corresponding contrast-retained image; An image rendering device, connected to the contrast preserving device, for performing image rendering processing on the received contrast preserving image to obtain and output a corresponding current rendered image; A grayscale correction device, connected to the image rendering device, for performing grayscale non-uniformity correction processing on the received current rendered image to obtain and output a corresponding grayscale corrected image; A multi-layer detection device is arranged at a live broadcast server and connected to the grayscale correction device, comprising a pixel resolution device, a numerical analysis device and a signal output device, wherein the numerical analysis device is respectively connected to the pixel resolution device and the signal output device, and the multi-layer detection device is used to perform the following detection operation on a single pixel of the received grayscale correction image as a current pixel: when the red channel value, green channel value and blue channel value of the current pixel in the RGB color space fall within the red channel value range, green channel value range and blue channel range corresponding to the human face, respectively, the current pixel is regarded as a valid pixel; otherwise, the current pixel is regarded as an invalid pixel; and when the number of valid pixels in the grayscale correction image exceeds or equals to a set number threshold, a human face extraction success signal is issued; and when the number of valid pixels in the grayscale correction image is less than the set number threshold, a human face extraction failure signal is issued; The special effects processing device is connected to the multi-layer detection device and is used to move the special effects playback position of the live broadcast picture most recently received and cached by the cache processing mechanism to an image area with a set number of pixels away from the position of the human face of the live broadcast picture most recently received and cached by the cache processing mechanism for playback when a human face extraction success signal is received.

2. The human face special effects live broadcast strategy analysis system as claimed in claim 1, characterized in that: The special effects processing device is also used to maintain the special effects playback position of the live broadcast picture most recently received and cached by the cache processing mechanism at the default playback position of the live broadcast picture most recently received and cached by the cache processing mechanism when a human face extraction failure signal is received.

3. The human face special effects live broadcast strategy analysis system as claimed in claim 2, characterized in that: The system further comprises: a signal sharpening mechanism, connected to the contrast preserving device, the image rendering device and the grayscale correction device, respectively, for performing signal sharpening on the respective output image signals of the contrast preserving device, the image rendering device and the grayscale correction device, so as to obtain respective signal sharpened images corresponding to the respective output image signals of the contrast preserving device, the image rendering device and the grayscale correction device; Among them, signal sharpening is performed on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device to obtain respective signal-sharpened images corresponding to the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device, including: the first level signal sharpening performs horizontal sharpening on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device.

4. The human face special effects live broadcast strategy analysis system as claimed in claim 3, characterized in that: Signal sharpening is performed on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device to obtain respective signal-sharpened images corresponding to the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device, and the second level signal sharpening is performed on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device in the vertical direction.

5. The human face special effects live broadcast strategy analysis system as claimed in claim 4, characterized in that: Signal sharpening is performed on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device to obtain respective signal-sharpened images corresponding to the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device, and also includes: third-level signal sharpening is performed on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device using the spatial differential method.

6. The human face special effects live broadcast strategy analysis system as claimed in claim 5, characterized in that: Signal sharpening is performed on the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device to obtain respective signal-sharpened images corresponding to the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device, and also includes: fourth-level signal sharpening is performed on the edges of the respective output image signals of the contrast retention device, the image rendering device and the grayscale correction device.

7. The human face special effects live broadcast strategy analysis system according to any one of claims 3 to 6, characterized in that: The system further comprises: The timing service device is connected to the contrast retaining device, the image rendering device and the grayscale correction device respectively, and is used to provide the timing services required by the contrast retaining device, the image rendering device and the grayscale correction device respectively.

8. The human face special effects live broadcast strategy analysis system as claimed in claim 7, characterized in that: A timing service device is respectively connected to the contrast retention device, the image rendering device and the grayscale correction device, and is used to respectively provide the contrast retention device, the image rendering device and the grayscale correction device with the timing services they need, including: the timing service device has a built-in quartz oscillation unit, is respectively connected to the contrast retention device, the image rendering device and the grayscale correction device, and is used to respectively provide the contrast retention device, the image rendering device and the grayscale correction device with the timing services they need.

9. The human face special effects live broadcast strategy analysis system according to claim 7, characterized in that: The timing service device is connected to the contrast retention device, the image rendering device and the grayscale correction device respectively, and is used to provide the contrast retention device, the image rendering device and the grayscale correction device with the timing services required by each device, including: using a SOC device to implement the timing service device.