Internet live broadcast scene judgment system

Abstract

The present invention relates to an Internet live broadcast scene judgment system, comprising: a proportion recognition device, which is arranged in a live broadcast test mechanism and is used to use an AI intelligent model to intelligently judge the area ratio of the image block where the human hair is located to the image block where the host's face is located based on various visual data of the human hair in the final analysis image; a state extraction device, which is used to send a proportion imbalance signal when the received area ratio is outside the set ratio value range. The Internet live broadcast scene judgment system of the present invention is compact in design and intelligent in operation. Since it can use an AI intelligent model to intelligently judge the area ratio of the image block where the human hair is located to the image block where the host's face is located based on various visual data of the human hair, it provides a guarantee for the image quality of the live broadcast screen where the host is located, and improves the competitiveness of the live broadcast platform where the host is located.

Description

Technical Field

[0001] The present invention relates to the field of Internet, and more specifically, to an Internet live broadcast scene judgment system. Background Art

[0002] In the past, mainstream media’s innovative live broadcasts and short videos all relied on commercial platforms to achieve dissemination effects in the mode of “content + platform + dissemination”. However, with the deepening of integration, building independent and controllable platforms has become the choice of many media. For example, some official media live clouds provide “live broadcast products” to a wide range of regions, thus creating a new atmosphere of live broadcast competition and popularizing the application of live broadcast as a mode of promoting audio and video content.

[0003] As the competition mechanism is gradually unfolding in the live broadcast industry, each live broadcast platform hopes to attract more viewers to become its fans with better picture quality and more creative content. Therefore, each live broadcast platform is constantly improving its live broadcast screen and content. For example, for the corresponding anchor, it is hoped that the proportion of the anchor's hair to the anchor's face area is within the set proportion range, thereby ensuring the live broadcast quality and effect of the live broadcast platform. However, the existing technology lacks corresponding detection solutions, resulting in the anchor's image failing to meet the expected requirements. Summary of the invention

[0004] In order to solve the technical problems in the prior art, the present invention provides an Internet live broadcast scene judgment system, which can confirm the image block where the human hair is located in the received targeted optimized anchor test image based on the color imaging characteristics corresponding to the human hair, analyze multiple geometric parameters corresponding to the geometric shape of the image block where the human hair is located and the multiple geometric parameters corresponding to the geometric shape of the image block where the anchor's face is located, and use an AI intelligent model to intelligently judge the area ratio of the image block where the human hair is located to the image block where the anchor's face is located according to the multiple geometric parameters corresponding to the geometric shape of the image block where the human hair is located, the multiple geometric parameters corresponding to the geometric shape of the image block where the anchor's face is located, and the horizontal resolution and vertical resolution of the final-level analyzed image, thereby providing guarantee for the picture quality of the live broadcast screen where the anchor is located.

[0005] According to the present invention, a system for determining a scene of an Internet live broadcast is provided, the system comprising:

[0006] A directional camera device is provided in the live broadcast test institution and is used to perform directional camera actions facing the anchor in the live broadcast scene to obtain and output corresponding real-time camera frames;

[0007] A primary analysis device, arranged in the live broadcast test mechanism and connected to the directional camera device, is used to perform image content sharpening processing based on the Roberts operator on the received real-time camera frame to obtain and output a corresponding primary analysis image;

[0008] A secondary analysis device, arranged in the live broadcast test mechanism and connected to the primary analysis device, is used to perform harmonic mean filtering on the received primary analysis image to obtain and output a corresponding secondary analysis image;

[0009] A final analysis device, arranged in the live broadcast test mechanism and connected to the secondary analysis device, is used to perform artifact removal processing on the received secondary analysis image to obtain and output a corresponding final analysis image;

[0010] A ratio recognition device is arranged in the live broadcast test mechanism and connected to the final analysis device, and is used to confirm the image block where the human hair in the received final analysis image is located based on the color imaging features corresponding to the human hair, wherein the color imaging features corresponding to the human hair are the hue component value interval, the brightness component value interval and the saturation component value interval under the HSB color space corresponding to the object, and the ratio recognition device is also used to parse the multiple geometric parameters corresponding to the geometric shape of the image block where the human hair is located and the multiple geometric parameters corresponding to the geometric shape of the image block where the anchor's face is located, and the image block where the human hair is located is determined according to the color imaging features corresponding to the human hair. The AI ​​intelligent model is used to intelligently determine the area ratio of the image block where the human hair is located to the image block where the host's face is located, wherein the multiple geometric parameters corresponding to the geometric shape of each image block include the roundness, convexity, aspect ratio and compactness corresponding to the geometric shape of the image block, and the horizontal resolution and vertical resolution of the final resolution image are the number of pixel columns and the number of pixel rows in the final resolution image, respectively.

[0011] A state extraction device is arranged in the live broadcast test mechanism and connected to the proportion recognition device, and is used to send a proportion imbalance signal when the area ratio of the received image block where the human hair is located to the image block where the host's face is located is outside the set proportion value interval, and is also used to send a proportion balance signal when the area ratio of the received image block where the human hair is located to the image block where the host's face is located is within the set proportion value interval;

[0012] Among them, the AI ​​intelligent model is a deep convolutional network that has completed a fixed number of multiple trainings, and the number of trainings is proportional to the total number of pixels occupied by the final level parsed image.

[0013] Therefore, the present invention has at least the following three beneficial technical effects:

[0014] Technical effect 1: confirming the image block where the human hair in the received targeted optimized anchor test image is located based on the color imaging features corresponding to the human hair, wherein the color imaging features corresponding to the human hair are the hue component value interval, brightness component value interval and saturation component value interval in the HSB color space corresponding to the object;

[0015] Technical effect 2: analyzing multiple geometric parameters corresponding to the geometric shape of the image block where the human hair is located and multiple geometric parameters corresponding to the geometric shape of the image block where the host's face is located, and using an AI intelligent model to intelligently determine the area ratio of the image block where the human hair is located to the image block where the host's face is located based on the multiple geometric parameters corresponding to the geometric shape of the image block where the human hair is located, the multiple geometric parameters corresponding to the geometric shape of the image block where the host's face is located, and the horizontal resolution and vertical resolution of the final-level analyzed image, wherein the multiple geometric parameters corresponding to the geometric shape of each image block include the roundness, convexity and concavity, aspect ratio and compactness corresponding to the geometric shape of the image block, and the horizontal resolution and vertical resolution of the final-level analyzed image are the number of pixel columns and the number of pixel rows in the final-level analyzed image, respectively;

[0016] Technical effect three: The AI ​​intelligent model used for intelligent judgment of area ratio is a deep convolutional network that has completed a fixed number of multiple trainings, and the number of trainings is proportional to the total number of pixels occupied by the anchor test image after targeted optimization, thereby completing the customized structural design of the AI ​​intelligent model.

[0017] The Internet live broadcast scene judgment system of the present invention is compact in design and intelligent in operation. Since it can intelligently judge the area ratio of the image block where the human hair is located to the image block where the host's face is located based on various visual data of the human hair, it provides guarantee for the image quality of the live broadcast screen where the host is located, and improves the competitiveness of the live broadcast platform where the host is located. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Those skilled in the art may better understand the numerous advantages of the present invention by referring to the accompanying drawings, in which:

[0019] Figure 1 It is a schematic diagram of the internal structure of the Internet live broadcast scene judgment system according to the first embodiment of the present invention.

[0020] Figure 2 It is a schematic diagram of the internal structure of an Internet live broadcast scene judgment system according to the second embodiment of the present invention.

[0021] Figure 3 It is a schematic diagram of the internal structure of an Internet live broadcast scene judgment system according to the third embodiment of the present invention. DETAILED DESCRIPTION

[0022] The implementation scheme of the Internet live broadcast scene judgment system of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Figure 1 1 is a schematic diagram of the internal structure of an Internet live broadcast scene judgment system according to a first embodiment of the present invention, the system comprising:

[0024] A directional camera device is provided in the live broadcast test institution and is used to perform directional camera actions facing the anchor in the live broadcast scene to obtain and output corresponding real-time camera frames;

[0025] A primary analysis device, arranged in the live broadcast test mechanism and connected to the directional camera device, is used to perform image content sharpening processing based on the Roberts operator on the received real-time camera frame to obtain and output a corresponding primary analysis image;

[0026] Specifically, the primary analysis device is arranged in the live broadcast test mechanism and connected to the directional camera device, and is used to perform image content sharpening processing based on the Roberts operator on the received real-time camera frame to obtain and output the corresponding primary analysis image, including: selecting an FPGA chip to implement the primary analysis device, which is arranged in the live broadcast test mechanism and connected to the directional camera device, and is used to perform image content sharpening processing based on the Roberts operator on the received real-time camera frame to obtain and output the corresponding primary analysis image;

[0027] A secondary analysis device, arranged in the live broadcast test mechanism and connected to the primary analysis device, is used to perform harmonic mean filtering on the received primary analysis image to obtain and output a corresponding secondary analysis image;

[0028] A final analysis device, arranged in the live broadcast test mechanism and connected to the secondary analysis device, is used to perform artifact removal processing on the received secondary analysis image to obtain and output a corresponding final analysis image;

[0029] A ratio recognition device is arranged in the live broadcast test mechanism and connected to the final analysis device, and is used to confirm the image block where the human hair in the received final analysis image is located based on the color imaging features corresponding to the human hair, wherein the color imaging features corresponding to the human hair are the hue component value interval, the brightness component value interval and the saturation component value interval under the HSB color space corresponding to the object, and the ratio recognition device is also used to parse the multiple geometric parameters corresponding to the geometric shape of the image block where the human hair is located and the multiple geometric parameters corresponding to the geometric shape of the image block where the anchor's face is located, and the image block where the human hair is located is determined according to the color imaging features corresponding to the human hair. The AI ​​intelligent model is used to intelligently determine the area ratio of the image block where the human hair is located to the image block where the host's face is located, wherein the multiple geometric parameters corresponding to the geometric shape of each image block include the roundness, convexity, aspect ratio and compactness corresponding to the geometric shape of the image block, and the horizontal resolution and vertical resolution of the final resolution image are the number of pixel columns and the number of pixel rows in the final resolution image, respectively.

[0030] A state extraction device is arranged in the live broadcast test mechanism and connected to the proportion recognition device, and is used to send a proportion imbalance signal when the area ratio of the received image block where the human hair is located to the image block where the host's face is located is outside the set proportion value interval, and is also used to send a proportion balance signal when the area ratio of the received image block where the human hair is located to the image block where the host's face is located is within the set proportion value interval;

[0031] The AI ​​intelligent model is a deep convolutional network that has completed a fixed number of multiple trainings, and the number of trainings is proportional to the total number of pixels occupied by the final resolution image;

[0032] Among them, the AI ​​intelligent model is a deep convolutional network that has completed a fixed number of multiple trainings and the number of trainings is proportional to the total number of pixels occupied by the final-level analytical image, including: using a numerical conversion function to represent a numerical conversion relationship in which the number of trainings is proportional to the total number of pixels occupied by the final-level analytical image.

[0033] Figure 2 It is a schematic diagram of the internal structure of an Internet live broadcast scene judgment system according to the second embodiment of the present invention.

[0034] and Figure 1 different, Figure 2 The Internet live broadcast scene judgment system in the embodiment may also include the following components:

[0035] a heat analysis device, connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, respectively, for measuring the instantaneous internal heat of each of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device;

[0036] Wherein, the heat analysis device is respectively connected with the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, and is used to respectively measure the instantaneous internal heat of each of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, including: the heat analysis device includes a plurality of heat measurement units, which are respectively connected with the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, so as to respectively measure the instantaneous internal heat of each of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device;

[0037] Wherein, the heat analysis device comprises a plurality of heat measurement units, which are respectively connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device to respectively measure the instantaneous internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, including: the plurality of heat measurement units are a plurality of heat sensors, which are respectively connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device to respectively measure the instantaneous internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device;

[0038] The plurality of heat measurement units are a plurality of heat sensors, which are used to be connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device respectively, so as to respectively measure the instant internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device respectively, including: the internal structures of the plurality of heat sensors are the same;

[0039] And wherein, the multiple heat measurement units are multiple heat sensors, which are used to be connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device respectively, so as to complete the respective measurements of the instantaneous internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, and also include: the multiple heat sensors have the same heat measurement upper limit value and heat measurement lower limit value.

[0040] Figure 3 It is a schematic diagram of the internal structure of an Internet live broadcast scene judgment system according to the third embodiment of the present invention.

[0041] and Figure 1 different, Figure 3 The Internet live broadcast scene judgment system in the embodiment may also include the following components:

[0042] an embedded processor, connected to the plurality of heat measurement units of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, respectively, for realizing on-site configuration of the real-time working mode of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device;

[0043] Wherein, the embedded processor is respectively connected to the plurality of heat measurement units of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, and is used to implement the on-site configuration of the real-time working mode of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, including: the embedded processor is a processor with an ARM 13 core;

[0044] And wherein, the embedded processor is respectively connected to multiple heat measurement units of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, and is used to realize the on-site configuration of the real-time working mode of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, including: the embedded processor has built-in registers, timers and static storage devices.

[0045] In addition, in the Internet live broadcast scene judgment system, a numerical conversion function is used to represent a numerical conversion relationship in which the number of training times is proportional to the total number of pixel points occupied by the final-level resolution image, including: in the numerical conversion function, the total number of pixel points occupied by the final-level resolution image is the input parameter of the numerical conversion function, and the number of training times is the output parameter of the numerical conversion function.

[0046] Therefore, the descriptions limiting the shapes, etc. disclosed above are examples for easy understanding of the present invention and are not intended to limit the present invention. Therefore, the descriptions of component names that exclude part or all of the limitations on their shapes, materials, etc. are also included in the present invention.

Claims

1. An Internet live broadcast scene judgment system, characterized in that: The system comprises: A directional camera device is provided in the live broadcast test institution and is used to perform directional camera actions facing the anchor in the live broadcast scene to obtain and output corresponding real-time camera frames; A primary analysis device, arranged in the live broadcast test mechanism and connected to the directional camera device, is used to perform image content sharpening processing based on the Roberts operator on the received real-time camera frame to obtain and output a corresponding primary analysis image; A secondary analysis device, arranged in the live broadcast test mechanism and connected to the primary analysis device, is used to perform harmonic mean filtering on the received primary analysis image to obtain and output a corresponding secondary analysis image; A final analysis device, arranged in the live broadcast test mechanism and connected to the secondary analysis device, is used to perform artifact removal processing on the received secondary analysis image to obtain and output a corresponding final analysis image; A ratio recognition device is arranged in the live broadcast test mechanism and connected to the final analysis device, and is used to confirm the image block where the human hair in the received final analysis image is located based on the color imaging features corresponding to the human hair, wherein the color imaging features corresponding to the human hair are the hue component value interval, the brightness component value interval and the saturation component value interval under the HSB color space corresponding to the object, and the ratio recognition device is also used to parse the multiple geometric parameters corresponding to the geometric shape of the image block where the human hair is located and the multiple geometric parameters corresponding to the geometric shape of the image block where the anchor's face is located, and the image block where the human hair is located is determined according to the color imaging features corresponding to the human hair. The AI ​​intelligent model is used to intelligently determine the area ratio of the image block where the human hair is located to the image block where the host's face is located, wherein the multiple geometric parameters corresponding to the geometric shape of each image block include the roundness, convexity, aspect ratio and compactness corresponding to the geometric shape of the image block, and the horizontal resolution and vertical resolution of the final resolution image are the number of pixel columns and the number of pixel rows in the final resolution image, respectively. A state extraction device is arranged in the live broadcast test mechanism and connected to the proportion recognition device, and is used to send a proportion imbalance signal when the area ratio of the received image block where the human hair is located to the image block where the host's face is located is outside the set proportion value interval, and is also used to send a proportion balance signal when the area ratio of the received image block where the human hair is located to the image block where the host's face is located is within the set proportion value interval; Among them, the AI ​​intelligent model is a deep convolutional network that has completed a fixed number of multiple trainings, and the number of trainings is proportional to the total number of pixels occupied by the final level parsed image.

2. The Internet live broadcast scene judgment system according to claim 1, characterized in that: The AI ​​intelligent model is a deep convolutional network that has completed a fixed number of multiple trainings, and the number of trainings is proportional to the total number of pixels occupied by the final-level analytical image, including: using a numerical conversion function to represent a numerical conversion relationship in which the number of trainings is proportional to the total number of pixels occupied by the final-level analytical image.

3. The Internet live broadcast scene judgment system according to claim 2, characterized in that: The system further comprises: a heat analysis device, connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, respectively, for measuring the instantaneous internal heat of each of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device; Among them, the thermal analysis device is respectively connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, and is used to respectively measure the instantaneous internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device. The thermal analysis device includes a plurality of heat measurement units, which are respectively connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device to complete the respective measurements of the instantaneous internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device.

4. The Internet live broadcast scene judgment system as claimed in claim 3, characterized in that: The thermal analysis device includes a plurality of heat measurement units, which are used to be respectively connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device to complete the respective measurement of the instantaneous internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, including: the plurality of heat measurement units are a plurality of heat sensors, which are used to be respectively connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device to complete the respective measurement of the instantaneous internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device.

5. The Internet live broadcast scene judgment system according to claim 4, characterized in that: The multiple heat measurement units are multiple heat sensors, which are used to be connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device respectively, so as to complete the respective measurements of the real-time internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, including: the internal structures of the multiple heat sensors are the same.

6. The Internet live broadcast scene judgment system according to claim 5, characterized in that: The multiple heat measurement units are multiple heat sensors, which are used to be connected to the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device respectively to complete the measurement of the real-time internal heat of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device respectively, and also include: the multiple heat sensors have the same heat measurement upper limit value and heat measurement lower limit value.

7. The Internet live broadcast scene judgment system according to claim 2, characterized in that: The system further comprises: The embedded processor is respectively connected to the multiple heat measurement units of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, and is used to realize the on-site configuration of the real-time working mode of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device.

8. The Internet live broadcast scene judgment system according to claim 7, characterized in that: The embedded processor is respectively connected to the multiple heat measurement units of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, and is used to realize the on-site configuration of the real-time working mode of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, including: the embedded processor is a processor with an ARM 13 core.

9. The Internet live broadcast scene judgment system according to claim 7, characterized in that: The embedded processor is respectively connected to the multiple heat measurement units of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, and is used to realize the on-site configuration of the real-time working mode of the primary analysis device, the secondary analysis device, the final analysis device and the ratio identification device, including: the embedded processor has built-in registers, timers and static storage devices.

Images