Live broadcast effect checking system

Abstract

The invention relates to a live broadcast effect checking system. The system comprises a user control mechanism, a content filtering device, a logarithmic enhancement device, an edge sharpening device, an intelligent detection device, a debugging response device and the like. The live broadcast effect checking system is intelligent in design and wide in application.

Description

Technical Field

[0001] The present invention relates to the field of live broadcast effect troubleshooting, and more specifically, to a live broadcast effect troubleshooting system. Background Art

[0002] Live broadcast will transform from the current media landscape into an information medium with tool attributes, or become a "pan-media". The "pan-media phenomenon" brought about by live broadcast cannot only focus on the driving role of technology, but also must examine the social interaction behind the technology. The social interaction brought about by live broadcast will affect politics, economy, culture and social life. For users, live broadcast blurs the boundaries between daily life and media life, shaping new habits and norms. Users' social life and live broadcast will gradually merge and interact. Live broadcast will become a media carrier that connects data, products, users, and even everything, and become a platform for social interconnection with the help of the Internet of Things. Summary of the invention

[0003] With the widespread application of live broadcasting, the requirements for live broadcasting quality are getting higher and higher, and the requirements for the makeup level of the anchor are also getting higher and higher. For example, if the anchor's lip line is too blurry, it will make the lips look dirty and need to be redrawn. However, there is no high-precision detection mechanism for whether the anchor's lip line is too blurry in the existing technology, resulting in the difficulty in meeting the detailed requirements of live broadcasting quality.

[0004] In order to solve technical problems in related fields, the present invention provides a live broadcast effect troubleshooting system, which can identify each brightness value corresponding to each pixel point occupied by the lip line in an edge-sharpened image, and identify each brightness value corresponding to each edge pixel point among the pixels, and input each brightness value corresponding to each pixel point occupied by the lip line in the edge-sharpened image, each brightness value corresponding to each edge pixel point, the total number of pixels occupied by the lip line in the edge-sharpened image, the upper brightness threshold of the lip line and the lower brightness threshold of the lip line into an AI intelligent model at the same time to intelligently detect the entity blur level corresponding to the lip line, wherein the AI ​​intelligent model is a deep neural network that has executed a set number of multiple learnings, thereby realizing intelligent identification of the entity blur level corresponding to the lip line, and providing key information for subsequent response operations when the host's lip line is too blurred and appears dirty and the lip line needs to be re-outlined.

[0005] According to the present invention, a live broadcast effect checking system is provided, the system comprising: The user control mechanism is arranged in the live broadcast debugging device, and is used to control the focus area of ​​the live broadcast camera mechanism under the manual control of the user, and to send a lip line detection request when the focus area is the anchor's lips; A content filtering device, arranged in the live broadcast debugging device and connected to the user control mechanism, is used to perform statistical sorting filtering processing on the output picture of the live broadcast camera mechanism with the focus area being the anchor's lips when receiving the lip line detection request, so as to obtain and output a corresponding content filtering image; a logarithmic enhancement device, connected to the content filtering device, configured to perform image content enhancement processing based on logarithmic transformation on the received content filtered image, so as to obtain and output a corresponding logarithmic enhanced image; An edge sharpening device, connected to the exponential enhancement device, for performing edge sharpening processing on the received exponentially enhanced image to obtain and output a corresponding edge sharpened image; An intelligent detection device is provided in the live debugging device and connected to the edge sharpening device, and is used to receive the edge sharpening image, identify the brightness values ​​corresponding to the pixels occupied by the lip line in the edge sharpening image, and identify the brightness values ​​corresponding to the edge pixels in the pixels, and input the brightness values ​​corresponding to the pixels occupied by the lip line in the edge sharpening image, the brightness values ​​corresponding to the edge pixels, the total number of the pixels occupied by the lip line in the edge sharpening image, the upper brightness threshold of the lip line, and the lower brightness threshold of the lip line into the AI ​​intelligent model at the same time to intelligently detect the entity blur level corresponding to the lip line, wherein the AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings; A debugging response device is connected to the intelligent detection device, and is used to send a notification signal corresponding to the host's lip line being too blurred and dirty and needing to redraw the lip line when the entity blur level corresponding to the received lip line is greater than the set level limit; otherwise, temporarily suspend sending the notification signal corresponding to the host's lip line being too blurred and dirty and needing to redraw the lip line; Among them, the brightness values ​​corresponding to each pixel point occupied by the lip line in the edge-sharpened image, the brightness values ​​corresponding to each edge pixel point, the total number of pixels occupied by the lip line in the edge-sharpened image, the upper brightness threshold of the lip line and the lower brightness threshold of the lip line are simultaneously input into the AI ​​intelligent model to intelligently detect the entity blur level corresponding to the lip line. The AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings, including: the higher the entity blur level corresponding to the lip line, the blurrier the lip line representing the host's lip position.

[0006] The live broadcast effect troubleshooting system of the present invention is intelligently designed and widely used. Since a targeted artificial intelligence model can be used to intelligently detect the entity blur level corresponding to the lip line based on various visual information of the lip line in the edge-sharpened image, an intelligent identification of the entity blur level corresponding to the lip line is achieved, providing key information for the subsequent response operation when the host's lip line is too blurred and looks dirty and the lip line needs to be redrawn. DETAILED DESCRIPTION

[0007] Example 1

[0008] The live broadcast effect checking system according to embodiment 1 of the present invention includes: The user control mechanism is arranged in the live broadcast debugging device, and is used to control the focus area of ​​the live broadcast camera mechanism under the manual control of the user, and to send a lip line detection request when the focus area is the anchor's lips; A content filtering device, arranged in the live broadcast debugging device and connected to the user control mechanism, is used to perform statistical sorting filtering processing on the output picture of the live broadcast camera mechanism with the focus area being the anchor's lips when receiving the lip line detection request, so as to obtain and output a corresponding content filtering image; Specifically, the content filtering device is arranged in the live debugging device and connected to the user control mechanism, and is used to perform statistical sorting filtering processing on the output picture of the live camera mechanism whose focus area is the anchor's lips when receiving the lip line detection request, so as to obtain and output the corresponding content filtering image, including: a programmable logic device can be selected to implement the content filtering device, which is arranged in the live debugging device and connected to the user control mechanism, and is used to perform statistical sorting filtering processing on the output picture of the live camera mechanism whose focus area is the anchor's lips when receiving the lip line detection request, so as to obtain and output the corresponding content filtering image; a logarithmic enhancement device, connected to the content filtering device, configured to perform image content enhancement processing based on logarithmic transformation on the received content filtered image, so as to obtain and output a corresponding logarithmic enhanced image; An edge sharpening device, connected to the exponential enhancement device, for performing edge sharpening processing on the received exponentially enhanced image to obtain and output a corresponding edge sharpened image; An intelligent detection device is provided in the live debugging device and connected to the edge sharpening device, and is used to receive the edge sharpening image, identify the brightness values ​​corresponding to the pixels occupied by the lip line in the edge sharpening image, and identify the brightness values ​​corresponding to the edge pixels in the pixels, and input the brightness values ​​corresponding to the pixels occupied by the lip line in the edge sharpening image, the brightness values ​​corresponding to the edge pixels, the total number of the pixels occupied by the lip line in the edge sharpening image, the upper brightness threshold of the lip line, and the lower brightness threshold of the lip line into the AI ​​intelligent model at the same time to intelligently detect the entity blur level corresponding to the lip line, wherein the AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings; A debugging response device is connected to the intelligent detection device, and is used to send a notification signal corresponding to the host's lip line being too blurred and dirty and needing to redraw the lip line when the entity blur level corresponding to the received lip line is greater than the set level limit; otherwise, temporarily suspend sending the notification signal corresponding to the host's lip line being too blurred and dirty and needing to redraw the lip line; Among them, the brightness values ​​corresponding to the pixels occupied by the lip line in the edge-sharpened image, the brightness values ​​corresponding to the edge pixels, the total number of pixels occupied by the lip line in the edge-sharpened image, the upper brightness threshold of the lip line and the lower brightness threshold of the lip line are simultaneously input into the AI ​​intelligent model to intelligently detect the entity blur level corresponding to the lip line, and the AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings, including: the higher the entity blur level corresponding to the lip line, the blurrier the lip line indicating the lip position of the host; Among them, the brightness values ​​corresponding to each pixel point occupied by the lip line in the edge-sharpened image, the brightness values ​​corresponding to each edge pixel point, the total number of pixels occupied by the lip line in the edge-sharpened image, the upper brightness threshold of the lip line and the lower brightness threshold of the lip line are simultaneously input into the AI ​​intelligent model to intelligently detect the entity blur level corresponding to the lip line. The AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings and also includes: the value of the set number is monotonically positively correlated with the resolution of the live camera mechanism.

[0009] Example 2

[0010] A live broadcast effect checking system is shown in accordance with Embodiment 2 of the present invention.

[0011] Different from Example 1, the live broadcast effect checking system in Example 2 may also include the following components: An MCU control device, connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, respectively, for providing configuration operations of working parameters for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner; The MCU control device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide the configuration operation of working parameters for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, including: the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device share the same working parameter configuration interface; Wherein, the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device use different configuration address data; Wherein, a parallel connection of a communication data link is established between each of the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device through a parallel data bus; Among them, the parallel connection of the communication data link established between each of the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device through a parallel data bus includes: the parallel data bus is one of an 8-bit parallel data bus, a 16-bit parallel data bus and a 32-bit parallel data bus.

[0012] Example 3

[0013] Embodiment 3 of the present invention shows a live broadcast effect checking system.

[0014] Different from Example 1, the live broadcast effect checking system in Example 3 may also include the following components: A power support device, connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, respectively, for providing power distribution support of different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner; The power support device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide power distribution support with different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, including: two or more devices with the same working voltage requirements among the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device use the same power supply line; Wherein, the power support device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide power distribution support with different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, and further includes: the power support device is an uninterruptible power supply device; The power support device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide power distribution support with different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in time-sharing, and further includes: providing different working voltages including 3.3V working voltage for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in time-sharing; And wherein, the power support device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide power distribution support with different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, and also includes: providing different working voltages including a 5V working voltage for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner.

[0015] In addition, in the live broadcast effect investigation system, each brightness value corresponding to each pixel point occupied by the lip line in the edge-sharpened image, each brightness value corresponding to each edge pixel point, the total number of each pixel point occupied by the lip line in the edge-sharpened image, the upper brightness threshold of the lip line, and the lower brightness threshold of the lip line are simultaneously input into the AI ​​intelligent model to intelligently detect the entity blur level corresponding to the lip line. The AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings and further includes: using the MATLAB toolbox to complete the simulation and testing of the data processing process of inputting each brightness value corresponding to each pixel point occupied by the lip line in the edge-sharpened image, each brightness value corresponding to each edge pixel point, the total number of each pixel point occupied by the lip line in the edge-sharpened image, the upper brightness threshold of the lip line, and the lower brightness threshold of the lip line into the AI ​​intelligent model to intelligently detect the entity blur level corresponding to the lip line; And wherein, each brightness value corresponding to each pixel point occupied by the lip line in the edge sharpened image, each brightness value corresponding to each edge pixel point, the total number of pixels occupied by the lip line in the edge sharpened image, the upper brightness threshold of the lip line and the lower brightness threshold of the lip line are simultaneously input into the AI ​​intelligent model to intelligently detect the entity blur level corresponding to the lip line. The AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings and also includes: the entity blur level corresponding to the lip line obtained by intelligent detection is in binary numerical representation.

[0016] Although the preferred embodiment of the present invention has been illustrated and described, it should be understood that the present invention is not limited thereto. Numerous modifications, changes, variations, substitutions and equivalents will occur to those skilled in the art without departing from the spirit and scope of the present invention as defined by the claims.

Claims

1. A live broadcast effect checking system, characterized in that: The system comprises: The user control mechanism is arranged in the live broadcast debugging device, and is used to control the focus area of ​​the live broadcast camera mechanism under the manual control of the user, and to send a lip line detection request when the focus area is the anchor's lips; A content filtering device, arranged in the live broadcast debugging device and connected to the user control mechanism, is used to perform statistical sorting filtering processing on the output picture of the live broadcast camera mechanism with the focus area being the anchor's lips when receiving the lip line detection request, so as to obtain and output a corresponding content filtering image; a logarithmic enhancement device, connected to the content filtering device, configured to perform image content enhancement processing based on logarithmic transformation on the received content filtered image, so as to obtain and output a corresponding logarithmic enhanced image; An edge sharpening device, connected to the exponential enhancement device, for performing edge sharpening processing on the received exponentially enhanced image to obtain and output a corresponding edge sharpened image; An intelligent detection device is provided in the live debugging device and connected to the edge sharpening device, and is used to receive the edge sharpening image, identify the brightness values ​​corresponding to the pixels occupied by the lip line in the edge sharpening image, and identify the brightness values ​​corresponding to the edge pixels in the pixels, and input the brightness values ​​corresponding to the pixels occupied by the lip line in the edge sharpening image, the brightness values ​​corresponding to the edge pixels, the total number of the pixels occupied by the lip line in the edge sharpening image, the upper brightness threshold of the lip line, and the lower brightness threshold of the lip line into the AI ​​intelligent model at the same time to intelligently detect the entity blur level corresponding to the lip line, wherein the AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings; A debugging response device is connected to the intelligent detection device, and is used to send a notification signal corresponding to the host's lip line being too blurred and dirty and needing to redraw the lip line when the entity blur level corresponding to the received lip line is greater than the set level limit; otherwise, temporarily suspend sending the notification signal corresponding to the host's lip line being too blurred and dirty and needing to redraw the lip line; Among them, the brightness values ​​corresponding to each pixel point occupied by the lip line in the edge-sharpened image, the brightness values ​​corresponding to each edge pixel point, the total number of pixels occupied by the lip line in the edge-sharpened image, the upper brightness threshold of the lip line and the lower brightness threshold of the lip line are simultaneously input into the AI ​​intelligent model to intelligently detect the entity blur level corresponding to the lip line. The AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings, including: the higher the entity blur level corresponding to the lip line, the blurrier the lip line representing the host's lip position.

2. The live broadcast effect checking system according to claim 1, characterized in that: The brightness values ​​corresponding to each pixel point occupied by the lip line in the edge-sharpened image, the brightness values ​​corresponding to each edge pixel point, the total number of pixels occupied by the lip line in the edge-sharpened image, the upper brightness threshold of the lip line and the lower brightness threshold of the lip line are simultaneously input into the AI ​​intelligent model to intelligently detect the entity blur level corresponding to the lip line. The AI ​​intelligent model is a deep neural network after executing a set number of multiple learnings and also includes: the value of the set number is monotonically positively correlated with the resolution of the live camera mechanism.

3. The live broadcast effect checking system according to claim 2, characterized in that: The system further comprises: An MCU control device, connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, respectively, for providing configuration operations of working parameters for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner; Among them, the MCU control device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide the configuration operation of working parameters for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, including: the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device share the same working parameter configuration interface.

4. The live broadcast effect checking system according to claim 3, characterized in that: The content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device use different configuration address data.

5. The live broadcast effect checking system according to claim 3, characterized in that: Establishing parallel connections of communication data links between the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device through parallel data buses; Among them, the parallel connection of the communication data link established between each of the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device through a parallel data bus includes: the parallel data bus is one of an 8-bit parallel data bus, a 16-bit parallel data bus and a 32-bit parallel data bus.

6. The live broadcast effect checking system according to claim 2, characterized in that: The system further comprises: A power support device, connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, respectively, for providing power distribution support of different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner; Among them, the power support device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide power distribution support with different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, including: among the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, two or more devices with the same working voltage requirements use the same power supply line.

7. The live broadcast effect checking system according to claim 6, characterized in that: The power support device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide power distribution support with different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, and also includes: the power support device is an uninterruptible power supply device.

8. The live broadcast effect checking system according to claim 7, characterized in that: The power support device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide power distribution support with different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, and also includes: providing different working voltages including a 3.3V working voltage for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner.

9. The live broadcast effect checking system according to claim 8, characterized in that: The power support device is respectively connected to the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device, and is used to provide power distribution support with different working voltages for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner, and also includes: providing different working voltages including a 5V working voltage for the content filtering device, the logarithmic enhancement device, the edge sharpening device and the intelligent detection device in a time-sharing manner.