Face state determination system

Abstract

The invention relates to a face state judgment system which comprises a signal capturing device, a midpoint filtering device, a curve processing device, a customized sharpening device, a model reconstruction device, an information analysis device, an intelligent detection device and an alarm execution device. The face state judgment system is compact in structure and wide in application.

Description

Technical Field

[0001] The present invention relates to the field of face recognition, and in particular to a face state judgment system. Background Art

[0002] Online live streaming has become an important form of e-commerce, namely live streaming with goods. The anchor promotes and sells products through live streaming, and the audience can directly purchase through the link, which not only increases the sales of products, but also drives the development of related industries. The rise of online live streaming has provided a large number of employment opportunities for society, including: anchors, planners, technical support and other professions. This not only alleviates employment pressure, but also provides many people with the opportunity to realize their self-worth. Enterprises and brands can use online live streaming to carry out activities such as product launches and brand promotions, thereby enhancing brand awareness and influence. At the same time, through cooperation with anchors, brands can also reach target consumers more directly and improve marketing effectiveness.

[0003] In order to create brand awareness and influence for online live broadcasts, there are strict requirements on the quality of the live broadcast images. For example, in order to highlight the facial effect, some anchors apply too much foundation on their faces and add too many special effects, resulting in the anchor's face not having normal blood color, and the live broadcast image appears seriously distorted. Therefore, a high-precision detection mechanism is needed to effectively identify the degree of deviation between the human face in the live broadcast image and the standard color corresponding to the human face blood color. Summary of the invention

[0004] In order to solve the technical problems in the related fields, the present invention provides a facial state judgment system, which can perform multiple training on a deep neural network to obtain a deep neural network model, wherein the deep neural network includes multiple hidden layers, a single input layer and a single output layer, and the number of the multiple hidden layers is positively correlated with the number of pixels of a customized sharpened image, thereby completing the structural customization of the deep neural network model, and converting the hue component value interval, brightness component value interval, saturation component value interval of the standard color corresponding to the blood color of the human face, the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face, the number of pixels of the customized sharpened image and the number of pixels of each pixel point occupied by the human face into the facial state judgment system. The total number is used as the item-by-item input content of the deep neural network model to run the deep neural network model and obtain the color abnormality value of the human face output by the deep neural network model. The larger the color abnormality value of the human face, the greater the deviation of the human face from the standard color corresponding to the blood color of the human face. An alarm execution device is also used to execute a blood color deficiency alarm operation to warn the anchor that the current foundation is too thick when the received color abnormality value of the human face is greater than or equal to a preset color abnormality threshold. It is also used to stop executing the blood color deficiency alarm operation and send a blood color recognition signal when the received color abnormality value of the human face is less than the preset color abnormality threshold, thereby completing the targeted control of the live broadcast effect based on the intelligent detection result of the anchor's facial blood color.

[0005] According to the present invention, a facial state determination system is provided, the system comprising: A signal capture device, arranged in the live broadcast test terminal, is used to perform an image signal capture operation on the face of the current anchor to obtain and output a corresponding real-time test image; A midpoint filtering device, which is arranged in the live test terminal and connected to the signal capture device, is used to perform midpoint filtering processing on the received real-time test image to obtain and output a corresponding midpoint filtered image; A curve processing device, connected to the midpoint filtering device, for adjusting the maximum curvature of each curve in the received midpoint filtering image to be lower than or equal to a preset curve maximum curvature threshold, so as to obtain and output a corresponding curvature adjusted image; A customized sharpening device, connected to the curve processing device, for performing a Kirsch operator-based image content sharpening process on the received curve processing image to obtain and output a corresponding customized sharpened image; A model reconstruction device, disposed in the live broadcast test terminal, for performing multiple trainings on a deep neural network to obtain a deep neural network model, wherein the deep neural network includes a plurality of hidden layers, a single input layer, and a single output layer, and the number of the plurality of hidden layers is positively correlated with the number of pixels of the customized sharpened image; An information analysis device, arranged in the live broadcast test terminal and connected to the customized sharpening device, is used to analyze the hue component value, brightness component value and saturation component value of each pixel point occupied by the human face in the customized sharpened image in the HSV space, so as to obtain the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face; An intelligent detection device is provided in the live broadcast test terminal and is connected to the model reconstruction device and the information analysis device, and is used to use the hue component value interval, brightness component value interval, saturation component value interval of the standard color corresponding to the human face blood color, the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face, the number of pixels of the customized sharpened image and the total number of each pixel point occupied by the human face as the input content of the deep neural network model item by item, so as to run the deep neural network model and obtain the color abnormality value of the human face output by the deep neural network model, wherein the larger the color abnormality value of the human face is, the greater the deviation of the human face from the standard color corresponding to the human face blood color is; An alarm execution device is arranged in the live broadcast test terminal and connected to the intelligent detection device, and is used to execute a blood deficiency alarm operation when the received color abnormality value of the human face is greater than or equal to a preset color abnormality threshold, and is also used to stop executing the blood deficiency alarm operation and send a blood color recognition signal when the received color abnormality value of the human face is less than the preset color abnormality threshold.

[0006] It can be seen that the present invention has at least the following three significant substantive features: Substantive feature 1: performing multiple training on a deep neural network to obtain a deep neural network model, wherein the deep neural network includes multiple hidden layers, a single input layer, and a single output layer, and the number of the multiple hidden layers is positively correlated with the number of pixels of the customized sharpened image, thereby completing the structural customization of the deep neural network model; Substantive feature 2: The hue component value interval, brightness component value interval, saturation component value interval of the standard color corresponding to the human face blood color, the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face, the number of pixels of the customized sharpened image and the total number of each pixel point occupied by the human face are used as the input content of the deep neural network model one by one, so as to run the deep neural network model and obtain the color abnormality value of the human face output by the deep neural network model. The larger the color abnormality value of the human face is, the greater the deviation of the human face from the standard color corresponding to the human face blood color is. Substantive feature three: The alarm execution device is used to execute the blood deficiency alarm operation to warn the anchor that the current foundation is too thick when the received color abnormality value of the human face is greater than or equal to the preset color abnormality threshold. It is also used to stop executing the blood deficiency alarm operation and send a blood recognition signal when the received color abnormality value of the human face is less than the preset color abnormality threshold, thereby completing the targeted control of the live broadcast effect based on the intelligent detection results of the anchor's facial blood color.

[0007] The facial state judgment system of the present invention has a compact structure and wide application. Since a deep neural network model can be used to intelligently identify abnormal color values ​​of the human face based on various customized and filtered basic information, targeted control of the live broadcast effect based on the intelligent detection results of the host's facial blood color can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The embodiments of the present invention will be described below with reference to the accompanying drawings, wherein: Figure 1 FIG. 1 is a diagram showing the internal structure of a facial state determination system according to a first embodiment of the present invention.

[0009] Figure 2 FIG. 1 is a diagram showing the internal structure of a facial state determination system according to a second embodiment of the present invention.

[0010] Figure 3 FIG. 4 is a diagram showing the internal structure of a facial state determination system according to a third embodiment of the present invention. DETAILED DESCRIPTION

[0011] The following is a detailed description of an embodiment of the face state judgment system of the present invention with reference to the accompanying drawings.

[0012] Figure 1 The internal structure diagram of the facial state judgment system according to the first embodiment of the present invention is shown, and the system includes: A signal capture device, arranged in the live broadcast test terminal, is used to perform an image signal capture operation on the face of the current anchor to obtain and output a corresponding real-time test image; A midpoint filtering device, which is arranged in the live test terminal and connected to the signal capture device, is used to perform midpoint filtering processing on the received real-time test image to obtain and output a corresponding midpoint filtered image; For example, the midpoint filtering device is arranged in the live test terminal and connected to the signal capture device, and is used to perform midpoint filtering processing on the received real-time test image to obtain and output the corresponding midpoint filtered image, including: selecting an ASIC device to implement the midpoint filtering device, which is arranged in the live test terminal and connected to the signal capture device, and is used to perform midpoint filtering processing on the received real-time test image to obtain and output the corresponding midpoint filtered image; A curve processing device, connected to the midpoint filtering device, for adjusting the maximum curvature of each curve in the received midpoint filtering image to be lower than or equal to a preset curve maximum curvature threshold, so as to obtain and output a corresponding curvature adjusted image; A customized sharpening device, connected to the curve processing device, for performing a Kirsch operator-based image content sharpening process on the received curve processing image to obtain and output a corresponding customized sharpened image; A model reconstruction device, disposed in the live broadcast test terminal, for performing multiple trainings on a deep neural network to obtain a deep neural network model, wherein the deep neural network includes a plurality of hidden layers, a single input layer, and a single output layer, and the number of the plurality of hidden layers is positively correlated with the number of pixels of the customized sharpened image; An information analysis device, arranged in the live broadcast test terminal and connected to the customized sharpening device, is used to analyze the hue component value, brightness component value and saturation component value of each pixel point occupied by the human face in the customized sharpened image in the HSV space, so as to obtain the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face; An intelligent detection device is provided in the live broadcast test terminal and is connected to the model reconstruction device and the information analysis device, and is used to use the hue component value interval, brightness component value interval, saturation component value interval of the standard color corresponding to the human face blood color, the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face, the number of pixels of the customized sharpened image and the total number of each pixel point occupied by the human face as the input content of the deep neural network model item by item, so as to run the deep neural network model and obtain the color abnormality value of the human face output by the deep neural network model, wherein the larger the color abnormality value of the human face is, the greater the deviation of the human face from the standard color corresponding to the human face blood color is; an alarm execution device, arranged in the live broadcast test terminal and connected to the intelligent detection device, for executing a blood color deficiency alarm operation when the received color abnormality value of the human face is greater than or equal to a preset color abnormality threshold, and for stopping the execution of the blood color deficiency alarm operation and issuing a blood color recognition signal when the received color abnormality value of the human face is less than the preset color abnormality threshold; Among them, the hue component value interval, brightness component value interval, saturation component value interval of the standard color corresponding to the human face blood color, the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face, the number of pixels of the customized sharpened image and the total number of each pixel point occupied by the human face are used as the input content of the deep neural network model item by item to run the deep neural network model to obtain the color abnormality value of the human face output by the deep neural network model. The larger the color abnormality value of the human face is, the greater the degree of deviation of the human face from the standard color corresponding to the human face blood color is, including: the hue component value interval of the standard color corresponding to the human face blood color is limited by a preset hue component upper limit threshold and a preset hue component lower limit threshold, the brightness component value interval of the standard color corresponding to the human face blood color is limited by a preset brightness component upper limit threshold and a preset brightness component lower limit threshold, and the saturation component value interval of the standard color corresponding to the human face blood color is limited by a preset saturation component upper limit threshold and a preset saturation component lower limit threshold.

[0013] Figure 2 FIG. 1 is a diagram showing the internal structure of a facial state determination system according to a second embodiment of the present invention.

[0014] exist Figure 2 In, with Figure 1 Differently, the facial state determination system shown in the second embodiment of the present invention may further include: A voltage supply interface, connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device, respectively, for providing the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with their respective required working voltages; The voltage supply interface is connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively, and is used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with the required working voltages respectively, including: the working voltages required by the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively include 3.3 volts and 5 volts; The voltage supply interface is connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively, and is used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with the required working voltages respectively, and further includes: the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device are arranged around the voltage supply interface and are isolated from the voltage supply interface by different electromagnetic shielding mechanisms respectively; The midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device are arranged around the voltage supply interface and are isolated from the voltage supply interface by using different electromagnetic shielding mechanisms respectively, including: the distances from the different electromagnetic shielding mechanisms respectively used by the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device to the voltage supply interface are equal; And wherein, the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device are arranged around the voltage supply interface and are isolated from the voltage supply interface by different electromagnetic shielding mechanisms respectively, and also include: the internal structures of the different electromagnetic shielding mechanisms respectively used by the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device are the same.

[0015] Figure 3 FIG. 4 is a diagram showing the internal structure of a facial state determination system according to a third embodiment of the present invention.

[0016] exist Figure 3 In, with Figure 1 Differently, the facial state determination system shown in the third embodiment of the present invention may further include: The radiation sensing device is connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively, and is used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with respective numerical sensing operations of real-time electromagnetic radiation quantities; The radiation sensing device is connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively, and is used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with respective numerical sensing operations of real-time electromagnetic radiation quantities, including: the radiation sensing device includes a plurality of radiation sensing units, which are used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with respective numerical sensing operations of real-time electromagnetic radiation quantities; The radiation sensing device includes a plurality of radiation sensing units, which are used to respectively provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with respective numerical sensing operations of the real-time electromagnetic radiation amount, including: the internal structures of the plurality of radiation sensing units are the same; And wherein, the radiation sensing device includes multiple radiation sensing units, which are used to provide respective numerical sensing operations of real-time electromagnetic radiation quantities for the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively, including: the multiple radiation sensing units are respectively equipped with respective radiation alarm components.

[0017] In addition, in the facial state judgment system, the hue component value interval of the standard color corresponding to the facial blood color of the human body is limited by a preset hue component upper limit threshold and a preset hue component lower limit threshold, the brightness component value interval of the standard color corresponding to the facial blood color of the human body is limited by a preset brightness component upper limit threshold and a preset brightness component lower limit threshold, and the saturation component value interval of the standard color corresponding to the facial blood color of the human body is limited by a preset saturation component upper limit threshold and a preset saturation component lower limit threshold, including: the value of any threshold among the preset hue component upper limit threshold, the preset hue component lower limit threshold, the preset brightness component upper limit threshold, the preset brightness component lower limit threshold, the preset saturation component upper limit threshold and the preset saturation component lower limit threshold are all between 0-255.

[0018] Although the present invention has been specifically shown and described with reference to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present invention as set forth in the appended claims. In addition, although elements of the present invention are described or claimed in the singular, the plural is also encompassed unless expressly stated to be limited to the singular.

Claims

1. A facial state judgment system, characterized in that: The system comprises: A signal capture device, arranged in the live broadcast test terminal, is used to perform an image signal capture operation on the face of the current anchor to obtain and output a corresponding real-time test image; A midpoint filtering device, which is arranged in the live test terminal and connected to the signal capture device, is used to perform midpoint filtering processing on the received real-time test image to obtain and output a corresponding midpoint filtered image; A curve processing device, connected to the midpoint filtering device, for adjusting the maximum curvature of each curve in the received midpoint filtering image to be lower than or equal to a preset curve maximum curvature threshold, so as to obtain and output a corresponding curvature adjusted image; A customized sharpening device, connected to the curve processing device, for performing a Kirsch operator-based image content sharpening process on the received curve processing image to obtain and output a corresponding customized sharpened image; A model reconstruction device, disposed in the live broadcast test terminal, for performing multiple trainings on a deep neural network to obtain a deep neural network model, wherein the deep neural network includes a plurality of hidden layers, a single input layer, and a single output layer, and the number of the plurality of hidden layers is positively correlated with the number of pixels of the customized sharpened image; An information analysis device, arranged in the live test terminal and connected to the customized sharpening device, is used to analyze the hue component value, brightness component value and saturation component value of each pixel point occupied by the human face in the customized sharpened image in the HSV space, so as to obtain the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face; An intelligent detection device is provided in the live broadcast test terminal and is connected to the model reconstruction device and the information analysis device, and is used to use the hue component value interval, brightness component value interval, saturation component value interval of the standard color corresponding to the human face blood color, the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face, the number of pixels of the customized sharpened image and the total number of each pixel point occupied by the human face as the input content of the deep neural network model item by item, so as to run the deep neural network model and obtain the color abnormality value of the human face output by the deep neural network model, wherein the larger the color abnormality value of the human face is, the greater the deviation of the human face from the standard color corresponding to the human face blood color is; An alarm execution device is arranged in the live broadcast test terminal and connected to the intelligent detection device, and is used to execute a blood deficiency alarm operation when the received color abnormality value of the human face is greater than or equal to a preset color abnormality threshold, and is also used to stop executing the blood deficiency alarm operation and send a blood color recognition signal when the received color abnormality value of the human face is less than the preset color abnormality threshold.

2. The facial state determination system according to claim 1, wherein: The hue component value interval, brightness component value interval, saturation component value interval of the standard color corresponding to the human face blood color, the hue component values, brightness component values ​​and saturation component values ​​corresponding to each pixel point occupied by the human face, the number of pixels of the customized sharpened image and the total number of each pixel point occupied by the human face are used as the input content of the deep neural network model item by item to run the deep neural network model to obtain the color abnormality value of the human face output by the deep neural network model. The larger the color abnormality value of the human face is, the greater the degree of deviation of the human face from the standard color corresponding to the human face blood color is, including: the hue component value interval of the standard color corresponding to the human face blood color is limited by a preset hue component upper limit threshold and a preset hue component lower limit threshold, the brightness component value interval of the standard color corresponding to the human face blood color is limited by a preset brightness component upper limit threshold and a preset brightness component lower limit threshold, and the saturation component value interval of the standard color corresponding to the human face blood color is limited by a preset saturation component upper limit threshold and a preset saturation component lower limit threshold.

3. The facial state determination system according to claim 2, characterized in that: The system further comprises: A voltage supply interface, connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device, respectively, for providing the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with their respective required working voltages; Among them, the voltage supply interface is connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively, and is used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with the required working voltages respectively, including: the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively required working voltages include 3.3 volts and 5 volts.

4. The facial state determination system according to claim 3, wherein: A voltage supply interface is respectively connected to a midpoint filtering device, a curve processing device, a customized sharpening device and a model reconstruction device, and is used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with their respective required working voltages. The midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device are also included. The midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device are arranged around the voltage supply interface and are respectively isolated from the voltage supply interface by different electromagnetic shielding mechanisms.

5. The facial state determination system according to claim 4, characterized in that: The midpoint filtering device, curve processing device, customized sharpening device and model reconstruction device are configured around the voltage supply interface and are isolated from the voltage supply interface by using different electromagnetic shielding mechanisms respectively, including: the midpoint filtering device, curve processing device, customized sharpening device and model reconstruction device are at equal distances from the voltage supply interface by the different electromagnetic shielding mechanisms respectively used.

6. The facial state determination system according to claim 4, wherein: The midpoint filtering device, curve processing device, customized sharpening device and model reconstruction device are configured around the voltage supply interface and are isolated from the voltage supply interface by using different electromagnetic shielding mechanisms respectively. The midpoint filtering device, curve processing device, customized sharpening device and model reconstruction device also have the same internal structure as the different electromagnetic shielding mechanisms respectively used by the midpoint filtering device, curve processing device, customized sharpening device and model reconstruction device.

7. The facial state determination system according to claim 2, characterized in that: The system further comprises: The radiation sensing device is connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively, and is used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with respective numerical sensing operations of real-time electromagnetic radiation quantities; Among them, the radiation sensing device is connected to the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device respectively, and is used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with respective numerical sensing operations of real-time electromagnetic radiation quantities, including: the radiation sensing device includes multiple radiation sensing units, which are used to provide the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device with respective numerical sensing operations of real-time electromagnetic radiation quantities.

8. The facial state determination system according to claim 7, characterized in that: The radiation sensing device includes a plurality of radiation sensing units, which are used to provide respective numerical sensing operations of real-time electromagnetic radiation quantities for a midpoint filtering device, a curve processing device, a custom sharpening device and a model reconstruction device, including: the internal structures of the plurality of radiation sensing units are the same.

9. The facial state determination system according to claim 7, characterized in that: The radiation sensing device includes a plurality of radiation sensing units, which are used to provide respective numerical sensing operations of real-time electromagnetic radiation quantities for the midpoint filtering device, the curve processing device, the customized sharpening device and the model reconstruction device, respectively. The operations include: the plurality of radiation sensing units are respectively equipped with respective radiation alarm components.

Images