Special effects strategy differentiation processing system for face recognition processing
Through the face recognition processing system, special effects processing is only performed when the anchor’s front face is detected, which solves the problem of indiscriminate and discontinuous special effects processing in the live broadcast room, and improves the quality of the live broadcast picture and the audience experience.
Patent Information
- Application Number
- CN202410118692.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-29
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-01-29
AI Technical Summary
In the live broadcast room, the existing technology cannot effectively distinguish and process the special effects of the anchor and the audience's face, resulting in the indiscriminate or discontinuous special effects processing, which brings discomfort to the audience.
A special effect strategy differentiation processing system for face recognition processing was designed. Through wired acquisition mechanism, multi-level processing mechanism, extraction equipment and special effect processing equipment, live broadcast environment images can be collected and processed in real time, and special effects can be performed only when the anchor's front face is detected. Handle to avoid abuse and misuse of special effects.
Customized face special effects processing for anchors is realized, avoiding the abuse and misuse of special effects, reducing the picture discomfort caused by special effects switching, and improving the quality of live broadcast pictures.
Smart Images

Figure CN117911238B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of face recognition, and in particular to a special effect strategy differentiation processing system for face recognition processing. Background Art
[0002] As a virtual scene, special effects can be made into any form. They can be separated like emoticons and science fiction videos and become part of people's creativity into reality. They can also be combined with real-life videos to beautify the picture. In addition to the beautifying effect of film and television works, the use of filter special effects in current short video platforms has played a beautifying role in faces and pictures, attracting more users to use them. For example, the embodiment of the invention application CN115278279A provides a method and system for processing audio and video data, wherein the method includes: obtaining audio and video data to be processed from the audio and video source, and the audio and video source includes professional audio and video acquisition equipment; according to the image processing configuration information obtained from the live broadcast management service system, the audio and video data is image processed to obtain streaming media data, and the image processing includes at least one of the following: beauty processing, filter processing, and subtitle processing; forwarding the streaming media data to the streaming media service system, so that the streaming media service system forwards the streaming media data to the corresponding user terminal according to the streaming media forwarding configuration information obtained from the live broadcast management service system.
[0003] However, there may be multiple people in the live broadcast room. If special effects processing is performed indiscriminately on all people, it is obviously impossible to highlight the anchor bound to the live broadcast room. If special effects processing is performed only on the anchor bound to the live broadcast room, when the anchor is in the process of switching between front face and side face, the special effects will also switch between being assigned to the anchor's face and not being assigned to the anchor's face, which will cause discomfort to the audience in the live broadcast room. Summary of the invention
[0004] The present invention proposes a special effect strategy differentiation processing system for face recognition processing, the system comprising:
[0005] A wired acquisition mechanism is arranged at the live broadcast site to perform signal acquisition of each frame of live broadcast environment image at uniform time intervals on the live broadcast environment, and output the current live broadcast environment image corresponding to the current moment in real time;
[0006] A multi-stage processing mechanism, including a directional sharpening device, a geometric correction device and a real-time filtering device, wherein the directional sharpening device is connected to the wired acquisition mechanism and is used to sequentially perform horizontal direction sharpening processing and vertical direction sharpening processing on the received current live broadcast environment image to obtain and output a corresponding directional sharpened image, the geometric correction device is connected to the directional sharpening device and is used to perform geometric correction processing on the received directional sharpened image to obtain and output a corresponding geometric correction image, and the real-time filtering device is connected to the geometric correction device and is used to perform Gaussian high-pass filtering processing on the received geometric correction image to obtain and output a corresponding real-time filtered image;
[0007] A first extraction device is connected to the multi-stage processing mechanism, and has a built-in static storage unit to pre-store an object contour reference image, wherein the object is the front face of the anchor corresponding to the live broadcast site, and uses an image area in the received real-time filtered image that best matches the object contour reference image and in which the number of pixel points occupying the real-time filtered image exceeds or equals a set number limit as the current reference area;
[0008] a second extraction device, connected to the first extraction device, for acquiring the current reference area and identifying each object pixel in the current reference area according to a preset target brightness value range in the current reference area;
[0009] a third extraction device, connected to the second extraction device, for performing a combination process on each object pixel point in the current reference area after removing isolated pixel points to obtain a pending object image block, and issuing a frontal face analysis success command when the proportion of the area occupied by the pending object image block in the real-time filtered image is greater than or equal to a preset proportion threshold, otherwise, issuing a frontal face analysis failure command;
[0010] A special effects processing device is connected to the third extraction device, and is used to perform facial special effects processing on the image blocks of the pending object when a frontal face analysis success command is received, and is also used to terminate the facial special effects processing on the image blocks of the pending object when a frontal face analysis failure command is received, wherein the facial special effects processing is a pre-set special effects processing type corresponding to the live broadcast site.
[0011] The technology of the present invention is based on a targeted visual processing mechanism and only executes the set facial special effects processing on the live broadcast scene when a customized detection mechanism is used to detect the presence of the host's face corresponding to the live broadcast scene in the live broadcast scene, thereby avoiding the abuse and misuse of the special effects of the live broadcast scene and solving the above-mentioned technical problems. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The embodiments of the present invention will be described below with reference to the accompanying drawings.
[0013] Figure 1 Schematic diagram of the internal structure of the special effects strategy differentiation processing system for face recognition processing according to implementation A of the present invention.
[0014] Figure 2 Schematic diagram of the internal structure of the special effects strategy differentiation processing system for face recognition processing according to implementation B of the present invention.
[0015] Figure 3 Schematic diagram of the internal structure of the special effects strategy differentiation processing system for face recognition processing according to implementation C of the present invention. DETAILED DESCRIPTION
[0016] The implementation scheme of the special effect strategy differentiation processing system for face recognition processing of the present invention will be described in detail below with reference to the accompanying drawings.
[0017] Implementation A
[0018] Figure 1 This is a schematic diagram of the internal structure of a special effect strategy differentiation processing system for face recognition processing according to Embodiment A of the present invention, wherein the system comprises:
[0019] A wired acquisition mechanism is arranged at the live broadcast site to perform signal acquisition of each frame of live broadcast environment image at uniform time intervals on the live broadcast environment, and output the current live broadcast environment image corresponding to the current moment in real time;
[0020] For example, a wired acquisition mechanism is provided at a live broadcast site to perform signal acquisition of each frame of live broadcast environment image at uniform time intervals on the live broadcast environment, and outputs the current live broadcast environment image corresponding to the current moment in real time, including: the wired acquisition mechanism includes a wired control interface, which is used to provide a control entry for signal acquisition of each frame of the live broadcast environment image;
[0021] A multi-stage processing mechanism, including a directional sharpening device, a geometric correction device and a real-time filtering device, wherein the directional sharpening device is connected to the wired acquisition mechanism and is used to sequentially perform horizontal direction sharpening processing and vertical direction sharpening processing on the received current live broadcast environment image to obtain and output a corresponding directional sharpened image, the geometric correction device is connected to the directional sharpening device and is used to perform geometric correction processing on the received directional sharpened image to obtain and output a corresponding geometric correction image, and the real-time filtering device is connected to the geometric correction device and is used to perform Gaussian high-pass filtering processing on the received geometric correction image to obtain and output a corresponding real-time filtered image;
[0022] A first extraction device is connected to the multi-stage processing mechanism, and has a built-in static storage unit to pre-store an object contour reference image, wherein the object is the front face of the anchor corresponding to the live broadcast site, and uses an image area in the received real-time filtered image that best matches the object contour reference image and in which the number of pixel points occupying the real-time filtered image exceeds or equals a set number limit as the current reference area;
[0023] a second extraction device, connected to the first extraction device, for acquiring the current reference area and identifying each object pixel in the current reference area according to a preset target brightness value range in the current reference area;
[0024] a third extraction device, connected to the second extraction device, for performing a combination process on each object pixel point in the current reference area after removing isolated pixel points to obtain a pending object image block, and issuing a frontal face analysis success command when the proportion of the area occupied by the pending object image block in the real-time filtered image is greater than or equal to a preset proportion threshold, otherwise, issuing a frontal face analysis failure command;
[0025] a special effects processing device connected to the third extraction device, and configured to execute special effects processing on the image blocks of the pending object when receiving a command indicating a success of the frontal face analysis, and to suspend the execution of special effects processing on the image blocks of the pending object when receiving a command indicating a failure of the frontal face analysis, wherein the special effects processing on the pending object is a pre-set special effects processing type corresponding to the live broadcast site;
[0026] Among them, when a command of successful frontal face analysis is received, facial special effects processing is performed on the image block of the pending object, and when a command of failed frontal face analysis is received, facial special effects processing is terminated on the image block of the pending object. The facial special effects processing is a pre-set special effects processing type corresponding to the live broadcast site, including: when a command of successful frontal face analysis is received, facial special effects processing is not performed on the image area outside the image block of the pending object in the real-time filtered image.
[0027] Implementation Plan B
[0028] Figure 2 Schematic diagram of the internal structure of the special effects strategy differentiation processing system for face recognition processing according to implementation B of the present invention.
[0029] The special effects strategy differentiation processing system for face recognition processing shown in embodiment B may include the following components:
[0030] A wired acquisition mechanism is arranged at the live broadcast site to perform signal acquisition of each frame of live broadcast environment image at uniform time intervals on the live broadcast environment, and output the current live broadcast environment image corresponding to the current moment in real time;
[0031] A multi-stage processing mechanism, including a directional sharpening device, a geometric correction device and a real-time filtering device, wherein the directional sharpening device is connected to the wired acquisition mechanism and is used to sequentially perform horizontal direction sharpening processing and vertical direction sharpening processing on the received current live broadcast environment image to obtain and output a corresponding directional sharpened image, the geometric correction device is connected to the directional sharpening device and is used to perform geometric correction processing on the received directional sharpened image to obtain and output a corresponding geometric correction image, and the real-time filtering device is connected to the geometric correction device and is used to perform Gaussian high-pass filtering processing on the received geometric correction image to obtain and output a corresponding real-time filtered image;
[0032] A first extraction device is connected to the multi-stage processing mechanism, and has a built-in static storage unit to pre-store an object contour reference image, wherein the object is the front face of the anchor corresponding to the live broadcast site, and uses an image area in the received real-time filtered image that best matches the object contour reference image and in which the number of pixel points occupying the real-time filtered image exceeds or equals a set number limit as the current reference area;
[0033] a second extraction device, connected to the first extraction device, for acquiring the current reference area and identifying each object pixel in the current reference area according to a preset target brightness value range in the current reference area;
[0034] a third extraction device, connected to the second extraction device, for performing a combination process on each object pixel point in the current reference area after removing isolated pixel points to obtain a pending object image block, and issuing a frontal face analysis success command when the proportion of the area occupied by the pending object image block in the real-time filtered image is greater than or equal to a preset proportion threshold, otherwise, issuing a frontal face analysis failure command;
[0035] a special effects processing device connected to the third extraction device, and configured to execute special effects processing on the image blocks of the pending object when receiving a command indicating a success of the frontal face analysis, and to suspend the execution of special effects processing on the image blocks of the pending object when receiving a command indicating a failure of the frontal face analysis, wherein the special effects processing on the pending object is a pre-set special effects processing type corresponding to the live broadcast site;
[0036] A height sensor device is connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device, respectively, and is used to measure the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively;
[0037] Wherein, the height sensor device is respectively connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device, and is used to respectively measure the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, including: the height sensor device includes a plurality of height measurement units, which are respectively connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device, so as to complete the measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device;
[0038] Wherein, the height sensor device includes a plurality of height measurement units, which are used to be connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, so as to complete the measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, including: the plurality of height measurement units are a plurality of height sensors, which are used to be connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, so as to complete the measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively;
[0039] Wherein, the plurality of height measurement units are a plurality of height sensors, which are used to be connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, so as to complete the respective measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, including: the plurality of height sensors have the same structure;
[0040] And wherein the multiple height measurement units are multiple height sensors, which are used to be connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, so as to complete the respective measurements of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, and also include: the multiple height sensors have the same height measurement upper limit value and height measurement lower limit value.
[0041] Implementation Plan C
[0042] Figure 3 Schematic diagram of the internal structure of the special effects strategy differentiation processing system for face recognition processing according to implementation C of the present invention.
[0043] The special effect strategy differentiation processing system for face recognition processing shown in embodiment C may include the following components:
[0044] A wired acquisition mechanism is arranged at the live broadcast site to perform signal acquisition of each frame of live broadcast environment image at uniform time intervals on the live broadcast environment, and output the current live broadcast environment image corresponding to the current moment in real time;
[0045] A multi-stage processing mechanism, including a directional sharpening device, a geometric correction device and a real-time filtering device, wherein the directional sharpening device is connected to the wired acquisition mechanism and is used to sequentially perform horizontal direction sharpening processing and vertical direction sharpening processing on the received current live broadcast environment image to obtain and output a corresponding directional sharpened image, the geometric correction device is connected to the directional sharpening device and is used to perform geometric correction processing on the received directional sharpened image to obtain and output a corresponding geometric correction image, and the real-time filtering device is connected to the geometric correction device and is used to perform Gaussian high-pass filtering processing on the received geometric correction image to obtain and output a corresponding real-time filtered image;
[0046] A first extraction device is connected to the multi-stage processing mechanism, and has a built-in static storage unit to pre-store an object contour reference image, wherein the object is the front face of the anchor corresponding to the live broadcast site, and uses an image area in the received real-time filtered image that best matches the object contour reference image and in which the number of pixel points occupying the real-time filtered image exceeds or equals a set number limit as the current reference area;
[0047] a second extraction device, connected to the first extraction device, for acquiring the current reference area and identifying each object pixel in the current reference area according to a preset target brightness value range in the current reference area;
[0048] a third extraction device, connected to the second extraction device, for performing a combination process on each object pixel point in the current reference area after removing isolated pixel points to obtain a pending object image block, and issuing a frontal face analysis success command when the proportion of the area occupied by the pending object image block in the real-time filtered image is greater than or equal to a preset proportion threshold, otherwise, issuing a frontal face analysis failure command;
[0049] a special effects processing device connected to the third extraction device, and configured to execute special effects processing on the image blocks of the pending object when receiving a command indicating a success of the frontal face analysis, and to suspend the execution of special effects processing on the image blocks of the pending object when receiving a command indicating a failure of the frontal face analysis, wherein the special effects processing on the pending object is a pre-set special effects processing type corresponding to the live broadcast site;
[0050] a playback processing device, connected to a plurality of height sensors of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, respectively, and used for synchronously playing the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively;
[0051] Wherein, the playback processing device is respectively connected to a plurality of height sensors of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, and is used to synchronously play the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, including: the playback processing device is a character display device;
[0052] And wherein, the playback processing device is respectively connected to multiple height sensors of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, and is used to synchronously play the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, including: the playback processing device is a voice playback device.
[0053] In addition, in the special effects strategy differentiation processing system for face recognition processing, when a frontal face analysis success command is received, facial special effects processing is performed on the image blocks of the pending object, and when a frontal face analysis failure command is received, facial special effects processing on the image blocks of the pending object is terminated. The facial special effects processing is a pre-set special effects processing type corresponding to the live broadcast site, and also includes: when a frontal face analysis failure command is received, facial special effects processing on the image blocks of the pending object is not performed on the real-time filtered image as a whole.
[0054] The technical points of the present invention shown in the embodiment of the present invention are:
[0055] 1. When the customized detection mechanism is used to detect that there is a front face of the host corresponding to the live broadcast scene in the live broadcast scene, the set facial special effects processing of the live broadcast scene is performed, thereby avoiding abuse and misuse of the special effects of the live broadcast scene;
[0056] Second, three extraction devices that execute different image processing modes are continuously used to complete the suspected object existence area that best matches the object contour reference image and has an area exceeding the limit in the real-time filtered image, and the number of object pixel points existing in the suspected object existence area is determined according to a preset object brightness value range in the suspected object existence area, and based on the number of object pixel points existing in the suspected object existence area, it is determined whether the suspected object existence area corresponds to the imaging area of the specific object;
[0057] 3. A multi-stage processing mechanism including directional sharpening equipment, geometric correction equipment and real-time filtering equipment is used to improve the imaging quality of the image and provide reliable data for subsequent host face detection.
[0058] Therefore, the special effects strategy differentiation processing system for face recognition processing of the present invention can solve the technical problems in the prior art that special effects are abused and misused, and that the switching of anchor special effects easily causes discomfort to the audience. On the basis of a targeted visual processing mechanism, it can execute the set facial special effects processing on the live broadcast scene only when a customized detection mechanism is used to detect the presence of the anchor's face corresponding to the live broadcast scene in the live broadcast scene. This can avoid the abuse and misuse of special effects while reducing the discomfort caused by the switching of special effects of the anchor bound to the live broadcast room.
[0059] As many apparently widely different embodiments of the present invention can be made without departing from the spirit and scope of the present invention, it is to be understood that the invention is not limited to the specific embodiments except as defined in the appended claims.
Claims
1. A special effect strategy differentiation processing system for face recognition processing, characterized in that: The system comprises: A wired acquisition mechanism is arranged at the live broadcast site to perform signal acquisition of each frame of live broadcast environment image at uniform time intervals on the live broadcast environment, and output the current live broadcast environment image corresponding to the current moment in real time; A multi-stage processing mechanism, including a directional sharpening device, a geometric correction device and a real-time filtering device, wherein the directional sharpening device is connected to the wired acquisition mechanism and is used to sequentially perform horizontal direction sharpening processing and vertical direction sharpening processing on the received current live broadcast environment image to obtain and output a corresponding directional sharpened image, the geometric correction device is connected to the directional sharpening device and is used to perform geometric correction processing on the received directional sharpened image to obtain and output a corresponding geometric correction image, and the real-time filtering device is connected to the geometric correction device and is used to perform Gaussian high-pass filtering processing on the received geometric correction image to obtain and output a corresponding real-time filtered image; A first extraction device is connected to the multi-stage processing mechanism, and has a built-in static storage unit to pre-store an object contour reference image, wherein the object is the front face of the anchor corresponding to the live broadcast site, and uses an image area in the received real-time filtered image that best matches the object contour reference image and in which the number of pixel points occupying the real-time filtered image exceeds or equals a set number limit as the current reference area; a second extraction device, connected to the first extraction device, for acquiring the current reference area and identifying each object pixel in the current reference area according to a preset target brightness value range in the current reference area; a third extraction device, connected to the second extraction device, for performing a combination process on each object pixel point in the current reference area after removing isolated pixel points to obtain a pending object image block, and issuing a frontal face analysis success command when the proportion of the area occupied by the pending object image block in the real-time filtered image is greater than or equal to a preset proportion threshold, otherwise, issuing a frontal face analysis failure command; a special effects processing device connected to the third extraction device, and configured to execute special effects processing on the image blocks of the pending object when receiving a command indicating a success of the frontal face analysis, and to suspend the execution of special effects processing on the image blocks of the pending object when receiving a command indicating a failure of the frontal face analysis, wherein the special effects processing on the pending object is a pre-set special effects processing type corresponding to the live broadcast site; When a frontal face analysis success command is received, facial special effects processing is performed on the pending object image block. When a frontal face analysis failure command is received, facial special effects processing is terminated on the pending object image block. The facial special effects processing is a pre-set special effects processing type corresponding to the live broadcast site, including: when a frontal face analysis success command is received, facial special effects processing is not performed on the image area outside the pending object image block in the real-time filtered image; when a frontal face analysis failure command is received, facial special effects processing is not performed on the pending object image block on the entire real-time filtered image.
2. The special effect strategy differentiation processing system for face recognition processing according to claim 1, characterized in that: The system further comprises: A height sensor device is connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device, respectively, and is used to measure the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively; Among them, the height sensor device is respectively connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device, and is used to respectively measure the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device. The height sensor device includes multiple height measurement units, which are respectively connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device to complete the measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device.
3. The special effect strategy differentiation processing system for face recognition processing as claimed in claim 2, characterized in that: The height sensor device includes multiple height measurement units, which are used to be connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, so as to complete the measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively. The multiple height measurement units are multiple height sensors, which are used to be connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, so as to complete the measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively.
4. The special effect strategy differentiation processing system for face recognition processing as claimed in claim 3, characterized in that: The multiple height measurement units are multiple height sensors, which are used to be connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, so as to complete the measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, including: the structures of the multiple height sensors are the same.
5. The special effect strategy differentiation processing system for face recognition processing as claimed in claim 4, characterized in that: The multiple height measurement units are multiple height sensors, which are used to be connected to the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively to complete the measurement of the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device respectively, and also include: the multiple height sensors have the same height measurement upper limit value and height measurement lower limit value.
6. The special effect strategy differentiation processing system for face recognition processing according to any one of claims 2 to 5, characterized in that: The system further comprises: The playback processing device is respectively connected to multiple height sensors of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, and is used to synchronously play the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device.
7. The special effect strategy differentiation processing system for face recognition processing according to claim 6, characterized in that: A playback processing device is respectively connected to multiple height sensors of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, and is used to synchronously play the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, including: the playback processing device is a character display device.
8. The special effect strategy differentiation processing system for face recognition processing as claimed in claim 6, characterized in that: A playback processing device is respectively connected to multiple height sensors of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, and is used to synchronously play the current real-time height values of the first extraction device, the second extraction device, the third extraction device and the special effect processing device, including: the playback processing device is a voice playback device.
