Live broadcast health degree and professional degree monitoring method and device, equipment and medium

Abstract

The invention provides a live broadcast health degree and professional degree monitoring method and device, equipment and a medium. The method comprises the following steps: acquiring video stream information; intercepting key frame information based on the video stream information; based on the key frame information, analyzing to obtain face information, limb information and article information; the face information, the limb information and the article information are analyzed, and whether the face information, the limb information and the article information meet preset conditions or not is judged; under the condition that at least one of the face information, the limb information and the article information does not meet a preset condition, performing early warning on the violation information according to a preset priority, and displaying related early warning information; wherein the violation information indicates information which does not meet a preset condition; processing the video stream information based on the early warning information to obtain a processed video stream; and pushing the processed video stream to an audience side for playing. According to the scheme, the health degree and professional degree of live broadcast can be ensured, the live broadcast quality and efficiency are improved, and the operation cost and risk are reduced.

Description

Technical Field

[0001] The present application relates to the technical field of e-commerce live broadcast monitoring, and in particular to a method, device, equipment and medium for monitoring the health and professionalism of a live broadcast. Background Art

[0002] With the rapid expansion of the live broadcast industry, live broadcast content has become more diverse and complex, which has also brought a series of problems to the anchor management and live broadcast quality of live broadcast companies. For example, if the anchor fails to start the broadcast at the scheduled time, the anchor's position is offset during the live broadcast, or the items displayed are too far or too close to the camera, the anchor wears competing clothing or accessories during the broadcast, or the anchor speaks banned words, causing the platform to deduct points, etc., it will cause many problems, including affecting the professionalism and image of the live broadcast, distracting the audience's attention, affecting the live broadcast effect, possibly causing live broadcast errors, reducing audience interaction, and affecting live broadcast data.

[0003] However, most live broadcast companies still rely on manual management or post-management by third-party platforms to monitor the health and professionalism of live broadcasts. This approach has many disadvantages: on the one hand, manual management requires a lot of human resources and is inefficient; on the other hand, post-management by third-party platforms can only deal with problems that have already occurred, and cannot achieve pre-prevention and real-time monitoring. Therefore, it is necessary to develop an effective live broadcast monitoring system to ensure the health and professionalism of live broadcasts. Summary of the invention

[0004] In order to overcome the defects of the above-mentioned prior art, the present application provides a method, device, equipment and medium for monitoring the health and professionalism of live broadcast, which can ensure the health and professionalism of live broadcast, improve the quality and efficiency of live broadcast, and reduce operating costs and risks.

[0005] In order to solve the above technical problems, this application provides the following technical solutions: According to a first aspect of an embodiment of the present application, a method for monitoring the health and professionalism of a live broadcast is provided, comprising: Get video stream information; Based on the video stream information, key frame information is intercepted; the key frame information carries key frame timeline information; Based on the key frame information, facial information, body information and object information are obtained by parsing; Analyze the face information, the body information, and the object information to determine whether the face information, the body information, and the object information meet preset conditions; In the case where at least one of the face information, the body information and the object information does not meet the preset condition, an early warning is issued for the violation information according to the preset priority, and relevant early warning information is displayed; wherein the violation information indicates the information that does not meet the preset condition; Processing the video stream information based on the warning information to obtain a processed video stream; The processed video stream is pushed to the audience end for playback.

[0006] In an exemplary embodiment, the extracting key frame information based on the video stream information includes: Using a key frame extraction algorithm to extract the key frame information from the video stream information; Among them, the key frame extraction algorithm adopts one of the methods based on shot boundaries, image content, motion analysis, clustering, deep learning, frame difference, and dynamic frame.

[0007] In an exemplary embodiment, the parsing based on the key frame information to obtain the face information, body information and object information includes: Performing image preprocessing on the key frame pictures in the key frame information to obtain preprocessed pictures; Locate the face area in the preprocessed image based on a face detection algorithm; locate the limb key points in the preprocessed image based on a limb detection algorithm; and locate the object area in the preprocessed image based on an object detection algorithm; Features are extracted from the face area to obtain the face information; posture estimation is performed based on the limb key points, and relevant features are extracted and classified to obtain the limb information; and features are extracted from the object area to obtain the object information.

[0008] In an exemplary embodiment, the analyzing the face information, the body information, and the object information to determine whether the face information, the body information, and the object information meet preset conditions respectively includes: Based on the key frame picture, obtain the picture width and picture height; The face information is analyzed based on the screen width and the live broadcast category identifier carried in the key frame information; and the limb information is analyzed based on the limb information library; and the item information is analyzed based on the competitor information library, the screen width and the screen height; Determine whether the ratio information between the face frame coordinates in the face information and the screen width meets the preset conditions; and determine whether the body information is an indecent gesture or action; and determine whether the item corresponding to the item information is a competing product, and determine whether the ratio information between the item frame coordinates in the item information and the screen width and the screen height meets the preset conditions.

[0009] In an exemplary embodiment, the method further comprises: Based on the video stream information, parse and obtain audio information; Analyze the audio information to determine whether the audio information meets the preset condition; When the audio information does not meet the preset condition, an early warning is issued for the violation information according to the preset priority, and the relevant early warning information is displayed.

[0010] In an exemplary embodiment, the step of issuing an early warning for violation information according to a preset priority level includes: According to the wearable competing products, audio information and other information, the illegal information is warned in descending order of priority; wherein the other information includes facial information, body information and item location.

[0011] In an exemplary embodiment, the method further comprises: Get live broadcast room data; Determine whether the live broadcast room is on according to the live broadcast room data; When the live broadcast room is started, the video stream information is obtained.

[0012] According to a second aspect of an embodiment of the present application, a device for monitoring the health and professionalism of a live broadcast is provided, and a method for monitoring the health and professionalism of a live broadcast is adopted as described in any one of the above, and the device includes: A video stream acquisition module is used to acquire video stream information; A key frame interception module, used to intercept key frame information based on the video stream information; the key frame information carries key frame timeline information; An information analysis module, used to analyze and obtain face information, body information and object information based on the key frame information; An information analysis module, used to analyze the face information, the body information and the object information, and respectively determine whether the face information, the body information and the object information meet preset conditions; An information warning module, used to issue a warning for violation information according to a preset priority and display relevant warning information when at least one of the face information, the body information and the object information does not meet the preset condition; wherein the violation information indicates information that does not meet the preset condition; A video stream processing module, used for processing the video stream information based on the warning information to obtain a processed video stream; The push module is used to push the processed video stream to the audience end for playback.

[0013] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a method for monitoring the health and professionalism of a live broadcast as described in any one of the above.

[0014] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement a live broadcast health and professionalism monitoring method as described in any one of the above.

[0015] By adopting the above technical solution, the present application has the following beneficial effects: By real-time monitoring of broadcast information, facial information, body information, object information and audio information, it is possible to promptly discover whether the host has not started the broadcast at the scheduled time, whether the host's position is offset during the live broadcast or the items displayed are too far or too close to the camera, whether the host is wearing competing clothing or accessories, whether the host speaks banned words, whether the host shows indecent gestures or actions, and perform timely pre-processing to avoid the outflow of live broadcast clips with greater impact to third-party live broadcast platforms. This can ensure the health and professionalism of live broadcasts, improve the quality and efficiency of live broadcasts, reduce operating costs and risks, and reduce the workload of manual review. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of a method for monitoring the health and professionalism of a live broadcast provided in an embodiment of the present application; Figure 2A schematic diagram of a practical application of a live broadcast face, body, and object monitoring method provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a facial information warning provided in an embodiment of the present application; Figure 4 A schematic diagram of a practical application of a live audio monitoring method provided in an embodiment of the present application; Figure 5 A schematic diagram of a practical application of a large-screen video display algorithm provided in an embodiment of the present application; Figure 6 A structural block diagram of a live broadcast health and professionalism monitoring device provided in an embodiment of the present application; Figure 7 A hardware structure block diagram of an electronic device that runs a live broadcast health and professionalism monitoring method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] As used herein, "one embodiment" or "embodiment" refers to a The specific features, structures or characteristics in at least one implementation of the present application. In the description of the embodiments of the present application, it is to be understood that the orientation or positional relationship indicated by the terms "upper", "lower", "top", "bottom", etc. is based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" can include one or more of the features explicitly or implicitly. Moreover, the terms "first", "second", etc. are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here.

[0020] See also Figure 1, which is a flow chart of a live broadcast health and professionalism monitoring method provided in an embodiment of the present application, the live broadcast health and professionalism monitoring method comprising: Step S1: Obtain video stream information; Step S2: based on the video stream information, intercept key frame information; the key frame information carries key frame timeline information; Step S3: Based on the key frame information, analyze and obtain face information, body information and object information; Step S4: analyzing the face information, the body information and the object information, and determining whether the face information, the body information and the object information meet preset conditions; Step S5: when at least one of the face information, the body information and the object information does not meet the preset condition, an early warning is issued for the violation information according to the preset priority, and relevant early warning information is displayed; wherein the violation information indicates the information that does not meet the preset condition; Step S6: Processing the video stream information based on the warning information to obtain a processed video stream; Step S7: Push the processed video stream to the viewer end for playback.

[0021] In a specific embodiment, before executing step S1, the live broadcast health and professionalism monitoring method may further include: Get live broadcast room data; Determine whether the live broadcast room is on according to the live broadcast room data; When the live broadcast room is on, the video stream information is obtained; when the live broadcast room is not on, the push information directly notifies the administrator of the corresponding live broadcast room.

[0022] In a specific embodiment, The above step S3 may further include: Based on the video stream information, parse and obtain audio information; The above step S4 may further include: Analyze the audio information to determine whether the audio information meets the preset condition; The above step S5 may further include: When the audio information does not meet the preset condition, an early warning is issued for the violation information according to the preset priority, and the relevant early warning information is displayed; That is, the above-mentioned step S5 can be: when at least one of the facial information, the limb information, the object information and the audio information does not meet the preset conditions, an alert is issued for the violation information according to the preset priority, and relevant warning information is displayed; wherein, the violation information indicates information that does not meet the preset conditions.

[0023] In a specific embodiment, the live broadcast health and professionalism monitoring method may also include: When the facial information, the body information, the object information and the audio information all meet the preset conditions, the video stream information is pushed to the viewer end for playback.

[0024] In an optional embodiment, the above step S2 may include: Using a key frame extraction algorithm to extract the key frame information from the video stream information; Among them, the key frame extraction algorithm adopts one of the methods based on shot boundaries, image content, motion analysis, clustering, deep learning, frame difference, and dynamic frame.

[0025] Specifically, the shot boundary-based method includes dividing the video into multiple shots and then extracting key frames from each shot; The image content-based method extracts key frames by analyzing the color, texture, shape and other features of the video frame; The motion analysis-based method extracts key frames by detecting motion information in the video; The clustering-based method groups similar video frames and then selects one or more frames from each group as key frames; commonly used clustering algorithms include K-means, hierarchical clustering, etc. The deep learning based method uses a deep learning model to extract key frames; The frame difference based method extracts key frames by calculating the differences (such as changes in pixel intensity) between adjacent frames; The dynamic frame-based method first determines the representative grayscale set of each pixel in the dynamic frame, and then determines the key frame by calculating the distance between each frame in the video and the dynamic frame.

[0026] In an optional embodiment, the facial information, the body information, the object information and the audio information in the above step S3 are obtained by synchronous analysis, and there is no priority relationship.

[0027] Specifically, the parsing based on the key frame information to obtain the face information, body information and object information may include: Performing image preprocessing on the key frame pictures in the key frame information to obtain preprocessed pictures; wherein the image preprocessing may include normalization and denoising; Locating the face area in the preprocessed image based on a face detection algorithm; locating the limb key points in the preprocessed image based on a limb detection algorithm; and locating the object area in the preprocessed image based on an object detection algorithm; wherein the face detection algorithm may include one of MTCNN (Multi-task Cascaded ConvolutionalNetworks), SSD (Single Shot MultiBox Detector) or YOLO (You Only Look Once); the limb detection algorithm may include one of OpenPose, HRNet, etc.; the object detection algorithm may include one of Faster R-CNN, SSD, YOLO, etc.; Features are extracted from the face area to obtain the face information; posture estimation is performed based on the limb key points, and relevant features are extracted and classified to obtain the limb information; and features are extracted from the object area to obtain the object information; wherein, a pre-trained deep learning model (such as FaceNet, VGGFace) can be used to obtain the face information; a posture estimation model or a custom algorithm can be used to perform posture estimation, and a pre-trained deep learning model can be used to obtain the limb information; a pre-trained deep learning model can be used to obtain the object information.

[0028] The parsing to obtain the audio information based on the video stream information may include: Extracting audio based on the video stream information; The audio is analyzed to obtain volume information, and the voice content in the audio is converted into text information.

[0029] In an optional embodiment, the analysis of the facial information, the body information, the object information and the audio information in the above step S4 is performed simultaneously without any priority relationship.

[0030] Specifically, analyzing the face information, the body information, and the object information to determine whether the face information, the body information, and the object information meet preset conditions may include: Based on the key frame picture, obtain the picture width and picture height; The face information is analyzed based on the screen width and the live broadcast category identifier carried in the key frame information; and the limb information is analyzed based on the limb information library; and the item information is analyzed based on the competitor information library, the screen width and the screen height; Determine whether the ratio information between the face frame coordinates in the face information and the screen width meets the preset conditions; and determine whether the body information is an indecent gesture or action; and determine whether the item corresponding to the item information is a competing product, and determine whether the ratio information between the item frame coordinates in the item information and the screen width and the screen height meets the preset conditions; Wherein, judging whether the ratio information between the face frame coordinates in the face information and the picture width meets the preset condition may include: Determine whether there is any coordinate loss in the face frame coordinates; In the case where there are missing coordinates in the face frame coordinates, it is determined that the face is out of the picture; in the case where there are no missing coordinates in the face frame coordinates, based on the relationship between the face frame coordinates and the picture width, it is determined that the picture meets the preset condition, or the picture is biased to the left, or the picture is biased to the right, and based on the live broadcast category identifier, it is determined that the distance between the anchor and the screen meets the preset condition, or the distance is too close, or the distance is too far.

[0031] The analyzing the audio information to determine whether the audio information meets the preset condition may include: The volume information and the text information are analyzed to determine whether the volume information meets a preset range and whether the text information contains prohibited words.

[0032] In an optional embodiment, since competitors rely on live streaming operations and are difficult to distinguish with the naked eye, abnormal sound will more directly affect the audience, while faces, limbs, and object positions have less impact on the live broadcast room and brand. Therefore, if there are multiple types of illegal information at the same time, the warning of competitors will be prioritized, followed by sound, and finally faces, limbs, and object positions. That is, in the above step S5, the warning of illegal information according to the preset priority may include: According to the wearable competing products, audio information and other information, the illegal information is warned in descending order of priority; wherein the other information includes facial information, body information and item location.

[0033] Specifically, for different warning objects, warning detection will be carried out at the same time, and a warning record will be kept for each warning information. It can be displayed from top to bottom according to the preset priority only in the warning display link, or the warning information can be displayed in pop-up windows in sequence according to the preset priority. Finally, the warning information will be collected into an interface, and the information will be pushed to directly notify the managers of the corresponding live broadcast room.

[0034] Since live broadcast is a dynamic process, for example, the anchor may leave the screen to take a sample, and the face may only be out of the screen for a few seconds. At this time, the warning information should not be issued. Therefore, the warning is divided into different levels, and different levels need to last for different periods of time before the level is upgraded. Warnings will be issued after the highest level.

[0035] The live broadcast room can be divided into areas for display according to its corresponding live broadcast category logo. When displaying warning information, in addition to displaying specific warning information, the display frame of the live broadcast room can also show the corresponding warning color.

[0036] In an optional embodiment, in the above step S6, for the warning information generated when the face information, the body information, the object information or the audio information does not meet the preset condition, different processing measures are corresponding when processing the video stream information, as follows: When the facial information does not meet the preset conditions, the corresponding warning information is pushed to notify the management personnel following the corresponding live broadcast room; wherein, the warning information generated when the facial information does not meet the preset conditions may include: the face is out of the picture, the picture is biased to the left, the picture is biased to the right, the anchor is too close to the screen, the anchor is too far from the screen, etc.

[0037] In the case that the limb information does not meet the preset conditions, the relevant frames are cut off and the video is spliced ​​to obtain the processed video stream, and the corresponding early warning information is pushed to notify the management personnel of the corresponding live broadcast room as a warning.

[0038] In the case where the item information does not meet the preset conditions, when the item information indicates that the anchor is wearing competing clothing or accessories during the broadcast, the corresponding warning information is pushed to notify the management personnel of the corresponding live broadcast room, and temporary replacement of the broadcast is adopted; when the item information indicates that the position of the item does not meet the preset conditions, the corresponding warning information is pushed to notify the management personnel of the corresponding live broadcast room; wherein, the warning information generated by the object position not meeting the preset conditions may include: the object is out of the picture, the picture is biased to the left, the picture is biased to the right, the picture is biased to the top, the picture is biased to the bottom, the object is too close to the screen, the object is too far from the screen, etc.

[0039] When the audio information does not meet the preset condition, processing the video stream information based on the warning information to obtain a processed video stream may include: In the case where the warning information indicates that the text information does not meet the preset condition, the corresponding warning information is pushed to notify the management personnel of the corresponding live broadcast room to warn, and the abnormal audio corresponding to the abnormal text is intercepted; the abnormal text indicates the part of the text information that does not meet the preset condition, and the abnormal audio carries abnormal timeline data; Based on the abnormal time axis data of the abnormal audio, the sample audio is processed to obtain a matching sample audio; Using the matching sample audio to replace the abnormal audio to obtain edited audio; The video stream information is processed based on the edited audio to obtain the processed video stream.

[0040] When the warning information indicates that the volume information does not meet the preset condition, determining whether the volume of the left channel and the right channel is within a preset range, and determining whether the absolute value of the difference between the volume of the left channel and the volume of the right channel meets a preset threshold; When the volume of the left channel or / and the right channel is not within the preset range, adjust the volume of the left channel or / and the right channel; when the absolute value of the difference between the volume of the left channel and the volume of the right channel does not meet the preset threshold, push the corresponding warning information to notify the management personnel of the corresponding live broadcast room; wherein, the warning information generated when the absolute value of the difference between the volume of the left channel and the volume of the right channel does not meet the preset threshold may include that the host is too far away from the microphone and the position is to the left, or that the host is too far away from the microphone and the position is to the right.

[0041] See also Figure 2 , which is a schematic diagram of a practical application of a live face, body, and object monitoring method provided by an embodiment of the present application; as shown in the figure, the live face, body, and object monitoring method may include: The live broadcast room uses cameras or other recording devices to broadcast live, and the live broadcast screen will first pass through the company's "this system" -> third-party live broadcast platform software -> finally reach the audience. Among them, "this system" will obtain the live video stream through the RTMP real-time data transmission protocol and enter the backend server. After live data interception -> AI recognition -> algorithm analysis, relevant data is obtained, and finally the monitoring data is returned to the live broadcast operation end through "this system" for viewing or intervention on the anchor.

[0042] In an optional embodiment, the live broadcast face, body, and object monitoring method may specifically include: Step 1: Live data capture; The live data is mainly obtained through the rtmp real-time data transmission protocol and then transmitted to the server's ffmpeg service process for data processing. The server will intercept the single frame data of the live video through ffmpeg; ffmpeg is an open source computer program that can be used to record and convert digital audio and video and convert them into streams.

[0043] Step 2: AI recognition; Face recognition and coordinate identification based on TensorFlow and face-api; Use TensorFlow Object Detection API to identify objects and identify object coordinates; preferably, use the company's huge product image library to train the model; Body detection and training are roughly the same as object recognition, mainly involving some indecent gestures or movements.

[0044] Step 3: Algorithm analysis; Face warning algorithm: Figure 3 As shown, according to step 2, the coordinates a and d of the face area of ​​the anchor during the live broadcast can be obtained; Through the a and d coordinates, a rectangular frame of the face in the video can be obtained. By determining the position of the rectangular frame, abnormal data can be obtained, as follows: (1) One of the coordinates a or d is missing -> the face is out of the picture; (2) Assuming the video width is w, if the x-axis coordinate of a (3) Assuming the video width is w, if the x-axis coordinate of d > ww*10, and the x-axis coordinate of a is not less than w*10%, the picture is biased to the right; (4) Based on the a and d coordinates, a rectangular frame can be drawn to define the range of the face. Based on the size of the video, the proportion of the face in the entire video can be determined. Through AI learning and daily accumulation, the safety values ​​of the range of faces in different categories during live broadcasts can be obtained. By comparing the values, it can be determined whether the host is too close or too far from the screen. Item warning algorithm: Object distance, similar to face recognition, the difference plus upper and lower distance judgment, 20% of the screen; Competitive product analysis: Combine the current category data of the live broadcast room with the product data identified in step 1 to check whether there are any competing wearable products on the broadcast; Limb warning algorithm; Identify indecent gestures or movements; Abnormal sound warning algorithm: Through AI learning and daily accumulation, the suitable range of volume n~m and the deviation value of left and right channels (left channel volume - right channel volume) x can be obtained; (1) If the volume of either the left or right channel is less than n or greater than m, the microphone volume is abnormal; (2) If the absolute value of the left channel volume minus the right channel volume is greater than x, the microphone volume is abnormal, usually because the host is too far away from the microphone and is positioned to the left or right; Banned word warning algorithm: like Figure 4 As shown, the live broadcast room is broadcasted through live broadcast hardware such as camera equipment and bees. The system receives data stream slices (i.e., video stream information) and separates the video from the voice. The voice will be uploaded to the server for banned word detection and returned to the system after the detection is completed. If there is banned word information, the system will mute and replace the original audio related words, and then synchronize the audio and video with the picture, and finally return to the live broadcast on the third-party platform. Because it is a live broadcast scene, the above processing of the system needs to be controlled within 5 seconds. Among them, in order to expand the banned word library, the crawler service regularly crawls banned word data on third-party websites or public cosmetics industry websites. Python+ffmpeg can be used for video stream slicing and audio and video separation. The open source voice processing model Whisper can be used to identify the time nodes when banned words and banned words appear. When the banned words are at the same time node, the recorded mute template (currently a blank sound) is used and superimposed on the original audio, so that a mute audio can be generated. ffmpeg can be used to synthesize the sliced ​​audio and video, and push the video to the third-party platform after integration. Step 4: Early warning; Live broadcast is a dynamic process. For example, the anchor may go out of the screen to pick up a sample for only a few seconds. At this time, the warning information cannot be issued. Therefore, the system divides the warning into different levels, and different levels need to last for different periods of time before the level is upgraded. Warnings will be issued after the highest level. Step 5: Display the algorithm on a large screen video; Purpose: To achieve the best display effect on the large screen of the live broadcast center, the same categories should be placed in the same row as much as possible, and those not broadcasted should be placed on both sides.

[0045] like Figure 5 As shown, the large-screen video display algorithm specifically includes: Use AI algorithms to distinguish live broadcast categories; if the live broadcast is a product live broadcast, it will be distinguished according to the product category, for example, it can be divided into 4 categories: clothing, shoes, jewelry, and cosmetics; Use crawler programs to determine whether the live broadcast room is on; Create array 1, array 2, and array 3, using categories as key values.

[0046] Optionally, the specific process may include: Get live broadcast room data; Determine whether the number of the same category in the array in the live broadcast is greater than 8; If yes, determine whether the broadcast has started; if not, add it to array 1, and finally get array 1 with category as key value, and traverse to get the number of vacancies m (8-number of live broadcast rooms) in each category; If yes, that is, it has been broadcasted, then it is added to array 2, and finally array 2 with category as key value is obtained, and the number of category live broadcast rooms n in array 2 is obtained; if no, that is, it has not been broadcasted, then it is added to array 3, and finally array 3 with category as key value is obtained; Traverse array 1 and array 2, search array 1 for the category with the smallest mn for each category in array 2, and fill the live broadcast information in array 1 until 8 are full; For arrays that are not yet full of 8, use the data in array 3 to fill them.

[0047] It can be seen from the above technical solutions of the embodiments of the present application that, in the embodiments of the present application, by real-time monitoring of the broadcast information, facial information, body information, object information and audio information, it is possible to promptly discover whether the host has not started the broadcast at the start time, whether the host's position is offset during the live broadcast or the object display is too far or too close to the camera, whether the host is wearing competing clothing or accessories during the broadcast, whether the host speaks banned words, whether the host shows indecent gestures or actions, and perform pre-processing in a timely manner to avoid the outflow of live broadcast clips with greater impact to third-party live broadcast platforms, which can ensure the health and professionalism of the live broadcast, improve the quality and efficiency of the live broadcast, reduce operating costs and risks, and reduce the workload of manual review.

[0048] Corresponding to the live broadcast health and professionalism monitoring method provided in the above-mentioned embodiment, the embodiment of the present application also provides a live broadcast health and professionalism monitoring device. Since the live broadcast health and professionalism monitoring device provided in the embodiment of the present application corresponds to the live broadcast health and professionalism monitoring method provided in the above-mentioned embodiment, the implementation method of the aforementioned live broadcast health and professionalism monitoring method is also applicable to the live broadcast health and professionalism monitoring device provided in this embodiment, and will not be described in detail in this embodiment.

[0049] See also Figure 6 , which is a structural block diagram of a live broadcast health and professionalism monitoring device provided in an embodiment of the present application; the device includes: 01: Video stream acquisition module, used to obtain video stream information; 02: a key frame interception module, used to intercept key frame information based on the video stream information; the key frame information carries key frame timeline information; 03: an information analysis module, used to analyze and obtain face information, body information and object information based on the key frame information; 04: an information analysis module, used to analyze the face information, the body information and the object information, and respectively determine whether the face information, the body information and the object information meet preset conditions; 05: an information warning module, used to issue a warning for violation information according to a preset priority and display relevant warning information when at least one of the face information, the body information and the object information does not meet the preset condition; wherein the violation information indicates information that does not meet the preset condition; 06: a video stream processing module, used to process the video stream information based on the warning information to obtain a processed video stream; 07: Push module, used to push the processed video stream to the audience end for playback.

[0050] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0051] The live broadcast health and professionalism monitoring device of the embodiment of the present application can promptly detect whether the host has not started the broadcast at the start time, whether the host's position is offset during the live broadcast or the object displayed is too far or too close to the camera, whether the host is wearing competing clothing or accessories, whether the host speaks banned words, whether the host shows indecent gestures or actions, and perform pre-processing in time to avoid the outflow of live broadcast clips with greater impact to third-party live broadcast platforms. It can ensure the health and professionalism of the live broadcast, improve the quality and efficiency of the live broadcast, reduce operating costs and risks, and reduce the workload of manual review.

[0052] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the live broadcast health and professionalism monitoring method provided in the above method embodiment.

[0053] The memory can be used to store software programs and modules. The processor executes various functional applications and realizes high-level autonomous driving by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs required for functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0054] The method embodiments provided in the embodiments of the present application may be executed in a computer terminal, a server or a similar computing device, that is, the above-mentioned electronic device may include a computer terminal, a server or a similar computing device. Figure 7 is a hardware structure block diagram of an electronic device running a live broadcast health and professionalism monitoring method provided in an embodiment of the present application, such as Figure 7 As shown, the internal structure of the electronic device may include but is not limited to: a processor, a network interface and a memory. The processor, network interface and memory in the electronic device may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0055] Among them, the processor (or CPU (Central Processing Unit)) is the computing core and control core of the electronic device. The network interface may optionally include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.). Memory is a memory device in an electronic device for storing programs and data. It is understandable that the memory here can be a high-speed RAM storage device or a non-volatile memory device, such as at least one disk storage device; optionally, it can also be at least one storage device located away from the aforementioned processor. The memory provides a storage space, which stores the operating system of the electronic device, which may include but is not limited to: Windows system (an operating system), Linux (an operating system), Android (Android, a mobile operating system) system, IOS (a mobile operating system) system, etc., and this application does not limit this; and, in the storage space, one or more instructions suitable for being loaded and executed by the processor are also stored, and these instructions can be one or more computer programs (including program codes). In the embodiment of this specification, the processor loads and executes one or more instructions stored in the memory to implement the live broadcast health and professionalism monitoring method provided in the above method embodiment.

[0056] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the live broadcast health and professionalism monitoring method provided in the method embodiment.

[0057] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0058] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. And the above describes a specific embodiment of this specification. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-small sample image classification and parallel processing are also possible or may be advantageous.

[0059] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0060] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0061] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for monitoring the health and professionalism of live broadcast, characterized in that: include: Get video stream information; Based on the video stream information, intercept key frame information; The key frame information carries key frame timeline information; Based on the key frame information, facial information, body information and object information are obtained by parsing; Analyze the face information, the body information, and the object information to determine whether the face information, the body information, and the object information meet preset conditions; In the case where at least one of the face information, the body information and the object information does not meet the preset condition, an early warning is issued for the violation information according to the preset priority, and relevant early warning information is displayed; wherein the violation information indicates the information that does not meet the preset condition; Processing the video stream information based on the warning information to obtain a processed video stream; The processed video stream is pushed to the audience end for playback.

2. A method for monitoring the health and professionalism of live broadcast according to claim 1, characterized in that: The step of intercepting key frame information based on the video stream information includes: Using a key frame extraction algorithm to extract the key frame information from the video stream information; Among them, the key frame extraction algorithm adopts one of the methods based on shot boundaries, image content, motion analysis, clustering, deep learning, frame difference, and dynamic frame.

3. A method for monitoring the health and professionalism of live broadcast according to claim 1, characterized in that: The facial information, body information and object information are obtained by parsing based on the key frame information, including: Performing image preprocessing on the key frame pictures in the key frame information to obtain preprocessed pictures; Locate the face area in the preprocessed image based on a face detection algorithm; locate the limb key points in the preprocessed image based on a limb detection algorithm; and locate the object area in the preprocessed image based on an object detection algorithm; Features are extracted from the face area to obtain the face information; posture estimation is performed based on the limb key points, and relevant features are extracted and classified to obtain the limb information; and features are extracted from the object area to obtain the object information.

4. A method for monitoring the health and professionalism of live broadcast according to claim 3, characterized in that: The analyzing the face information, the body information and the object information to determine whether the face information, the body information and the object information meet preset conditions respectively includes: Based on the key frame picture, obtain the picture width and picture height; The face information is analyzed based on the screen width and the live broadcast category identifier carried in the key frame information; and the limb information is analyzed based on the limb information library; and the item information is analyzed based on the competitor information library, the screen width and the screen height; Determine whether the ratio information between the face frame coordinates in the face information and the screen width meets the preset conditions; and determine whether the body information is an indecent gesture or action; and determine whether the item corresponding to the item information is a competing product, and determine whether the ratio information between the item frame coordinates in the item information and the screen width and the screen height meets the preset conditions.

5. A method for monitoring the health and professionalism of live broadcasting according to any one of claims 1 to 4, characterized in that: The method further comprises: Based on the video stream information, parse and obtain audio information; Analyze the audio information to determine whether the audio information meets the preset condition; When the audio information does not meet the preset condition, an early warning is issued for the violation information according to the preset priority, and the relevant early warning information is displayed.

6. A method for monitoring the health and professionalism of live broadcasting according to claim 5, characterized in that: The warning of violation information according to the preset priority includes: According to the wearable competing products, audio information and other information, the illegal information is warned in descending order of priority; wherein the other information includes facial information, body information and item location.

7. A method for monitoring the health and professionalism of live broadcasting according to claim 6, characterized in that: The method further comprises: Get live broadcast room data; Determine whether the live broadcast room is on according to the live broadcast room data; When the live broadcast room is started, the video stream information is obtained.

8. A device for monitoring the health and professionalism of a live broadcast, using a method for monitoring the health and professionalism of a live broadcast as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A video stream acquisition module is used to acquire video stream information; A key frame interception module, used to intercept key frame information based on the video stream information; the key frame information carries key frame timeline information; An information analysis module, used to analyze and obtain face information, body information and object information based on the key frame information; An information analysis module, used to analyze the face information, the body information and the object information, and respectively determine whether the face information, the body information and the object information meet preset conditions; An information warning module, used to issue a warning for violation information according to a preset priority and display relevant warning information when at least one of the face information, the body information and the object information does not meet the preset condition; wherein the violation information indicates information that does not meet the preset condition; A video stream processing module, used for processing the video stream information based on the warning information to obtain a processed video stream; The push module is used to push the processed video stream to the audience end for playback.

9. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a live broadcast health and professionalism monitoring method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement a live broadcast health and professionalism monitoring method as described in any one of claims 1 to 7.

Images