Live broadcast on-hook detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a live broadcast on-hook detection method, a live broadcast on-hook detection device, electronic equipment and a storage medium, which are used for solving the technical problem that misjudgment is easy to generate in a mode of detecting on-hook by means of an image algorithm. The method comprises the following steps: pulling an anchor microphone audio of a target anchor from a data storage service in real time; identifying an anchor voiceprint of the anchor microphone audio; when the target anchor is a registered anchor, acquiring an anchor registration voiceprint of the target anchor; calculating the voiceprint similarity between the anchor voiceprint and the anchor registration voiceprint; and judging whether the anchor hangs up or not according to the voiceprint similarity.

Description

Technical Field

[0001] The present invention relates to the technical field of hang-up detection, and in particular to a live broadcast hang-up detection method, device, electronic equipment and storage medium. Background Art

[0002] Live streaming is an activity for the host to communicate and interact with the audience. The audience hopes to see high-quality content from the host, and the host also hopes to attract more viewers through his or her own efforts. The platform also hopes that the host can be more active and interact more with the audience, rather than sitting there quietly, doing other things, or even leaving the live camera for a long time.

[0003] At present, many live broadcast platforms calculate the host’s live broadcast time as part of their salary. On the other hand, if you want to recommend high-quality live broadcasts to the audience, you need to monitor whether the host is idle.

[0004] However, anchors usually try to evade system monitoring. The most common method is to keep playing songs and posting recorded videos / gifs.

[0005] The current mainstream way to monitor whether the anchor is offline is to detect and monitor through image algorithms:

[0006] 1) By calculating the similarity of consecutive frames, determine whether the live broadcast screen is dynamic. When the similarity of two frames exceeds the threshold, that is, the screen has basically not changed, it is judged as idle; otherwise, it is judged as normal live broadcast status.

[0007] 2) Use face detection to determine whether there is a face in the live broadcast room.

[0008] However, if the host plays a pre-recorded animated image in a loop or plays a recorded video, the image algorithm will mistake the host who is idle as a normal live broadcast. Summary of the invention

[0009] The present invention provides a live broadcast hang-up detection method, device, electronic device and storage medium, which are used to solve the technical problem that the method of detecting hang-up by relying on image algorithm is prone to misjudgment.

[0010] The present invention provides a live broadcast hang-up detection method, which is applied to voiceprint verification algorithm service; the method comprises:

[0011] Pull the target anchor's microphone audio from the data storage service in real time;

[0012] Identify the host's voiceprint of the host's microphone audio;

[0013] When the target anchor is a registered anchor, obtaining the anchor registration voiceprint of the target anchor;

[0014] Calculating the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint;

[0015] Whether the host is offline is determined according to the voiceprint similarity.

[0016] Optionally, the step of identifying the host's voiceprint of the host's microphone audio includes:

[0017] Extracting audio data of a preset first duration from the host microphone audio;

[0018] Performing audio interception on the audio data with a preset window length and a preset window offset to obtain a plurality of audio clips;

[0019] Each of the audio clips is converted into the host's voiceprint through voiceprint recognition.

[0020] Optionally, when the target anchor is a registered anchor, before the step of obtaining the anchor registration voiceprint of the target anchor, the step further includes:

[0021] Obtain the historical audio of the target anchor;

[0022] Separating vocal audio and background audio from the anchor's historical audio;

[0023] Segmenting the human voice audio using a voice activity detection algorithm to obtain human voice segments;

[0024] Separating the background sound segment corresponding to each of the human voice segments from the background audio;

[0025] Determine in turn whether the volume of the background sound segment corresponding to each of the human voice segments is lower than a preset threshold;

[0026] If yes, retain the anchor's historical audio segment corresponding to the human voice segment as clean audio;

[0027] Clustering all the clean audios to obtain a number of clean audio clusters;

[0028] The clean audio cluster with the largest number of clean audios is taken as the target cluster;

[0029] Based on the center of the target cluster, a preset number of clean audios are selected as registration audios;

[0030] The voiceprint of the registration audio is extracted as the anchor registration voiceprint of the target anchor.

[0031] Optionally, the step of judging whether the anchor is off-hook according to the voiceprint similarity includes:

[0032] Determine whether there is at least one host voiceprint corresponding to the audio segment in the audio data, and the voiceprint similarity between the host's registered voiceprint and the host's registered voiceprint is greater than a preset similarity threshold;

[0033] If yes, it is determined that the audio data contains the voice of the target host, and a first identifier is marked;

[0034] If not, it is determined that the audio data does not contain the target host's voice, and a second identifier is marked;

[0035] Determining whether the plurality of audio data segments within a preset second duration are all marked with a second identifier;

[0036] If not, it is determined that the target anchor is in a non-on-hook state;

[0037] If so, it is determined that the target anchor is in an on-hook state.

[0038] Optionally, it also includes:

[0039] When the target anchor is not a registered anchor, it is determined that the target anchor is in a non-on-hook state.

[0040] The present invention also provides a live broadcast hang-up detection device, which is applied to voiceprint verification algorithm service; the device comprises:

[0041] The host microphone audio pulling module is used to pull the host microphone audio of the target host from the data storage service in real time;

[0042] A host voiceprint recognition module, used to recognize the host voiceprint of the host microphone audio;

[0043] The anchor registration voiceprint acquisition module is used to acquire the anchor registration voiceprint of the target anchor when the target anchor is a registered anchor;

[0044] A voiceprint similarity calculation module, used to calculate the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint;

[0045] The on-hook judgment module is used to judge whether the anchor is on-hook according to the voiceprint similarity.

[0046] Optionally, the anchor voiceprint recognition module includes:

[0047] An audio data extraction submodule, used to extract audio data of a preset first duration from the host microphone audio;

[0048] An audio interception submodule, used to intercept the audio data with a preset window length and a preset window offset to obtain a plurality of audio clips;

[0049] The host voiceprint recognition submodule is used to convert each of the audio clips into the host voiceprint through voiceprint recognition.

[0050] Optionally, it also includes:

[0051] The anchor historical audio acquisition module is used to acquire the anchor historical audio of the target anchor;

[0052] An audio separation module, used to separate human voice audio and background audio from the anchor's historical audio;

[0053] A human voice segmentation module, used for segmenting the human voice audio by a voice activity detection algorithm to obtain human voice segments;

[0054] A background sound segmentation module, used to separate the background sound segment corresponding to each of the human voice segments from the background audio;

[0055] A volume determination module, used to determine in turn whether the volume of the background sound segment corresponding to each of the human voice segments is lower than a preset threshold;

[0056] A clean audio retention module, used for retaining the anchor's historical audio segment corresponding to the human voice segment as clean audio;

[0057] A clustering module, used for clustering all the clean audios to obtain a plurality of clean audio clusters;

[0058] A target cluster determination module, used for taking the clean audio cluster with the largest number of clean audios as the target cluster;

[0059] A registration audio selection module, configured to select a preset number of clean audios as registration audios based on the center of the target cluster;

[0060] The anchor registration voiceprint extraction module is used to extract the voiceprint of the registration audio as the anchor registration voiceprint of the target anchor.

[0061] The present invention also provides an electronic device, the device comprising a processor and a memory:

[0062] The memory is used to store program code and transmit the program code to the processor;

[0063] The processor is used to execute the live broadcast hang-up detection method as described in any one of the above items according to the instructions in the program code.

[0064] The present invention also provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the live broadcast hang-up detection method as described in any one of the above items.

[0065] It can be seen from the above technical solutions that the present invention has the following advantages: the present invention provides a method for detecting live broadcast hang-up, and specifically discloses: pulling the target anchor's microphone audio from the data storage service in real time; identifying the anchor's voiceprint of the anchor's microphone audio; when the target anchor is a registered anchor, obtaining the anchor's registered voiceprint of the target anchor; calculating the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint; judging whether the anchor is hang-up based on the voiceprint similarity. The present invention collects audio data from the anchor's microphone, extracts the anchor's voiceprint from it, and then calculates the similarity between it and the anchor's registered voiceprint to judge whether the anchor's voice exists in the data collected by the anchor's microphone, so as to judge whether the anchor has hang-up behavior. This avoids the situation where relying on image algorithms to detect hang-up behavior is prone to misjudgment, and improves the accuracy of hang-up behavior detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0067] Figure 1 A flowchart of a live broadcast hang-up detection method provided by an embodiment of the present invention;

[0068] Figure 2 A flowchart of a live broadcast hang-up detection method provided by another embodiment of the present invention;

[0069] Figure 3 A schematic diagram of obtaining clean audio segments provided by the host in an embodiment of the present invention;

[0070] Figure 4 A live broadcast hang-up detection flow chart provided by an embodiment of the present invention;

[0071] Figure 5 A structural block diagram of a live broadcast hang-up detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The embodiments of the present invention provide a live broadcast hang-up detection method, device, electronic device and storage medium, which are used to solve the technical problem that the method of relying on image algorithm to detect hang-up is prone to misjudgment.

[0073] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] See also Figure 1 , Figure 1 A flowchart of the steps of a live broadcast hang-up detection method provided in an embodiment of the present invention.

[0075] A live broadcast hang-up detection method provided by the present invention is applied to a voiceprint verification algorithm service and may specifically include the following steps:

[0076] Step 101, pull the target anchor's microphone audio from the data storage service in real time;

[0077] In an embodiment of the present invention, whether the host is on-hook can be determined by detecting in real time whether the host is speaking. Specifically, the target host's voice can be collected in real time through the target host's microphone to form the host microphone audio and upload it to the data storage service.

[0078] When the voiceprint verification algorithm service detects that new host microphone audio is generated in the data storage service, it can pull the host microphone audio of the target host from the data storage service in real time to determine whether the host's voice exists in the host microphone audio, thereby determining whether the target host is offline.

[0079] Step 102, identifying the host's voiceprint from the host's microphone audio;

[0080] Step 103: when the target anchor is a registered anchor, obtain the anchor registration voiceprint of the target anchor;

[0081] Voiceprint is a static sound spectrum image in coordinates that can be observed, compared and identified by converting human speech into electromagnetic vibrations using electroacoustic instruments, and then displayed on a screen or printed on paper, i.e., a spectrogram of speech. Voiceprints of different people are different, so different speakers can be identified based on voiceprints.

[0082] Voiceprint recognition mainly determines the speaker by comparing two voiceprints. In the embodiment of the present invention, the two parties to be compared can be the anchor's voiceprint of the anchor's microphone audio and the anchor's registered voiceprint of the target anchor.

[0083] The anchor registration voiceprint is the voiceprint of the audio information collected when the target anchor registers.

[0084] Step 104, calculating the voiceprint similarity between the host's voiceprint and the host's registered voiceprint;

[0085] Step 105, judging whether the host is offline based on the voiceprint similarity.

[0086] After collecting the target anchor's voiceprint and anchor's registered voiceprint, the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint can be calculated. The voiceprint similarity can be used to determine whether the voiceprint information collected by the microphone is the information of the corresponding registered anchor, thereby determining whether the target anchor is offline.

[0087] The present invention collects audio data from the host's microphone, extracts the host's voiceprint from it, and then calculates the similarity between it and the host's registered voiceprint to determine whether the host's voice is present in the data collected by the host's microphone, so as to determine whether the host has hung up. This avoids the situation where relying on image algorithms to detect hanging up behavior is prone to misjudgment, and improves the accuracy of hanging up behavior detection.

[0088] See also Figure 2 , Figure 2 A flowchart of a method for detecting a live broadcast hang-up provided by another embodiment of the present invention. Specifically, the following steps may be included:

[0089] Step 201, pull the target anchor's microphone audio from the data storage service in real time;

[0090] In an embodiment of the present invention, whether the host is on-hook can be determined by detecting in real time whether the host is speaking. Specifically, the target host's voice can be collected in real time through the target host's microphone to form the host microphone audio and upload it to the data storage service.

[0091] When the voiceprint verification algorithm service detects that new host microphone audio is generated in the data storage service, it can pull the host microphone audio of the target host from the data storage service in real time to determine whether the host's voice exists in the host microphone audio, thereby determining whether the target host is offline.

[0092] Step 202, extracting audio data of a preset first duration from the host's microphone audio;

[0093] Step 203, performing audio interception on the audio data with a preset window length and a preset window offset to obtain a plurality of audio clips;

[0094] In an embodiment of the present invention, after the target anchor's microphone audio is pulled from the data storage service, a number of audio data of a preset first duration can be extracted from the anchor's microphone audio, and then the audio data is intercepted with a preset window length and a preset window offset to obtain a number of audio segments. The preset window length is greater than the preset window offset.

[0095] For example, multiple audio data of a minutes are obtained from the host's microphone audio, and each audio data has a corresponding host ID; then each audio data is intercepted with a window length of b seconds and a window offset of c (c < b) seconds to obtain several audio clips of b seconds.

[0096] Step 204, converting each audio clip into a host's voiceprint through voiceprint recognition;

[0097] After obtaining the audio clips, each audio clip can be converted into a voiceprint feature through a voiceprint recognition algorithm as the host voiceprint of the target host.

[0098] It should be noted that those skilled in the art may use any conventional voiceprint recognition algorithm to perform voiceprint recognition, and the embodiments of the present invention do not impose any specific limitation on the selection of the voiceprint recognition algorithm.

[0099] Step 205: when the target anchor is a registered anchor, obtain the anchor registration voiceprint of the target anchor;

[0100] In actual scenarios, new anchors do not generate historical audio data, so voiceprint judgment cannot be performed based on historical audio. Therefore, in the embodiment of the present invention, the anchor who has performed live broadcast is mainly judged to be on-hook. After the anchor has performed live broadcast, voiceprint information can be collected for voiceprint registration to obtain the corresponding anchor registration voiceprint, so as to perform subsequent on-hook judgment based on the anchor registration voiceprint.

[0101] In one example, the target anchor's anchor registration voiceprint generation process is as follows:

[0102] S101, obtaining the historical audio of the target anchor;

[0103] S102, separating the human voice audio and the background audio from the anchor's historical audio;

[0104] S103, segmenting the human voice audio using a voice activity detection algorithm to obtain human voice segments;

[0105] S104, separating the background sound segment corresponding to each voice segment from the background audio;

[0106] S105, determining in turn whether the volume of the background sound segment corresponding to each vocal segment is lower than a preset threshold;

[0107] S106: If yes, retain the anchor's historical audio segment corresponding to the human voice segment as the clean audio;

[0108] S107, clustering all clean audios to obtain a number of clean audio clusters;

[0109] S108, taking the clean audio cluster with the largest number of clean audios as the target cluster;

[0110] S109, based on the center of the target cluster, selecting a preset number of clean audios as registration audios;

[0111] S110, extracting the voiceprint of the registration audio as the anchor registration voiceprint of the target anchor.

[0112] In the specific implementation, Figure 3 As shown, first, the historical audio of the target anchor can be collected, and then the clean human voice can be filtered out from it. The process can be: through the sound source separation algorithm, the anchor historical audio is separated into human voice audio and background audio, and then the human voice audio is segmented by the voice activity detection (VAD) algorithm to obtain human voice segment. Then, the corresponding background sound segment is obtained by the timestamp of the human voice segment, and it is determined whether the volume of the background sound segment corresponding to each human voice segment is lower than the preset threshold. If so, the anchor historical audio segment corresponding to the human voice segment is retained as the clean audio, otherwise, it is not retained.

[0113] Since not all clean audio collected is the target host's voice audio, further screening is required. When a sufficient number of clean audios are collected (the specific number can be determined according to actual needs), the cluster with the largest number of audios can be selected as the target cluster by clustering, the center of the target cluster is calculated, and a preset number of clean audios near the center are selected as the registered audios. Then the voiceprint of each registered audio is extracted as the host's registered voiceprint.

[0114] In one example, a registered anchor will have a corresponding voiceprint registration record, while those without a record will wait until enough audio is collected before registering. Only registered anchors will undergo voiceprint verification, while those who are not registered will not undergo voiceprint verification.

[0115] It should be noted that since unregistered anchors will not undergo voiceprint verification, when the target anchor is not a registered anchor, it can be determined that the target anchor is not in an idle state.

[0116] Step 206, calculating the voiceprint similarity between the host's voiceprint and the host's registered voiceprint;

[0117] Step 207, judging whether the host is offline based on the voiceprint similarity.

[0118] After collecting the target anchor's voiceprint and anchor's registered voiceprint, the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint can be calculated. The voiceprint similarity can be used to determine whether the voiceprint information collected by the microphone is the information of the corresponding registered anchor, thereby determining whether the target anchor is offline.

[0119] In one example, the step of determining whether the anchor is offline based on the voiceprint similarity may include the following sub-steps:

[0120] S71, determining whether there is at least one audio segment in the audio data corresponding to the host's voiceprint and the host's registered voiceprint have a voiceprint similarity greater than a preset similarity threshold;

[0121] S72: If yes, determine that the audio data contains the target host's voice, and mark it with a first identifier;

[0122] S73: if not, it is determined that the audio data does not contain the target host's voice, and a second identifier is marked;

[0123] S74, determining whether the plurality of audio data segments within the preset second duration are all marked with a second identifier;

[0124] S75, if not, determining that the target anchor is in a non-on-hook state;

[0125] S76: If yes, determine that the target anchor is in the on-hook state.

[0126] In a specific implementation, it is possible to calculate whether the voiceprint similarity between the host's voiceprint corresponding to each audio clip and the host's registered voiceprint is greater than a preset similarity threshold. If so, the audio clip is judged to be the host's voice. If not, it is marked as not the host's voice. When there is at least one audio clip in the audio data whose voiceprint similarity between the host's voiceprint corresponding to the host and the host's registered voiceprint is greater than the preset similarity threshold, it is determined that the audio data contains the target host's voice. At this time, it can be marked with a first identifier (such as 1), otherwise it can be marked with a second identifier (such as 0). Then it is determined whether multiple audio data within the preset second time length are all marked with the second identifier. If so, it is determined that the host is on-hook, otherwise it is determined that the host is not on-hook.

[0127] For example, when the similarity between the host's voiceprint in an audio clip and the host's registered voiceprint exceeds the preset similarity threshold, it is marked as 1, otherwise it is marked as 0. If at least one of the b-second audio clips in the a-minute audio data is marked as 1, then this a-minute audio data is judged to contain the host's voice and is marked as 1, otherwise it is marked as 0. If the audio data in the audio of a host for d consecutive minutes (the second preset duration) is marked as 0, then the host is judged to be in the on-hook state, otherwise it is in the non-on-hook state. It should be noted that if the host starts broadcasting for less than d minutes, it is also considered to be in the non-on-hook state, and the second preset duration d can be 20-30 minutes.

[0128] The present invention collects audio data from the host's microphone, extracts the host's voiceprint from it, and then calculates the similarity between it and the host's registered voiceprint to determine whether the host's voice is present in the data collected by the host's microphone, so as to determine whether the host has hung up. This avoids the situation where relying on image algorithms to detect hanging up behavior is prone to misjudgment, and improves the accuracy of hanging up behavior detection.

[0129] For ease of understanding, the following is an example of how to detect when a live broadcast is idle:

[0130] See also Figure 4 , Figure 4 A live broadcast hang-up detection flow chart provided for an embodiment of the present invention.

[0131] like Figure 4 As shown, after the anchor starts broadcasting, the collected anchor microphone audio is first sliced ​​to obtain audio slices; then the voiceprint in the anchor microphone audio is extracted through the voiceprint recognition model to obtain the anchor voiceprint; then the voiceprint similarity between the anchor voiceprint and the anchor registered voiceprint is calculated; it is determined in turn whether there is at least one audio segment of each audio data whose voiceprint similarity is greater than a preset similarity threshold, if so, mark the first identifier and determine that the anchor is not on-hook; if not, mark the second identifier; then determine that all audio data of the anchor within d consecutive minutes are the second identifier, if not, determine that the anchor is not on-hook, if so, determine that the anchor is on-hook.

[0132] See also Figure 5 , Figure 5 A structural block diagram of a live broadcast hang-up detection device provided in an embodiment of the present invention.

[0133] The embodiment of the present invention provides a live broadcast hang-up detection device, which is applied to voiceprint verification algorithm service; the device includes:

[0134] The host microphone audio pulling module 501 is used to pull the host microphone audio of the target host from the data storage service in real time;

[0135] The host voiceprint recognition module 502 is used to recognize the host voiceprint of the host microphone audio;

[0136] The anchor registration voiceprint acquisition module 503 is used to acquire the anchor registration voiceprint of the target anchor when the target anchor is a registered anchor;

[0137] A voiceprint similarity calculation module 504 is used to calculate the voiceprint similarity between the host's voiceprint and the host's registered voiceprint;

[0138] The on-hook judgment module 505 is used to judge whether the anchor is on-hook according to the voiceprint similarity.

[0139] In the embodiment of the present invention, the anchor voiceprint recognition module 502 includes:

[0140] An audio data extraction submodule, used to extract audio data of a preset first duration from the host's microphone audio;

[0141] The audio interception submodule is used to intercept the audio data with a preset window length and a preset window offset to obtain a plurality of audio clips;

[0142] The host voiceprint recognition submodule is used to convert each audio clip into the host's voiceprint through voiceprint recognition.

[0143] In an embodiment of the present invention, it also includes:

[0144] The anchor historical audio acquisition module is used to obtain the anchor historical audio of the target anchor;

[0145] The audio separation module is used to separate the human voice audio and background audio from the anchor's historical audio;

[0146] A human voice segmentation module is used to segment human voice audio using a voice activity detection algorithm to obtain human voice segments;

[0147] A background sound segmentation module is used to separate the background sound segment corresponding to each voice segment from the background audio;

[0148] A volume determination module is used to determine in turn whether the volume of the background sound segment corresponding to each vocal segment is lower than a preset threshold;

[0149] A clean audio retention module is used to retain the anchor's historical audio fragment corresponding to the human voice fragment as clean audio;

[0150] A clustering module is used to cluster all clean audios to obtain several clean audio clusters;

[0151] A target cluster determination module, used for taking the clean audio cluster with the largest number of clean audios as the target cluster;

[0152] A registration audio selection module, used for selecting a preset number of clean audios as registration audios based on the center of the target cluster;

[0153] The anchor registration voiceprint extraction module is used to extract the voiceprint of the registration audio as the anchor registration voiceprint of the target anchor.

[0154] In the embodiment of the present invention, the on-hook determination module 505 includes:

[0155] The first judgment submodule is used to judge whether there is at least one audio segment in the audio data corresponding to the host's voiceprint and the voiceprint similarity of the host's registered voiceprint is greater than a preset similarity threshold;

[0156] A first identification marking submodule is used to determine that the audio data contains the voice of the target anchor and mark the first identification;

[0157] A second identification marking submodule is used to determine that the audio data does not contain the voice of the target host, and mark the second identification if no;

[0158] A second determination submodule is used to determine whether the plurality of audio data segments within a preset second duration are all marked with a second identifier;

[0159] The non-on-hook state determination submodule is used to determine whether the target anchor is in the non-on-hook state if no;

[0160] The on-hook status determination submodule is used to determine if the target anchor is in the on-hook status.

[0161] In an embodiment of the present invention, it also includes:

[0162] The non-hang-up state determination module is used to determine that the target anchor is in a non-hang-up state when the target anchor is not a registered anchor.

[0163] An embodiment of the present invention further provides an electronic device, the device comprising a processor and a memory:

[0164] The memory is used to store the program code and transmit the program code to the processor;

[0165] The processor is used to execute the live broadcast hang-up detection method of the embodiment of the present invention according to the instructions in the program code.

[0166] An embodiment of the present invention also provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the live broadcast hang-up detection method of the embodiment of the present invention.

[0167] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0169] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0170] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0174] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0175] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A live broadcast hang-up detection method, characterized in that: Applied to voiceprint verification algorithm service; the method includes: Pull the target anchor's microphone audio from the data storage service in real time; Identify the host's voiceprint of the host's microphone audio; When the target anchor is a registered anchor, obtaining the anchor registration voiceprint of the target anchor; Calculating the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint; Whether the host is offline is determined according to the voiceprint similarity.

2. The method according to claim 1, characterized in that The step of identifying the host's voiceprint of the host's microphone audio includes: Extracting audio data of a preset first duration from the host microphone audio; Performing audio interception on the audio data with a preset window length and a preset window offset to obtain a plurality of audio clips; Each of the audio clips is converted into the host's voiceprint through voiceprint recognition.

3. The method according to claim 1, characterized in that When the target anchor is a registered anchor, before the step of obtaining the anchor registration voiceprint of the target anchor, the step further includes: Obtain the historical audio of the target anchor; Separating vocal audio and background audio from the anchor's historical audio; Segmenting the human voice audio using a voice activity detection algorithm to obtain human voice segments; Separating the background sound segment corresponding to each of the human voice segments from the background audio; Determine in turn whether the volume of the background sound segment corresponding to each of the human voice segments is lower than a preset threshold; If yes, retain the anchor's historical audio segment corresponding to the human voice segment as clean audio; Clustering all the clean audios to obtain a number of clean audio clusters; The clean audio cluster with the largest number of clean audios is taken as the target cluster; Based on the center of the target cluster, a preset number of clean audios are selected as registration audios; The voiceprint of the registration audio is extracted as the anchor registration voiceprint of the target anchor.

4. The method according to any one of claims 2 to 3, characterized in that: The step of judging whether the anchor is offline according to the voiceprint similarity includes: Determine whether there is at least one host voiceprint corresponding to the audio segment in the audio data, and the voiceprint similarity between the host's registered voiceprint and the host's registered voiceprint is greater than a preset similarity threshold; If yes, it is determined that the audio data contains the voice of the target host, and a first identifier is marked; If not, it is determined that the audio data does not contain the target host's voice, and a second identifier is marked; Determining whether the plurality of audio data segments within a preset second duration are all marked with a second identifier; If not, it is determined that the target anchor is in a non-on-hook state; If so, it is determined that the target anchor is in an on-hook state.

5. The method according to claim 1, characterized in that Also includes: When the target anchor is not a registered anchor, it is determined that the target anchor is in a non-on-hook state.

6. A live broadcast hang-up detection device, characterized in that: Applied to voiceprint verification algorithm service; the device includes: The host microphone audio pulling module is used to pull the host microphone audio of the target host from the data storage service in real time; A host voiceprint recognition module, used to recognize the host voiceprint of the host microphone audio; The anchor registration voiceprint acquisition module is used to acquire the anchor registration voiceprint of the target anchor when the target anchor is a registered anchor; A voiceprint similarity calculation module, used to calculate the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint; The on-hook judgment module is used to judge whether the anchor is on-hook according to the voiceprint similarity.

7. The device according to claim 6, characterized in that The anchor voiceprint recognition module includes: An audio data extraction submodule, used to extract audio data of a preset first duration from the host microphone audio; An audio interception submodule, used to intercept the audio data with a preset window length and a preset window offset to obtain a plurality of audio clips; The host voiceprint recognition submodule is used to convert each of the audio clips into the host voiceprint through voiceprint recognition.

8. The device according to claim 6, characterized in that Also includes: A host historical audio acquisition module is used to acquire the host historical audio of the target host; An audio separation module, used to separate human voice audio and background audio from the anchor's historical audio; A human voice segmentation module, used for segmenting the human voice audio by a voice activity detection algorithm to obtain human voice segments; A background sound segmentation module, used to separate the background sound segment corresponding to each of the human voice segments from the background audio; A volume determination module, used to determine in turn whether the volume of the background sound segment corresponding to each of the human voice segments is lower than a preset threshold; A clean audio retention module, used for retaining the anchor's historical audio segment corresponding to the human voice segment as clean audio; A clustering module, used for clustering all the clean audios to obtain a plurality of clean audio clusters; A target cluster determination module, used for taking the clean audio cluster with the largest number of clean audios as the target cluster; A registration audio selection module, configured to select a preset number of clean audios as registration audios based on the center of the target cluster; The anchor registration voiceprint extraction module is used to extract the voiceprint of the registration audio as the anchor registration voiceprint of the target anchor.

9. An electronic device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the live broadcast hang-up detection method described in any one of claims 1-5 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the live broadcast hang-up detection method described in any one of claims 1-5.

Images