Live broadcast hang-up detection method, device, electronic device and storage medium

Abstract

The present invention discloses a live broadcast idle detection method, device, electronic device, and storage medium, which are used to solve the technical problem that idle detection methods relying on image algorithms are prone to misjudgment. The present invention comprises the following steps: pulling the target host's microphone audio from a data storage service in real time; identifying the host's voiceprint from the microphone audio; when the target host is a registered host, obtaining the target host's registered voiceprint; calculating the voiceprint similarity between the host's voiceprint and the registered voiceprint; and determining whether the host is idle based on the voiceprint similarity.

Description

Technical Field

[0001] The present invention relates to the technical field of hang-up detection, and in particular to a live broadcast hang-up detection method, device, electronic device and storage medium. Background Art

[0002] Live streaming is an activity for the host to communicate and interact with the audience. The audience hopes to see high-quality content from the host, and the host also hopes to attract more viewers through his or her own efforts. The platform also hopes that the host can be more active and interact more with the audience, rather than sitting there quietly, doing other things, or even leaving the live camera for a long time.

[0003] At present, many live broadcast platforms calculate the length of the anchor's live broadcast as part of the salary. On the other hand, in order to recommend high-quality live broadcasts to the audience, it is necessary to monitor whether the anchor is idle.

[0004] However, streamers often try to evade system monitoring, with the most common method being to constantly play music and include recorded videos or GIFs.

[0005] The current mainstream way to monitor whether a streamer is idle is to detect and monitor using image algorithms:

[0006] 1) Determine whether the live broadcast is dynamic by calculating the similarity between consecutive frames. If the similarity between two frames exceeds a threshold, meaning the images are essentially unchanged, the system is considered idle; otherwise, the system is considered to be in a normal live broadcast state.

[0007] 2) Use face detection to determine whether there is a face in the live broadcast room.

[0008] However, if the host plays a pre-recorded animated image in a loop or plays a recorded video, the image algorithm will mistakenly think that the host is broadcasting normally. Summary of the Invention

[0009] The present invention provides a live broadcast hang-up detection method, device, electronic device and storage medium, which are used to solve the technical problem that the method of relying on image algorithm to detect hang-up is prone to misjudgment.

[0010] The present invention provides a live broadcast idle detection method, which is applied to a voiceprint verification algorithm service; the method comprises:

[0011] Pull the target anchor's microphone audio from the data storage service in real time;

[0012] Identify the host's voiceprint based on the host's microphone audio;

[0013] When the target anchor is a registered anchor, obtaining the anchor registration voiceprint of the target anchor;

[0014] Calculating the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint;

[0015] Determine whether the host is idle based on the voiceprint similarity.

[0016] Optionally, the step of identifying the host's voiceprint from the host's microphone audio includes:

[0017] Extracting audio data of a preset first duration from the host microphone audio;

[0018] Performing audio interception on the audio data using a preset window length and a preset window offset to obtain a plurality of audio segments;

[0019] Each of the audio clips is converted into the host's voiceprint through voiceprint recognition.

[0020] Optionally, when the target anchor is a registered anchor, before the step of obtaining the anchor registration voiceprint of the target anchor, the method further includes:

[0021] Obtain the target anchor's historical audio;

[0022] Separating human voice audio and background audio from the anchor's historical audio;

[0023] Segmenting the human voice audio using a voice activity detection algorithm to obtain human voice segments;

[0024] Separating a background sound segment corresponding to each of the human voice segments from the background audio;

[0025] Determine in turn whether the volume of the background sound segment corresponding to each of the human voice segments is lower than a preset threshold;

[0026] If yes, retain the host's historical audio segment corresponding to the vocal segment as the clean audio;

[0027] Clustering all the clean audios to obtain a plurality of clean audio clusters;

[0028] The clean audio cluster with the largest number of clean audios is selected as the target cluster;

[0029] Based on the center of the target cluster, a preset number of clean audios are selected as registration audios;

[0030] The voiceprint of the registration audio is extracted as the anchor registration voiceprint of the target anchor.

[0031] Optionally, the step of determining whether the host is offline based on the voiceprint similarity includes:

[0032] Determine whether there is at least one anchor voiceprint corresponding to the audio segment in the audio data, and the similarity between the anchor's registered voiceprint and the anchor's voiceprint is greater than a preset similarity threshold;

[0033] If so, it is determined that the audio data contains the voice of the target host, and a first identifier is marked;

[0034] If not, it is determined that the audio data does not contain the target host's voice, and a second identifier is marked;

[0035] Determining whether the plurality of audio data segments within the preset second duration are all marked with the second identifier;

[0036] If not, it is determined that the target anchor is in a non-on-hook state;

[0037] If so, it is determined that the target anchor is in an on-hook state.

[0038] Optionally, it also includes:

[0039] When the target anchor is not a registered anchor, it is determined that the target anchor is in a non-on-hook state.

[0040] The present invention also provides a live broadcast hang-up detection device, which is applied to voiceprint verification algorithm services; the device includes:

[0041] The host microphone audio pulling module is used to pull the target host's microphone audio from the data storage service in real time;

[0042] An anchor voiceprint recognition module, used to recognize the anchor's voiceprint of the anchor's microphone audio;

[0043] The anchor registration voiceprint acquisition module is used to obtain the anchor registration voiceprint of the target anchor when the target anchor is a registered anchor;

[0044] A voiceprint similarity calculation module, configured to calculate the voiceprint similarity between the host's voiceprint and the host's registered voiceprint;

[0045] The on-hook judgment module is used to judge whether the anchor is on-hook based on the voiceprint similarity.

[0046] Optionally, the anchor voiceprint recognition module includes:

[0047] An audio data extraction submodule, configured to extract audio data of a preset first duration from the host's microphone audio;

[0048] An audio interception submodule, configured to intercept the audio data using a preset window length and a preset window offset to obtain a plurality of audio segments;

[0049] The host voiceprint recognition submodule is used to convert each of the audio clips into the host voiceprint through voiceprint recognition.

[0050] Optionally, it also includes:

[0051] The anchor historical audio acquisition module is used to obtain the anchor historical audio of the target anchor;

[0052] An audio separation module, used to separate human voice audio and background audio from the anchor's historical audio;

[0053] A human voice segmentation module, configured to segment the human voice audio using a voice activity detection algorithm to obtain human voice segments;

[0054] A background sound segmentation module is used to separate the background sound segment corresponding to each of the human voice segments from the background audio;

[0055] A volume determination module is used to determine in turn whether the volume of the background sound segment corresponding to each of the human voice segments is lower than a preset threshold;

[0056] A clean audio retention module is used to retain the anchor's historical audio fragment corresponding to the human voice fragment as clean audio;

[0057] A clustering module, configured to cluster all the clean audios to obtain a plurality of clean audio clusters;

[0058] a target cluster determination module, configured to select the clean audio cluster with the largest number of clean audios as the target cluster;

[0059] a registration audio selection module, configured to select a preset number of clean audios as registration audios based on the center of the target cluster;

[0060] The anchor registration voiceprint extraction module is used to extract the voiceprint of the registration audio as the anchor registration voiceprint of the target anchor.

[0061] The present invention further provides an electronic device, comprising a processor and a memory:

[0062] The memory is used to store program code and transmit the program code to the processor;

[0063] The processor is used to execute the live broadcast hang-up detection method as described in any one of the above items according to the instructions in the program code.

[0064] The present invention also provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the live broadcast hang-up detection method as described in any one of the above items.

[0065] It can be seen from the above technical solutions that the present invention has the following advantages: the present invention provides a method for detecting hang-up during live broadcast, and specifically discloses: pulling the target anchor's microphone audio from the data storage service in real time; identifying the anchor's voiceprint of the anchor's microphone audio; when the target anchor is a registered anchor, obtaining the anchor's registered voiceprint of the target anchor; calculating the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint; judging whether the anchor is hang-up based on the voiceprint similarity. The present invention collects audio data from the anchor's microphone, extracts the anchor's voiceprint from it, and then calculates the similarity between it and the anchor's registered voiceprint to judge whether the anchor's voice exists in the data collected by the anchor's microphone, so as to judge whether the anchor has hang-up behavior. This avoids the situation where relying on image algorithms to detect hang-up behavior easily leads to misjudgment, and improves the accuracy of hang-up behavior detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 A flowchart of a method for detecting idle playback during live streaming provided by an embodiment of the present invention;

[0068] Figure 2 A flowchart of a method for detecting idle playback during live streaming provided by another embodiment of the present invention;

[0069] Figure 3 A schematic diagram of obtaining clean audio fragments provided by the host in an embodiment of the present invention;

[0070] Figure 4 A live broadcast hang-up detection flow chart provided by an embodiment of the present invention;

[0071] Figure 5 This is a structural block diagram of a live broadcast hang-up detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0072] Embodiments of the present invention provide a live broadcast hang-up detection method, device, electronic device and storage medium, which are used to solve the technical problem that relying on image algorithms to detect hang-up is prone to misjudgment.

[0073] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0074] See also Figure 1 , Figure 1 A flowchart of the steps of a live broadcast hang-up detection method provided by an embodiment of the present invention.

[0075] The present invention provides a live broadcast idle detection method, which is applied to a voiceprint verification algorithm service and may specifically include the following steps:

[0076] Step 101: Pull the target anchor's microphone audio from the data storage service in real time;

[0077] In an embodiment of the present invention, whether the host is on-hook can be determined by detecting in real time whether the host is speaking. Specifically, the target host's voice can be collected in real time through the target host's microphone to form the host microphone audio and upload it to the data storage service.

[0078] When the voiceprint verification algorithm service detects new host microphone audio in the data storage service, it can pull the target host's microphone audio from the data storage service in real time to determine whether the target host's voice exists in the host microphone audio, and thus determine whether the target host is offline.

[0079] Step 102: Identify the host's voiceprint from the host's microphone audio;

[0080] Step 103: When the target anchor is a registered anchor, obtain the anchor registration voiceprint of the target anchor;

[0081] A voiceprint is a static, coordinate-based sound spectrum image, or spectrogram, of a person's speech, converted into electromagnetic vibrations using electroacoustic instruments. This image, displayed on a screen or printed on paper, can be observed, compared, and identified. Voiceprints vary from person to person, making them useful for identifying different speakers.

[0082] Voiceprint recognition mainly determines the speaker by comparing two voiceprints. In this embodiment of the present invention, the two parties to be compared can be the voiceprint of the host's microphone audio and the registered voiceprint of the target host.

[0083] The host registration voiceprint is the voiceprint of the audio information collected when the target host registers.

[0084] Step 104: Calculate the voiceprint similarity between the host's voiceprint and the host's registered voiceprint;

[0085] Step 105: Determine whether the host is offline based on the voiceprint similarity.

[0086] After collecting the target anchor's voiceprint and the anchor's registered voiceprint, the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint can be calculated. The voiceprint similarity can be used to determine whether the voiceprint information collected by the microphone is the information of the corresponding registered anchor, thereby determining whether the target anchor is AFK.

[0087] This method collects audio data from the host's microphone, extracts the host's voiceprint from it, and then calculates the similarity between the voiceprint and the host's registered voiceprint to determine whether the host's voice is present in the data collected by the host's microphone, thereby determining whether the host has hung up. This avoids the misjudgment that can easily occur when relying on image algorithms to detect hang-up behavior, and improves the accuracy of hang-up behavior detection.

[0088] See also Figure 2 , Figure 2 This is a flowchart of a method for detecting idle playback during live streaming provided by another embodiment of the present invention. Specifically, the following steps may be included:

[0089] Step 201: Pull the target anchor's microphone audio from the data storage service in real time;

[0090] In an embodiment of the present invention, whether the host is on-hook can be determined by detecting in real time whether the host is speaking. Specifically, the target host's voice can be collected in real time through the target host's microphone to form the host microphone audio and upload it to the data storage service.

[0091] When the voiceprint verification algorithm service detects new host microphone audio in the data storage service, it can pull the target host's microphone audio from the data storage service in real time to determine whether the target host's voice exists in the host microphone audio, and thus determine whether the target host is offline.

[0092] Step 202: extracting audio data of a preset first duration from the host's microphone audio;

[0093] Step 203: intercepting the audio data using a preset window length and a preset window offset to obtain a plurality of audio segments;

[0094] In an embodiment of the present invention, after the target host's microphone audio is pulled from the data storage service, a plurality of audio data of a preset first duration can be extracted from the host's microphone audio. Then, audio data is intercepted using a preset window length and a preset window offset to obtain a plurality of audio segments. The preset window length is greater than the preset window offset.

[0095] For example, multiple audio data of a minutes each are obtained from the host's microphone audio, each audio data has a corresponding host ID; then each audio data is intercepted with a window length of b seconds and a window offset of c (c < b) seconds to obtain several audio segments of b seconds.

[0096] Step 204: convert each audio clip into the host's voiceprint through voiceprint recognition;

[0097] After obtaining the audio clips, each audio clip can be converted into a voiceprint feature through a voiceprint recognition algorithm as the voiceprint of the target anchor.

[0098] It should be noted that those skilled in the art may use any conventional voiceprint recognition algorithm to perform voiceprint recognition, and the embodiments of the present invention do not impose any specific limitation on the selection of the voiceprint recognition algorithm.

[0099] Step 205: When the target anchor is a registered anchor, obtain the anchor registration voiceprint of the target anchor;

[0100] In actual scenarios, new hosts do not generate historical audio data, so voiceprint identification cannot be performed based on historical audio. Therefore, in this embodiment of the present invention, idleness determination is mainly performed on hosts who have already performed live broadcasts. After the host has performed a live broadcast, voiceprint information can be collected for voiceprint registration, and the corresponding host registration voiceprint can be obtained, so that subsequent idleness determination can be made based on the host registration voiceprint.

[0101] In one example, the target anchor's registration voiceprint generation process is as follows:

[0102] S101, obtaining the target anchor's historical audio;

[0103] S102, separating the human voice audio and background audio from the anchor's historical audio;

[0104] S103, segmenting the human voice audio using a voice activity detection algorithm to obtain human voice segments;

[0105] S104, separating the background sound fragment corresponding to each voice fragment from the background audio;

[0106] S105, determining in sequence whether the volume of the background sound segment corresponding to each vocal segment is lower than a preset threshold;

[0107] S106: If yes, retain the host's historical audio segment corresponding to the human voice segment as the clean audio;

[0108] S107, clustering all clean audios to obtain a number of clean audio clusters;

[0109] S108, taking the clean audio cluster with the largest number of clean audios as the target cluster;

[0110] S109, based on the center of the target cluster, selecting a preset number of clean audios as registration audios;

[0111] S110: Extract the voiceprint of the registration audio as the target anchor's registration voiceprint.

[0112] In the specific implementation, Figure 3 As shown, the target host's historical audio can be collected and then clean vocals can be filtered out. This process can include: using a sound source separation algorithm to separate the host's historical audio into vocal audio and background audio. Then, using a voice activity detection (VAD) algorithm, the vocal audio is segmented to obtain vocal segments. Next, the corresponding background audio segments are obtained based on the timestamps of the vocal segments. A determination is made as to whether the volume of the background audio segment corresponding to each vocal segment is below a preset threshold. If so, the corresponding segment of the host's historical audio is retained as clean audio. Otherwise, it is discarded.

[0113] Because not all clean audio collected is the target streamer's voice, further screening is required. Once a sufficient amount of clean audio has been collected (the specific amount can be determined based on actual needs), clustering is performed to select the cluster with the largest number of audio clips as the target cluster. The center of this target cluster is calculated, and a preset number of clean audio clips near the center are selected as registration audio clips. The voiceprint of each registration audio clip is then extracted as the streamer's registration voiceprint.

[0114] In one example, a registered streamer will have a corresponding voiceprint registration record, while those without a record will wait until they have collected enough audio before registering. Only registered streamers will undergo voiceprint verification, while those who have not registered will not undergo voiceprint verification.

[0115] It should be noted that since unregistered anchors will not undergo voiceprint verification, when the target anchor is not a registered anchor, it can be determined that the target anchor is in a non-idle state.

[0116] Step 206: Calculate the voiceprint similarity between the host's voiceprint and the host's registered voiceprint;

[0117] Step 207: Determine whether the host is offline based on the voiceprint similarity.

[0118] After collecting the target anchor's voiceprint and anchor's registered voiceprint, the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint can be calculated. The voiceprint similarity can be used to determine whether the voiceprint information collected by the microphone is the information of the corresponding registered anchor, thereby determining whether the target anchor is AFK.

[0119] In one example, the step of determining whether the anchor is offline based on voiceprint similarity may include the following sub-steps:

[0120] S71, determining whether there is at least one audio segment in the audio data corresponding to the host's voiceprint and the host's registered voiceprint have a voiceprint similarity greater than a preset similarity threshold;

[0121] S72: If yes, determine that the audio data contains the target host's voice and mark it with a first identifier;

[0122] S73: If not, determine that the audio data does not contain the target host's voice, and mark the second identifier;

[0123] S74, determining whether the plurality of audio data segments within the preset second duration are all marked with a second identifier;

[0124] S75, if not, determining that the target anchor is in a non-on-hook state;

[0125] S76: If yes, determine that the target anchor is in the on-hook state.

[0126] In a specific implementation, the similarity between the host's voiceprint corresponding to each audio clip and the host's registered voiceprint can be calculated to see if it exceeds a preset similarity threshold. If so, the audio clip is determined to be the host's voice; if not, it is marked as not the host's voice. When there is at least one audio clip in the audio data whose corresponding host's voiceprint has a voiceprint similarity greater than a preset similarity threshold and the host's registered voiceprint, the audio data is determined to contain the target host's voice. In this case, a first identifier (such as 1) can be marked for it; otherwise, a second identifier (such as 0) can be marked. Then, it is determined whether multiple audio data within a preset second time length are all marked with the second identifier. If so, the host is determined to be on-hook; otherwise, the host is determined to be on-hook.

[0127] For example, when the voiceprint similarity between the host's voiceprint in an audio clip and the host's registered voiceprint exceeds the preset similarity threshold, it is marked as 1, otherwise it is marked as 0. If at least one of the b-second audio clips in the a-minute audio data is marked as 1, then this a-minute audio data is judged to contain the host's voice and is marked as 1, otherwise it is marked as 0. If the audio data in the audio of a host for d consecutive minutes (the second preset duration) are all marked as 0, then the host is judged to be in the on-hook state, otherwise it is in the non-on-hook state. It should be noted that if the host starts broadcasting for less than d minutes, it is also considered to be in the non-on-hook state, and the second preset duration d can be 20-30 minutes.

[0128] This method collects audio data from the host's microphone, extracts the host's voiceprint from it, and then calculates the similarity between the voiceprint and the host's registered voiceprint to determine whether the host's voice is present in the data collected by the host's microphone, thereby determining whether the host has hung up. This avoids the misjudgment that can easily occur when relying on image algorithms to detect hang-up behavior, and improves the accuracy of hang-up behavior detection.

[0129] For easier understanding, the following describes the live broadcast idle detection process through a specific example:

[0130] See also Figure 4 , Figure 4 A live broadcast hang-up detection flow chart provided in an embodiment of the present invention.

[0131] like Figure 4 As shown, after the anchor starts broadcasting, the collected anchor microphone audio is first sliced ​​to obtain audio slices; then the voiceprint in the anchor microphone audio is extracted through the voiceprint recognition model to obtain the anchor voiceprint; then the voiceprint similarity between the anchor voiceprint and the anchor registered voiceprint is calculated; in turn, it is determined whether there is at least one audio segment of each audio data whose voiceprint similarity is greater than the preset similarity threshold. If so, a first identifier is marked to determine that the anchor is not on-hook; if not, a second identifier is marked; then it is determined that all audio data of the anchor within d consecutive minutes are the second identifier. If not, it is determined that the anchor is not on-hook, and if so, it is determined that the anchor is on-hook.

[0132] See also Figure 5 , Figure 5 This is a structural block diagram of a live broadcast hang-up detection device provided by an embodiment of the present invention.

[0133] An embodiment of the present invention provides a live broadcast hang-up detection device, which is applied to a voiceprint verification algorithm service; the device includes:

[0134] The host microphone audio pulling module 501 is used to pull the host microphone audio of the target host from the data storage service in real time;

[0135] Anchor voiceprint recognition module 502, used to recognize the anchor's voiceprint from the anchor's microphone audio;

[0136] The anchor registration voiceprint acquisition module 503 is used to acquire the anchor registration voiceprint of the target anchor when the target anchor is a registered anchor;

[0137] The voiceprint similarity calculation module 504 is used to calculate the voiceprint similarity between the host's voiceprint and the host's registered voiceprint;

[0138] The on-hook judgment module 505 is used to judge whether the anchor is on-hook based on the voiceprint similarity.

[0139] In this embodiment of the present invention, the host voiceprint recognition module 502 includes:

[0140] An audio data extraction submodule is used to extract audio data of a preset first duration from the host's microphone audio;

[0141] The audio interception submodule is used to intercept the audio data with a preset window length and a preset window offset to obtain a plurality of audio clips;

[0142] The host voiceprint recognition submodule is used to convert each audio clip into the host's voiceprint through voiceprint recognition.

[0143] In an embodiment of the present invention, the following further comprises:

[0144] The anchor historical audio acquisition module is used to obtain the anchor historical audio of the target anchor;

[0145] Audio separation module, used to separate human voice audio and background audio from the anchor's historical audio;

[0146] The human voice segmentation module is used to segment the human voice audio using a voice activity detection algorithm to obtain human voice segments;

[0147] Background sound segmentation module, used to separate the background sound segment corresponding to each voice segment from the background audio;

[0148] A volume determination module is used to determine in turn whether the volume of the background sound segment corresponding to each vocal segment is lower than a preset threshold;

[0149] A clean audio retention module is used to retain the host's historical audio fragments corresponding to the human voice fragments as clean audio;

[0150] The clustering module is used to cluster all clean audios to obtain several clean audio clusters;

[0151] a target cluster determination module, configured to select the clean audio cluster with the largest number of clean audios as the target cluster;

[0152] A registration audio selection module is used to select a preset number of clean audios as registration audios based on the center of the target cluster;

[0153] The anchor registration voiceprint extraction module is used to extract the voiceprint of the registration audio as the anchor registration voiceprint of the target anchor.

[0154] In this embodiment of the present invention, the on-hook determination module 505 includes:

[0155] The first judgment submodule is used to determine whether there is at least one audio segment in the audio data whose corresponding voiceprint of the anchor is greater than a preset similarity threshold value with the voiceprint similarity of the anchor's registered voiceprint;

[0156] A first identification marking submodule is used to determine that the audio data contains the voice of the target host and mark the first identification;

[0157] A second identification marking submodule is used to determine that the audio data does not contain the voice of the target host, and mark the second identification;

[0158] A second judgment submodule is used to judge whether the plurality of audio data segments within a preset second duration are all marked with a second identifier;

[0159] The non-hang-up state determination submodule is used to determine whether the target anchor is in the non-hang-up state if no;

[0160] The on-hook status determination submodule is used to determine if the target anchor is in the on-hook state.

[0161] In an embodiment of the present invention, the following further comprises:

[0162] The non-hang-up state determination module is used to determine that the target anchor is in a non-hang-up state when the target anchor is not a registered anchor.

[0163] An embodiment of the present invention further provides an electronic device, the device including a processor and a memory:

[0164] The memory is used to store program codes and transmit the program codes to the processor;

[0165] The processor is used to execute the live broadcast hang-up detection method of an embodiment of the present invention according to the instructions in the program code.

[0166] An embodiment of the present invention also provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the live broadcast hang-up detection method of the embodiment of the present invention.

[0167] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0169] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0174] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0175] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A live broadcast hang-up detection method, characterized in that: Applied to voiceprint verification algorithm service; the method includes: Pull the target anchor's microphone audio from the data storage service in real time; Identify the host's voiceprint based on the host's microphone audio; When the target anchor is a registered anchor, obtaining the anchor registration voiceprint of the target anchor; Calculating the voiceprint similarity between the anchor's voiceprint and the anchor's registered voiceprint; Determine whether the anchor is idle based on the voiceprint similarity; Wherein, when the target anchor is a registered anchor, before the step of obtaining the anchor registration voiceprint of the target anchor, the method further includes: Obtain the target anchor's historical audio; Separating human voice audio and background audio from the anchor's historical audio; Segmenting the human voice audio using a voice activity detection algorithm to obtain human voice segments; Separating a background sound segment corresponding to each of the human voice segments from the background audio; Determine in turn whether the volume of the background sound segment corresponding to each of the human voice segments is lower than a preset threshold; If yes, retain the host's historical audio segment corresponding to the vocal segment as the clean audio; Clustering all the clean audios to obtain a plurality of clean audio clusters; The clean audio cluster with the largest number of clean audios is selected as the target cluster; Based on the center of the target cluster, a preset number of clean audios are selected as registration audios; The voiceprint of the registration audio is extracted as the anchor registration voiceprint of the target anchor.

2. The method according to claim 1, characterized in that The step of identifying the host's voiceprint based on the host's microphone audio includes: Extracting audio data of a preset first duration from the host microphone audio; Performing audio interception on the audio data using a preset window length and a preset window offset to obtain a plurality of audio segments; Each of the audio clips is converted into the host's voiceprint through voiceprint recognition.

3. The method according to claim 2, characterized in that The step of determining whether the anchor is offline based on the voiceprint similarity includes: Determine whether there is at least one anchor voiceprint corresponding to the audio segment in the audio data, and the similarity between the anchor's registered voiceprint and the anchor's voiceprint is greater than a preset similarity threshold; If so, it is determined that the audio data contains the voice of the target host, and a first identifier is marked; If not, it is determined that the audio data does not contain the target host's voice, and a second identifier is marked; Determining whether the plurality of audio data segments within the preset second duration are all marked with the second identifier; If not, it is determined that the target anchor is in a non-on-hook state; If so, it is determined that the target anchor is in an on-hook state.

4. The method according to claim 1, wherein Also includes: When the target anchor is not a registered anchor, it is determined that the target anchor is in a non-on-hook state.

5. A live broadcast hang-up detection device, characterized in that: Applied to voiceprint verification algorithm service; the device includes: The host microphone audio pulling module is used to pull the target host's microphone audio from the data storage service in real time; An anchor voiceprint recognition module, used to recognize the anchor's voiceprint of the anchor's microphone audio; The anchor registration voiceprint acquisition module is used to obtain the anchor registration voiceprint of the target anchor when the target anchor is a registered anchor; A voiceprint similarity calculation module, configured to calculate the voiceprint similarity between the host's voiceprint and the host's registered voiceprint; An on-hook judgment module is used to judge whether the anchor is on-hook based on the voiceprint similarity; Among them, also include: The anchor historical audio acquisition module is used to obtain the anchor historical audio of the target anchor; An audio separation module, used to separate human voice audio and background audio from the anchor's historical audio; A human voice segmentation module, configured to segment the human voice audio using a voice activity detection algorithm to obtain human voice segments; A background sound segmentation module is used to separate the background sound segment corresponding to each of the human voice segments from the background audio; A volume determination module is used to determine in turn whether the volume of the background sound segment corresponding to each of the human voice segments is lower than a preset threshold; A clean audio retention module is used to retain the anchor's historical audio fragment corresponding to the human voice fragment as clean audio; A clustering module, configured to cluster all the clean audios to obtain a plurality of clean audio clusters; a target cluster determination module, configured to select the clean audio cluster with the largest number of clean audios as the target cluster; a registration audio selection module, configured to select a preset number of clean audios as registration audios based on the center of the target cluster; The anchor registration voiceprint extraction module is used to extract the voiceprint of the registration audio as the anchor registration voiceprint of the target anchor.

6. The device according to claim 5, characterized in that The anchor voiceprint recognition module includes: An audio data extraction submodule, configured to extract audio data of a preset first duration from the host's microphone audio; An audio interception submodule, configured to intercept the audio data using a preset window length and a preset window offset to obtain a plurality of audio segments; The host voiceprint recognition submodule is used to convert each of the audio clips into the host voiceprint through voiceprint recognition.

7. An electronic device, characterized in that: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the live broadcast hang-up detection method described in any one of claims 1-4 according to the instructions in the program code.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used by the processor to execute the live broadcast hang-up detection method described in any one of claims 1-4.

Images