Singing detection method, singing detection device and electronic equipment

Abstract

The invention relates to the field of video processing, in particular to a singing detection method and device and electronic equipment. The method comprises the following steps: establishing a sample voiceprint library comprising user voiceprint information; acquiring a real-time audio stream, and judging whether the real-time audio stream is a music clip or not; performing voiceprint information extraction on the music clip to obtain a voiceprint result; and according to the sample voiceprint library and the voiceprint result, judging whether the music clip is singing by the user. And determining that the user is singing in the music clip, and adding a singing tag for the user. According to the embodiment of the invention, singing detection is carried out on the real-time audio stream, whether the user is singing is determined, and a singing tag can be added to the user after determining that the music clip is singing by the user. The experience feeling of watching users is improved, and interested video contents can be selected according to own preferences.

Description

Technical Field

[0001] The present invention relates to the field of video processing, and more specifically, to a method, device and electronic device for detecting singing. Background Art

[0002] With the development of multimedia front-end technology, people's lives are changing more and more, and the types of entertainment activities they participate in are diverse and in any form. Live streaming is one of the most expressive and appealing forms of Internet products. Now more and more people are participating in live streaming entertainment and are keen to share their lives and habits through videos. The richness and diversity of video content have brought great convenience to users, but also brought difficulties in selection. When browsing videos, users often hope to quickly find the content they are interested in. For example, music lovers may prefer to watch videos that include live singing rather than those that only play background music.

[0003] Currently, when people watch live broadcasts, they need to enter the live broadcast room to know the host’s live broadcast content, which is not only difficult to distinguish, but also easy to waste unnecessary time. Summary of the invention

[0004] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a method, device and electronic device for detecting singing, so as to achieve the effect of detecting in real time that a user is singing.

[0005] According to a first aspect of the present application, a method for detecting singing is provided, comprising:

[0006] Establishing a sample voiceprint library including user voiceprint information;

[0007] Get real-time audio stream;

[0008] Determining whether the real-time audio stream is a music clip;

[0009] Extracting voiceprint information from the music clip to obtain a voiceprint result;

[0010] It is determined whether the music clip is sung by the user according to the sample voiceprint library and the voiceprint result.

[0011] Optionally, the determining whether the real-time audio stream is a music clip includes:

[0012] Obtaining the speech rate and music duration of the real-time audio stream;

[0013] Whether the real-time audio stream is a music clip is determined according to the speech rate and music duration of the real-time audio stream.

[0014] Optionally, the acquiring the speech rate and music duration of the real-time audio stream includes:

[0015] Segmenting the real-time audio stream according to a preset time length to obtain segmented segments;

[0016] Obtain the text corresponding to the audio in each of the segmented segments and the timestamp of each text;

[0017] Obtaining the speech rate of each segmented segment according to the number of characters in each segmented segment and the timestamps of all characters;

[0018] Music detection is performed on each of the segmented segments to obtain the music duration of each of the segmented segments.

[0019] The real-time audio stream is segmented according to a preset duration to obtain segmented segments. The preset duration regulates the length of the segmented segments, making it easy to obtain the text corresponding to the audio in each segmented segment and the timestamp of each text, thereby improving processing efficiency. Music detection is performed on each segmented segment to obtain the duration of the music in each segmented segment. The detection is targeted and is conducive to analyzing the segmented segments.

[0020] Optionally, judging whether the real-time audio stream is a music clip according to the speech rate and the duration of the music in the real-time audio stream specifically includes:

[0021] Determining whether each segmented segment is a music segment according to the speech speed and the duration of the music in each segmented segment;

[0022] The extracting voiceprint information from the music clip to obtain a voiceprint result includes:

[0023] splicing the segmented segments determined to be music segments according to the timestamps to obtain spliced ​​audio;

[0024] The voiceprint information in the spliced ​​audio is extracted to obtain a voiceprint result.

[0025] According to the speech speed and duration of each segmented segment, it can be determined whether the real-time audio stream is a music segment, and the possibility of a speech segment can be excluded. The segmented segments determined to be music segments are spliced ​​to obtain spliced ​​audio, which effectively ensures the continuity of the segments and reduces the impact of idle segments. The voiceprint extraction result can reflect the voiceprint characteristics of the real-time audio stream, which is convenient for user personality authentication of the real-time audio stream.

[0026] Optionally, judging whether the user is singing in the music clip according to the sample voiceprint library and the voiceprint result includes:

[0027] Preset voiceprint similarity score threshold;

[0028] The voiceprint result is input into the sample voiceprint library for retrieval to obtain a voiceprint similarity score;

[0029] If the voiceprint similarity score is greater than the voiceprint similarity score threshold, the corresponding user is determined according to the voiceprint similarity score, and it is determined that the user is singing in the music clip.

[0030] The voiceprint similarity score reflects the similarity between the voiceprint result and the user voiceprint information of the sample voiceprint library, and can determine the source of the voiceprint result. A voiceprint similarity score threshold is preset, and the voiceprint result is analyzed using the sample voiceprint library to obtain a voiceprint similarity score, and the two are compared. When the voiceprint similarity score is greater than the voiceprint similarity score threshold, it is determined that the voiceprint result and the user sample information of the sample voiceprint library are from the same source, and the user can be determined to be the user of the sample voiceprint library.

[0031] Optionally, when it is determined that the music clip is sung by the user, a singing tag is added to the user.

[0032] Adding a singing tag to the user can display the user's status, making it easier for other users to identify his status.

[0033] Optionally, when it is determined that the music clip is sung by the user, adding a singing tag for the user includes:

[0034] Maintaining a detection queue for temporarily storing music clips, wherein one user corresponds to one detection queue;

[0035] When it is determined that the music clip is a song sung by the user, the start time point of the user singing in the music clip is obtained.

[0036] According to the starting time point and the detection queue corresponding to the user, it is determined to add a singing tag for the user and add the music clip to the detection queue.

[0037] Optionally, determining to add a singing tag for the user and adding the music clip to the detection queue according to the start time point and the detection queue corresponding to the user is specifically as follows:

[0038] Before the music clip is added to the detection queue, the start time point of the music clip at the end of the detection queue corresponding to the user is first obtained as the first time point, and it is determined whether the start time point of the music clip and the first time point exceed a certain time interval;

[0039] If the time interval exceeds the certain time interval, the number of music clips in the detection queue corresponding to the user is counted; if the time interval does not exceed the certain time interval, the music clip is added to the end of the detection queue;

[0040] If the number of the music clips is greater than or equal to a preset clip number threshold, a singing tag is added to the user, and the detection queue corresponding to the user is cleared, and the music clip is added to the cleared detection queue corresponding to the user.

[0041] The detection queue uses the time point as the detection point, which can more accurately judge the real-time nature of the user's singing, thereby determining the queue entry form of the music clip. The characteristics of the time point can be used to effectively perform the operation of adding singing tags, which can improve the real-time nature of the judgment of the user's singing.

[0042] According to a second aspect of the present application, a singing detection device is provided, comprising:

[0043] A sample creation module, used to create a sample voiceprint library including user voiceprint information;

[0044] An acquisition module, used for acquiring the real-time audio stream;

[0045] A first judging module, used to judge whether the real-time audio stream is a music clip;

[0046] An extraction module, used to extract voiceprint information from the music clip to obtain a voiceprint result;

[0047] A second judgment module is used to judge whether the user is singing in the music clip according to the sample voiceprint library and the voiceprint result;

[0048] The adding module is used to add a singing tag for the user when it is determined that the music clip is sung by the user.

[0049] The singing detection device, each module is an indispensable part of the whole process, and the modules cooperate with each other. The sample establishment module obtains the sample voiceprint library, which provides the source of sample voiceprint information of the user. The acquisition module is used to obtain the real-time audio stream, which comes from the user, and the subsequent analysis will be targeted at the real-time audio stream. The real-time audio stream first passes through the first judgment module to determine whether the real-time audio stream is a music clip. When the real-time audio stream is determined to be a music clip, it enters the extraction module. The extraction module is used to extract the voiceprint information of the music clip to obtain the voiceprint result, which is an effective reference for the subsequent judgment of the source of the real-time audio. The voiceprint result is further input into the second judgment module. According to the voiceprint result and the sample voiceprint library, it can be determined whether the singing clip comes from the sample voiceprint library, so as to facilitate the determination of the corresponding user.

[0050] According to a third aspect of the present application, an electronic device is provided, including:

[0051] a memory for storing one or more computer programs;

[0052] The processor, when the one or more computer programs are executed by the processor, implements a method for detecting singing as described in the first aspect above.

[0053] The electronic device provides a complete set of equipment to facilitate the detection of audio and confirm whether it is the real-time singing of the user. The equipment processes the real-time audio stream quickly, accurately and effectively, and has certain advantages.

[0054] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for detecting singing as described in the first aspect above when executed.

[0055] Based on any of the above aspects, the singing detection method, device, and electronic device provided in the embodiments of the present application can process and deeply analyze the audio stream to determine whether the implemented audio stream comes from the music sung by the user in real time. When it is determined that the user is singing, the present application can also add a singing tag for the user, so that the audio and video platform can add singing tags to users such as live broadcast rooms as real-time user status, so as to provide users with more personalized content recommendations, helping users to find the audio and video and live broadcast rooms of interest to them more quickly, thereby improving viewing efficiency and satisfaction. At the same time, this also provides a platform for video creators to showcase their talents, making their efforts easier to be discovered and appreciated by the audience. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 A schematic application scenario diagram of a singing detection method provided in this embodiment.

[0058] Figure 2 A flow chart of a singing detection method provided in this embodiment.

[0059] Figure 3 A schematic diagram of the sub-steps of adding labels S150 for detecting a singing method provided in this embodiment.

[0060] Figure 4 A schematic diagram of the functional modules of a singing detection device provided in this embodiment.

[0061] Figure 5 This is a schematic diagram of the structure of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0062] The drawings of this application are only used for illustrative purposes and should not be construed as limiting the present application. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0063] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0064] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0065] At present, many audio and video platforms do not process audio and video effectively, which causes users to spend a lot of time browsing audio and video that they are not interested in when they are engaged in entertainment activities, which reduces the experience. Audio and video that have not been effectively processed appear on the page, and you need to click to enter the audio and video to identify the audio and video content. This process requires viewers to not only screen the audio and video but also reject the attraction of other audio and video. For viewers with special needs, conventional audio and video will make it difficult for them to find a type of video they like. Therefore, detection of singing and other processing is imperative for online videos.

[0066] This embodiment provides a technical solution that can solve the above-mentioned problem. The specific implementation methods of this application are described in detail below in conjunction with the accompanying drawings.

[0067] For example, a schematic diagram of a singing detection application scenario is provided in an embodiment of the present application. Figure 1 As shown, the application scenario includes at least a server 100 and a terminal 200 that can communicate with the server 100. The server 100 has the function of transmitting and receiving audio and video, and also has the function of analyzing and processing the received audio and video; the terminal device 200 has the function of transmitting and receiving audio and video, and also has the function of analyzing and processing the audio and video, and can also have the function of playing audio and video.

[0068] It is understandable that the server 100 can be an independent electronic device or a cluster of multiple electronic devices; the terminal 200 can be a smart phone terminal, a personal computer, a tablet computer, a car terminal, etc., but is not limited thereto.

[0069] In an practicable manner, the server 100 and the terminal 200 can respectively execute the singing detection method provided in the embodiment of the present application, or, optionally, the singing detection method provided in the embodiment of the present application is partially executed in the server 100 and partially executed in the terminal 200.

[0070] like Figure 2 As shown, this embodiment provides a singing detection method, which may include the following steps:

[0071] S110, establishing a sample voiceprint library including user voiceprint information;

[0072] In this embodiment, the user voiceprint information may include user singing voiceprint samples, user speaking voiceprint samples, etc. The user voiceprint information may involve fundamental frequency, sound intensity, timbre, speaking speed, rhythm, etc.

[0073] In the specific implementation process, the sample voiceprint library can be formed by collecting voiceprint information of different users. When the singing detection method of the present application is applied to an audio and video platform, the user can be a user of the audio and video platform.

[0074] In an optional implementation, voiceprint information including user fundamental frequency, sound intensity, timbre, speaking speed, rhythm, etc. is manually collected in advance, and the collected voiceprint information is used as sample voiceprints. A sample voiceprint library is established based on the sample voiceprints, and the sample voiceprints of the sample voiceprint library include but are not limited to the following.

[0075] S120, obtaining a real-time audio stream, and determining whether the real-time audio stream is a music clip;

[0076] In this embodiment, the real-time audio stream is obtained and it is determined whether the real-time audio stream is a music clip. The real-time audio stream can be the audio in the user's video or an independent audio. The time length of the real-time audio stream should not be too long. A real-time audio stream of any time length such as 5 minutes or 10 minutes can be obtained, which is not specifically limited in this embodiment. The method for determining whether the real-time audio stream is a music clip can be through pitch testing, rhythm analysis, spectrum analysis channel, formant analysis energy analysis or based on machine learning models and deep learning models.

[0077] In an optional implementation, a real-time audio stream in a user video is collected, and the speech rate and music duration of the real-time audio stream are obtained to determine whether the real-time audio stream is a music clip.

[0078] In a more preferred embodiment, the specific step of obtaining the speech rate and music duration of the real-time audio stream may include:

[0079] The real-time audio stream is segmented according to a preset time length to obtain segmented segments, and the preset time length is set according to the specific situation and can be any time period. For example, 5 seconds is set as the preset time length. According to the preset time length, a plurality of segmented segments with a time length of 5 seconds are obtained.

[0080] The segmented segments are subjected to character recognition to obtain the number of characters in the segmented segments and the timestamps of the corresponding characters. The character recognition can be performed by text recognition. Since the real-time audio stream is obtained, the real-time audio stream can carry time information. By extracting the time information in the real-time audio stream, the time information corresponding to each character can be obtained as a timestamp.

[0081] The speech rate of the segmented segment is obtained according to the number of characters in the segmented segment and the timestamps of all characters. In the specific implementation process, the total length of the timestamps corresponding to all characters can be obtained according to the timestamps of all characters, and the average speech rate corresponding to the output characters in each segmented segment can be calculated according to the number of characters and the total length of the timestamps, and the average speech rate is used as the speech rate of the segmented segment.

[0082] Music detection is performed on the segmented segments to obtain the duration of the music in the segmented segments.

[0083] In the specific implementation process, judging whether the real-time audio stream is a music clip based on the speech rate and music duration of the real-time audio stream is specifically judging whether each segmented segment is a music clip based on the speech rate and music duration of each segmented segment. In a preferred embodiment, the speech rate threshold and the music duration threshold are set according to the actual situation, and the speech rate and music duration of each segmented segment are compared with the speech rate threshold and the music duration threshold respectively, so as to judge whether each segmented segment is a music clip. Exemplarily, the speech rate threshold can be set to three words, three and a half words, four words or other values ​​per second, and the music duration can be set to three seconds, four seconds or other time lengths.

[0084] Exemplarily, when the speech rate of the segmented segment is less than 3.6 words per second and the duration of the music in the segmented segment is greater than four seconds, the segmented segment is judged to be a music segment. The judgment method includes but is not limited to this and is not limited here.

[0085] S130, extracting voiceprint information from the music clip to obtain a voiceprint result;

[0086] In this embodiment, the voiceprint information of the music clip is extracted to obtain a voiceprint result, and the voiceprint information extraction includes fundamental frequency extraction, Mel-frequency cepstral coefficient extraction, vocal tract feature extraction, speech rate and rhythm feature extraction, etc.

[0087] In an optional implementation, after determining that the real-time audio stream is a music clip, the real-time audio stream is intercepted according to the timestamp of each character in the segmented clip. The intercepted result obtained contains the audio corresponding to the text portion of the real-time audio stream. The intercepted results obtained are spliced ​​to obtain spliced ​​audio, and the spliced ​​audio is sampled. In this implementation, the sampling time length is set to two seconds. The spliced ​​audio is sampled according to a time length of two seconds, and the voiceprint information vector therein is extracted as the voiceprint result, and the voiceprint information vector that does not meet the two-second extraction condition is ignored. The voiceprint result is obtained by the sampling and extraction method, and the extraction method includes but is not limited to this.

[0088] S140, judging whether the user is singing in the music clip according to the sample voiceprint library and the voiceprint result;

[0089] In this embodiment, the judgment of whether the user is singing in the music clip is based on the sample voiceprint library and the voiceprint result. The music clip can be processed based on a deep learning model to determine whether the user is singing.

[0090] In an optional implementation, the user voiceprint information in the sample voiceprint library may be a voiceprint vector.

[0091] The obtained voiceprint results are input into the sample voiceprint library for retrieval, and the retrieval target is the voiceprint information of the user manually entered in advance. One or more similar or identical voiceprint information are extracted according to the retrieval results, and the similarity between the voiceprint result and one or more similar or identical voiceprint information is calculated to obtain one or more voiceprint similarity scores corresponding to each voiceprint result. When the voiceprint similarity scores include multiple ones, the obtained voiceprint similarity scores can be averaged as the voiceprint similarity score of the current music clip. When judging whether the music clip is the user singing, a voiceprint result threshold can be preset, such as a voiceprint similarity score threshold. When the voiceprint similarity score is greater than the voiceprint similarity score threshold, it is considered that the music clip is the user singing. The judgment method includes but is not limited to this.

[0092] S150: When it is determined that the music clip is a song sung by the user, a singing tag is added for the user.

[0093] In this embodiment, when it is determined that the music clip is a user singing, a singing tag is added to the user. The way of adding tags is that when it is determined that the music clip is a user singing, the music clip is directly tagged, and the idle music part is not excluded. When the embodiment of the present application is applied to a video platform, such as a live broadcast room, a singing tag can be added to each live broadcast room host, so that other users can understand the status of the live broadcast room host in real time.

[0094] In an optional embodiment, a detection queue for temporarily storing music clips is maintained, and one detection queue corresponds to one user; when it is determined that the music clip is a song sung by the user, the start time point of the user's singing in the music clip is obtained, and based on the start time point and the detection queue corresponding to the user, it is determined to add a singing tag for the user and add the music clip to the detection queue. The method of adding tags is not limited here.

[0095] In an optional embodiment, illustratively, Figure 3 A flowchart of adding labels for detecting a singing method provided in an embodiment of the present application is provided, and the specific steps of step S150 may include:

[0096] S151, before the music clip is added to the detection queue, first obtain the start time point of the music clip at the end of the detection queue corresponding to the user as the first time point, and determine whether the start time point of the music clip and the first time point exceed a certain time interval;

[0097] S152, if the time interval exceeds the certain time interval, counting the number of music clips in the detection queue corresponding to the user; if the time interval does not exceed the certain time interval, adding the music clip to the end of the detection queue;

[0098] S153. If the number of the music clips is greater than or equal to a preset clip number threshold, a singing tag is added to the user, and the detection queue corresponding to the user is cleared, and the music clip is added to the detection queue corresponding to the user after the clearing.

[0099] According to the method of adding labels, the real-time audio stream corresponding to the music clip judged to be sung by the user can be effectively labeled, excluding the idle music part, so that the viewing user can grasp the video dynamics in real time.

[0100] In this embodiment, the tag information of the user singing is updated in real time by maintaining a detection queue. The purpose is to make the user's singing status real-time, judge the singing status more accurately within a certain period of time, and improve the accuracy of the judgment.

[0101] like Figure 4 As shown, the embodiment of the present application further provides a singing detection device 210. Optionally, the singing detection device 210 may include:

[0102] A sample creation module 211 is used to create a sample voiceprint library including user voiceprint information;

[0103] In this embodiment, the sample creation module 211 can be used to execute Figure 2 As shown in step S110 , for a detailed description of the sample creation module 211 , reference may be made to the description of step S110 .

[0104] An acquisition module 212, used to acquire the real-time audio stream;

[0105] In this embodiment, the acquisition module 212 can be used to execute Figure 2 As shown in step S120, for a detailed description of the acquisition module 212, reference may be made to the description of step S120.

[0106] A first judging module 213, used to judge whether the real-time audio stream is a music clip;

[0107] In this embodiment, the first determination module 213 may be used to execute Figure 2 As shown in step S120, for the specific description of the first determining module 213, reference may be made to the description of step S120.

[0108] An extraction module 214 is used to extract the voiceprint of the music clip to obtain a voiceprint result;

[0109] In this embodiment, the extraction module 214 can be used to perform Figure 2As shown in step S130 , for a detailed description of the extraction module 214 , reference may be made to the description of step S130 .

[0110] A second judgment module 215 is used to judge whether the user is singing in the music clip according to the sample voiceprint library and the voiceprint result;

[0111] In this embodiment, the second determination module 215 may be used to execute Figure 2 As shown in step S140, for the specific description of the second determination module 215, reference may be made to the description of step S140.

[0112] The adding module 216 is used to add a singing tag for the user when it is determined that the music clip is sung by the user.

[0113] In this embodiment, the adding module 216 can be used to perform Figure 2 As shown in step S150, for the detailed description of the adding module 216, reference may be made to the description of step S150.

[0114] It can be understood that the above-mentioned device embodiment and the above-mentioned method embodiment can correspond to each other, and similar descriptions of the device embodiment can refer to the method embodiment. To avoid repetition, it will not be repeated here. A singing detection device provided in an embodiment of the present application can execute a singing detection method provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method. The functional modules of the singing detection device can be implemented in the form of hardware, can be implemented by instructions in the form of software, and can also be implemented by a combination of hardware and software modules.

[0115] Specifically, each step of the method embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor. The steps of the singing detection method in combination with the embodiment of the present application can be directly embodied as a hardware coding processor, or a combination of hardware and software modules in the coding processor. Optionally, the software module can be located in a random access memory, a read-only memory, a programmable read-only memory, a flash memory, an electrically erasable programmable memory, a register, and other storage media. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in combination with its hardware.

[0116] The present application embodiment provides an electronic device 710, whose structure is as follows: Figure 5 The electronic device 710 may be Figure 1 The server 100 or the terminal 200 is shown.

[0117] like Figure 5As shown, the electronic device 710 includes a memory 711, a processor 712, a communication module 713 and an input / output interface 714, etc. Optionally, the memory 711, the processor 712, the communication module 713 and the input / output interface 714 can be connected and communicated through a bus 715.

[0118] The memory 711 is used to store one or more computer programs and transmit the code of the computer program to the processor 712; when the one or more computer programs are executed by the processor 711, the singing detection method in the embodiment of the present application is implemented.

[0119] Optionally, the electronic device 710 can be connected to a network via a communication module 713 to communicate with other devices, such as a terminal or a server, through the network to achieve data interaction. The electronic device 710 can be various forms of digital computers, such as desktop computers, servers, workstations, mainframe computers or other types of computers. The electronic device 710 can also be various forms of mobile terminals, such as smart phones, tablet computers, wearable devices (such as helmets, glasses, watches, etc.) and other similar mobile terminals.

[0120] Optionally, the electronic device 710 can be connected to the required input / output devices, such as a keyboard, a display device, etc., through the input / output interface 714. The electronic device 710 itself can have a display device, and can also be connected to other display devices through the input / output interface 714. Optionally, a storage device, such as a hard disk, can also be connected through the input / output interface 714, so that data in the electronic device 710 can be stored in the storage device, or data in the storage device can be read, and data in the storage device can also be stored in the memory 711. It can be understood that the input / output interface 714 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 714 can be a component of the electronic device 710, or it can be an external device connected to the electronic device 710 when needed.

[0121] Optionally, the memory 711 may be a volatile memory and / or a non-volatile memory, the volatile memory may be a random access memory, etc., and the non-volatile memory may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory or a flash memory, etc.

[0122] Optionally, the computer program stored in the processor 711 may be divided into one or more modules, which are stored in the memory 711 and executed by the processor 712 to complete the method provided in the embodiment. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the computer program instruction segments are used to describe the execution process of the computer program in the electronic device 710.

[0123] Optionally, the processor 712 may be various general and / or special processing components with processing and computing capabilities. Some examples of the processor 712 include, but are not limited to, a central processing unit, a graphics processing unit, a digital signal processor, various special artificial intelligence computing chips, various processors running machine learning model algorithms, and any appropriate controller, microcontroller, processor, etc. The processor 712 executes various methods and processes of this embodiment, illustratively, such as a method for detecting singing in an embodiment of the present application.

[0124] Optionally, the bus 715 may include a path for transmitting information. The bus 715 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Depending on the function, the bus 715 may be divided into an address bus, a data bus, a control bus, etc. In an optional implementation, the embodiment of the present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer is able to execute the method of the above method embodiment. Part or all of the computer program can be loaded and / or installed on the memory 711 of the electronic device 710. When the computer program is executed by the processor 712, one or more steps of a method for detecting singing in an embodiment of the present application can be executed.

[0125] Optionally, the computer-readable storage medium may be a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, or the like.

[0126] Obviously, the above embodiments of the present application are merely examples for clearly illustrating the technical solution of the present application, and are not intended to limit the specific implementation methods of the present application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A method for detecting singing, characterized in that, include: Establishing a sample voiceprint library including user voiceprint information; Get real-time audio stream; Determining whether the real-time audio stream is a music clip; Extracting voiceprint information from the music clip to obtain a voiceprint result; It is determined whether the music clip is sung by the user according to the sample voiceprint library and the voiceprint result.

2. A method for detecting singing according to claim 1, characterized in that: The determining whether the real-time audio stream is a music clip comprises: Obtaining the speech rate and music duration of the real-time audio stream; Whether the real-time audio stream is a music clip is determined according to the speech rate and music duration of the real-time audio stream.

3. A method for detecting singing according to claim 2, characterized in that: The acquiring the speech rate and music duration of the real-time audio stream includes: Segmenting the real-time audio stream according to a preset time length to obtain segmented segments; Obtain the text corresponding to the audio in each of the segmented segments and the timestamp of each text; Obtaining the speech rate of each segmented segment according to the number of characters in each segmented segment and the timestamps of all characters; Music detection is performed on each of the segmented segments to obtain the music duration of each of the segmented segments.

4. A method for detecting singing according to claim 3, characterized in that: The determining whether the real-time audio stream is a music clip according to the speech speed and the duration of the music of the real-time audio stream specifically includes: Determining whether each segmented segment is a music segment according to the speech speed and the duration of the music in each segmented segment; The extracting voiceprint information from the music clip to obtain a voiceprint result includes: splicing the segmented segments determined to be music segments according to the timestamps to obtain spliced ​​audio; The voiceprint information in the spliced ​​audio is extracted to obtain a voiceprint result.

5. A method for detecting singing according to claim 1, characterized in that: The step of judging whether the music clip is sung by the user according to the sample voiceprint library and the voiceprint result includes: Preset voiceprint similarity score threshold; Analyzing the voiceprint result using the sample voiceprint library to obtain a voiceprint similarity score; If the voiceprint similarity score is greater than the voiceprint similarity score threshold, the corresponding user is determined according to the voiceprint similarity score, and it is determined that the user is singing in the music clip.

6. A method for detecting singing according to any one of claims 1 to 5, characterized in that: The method further comprises: When it is determined that the music clip is sung by the user, a singing tag is added for the user.

7. A method for detecting singing according to claim 1, characterized in that: When it is determined that the music clip is sung by the user, adding a singing tag for the user includes: Maintaining a detection queue for temporarily storing music clips, wherein one user corresponds to one detection queue; When it is determined that the music clip is a song sung by the user, the start time point of the user singing in the music clip is obtained. According to the starting time point and the detection queue corresponding to the user, it is determined to add a singing tag for the user and add the music clip to the detection queue.

8. A method for detecting singing according to claim 7, characterized in that: Determining to add a singing tag for the user and adding the music clip to the detection queue according to the start time point and the detection queue corresponding to the user includes: Before the music clip is added to the detection queue, the start time point of the music clip at the end of the detection queue corresponding to the user is first obtained as the first time point, and it is determined whether the start time point of the music clip and the first time point exceed a certain time interval; If the time interval exceeds the certain time interval, the number of music clips in the detection queue corresponding to the user is counted; if the time interval does not exceed the certain time interval, the music clip is added to the end of the detection queue; If the number of the music clips is greater than or equal to a preset clip number threshold, a singing tag is added to the user, and the detection queue corresponding to the user is cleared, and the music clip is added to the cleared detection queue corresponding to the user.

9. A device for detecting singing, characterized in that: The device comprises: A sample creation module, used to create a sample voiceprint library including user voiceprint information; An acquisition module, used for acquiring the real-time audio stream; A first judging module, used to judge whether the real-time audio stream is a music clip; An extraction module, used to extract voiceprint information from the music clip to obtain a voiceprint result; A second judgment module is used to judge whether the music clip is sung by the user according to the sample voiceprint library and the voiceprint result; The adding module is used to add a singing tag for the user when it is determined that the music clip is sung by the user.

10. An electronic device, characterized in that: include: a memory for storing one or more computer programs; A processor, when the one or more computer programs are executed by the processor, implements a method for detecting singing as described in any one of claims 1-8.

Images