Singing object identification method and device, electronic device and storage medium

Abstract

The embodiment of the present application provides a method and device, electronic device and storage medium for identifying a singing object, which belongs to the field of artificial intelligence technology. The method includes: obtaining target audio data of a target singing object; extracting spectral features of the target audio data to obtain Mel-cepstrum features; extracting timbre features of the target audio data to obtain target timbre features; extracting mid-level features of the target audio data to obtain music characteristic features; performing a first fusion process on the Mel-cepstrum features and the target timbre features to obtain a first fused audio feature; performing a second fusion process on the first fused audio feature and the music characteristic feature to obtain a second fused audio feature; predicting the second fused audio feature through a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to characterize the identity of the target singing object. The embodiment of the present application can improve the recognition accuracy of the singing object.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a singing object recognition method and device, electronic equipment and storage medium. Background Art

[0002] With the development of metaverse technology, all aspects of daily life can be extended to a world that combines the real and the virtual through the metaverse. However, as the number of singers in the metaverse increases, commonly used recognition methods often find it difficult to accurately identify the identities of the singers. Therefore, how to improve the accuracy of singer identification has become a technical problem that needs to be solved urgently. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to propose a singing object recognition method and device, electronic device and storage medium, aiming to improve the recognition accuracy of the singing object.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a method for identifying a singing object, the method comprising:

[0005] Obtain target audio data of a target singing object;

[0006] Extracting spectral features of the target audio data to obtain Mel-frequency cepstrum features;

[0007] Extracting timbre features from the target audio data to obtain target timbre features;

[0008] Performing mid-level feature extraction on the target audio data to obtain music characteristic features;

[0009] Performing a first fusion process on the Mel-cepstrum feature and the target timbre feature to obtain a first fused audio feature;

[0010] Performing a second fusion process on the first fused audio feature and the music characteristic feature to obtain a second fused audio feature;

[0011] The second fused audio feature is predicted and processed by a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to characterize the identity of the target singing object.

[0012] In some embodiments, the step of extracting spectral features from the target audio data to obtain Mel-cepstrum features includes:

[0013] Performing sound spectrum calculation on the target audio data by short-time Fourier transform to obtain a target spectrogram;

[0014] The target spectrum is filtered using a preset Mel-cepstrum filter to obtain the Mel-cepstrum feature.

[0015] In some embodiments, the step of extracting timbre features from the target audio data to obtain target timbre features includes:

[0016] Segmenting the target audio data using a preset audio segmentation model to obtain multiple target audio segments;

[0017] Perform feature extraction on each target audio segment using the audio segmentation model to obtain a target audio latent vector;

[0018] Performing feature calculation on all the target audio latent vectors to obtain a target audio mean vector and a target audio variance vector;

[0019] Concatenating the target audio variance vector and the target audio mean vector to obtain a target timbre latent vector;

[0020] The target timbre latent vector is predicted and processed by a preset function to obtain the target timbre feature.

[0021] In some embodiments, the step of extracting mid-level features from the target audio data to obtain music characteristic features includes:

[0022] Inputting the target audio data into a preset feature extraction model, wherein the feature extraction model includes a first convolutional layer, a second convolutional layer, and a third convolutional layer;

[0023] Extracting music mid-level features from the target audio data through the first convolutional layer to obtain first music mid-level features;

[0024] Performing music mid-level feature extraction on the target audio data through the second convolutional layer to obtain second music mid-level features;

[0025] Performing music mid-level feature extraction on the target audio data through the third convolutional layer to obtain a third music mid-level feature;

[0026] The first music middle-level feature, the second music middle-level feature, and the third music middle-level feature are concatenated to obtain the music characteristic feature.

[0027] In some embodiments, the character recognition model includes a GRU layer and a fully connected layer, and the step of performing prediction processing on the second fused audio feature using a preset character recognition model to obtain a target identity label of the target singing object includes:

[0028] Performing time series feature extraction on the second fused audio feature through the GRU layer to obtain a fused time series feature vector;

[0029] The fused time series feature vector is subjected to label prediction processing through the fully connected layer to obtain a target identity label of the target singing object.

[0030] In some embodiments, the step of performing time series feature extraction on the second fused audio feature through the GRU layer to obtain a fused time series feature vector includes:

[0031] Performing time series feature extraction on the second fused audio feature through the first gated recurrent unit of the GRU layer to obtain an initial time series feature vector;

[0032] The initial time series feature vector is subjected to time series feature extraction by the second gated recurrent unit of the GRU layer to obtain the fused time series feature vector.

[0033] In some embodiments, the step of performing label prediction processing on the fused time series feature vector through the fully connected layer to obtain the target identity label of the target singing object includes:

[0034] Performing label probability calculation on the fused time series feature vector using the classification function of the fully connected layer and the preset identity label to obtain a label probability vector corresponding to each preset identity label;

[0035] Select the preset identity label corresponding to the label probability vector with the largest value to obtain the candidate identity label;

[0036] The target identity tag is obtained according to the candidate identity tag.

[0037] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a singing object recognition device, the device comprising:

[0038] A data acquisition module is used to acquire target audio data of a target singing object;

[0039] A spectrum feature extraction module is used to extract spectrum features of the target audio data to obtain Mel-cepstrum features;

[0040] A timbre feature extraction module is used to extract timbre features from the target audio data to obtain target timbre features;

[0041] A mid-level feature extraction module is used to extract mid-level features from the target audio data to obtain music characteristic features;

[0042] A first fusion module is used to perform a first fusion process on the Mel-cepstrum feature and the target timbre feature to obtain a first fused audio feature;

[0043] A second fusion module is configured to perform a second fusion process on the first fused audio feature and the music characteristic feature to obtain a second fused audio feature;

[0044] A prediction module is used to perform prediction processing on the second fused audio feature through a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to characterize the identity of the target singing object.

[0045] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and runnable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the method described in the first aspect above is implemented.

[0046] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the first aspect above.

[0047] The present application proposes a method, device, electronic device, and storage medium for identifying a singing object. The method obtains target audio data of a target singing object; further, extracts spectral features from the target audio data to obtain Mel-frequency cepstrum features; extracts timbre features from the target audio data to obtain target timbre features; and extracts mid-level features from the target audio data to obtain music characteristic features. This method can more conveniently determine the timbre characteristic information, music characteristic information, and other contents of the target singing object. Further, the Mel-frequency cepstrum features and the target timbre features are subjected to a first fusion process to obtain a first fused audio feature. This method enables the timbre characteristic information to be incorporated into the subsequent recognition process, eliminating the influence of musical accompaniment or background music on the recognition effect. Further, the first fused audio feature and the music characteristic feature are subjected to a second fusion process to obtain a second fused audio feature, thereby enabling the music characteristic information to be incorporated into the subsequent recognition process, increasing the recognition of music types and styles and improving recognition accuracy. Finally, the second fused audio feature is predicted using a preset character recognition model to obtain a target identity tag for the target singing object, wherein the target identity tag is used to characterize the identity of the target singing object. This makes it easy to determine the identity of the target singer, effectively solving the problem of difficulty in identifying the singer due to the increase in the number of singers in the metaverse, and improving the accuracy of identifying the singer. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a singing object identification method provided in an embodiment of the present application;

[0049] Figure 2 yes Figure 1 Flowchart of step S102 in FIG.

[0050] Figure 3 yes Figure 1 Flowchart of step S103 in FIG.

[0051] Figure 4 yes Figure 1 Flowchart of step S104 in FIG.

[0052] Figure 5 yes Figure 1 Flowchart of step S107 in FIG.

[0053] Figure 6 yes Figure 5 Flowchart of step S501 in FIG.

[0054] Figure 7 yes Figure 5 Flowchart of step S502 in FIG.

[0055] Figure 81 is a schematic structural diagram of a singing object recognition device provided in an embodiment of the present application;

[0056] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0060] First, let’s analyze some of the terms used in this application:

[0061] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0062] Natural language processing (NLP): NLP uses computers to process, understand, and apply human languages ​​(such as Chinese and English). A branch of artificial intelligence, NLP is an interdisciplinary field between computer science and linguistics, often referred to as computational linguistics. Natural language processing encompasses grammatical analysis, semantic analysis, and discourse comprehension. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intent recognition, information extraction and filtering, text classification and clustering, public opinion analysis, and opinion mining. It encompasses data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research related to language processing, and linguistics research related to language computing.

[0063] Information Extraction (NER): A text processing technology that extracts specified types of entity, relationship, event, and other factual information from natural language text and forms structured data output. Information extraction is a technology that extracts specific information from text data. Text data is composed of some specific units, such as sentences, paragraphs, and chapters. Text information is composed of some small specific units, such as characters, words, phrases, sentences, paragraphs, or a combination of these specific units. Extracting noun phrases, names, place names, etc. from text data is all text information extraction. Of course, the information extracted by text information extraction technology can be of various types.

[0064] The Metaverse is a virtual world connected and created through technological means, mirroring and interacting with the real world. It is a digital living space with a new social system. The Metaverse is essentially a virtualization and digitization of the real world, requiring significant changes to content production, economic systems, user experience, and physical content. However, the development of the Metaverse is gradual, ultimately taking shape through the continuous integration and evolution of numerous tools and platforms, supported by shared infrastructure, standards, and protocols. It provides an immersive experience based on extended reality technology, generates a mirror image of the real world using digital twin technology, and builds an economic system based on blockchain technology. It seamlessly integrates the virtual and real worlds across economic, social, and identity systems, and allows every user to create content and edit the world.

[0065] Fourier transform: This function can be expressed as a linear combination of trigonometric functions (sine and / or cosine functions) or their integrals. In different research fields, the Fourier transform has many different variations, such as the continuous Fourier transform and the discrete Fourier transform.

[0066] Mel-Frequency Cipstal Coefficients (MFCCs) are a set of key coefficients used to construct the Mel-Frequency Cepstrum. A segment of a music signal can be used to generate a cepstrum that adequately represents the signal. The Mel-Frequency Cepstrum coefficients are the cepstrum (the spectrum of the spectrum) derived from this cepstrum. Unlike conventional cepstrum, the Mel-Frequency Cepstrum's most distinctive feature is that the frequency bands on the Mel-Frequency Cepstrum are evenly distributed on the Mel scale. This means that compared to the commonly seen linear cepstrum representation, these frequency bands are more closely aligned with the nonlinear human auditory system. For example, Mel-Frequency Cepstrum is often used in audio compression techniques.

[0067] Pooling: It is essentially a kind of sampling. It selects a certain method to reduce the dimension and compress the input feature map to speed up the operation. The most common pooling process is Max Pooling.

[0068] Activation Function: It is a function that runs on the neurons of the artificial neural network and is responsible for mapping the input of the neuron to the output.

[0069] Vector Quantization (VQ): It is a method of clustering continuous data into discrete data in a way similar to clustering, thereby reducing the amount of data that needs to be stored and achieving the purpose of data compression.

[0070] Encoder: Converts the input sequence into a fixed-length vector.

[0071] Decoder: It converts the previously generated fixed vector into an output sequence. The input sequence can be text, voice, image, or video; the output sequence can be text or image.

[0072] Gated Recurrent Unit (GRU): GRU is a type of recurrent neural network (RNN). Like LSTM (Long-Short Term Memory), it is designed to solve problems such as long-term memory and gradients in backpropagation.

[0073] Softmax function: The Softmax function is a normalized exponential function that can "compress" a K-dimensional vector z containing any real number into another K-dimensional real vector σ(z) so that the range of each element is between (0,1) and the sum of all elements is 1. This function is often used in multi-classification problems.

[0074] With the development of metaverse technology, all aspects of daily life can be extended to a world that combines the real and the virtual through the metaverse. However, as the number of singers in the metaverse increases, commonly used recognition methods often find it difficult to accurately identify the identities of the singers. Therefore, how to improve the accuracy of singer identification has become a technical problem that needs to be solved urgently.

[0075] Based on this, the embodiments of the present application provide a singing object recognition method, a singing object recognition device, an electronic device and a storage medium, aiming to improve the recognition accuracy of the singing object.

[0076] The singing object identification method, singing object identification device, electronic device and storage medium provided in the embodiments of the present application are specifically explained through the following embodiments. First, the singing object identification method in the embodiments of the present application is described.

[0077] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0078] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0079] The singing object recognition method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The singing object recognition method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the singing object recognition method, etc., but is not limited to the above forms.

[0080] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0081] Figure 1 This is an optional flowchart of the singing object recognition method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S107.

[0082] Step S101, obtaining target audio data of a target singing object;

[0083] Step S102, extracting spectral features of the target audio data to obtain Mel-frequency cepstrum features;

[0084] Step S103, extracting timbre features from the target audio data to obtain target timbre features;

[0085] Step S104, extracting mid-level features from the target audio data to obtain music characteristic features;

[0086] Step S105, performing a first fusion process on the Mel-frequency cepstrum feature and the target timbre feature to obtain a first fused audio feature;

[0087] Step S106, performing a second fusion process on the first fused audio feature and the music characteristic feature to obtain a second fused audio feature;

[0088] Step S107, predicting and processing the second fused audio feature through a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to represent the identity of the target singing object.

[0089] Steps S101 to S107 shown in the embodiment of the present application are as follows: obtaining target audio data of the target singing object; further, extracting spectral features of the target audio data to obtain Mel-frequency cepstrum features; extracting timbre features of the target audio data to obtain target timbre features; extracting mid-level features of the target audio data to obtain music characteristic features, which can more conveniently determine the timbre characteristic information, music characteristic information, etc. of the target singing object. A first fusion process is performed on the Mel-frequency cepstrum features and the target timbre features to obtain a first fused audio feature. In this way, the timbre characteristic information can be incorporated into the subsequent recognition process to eliminate the influence of music accompaniment or background music on the recognition effect. A second fusion process is performed on the first fused audio feature and the music characteristic feature to obtain a second fused audio feature, so that the music characteristic information can be incorporated into the subsequent recognition process, thereby increasing the recognition of music types and music styles and improving recognition accuracy. The second fused audio feature is predicted by a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to characterize the identity of the target singing object. This makes it easy to determine the identity of the target singer, effectively solving the problem of difficulty in identifying the singer due to the increase in the number of singers in the metaverse, and improving the accuracy of identifying the singer.

[0090] In step S101 of some embodiments, a web crawler can be programmed to set up a data source and then crawl data in a targeted manner to obtain target audio data of a target singer. The data source can be various types of online platforms, social media, or certain specific audio databases, and the target audio data can be music materials, speech reports, chat conversations, etc. of the target singer. Other methods can also be used to obtain the target audio data, but are not limited to these.

[0091] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0092] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S202:

[0093] Step S201, performing spectrum calculation on the target audio data by short-time Fourier transform to obtain a target spectrogram;

[0094] Step S202 : filtering the target spectrogram using a preset Mel-cepstrum filter to obtain Mel-cepstrum features.

[0095] In step S201 of some embodiments, a short-time Fourier transform (SFT) is used to calculate the sound spectrum of the target audio data to obtain a target spectrogram. Specifically, the target audio data is subjected to signal framing and windowing to obtain multiple audio frames. A short-time Fourier transform is then performed on each audio frame to convert the time domain features of the audio frame into frequency domain features. Finally, the frequency domain features of each frame are stacked in the time dimension to obtain the target spectrogram.

[0096] In step S202 of some embodiments, the target spectrogram is filtered using a 64-dimensional Mel-cepstrum filter bank. First, a logarithmic operation is performed on the target spectrogram to obtain a target log spectrum. The target log spectrum is then inverse Fourier transformed to obtain a target Mel-cepstrum. Furthermore, feature extraction is performed on the target Mel-cepstrum to obtain Mel-cepstrum features, where the feature dimension of the Mel-cepstrum features is T*64, where T is the number of frames of the target audio data.

[0097] Through the above steps S201 to S202, the target audio data can be easily converted into spectral features, and the spectral features are filtered to obtain Mel-frequency cepstrum features, so that the identity of the target singer can be identified through the spectral features, thereby improving the recognition accuracy.

[0098] See also Figure 3In some embodiments, step S103 may include but is not limited to steps S301 to S305:

[0099] Step S301, segmenting the target audio data using a preset audio segmentation model to obtain multiple target audio segments;

[0100] Step S302: extract features from each target audio segment using an audio segmentation model to obtain a target audio latent vector.

[0101] Step S303: perform feature calculation on all target audio latent vectors to obtain a target audio mean vector and a target audio variance vector;

[0102] Step S304: concatenate the target audio variance vector and the target audio mean vector to obtain a target timbre latent vector;

[0103] Step S305 , predicting the target timbre latent vector using a preset function to obtain the target timbre feature.

[0104] In step S301 of some embodiments, a preset audio segmentation model may be constructed based on an X-Vector network structure, and the audio segmentation model may include at least one DNN (Deep Neural Networks) layer. The target audio data is segmented using the audio segmentation model and preset segment lengths, dividing the target audio data into multiple target audio segments of equal length based on different audio time periods.

[0105] In step S302 of some embodiments, feature extraction is performed on each target audio segment through the DNN layer of the audio segmentation model to obtain audio feature information in the target audio segment and obtain a target audio latent vector corresponding to each target audio segment.

[0106] In step S303 of some embodiments, when performing feature calculation on all target audio latent vectors, firstly, the mean of all target audio latent vectors is calculated to obtain the target audio mean vector; then, the difference between each target audio latent vector and the target audio mean vector is calculated to obtain the target audio variance vector corresponding to each target audio latent vector.

[0107] In step S304 of some embodiments, vector concatenation is performed on the target audio variance vector and the target audio mean vector to obtain a target timbre latent vector.

[0108] In step S305 of some embodiments, the preset function is a softmax function, which can create a probability distribution of the target timbre latent vector on each preset reference timbre feature through the softmax function. The probability distribution reflects the possibility that the target timbre latent vector belongs to each preset reference timbre feature, and the reference timbre feature with the highest probability of the target timbre latent vector belonging to is selected as the target timbre feature. The feature dimension of the target timbre feature can be 512. This target timbre feature can more accurately reflect the timbre characteristics of the target singer, which include information such as the pitch, frequency, volume, and sound quality of the target singer when speaking.

[0109] See also Figure 4 In some embodiments, step S104 may include but is not limited to steps S401 to S405:

[0110] Step S401: input the target audio data into a preset feature extraction model, wherein the feature extraction model includes a first convolutional layer, a second convolutional layer, and a third convolutional layer;

[0111] Step S402: extracting music mid-level features from the target audio data through a first convolutional layer to obtain first music mid-level features;

[0112] Step S403, extracting music mid-level features from the target audio data through the second convolutional layer to obtain second music mid-level features;

[0113] Step S404: extracting music mid-level features from the target audio data through the third convolutional layer to obtain third music mid-level features;

[0114] Step S405 , performing concatenation processing on the first music middle-level feature, the second music middle-level feature, and the third music middle-level feature to obtain music characteristic features.

[0115] In step S401 of some embodiments, the target audio data is input into a preset feature extraction model, wherein the feature extraction model can be constructed based on the Inception V3 network structure, and the feature extraction model includes an input layer, an intermediate layer, a first convolutional layer, a second convolutional layer, and a third convolutional layer, wherein the input layer and the intermediate layer can also be convolutional structures.

[0116] In step S402 of some embodiments, music mid-level features are extracted from the target audio data through the first convolutional layer to capture the music characteristic information in the target audio data and obtain the first music mid-level features, the feature dimension of which is 128 dimensions.

[0117] In step S403 of some embodiments, the target audio data is subjected to music mid-level feature extraction through the second convolutional layer to capture the music characteristic information in the target audio data and obtain the second music mid-level feature, wherein the feature dimension of the second music mid-level feature is 128 dimensions.

[0118] In step S404 of some embodiments, the target audio data is subjected to music mid-level feature extraction through the third convolutional layer to capture the music characteristic information in the target audio data and obtain the third music mid-level feature, wherein the feature dimension of the third music mid-level feature is 256 dimensions.

[0119] In step S405 of some embodiments, vector concatenation is performed on the first music mid-level feature, the second music mid-level feature, and the third music mid-level feature in vector form to obtain a music characteristic feature, and the feature dimension of the music characteristic feature is 512 dimensions.

[0120] Through the above-mentioned steps S401 to S405, the target audio data can be better transferred for learning, the mid-level feature information of the music in the target audio data can be extracted, and the music characteristic features can be obtained. Through the music characteristic features, the music characteristic information in the target audio data can be more comprehensively reflected, so that the music characteristic content can be integrated into the process of singing object recognition, thereby improving the recognition accuracy of the singing object.

[0121] It should be explained that features in the field of music can be roughly divided into three levels. Among them, low-level music features have clearly defined concepts, including music beats, music chords, etc. High-level music features are not clearly defined and are subjective concepts. High-level music features include the singer's emotions, music genres, music similarities, etc. High-level music features can only be defined by considering all aspects of music. Mid-level music features refer to music features that are between low-level music features and high-level music features. Mid-level music features include music speed, rhythm stability, music melody, music rhythm complexity, music perceptibility, etc. Mid-level music features can be used to improve music emotion recognition, music retrieval, and music classification.

[0122] In step S105 of some embodiments, the first fusion processing of the Mel-cepstral feature and the target timbre feature includes two stages. The first stage is to perform splicing processing on the Mel-cepstral feature and the target timbre feature. Specifically, it can be to perform vector addition on the Mel-cepstral feature and the target timbre feature in vector form to obtain a preliminary fusion feature. The second stage is to perform multiple convolution processing on the initial fusion feature to obtain a first fusion audio feature. Specifically, in the singing object recognition method of the embodiment of the present application, the process of performing convolution processing on the initial fusion feature includes four convolution processings, namely, performing feature extraction on the initial fusion feature through the first convolution network to obtain a first fusion feature, then performing feature extraction on the first fusion feature through the second convolution network to obtain a second fusion feature, and then performing feature extraction on the second fusion feature through the third convolution network to obtain a third fusion feature. Finally, performing feature extraction on the third fusion feature through the fourth convolution network to obtain a first fusion audio feature.

[0123] It should be noted that the processing process of each of the above-mentioned convolutional networks includes stages such as convolution operations, pooling operations, and activation operations. This method can achieve dimensionality reduction and feature extraction of the initial fused features, thereby obtaining a first fused audio feature that meets the requirements. The first convolutional network, the second convolutional network, the third convolutional network, and the fourth convolutional network can be the same network structure. For example, each convolutional network includes a convolution layer, a pooling layer, and an activation layer. The convolution kernel size of the convolution layer can be 3×3 or 1×1, and the number of channels can be 128. The activation function of the activation layer can be a Relu function, a Sigmoid function, etc., without limitation.

[0124] In step S106 of some embodiments, when performing the second fusion processing on the first fused audio feature and the music characteristic feature, vector addition can be performed on the first fused audio feature and the music characteristic feature in vector form to achieve the merging of the first fused audio feature and the music characteristic feature, thereby obtaining the second fused audio feature.

[0125] See also Figure 5 In some embodiments, the person recognition model includes a GRU layer and a fully connected layer. Step S107 may include but is not limited to steps S501 to S502:

[0126] Step S501, performing time series feature extraction on the second fused audio feature through the GRU layer to obtain a fused time series feature vector;

[0127] Step S502: perform label prediction processing on the fused time series feature vector through a fully connected layer to obtain a target identity label of the target singing object.

[0128] In step S501 of some embodiments, when the time series feature of the second fused audio feature is extracted through the GRU layer of the character recognition model, the time series dimension feature of the second fused audio feature can be merged with the audio state feature to output a fused time series feature vector. The fused time series feature vector integrates the time series relationship of the spectral features, timbre features and music characteristics of the target audio data, and can better support the subsequent identity recognition of the target singing object, thereby improving the recognition accuracy.

[0129] In step S502 of some embodiments, the label probability of the fused time series feature vector is calculated by the classification function of the fully connected layer to obtain a label probability vector corresponding to each preset identity label, and the preset identity label corresponding to the label probability vector with the largest value is selected as the target identity label of the target singing object, so as to determine the identity of the target singing object based on the target identity label. The target identity label can not only characterize who the target singing object is, but also characterize whether the target singing object is a virtual character or a real person.

[0130] See also Figure 6 In some embodiments, step S501 includes but is not limited to steps S601 to S602:

[0131] Step S601, performing time series feature extraction on the second fused audio feature through the first gated recurrent unit of the GRU layer to obtain an initial time series feature vector;

[0132] Step S602 , performing time series feature extraction on the initial time series feature vector through the second gated recurrent unit of the GRU layer to obtain a fused time series feature vector.

[0133] In step S601 of some embodiments, the time series feature of the second fused audio feature is extracted through the first gated recurrent unit of the GRU layer to capture the time series dimension feature of the second fused audio feature, and then the time series dimension feature of the second fused audio feature is fused with the first audio state feature in the first gated recurrent unit to obtain an initial time series feature vector.

[0134] In step S602 of some embodiments, the initial timing feature vector is subjected to timing feature extraction through the second gated recurrent unit of the GRU layer to capture the time series dimension features of the initial timing feature vector, and then the time series dimension features of the initial timing feature vector are fused with the second audio state features in the second gated recurrent unit to obtain a fused timing feature vector.

[0135] It should be noted that the structures of the first gated recurrent unit and the second gated recurrent unit can be exactly the same or different, without limitation. For example, in some specific embodiments, the first gated recurrent unit and the second gated recurrent unit both have a 32-unit structure, and the random activation parameter is set to 0.5.

[0136] In the above steps S601 to S602, two gated recurrent units are used to extract the timing features of the second fused audio features, which can better capture the timing feature information of the target audio data in terms of timbre features, musical characteristics and spectral features, thereby providing better data support for the subsequent prediction process of the singing object, thereby improving the recognition accuracy of the singing object.

[0137] See also Figure 7 In some embodiments, step S502 may include but is not limited to steps S701 to S703:

[0138] Step S701, performing label probability calculation on the fused time series feature vector using the classification function of the fully connected layer and the preset identity label to obtain a label probability vector corresponding to each preset identity label;

[0139] Step S702: Select the preset identity label corresponding to the label probability vector with the largest value to obtain a candidate identity label;

[0140] Step S703: Obtain a target identity tag according to the candidate identity tags.

[0141] In step S701 of some embodiments, the classification function may be a probabilistic function such as a softmax function, and the preset identity tags may be extracted from different data sources, for example, basic information about various people, including identity information, personal information, and related audio and video data, may be obtained from online media and social platforms. Taking the softmax function as an example, the softmax function may be used to create a probability distribution of the fused time series feature vector on each preset identity tag. The probability distribution reflects the probability that the fused time series feature vector belongs to each preset identity tag, thereby obtaining a tag probability vector corresponding to each preset identity tag.

[0142] In steps S702 and S703 of some embodiments, the size of the label probability vector can intuitively reflect the possibility that the fused time series feature vector belongs to each preset identity tag. The larger the value of the label probability vector, the higher the degree of matching of the preset identity tag corresponding to the fused time series feature vector, indicating that the fused time series feature vector is more likely to come from the person corresponding to this preset identity tag. Therefore, the preset identity tag corresponding to the label probability vector with the largest value is selected to obtain one or more candidate identity tags, and then an identity tag is selected from the candidate identity tags as the target identity tag, so that the identity of the target singing object is represented by the target identity tag.

[0143] The above steps S701 to S703 can conveniently quantify the possibility that the fused time series feature vector belongs to each preset identity label through the classification function to obtain a label probability vector, and then select the most appropriate preset identity label as the target identity label according to the size of the label probability vector, thereby confirming the identity of the target singing object according to the target identity label, thereby improving the recognition accuracy of the singing object.

[0144] The singing object recognition method of the embodiment of the present application obtains the target audio data of the target singing object; further, extracts the spectrum feature of the target audio data to obtain the Mel-frequency cepstrum feature; extracts the timbre feature of the target audio data to obtain the target timbre feature; extracts the middle-level feature of the target audio data to obtain the music characteristic feature, which can more conveniently determine the timbre feature information, music characteristic information and other contents of the target singing object. Further, the Mel-frequency cepstrum feature and the target timbre feature are subjected to a first fusion process to obtain a first fused audio feature. In this way, the timbre feature information can be incorporated into the subsequent recognition process to eliminate the influence of music accompaniment or background music on the recognition effect; further, the first fused audio feature and the music characteristic feature are subjected to a second fusion process to obtain a second fused audio feature, so that the music characteristic information can be incorporated into the subsequent recognition process, thereby increasing the recognition of music types and music styles and improving the recognition accuracy. Finally, the second fused audio feature is predicted by a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to characterize the identity of the target singing object. This makes it easy to determine the identity of the target singer, effectively solving the problem of difficulty in identifying the singer due to the increase in the number of singers in the metaverse, and improving the accuracy of identifying the singer.

[0145] See also Figure 8 The present application also provides a device for identifying a singing object, which can implement the above singing object identification method. The device includes:

[0146] The data acquisition module 801 is used to acquire target audio data of a target singing object;

[0147] The spectrum feature extraction module 802 is used to extract spectrum features of the target audio data to obtain Mel-cepstrum features;

[0148] The timbre feature extraction module 803 is used to extract the timbre features of the target audio data to obtain the target timbre features;

[0149] A mid-level feature extraction module 804 is used to extract mid-level features from the target audio data to obtain music characteristic features;

[0150] A first fusion module 805 is configured to perform a first fusion process on the Mel-cepstrum feature and the target timbre feature to obtain a first fused audio feature;

[0151] A second fusion module 806 is configured to perform a second fusion process on the first fused audio feature and the music characteristic feature to obtain a second fused audio feature;

[0152] The prediction module 807 is used to perform prediction processing on the second fused audio feature through a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to represent the identity of the target singing object.

[0153] In some embodiments, the spectrum feature extraction module 802 includes:

[0154] A sound spectrum calculation unit, used to perform sound spectrum calculation on the target audio data through short-time Fourier transform to obtain a target spectrum graph;

[0155] The filtering unit is used to filter the target spectrum through a preset Mel-cepstrum filter to obtain Mel-cepstrum features.

[0156] In some embodiments, the timbre feature extraction module 803 includes:

[0157] A segmentation unit, configured to segment the target audio data using a preset audio segmentation model to obtain a plurality of target audio segments;

[0158] A feature extraction unit is used to extract features from each target audio segment using an audio segmentation model to obtain a target audio latent vector;

[0159] A feature calculation unit is used to perform feature calculation on all target audio latent vectors to obtain a target audio mean vector and a target audio variance vector;

[0160] A first concatenation unit is configured to concatenate the target audio variance vector and the target audio mean vector to obtain a target timbre latent vector;

[0161] The prediction unit is used to predict the target timbre latent vector through a preset function to obtain the target timbre feature.

[0162] In some embodiments, the mid-level feature extraction module 804 includes:

[0163] An input unit, configured to input target audio data into a preset feature extraction model, wherein the feature extraction model includes a first convolutional layer, a second convolutional layer, and a third convolutional layer;

[0164] A first extraction unit is configured to extract music mid-level features from the target audio data through a first convolutional layer to obtain first music mid-level features;

[0165] A first extraction unit is configured to extract music mid-level features from the target audio data through a second convolutional layer to obtain second music mid-level features;

[0166] A first extraction unit is configured to extract music mid-level features from the target audio data through a third convolutional layer to obtain third music mid-level features;

[0167] The second splicing unit is used to splice the first music middle-level feature, the second music middle-level feature, and the third music middle-level feature to obtain music characteristic features.

[0168] In some embodiments, the person recognition model includes a GRU layer and a fully connected layer, and the prediction module 807 includes:

[0169] A time series feature extraction unit, configured to perform time series feature extraction on the second fused audio feature through a GRU layer to obtain a fused time series feature vector;

[0170] The label prediction unit is used to perform label prediction processing on the fused time series feature vector through the fully connected layer to obtain the target identity label of the target singing object.

[0171] In some embodiments, the temporal feature extraction unit includes:

[0172] A first time series feature extraction subunit is used to extract time series features from the second fused audio features through the first gated recurrent unit of the GRU layer to obtain an initial time series feature vector;

[0173] The second time series feature extraction subunit is used to extract time series features from the initial time series feature vector through the second gated recurrent unit of the GRU layer to obtain a fused time series feature vector.

[0174] In some embodiments, the label prediction unit includes:

[0175] The probability calculation subunit is used to calculate the label probability of the fused time series feature vector through the classification function of the fully connected layer and the preset identity label, and obtain the label probability vector corresponding to each preset identity label;

[0176] The label selection subunit is used to select the preset identity label corresponding to the label probability vector with the largest value to obtain the candidate identity label;

[0177] The label determination subunit is used to obtain the target identity label based on the candidate identity labels.

[0178] The specific implementation of the singing object identification device is basically the same as the specific embodiment of the above-mentioned singing object identification method, and will not be repeated here.

[0179] The present application also provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the above-described method for identifying a singing object is implemented. The electronic device may be any intelligent terminal, such as a tablet computer or an in-vehicle computer.

[0180] See also Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0181] The processor 901 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0182] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program codes are stored in the memory 902 and are called by the processor 901 to execute the singing object recognition method of the embodiments of this application.

[0183] Input / output interface 903, used to implement information input and output;

[0184] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0185] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0186] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0187] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned singing object recognition method.

[0188] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0189] The embodiment of the present application provides a singing object recognition method, singing object recognition device, electronic device and storage medium, which obtains the target audio data of the target singing object; further, extracts the spectrum feature of the target audio data to obtain the Mel-frequency cepstrum feature; extracts the timbre feature of the target audio data to obtain the target timbre feature; extracts the middle-level feature of the target audio data to obtain the music characteristic feature, which can more conveniently determine the timbre feature information, music characteristic information and other contents of the target singing object. Further, the Mel-frequency cepstrum feature and the target timbre feature are subjected to a first fusion processing to obtain a first fused audio feature. This method can enable the timbre feature information to be incorporated into the subsequent recognition process, eliminating the influence of music accompaniment or background music on the recognition effect; further, the first fused audio feature and the music characteristic feature are subjected to a second fusion processing to obtain a second fused audio feature, so that the music characteristic information can be incorporated into the subsequent recognition process, increasing the recognition of music type and music style and improving the recognition accuracy. Finally, the second fused audio feature is predicted by a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to characterize the identity of the target singing object. This makes it easy to determine the identity of the target singer, effectively solving the problem of difficulty in identifying the singer due to the increase in the number of singers in the metaverse, and improving the accuracy of identifying the singer.

[0190] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0191] It will be understood by those skilled in the art that Figure 1-7 The technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figures, or a combination of certain steps, or different steps.

[0192] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0193] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0194] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0195] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0197] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0198] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0199] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0200] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A singing object recognition method, characterized in that: The method comprises: Obtain target audio data of a target singing object; Extracting spectral features of the target audio data to obtain Mel-frequency cepstrum features; Extracting timbre features from the target audio data to obtain target timbre features; Performing mid-level feature extraction on the target audio data to obtain music characteristic features; Performing a first fusion process on the Mel-cepstrum feature and the target timbre feature to obtain a first fused audio feature; Performing a second fusion process on the first fused audio feature and the music characteristic feature to obtain a second fused audio feature; Predicting the second fused audio feature using a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to represent the identity of the target singing object; The step of extracting the timbre feature of the target audio data to obtain the target timbre feature includes: Segmenting the target audio data using a preset audio segmentation model to obtain a plurality of target audio segments of the same length; extracting features from each of the target audio segments using the audio segmentation model to obtain a target audio latent vector; Performing mean calculation on all the target audio latent vectors to obtain a target audio mean vector, and performing difference calculation on each target audio latent vector and the target audio mean vector to obtain a target audio variance vector; The target audio variance vector and the target audio mean vector are concatenated to obtain a target timbre latent vector; and the target timbre latent vector is predicted using a softmax function to obtain the target timbre feature.

2. The method for identifying a singing object according to claim 1, wherein: The step of extracting spectral features from the target audio data to obtain Mel-cepstrum features includes: Performing sound spectrum calculation on the target audio data by short-time Fourier transform to obtain a target spectrogram; The target spectrum is filtered using a preset Mel-cepstrum filter to obtain the Mel-cepstrum feature.

3. The method for identifying a singing object according to claim 1, wherein: The step of extracting mid-level features from the target audio data to obtain music characteristic features includes: Inputting the target audio data into a preset feature extraction model, wherein the feature extraction model includes a first convolutional layer, a second convolutional layer, and a third convolutional layer; Extracting music mid-level features from the target audio data through the first convolutional layer to obtain first music mid-level features; Performing music mid-level feature extraction on the target audio data through the second convolutional layer to obtain second music mid-level features; Performing music mid-level feature extraction on the target audio data through the third convolutional layer to obtain a third music mid-level feature; The first music middle-level feature, the second music middle-level feature, and the third music middle-level feature are concatenated to obtain the music characteristic feature.

4. The method for identifying a singing object according to any one of claims 1 to 3, wherein: The character recognition model includes a GRU layer and a fully connected layer. The step of performing prediction processing on the second fused audio feature using the preset character recognition model to obtain a target identity label of the target singing object includes: Performing time series feature extraction on the second fused audio feature through the GRU layer to obtain a fused time series feature vector; The fused time series feature vector is subjected to label prediction processing through the fully connected layer to obtain a target identity label of the target singing object.

5. The method for identifying a singing object according to claim 4, wherein: The step of performing time series feature extraction on the second fused audio feature through the GRU layer to obtain a fused time series feature vector includes: Performing time series feature extraction on the second fused audio feature through the first gated recurrent unit of the GRU layer to obtain an initial time series feature vector; The initial time series feature vector is subjected to time series feature extraction by the second gated recurrent unit of the GRU layer to obtain the fused time series feature vector.

6. The method for identifying a singing object according to claim 4, wherein: The step of performing label prediction processing on the fused time series feature vector through the fully connected layer to obtain the target identity label of the target singing object includes: Performing label probability calculation on the fused time series feature vector using the classification function of the fully connected layer and the preset identity label to obtain a label probability vector corresponding to each preset identity label; Select the preset identity label corresponding to the label probability vector with the largest value to obtain the candidate identity label; The target identity tag is obtained according to the candidate identity tag.

7. A singing object recognition device, characterized in that: The device comprises: A data acquisition module is used to acquire target audio data of a target singing object; A spectrum feature extraction module is used to extract spectrum features of the target audio data to obtain Mel-cepstrum features; A timbre feature extraction module is used to extract timbre features from the target audio data to obtain target timbre features; A mid-level feature extraction module is used to extract mid-level features from the target audio data to obtain music characteristic features; A first fusion module is used to perform a first fusion process on the Mel-cepstrum feature and the target timbre feature to obtain a first fused audio feature; A second fusion module is configured to perform a second fusion process on the first fused audio feature and the music characteristic feature to obtain a second fused audio feature; A prediction module, configured to perform prediction processing on the second fused audio feature using a preset character recognition model to obtain a target identity tag of the target singing object, wherein the target identity tag is used to represent the identity of the target singing object; The step of extracting the timbre feature of the target audio data to obtain the target timbre feature includes: Segmenting the target audio data using a preset audio segmentation model to obtain a plurality of target audio segments of the same length; extracting features from each of the target audio segments using the audio segmentation model to obtain a target audio latent vector; Performing mean calculation on all the target audio latent vectors to obtain a target audio mean vector, and performing difference calculation on each target audio latent vector and the target audio mean vector to obtain a target audio variance vector; The target audio variance vector and the target audio mean vector are concatenated to obtain a target timbre latent vector; and the target timbre latent vector is predicted using a softmax function to obtain the target timbre feature.

8. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the singing object recognition method as described in any one of claims 1 to 6 are realized.

9. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the singing object recognition method described in any one of claims 1 to 6.

Images