Singing intonation acquisition method and device, storage medium and electronic equipment

Through the deep learning model, the singing audio data is processed, the lead and decorative notes are integrated, and the pitch sequence is generated, which solves the inconvenience and subjective influence of singing pitch judgment in the prior art, and achieves higher accuracy and interpretability. In electronic equipment, the efficiency of vocal education has been improved.

CN119028298APending Publication Date: 2024-11-26SUNLAND INFORMATION TECH (SHANGHAI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310610561.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the prior art, the judgment of singing pitch depends on specialists, the process is not convenient and fast enough and is easily subjectively affected, and the existing solutions are based solely on the pitch and energy characteristics, resulting in insufficient accuracy and interpretability of pitch.

Method used

Deep learning model, including singing transcription model and singing skills recognition model, process singing audio data, integrate lead notes and decorative notes, generate the first note sequence, and obtain singing pitch based on standard note sequences and integrated note sequences. Decorative notes are used as a basis to eliminate octave deviations and provide greater accuracy and interpretability.

Benefits of technology

It realizes the automatic acquisition of singing pitch, avoids human subjective errors, improves the accuracy and interpretability of pitch, is convenient and fast, is suitable for electronic devices, and enhances the efficiency of vocal education.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028298A_ABST
    Figure CN119028298A_ABST
Patent Text Reader

Abstract

The invention provides a singing intonation acquisition method and device, a storage medium and electronic equipment. The singing intonation acquisition method comprises the following steps: acquiring singing audio data; processing the singing audio data by using a deep learning model to integrate main sound notes and decorative sound notes of the singing audio data so as to obtain a first note sequence, the decorative sound notes being related to singing skills; and obtaining singing intonation according to a standard note sequence and the first note sequence. The singing intonation obtaining method can accurately obtain the singing intonation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the technical field of sound processing, and relates to an audio data processing method, and in particular to a singing pitch acquisition method, device, storage medium and electronic device. Background Art

[0002] Pitch refers to the accuracy of pitch when singing. In music, each note has a fixed pitch. If you deviate from this pitch when singing, you will have a pitch problem. Pitch problems may manifest as high or low pitch, inaccurate notes, inharmonious intervals, etc. In vocal education, pitch training is a very important aspect. In the prior art, vocal teachers and other specialists usually judge whether the singer's pitch is accurate. However, this method is not convenient and fast enough, and the judgment result is easily affected by the subjective judgment of the specialists. Summary of the invention

[0003] The embodiments of the present application provide a singing pitch acquisition method, device, storage medium and electronic device, which are used to solve the above-mentioned problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a method for acquiring singing pitch. The method comprises: acquiring singing audio data; processing the singing audio data using a deep learning model to integrate the main notes and the ornament notes of the singing audio data to obtain a first note sequence, wherein the ornament notes are related to singing skills; and acquiring singing pitch according to a standard note sequence and the first note sequence.

[0005] In an implementation of the first aspect, the deep learning model includes a singing transcription model and a singing skill recognition model, and the deep learning model is used to process the singing audio data to integrate the tonic notes and the grace notes of the singing audio data to obtain an integrated note sequence, including: using the singing transcription model to process the singing audio data to obtain a second note sequence; using the singing skill recognition model to process the second note sequence to obtain the positions of the grace notes and the tonic notes in the second note sequence; integrating the tonic notes and the grace notes according to their positions in the second note sequence to obtain the first note sequence.

[0006] In an implementation of the first aspect, the tonic note and the ornament note are integrated according to their positions in the second note sequence to obtain the first note sequence, including: merging the tonic note and its corresponding ornament note into an integrated note according to their positions in the second note sequence, the pitch of the integrated note being the pitch of the tonic note, the start time of the integrated note being the minimum start time of the tonic note and its corresponding ornament note, and the end time of the integrated note being the maximum end time of the tonic note and its corresponding ornament note, and each of the integrated notes constitutes the first note sequence.

[0007] In an implementation of the first aspect, the training method of the singing transcription model and the singing skill recognition model includes: obtaining training audio data and its note information labeling results and singing skill labeling results, the singing skill labeling results including the positions of the tonic note and the ornament note in a third note sequence, and the third note sequence is the note sequence of the training audio data; using the training audio data and its note information labeling results to train the singing transcription model; using the third note sequence and the singing skill labeling results to train the singing skill recognition model.

[0008] In an implementation of the first aspect, before the singing audio data is processed by a deep learning model, the singing pitch acquisition method further includes: preprocessing the singing audio data so that the data structure and data format of the singing audio data are consistent with the training audio data of the deep learning model.

[0009] In an implementation of the first aspect, before acquiring the singing pitch according to the standard note sequence and the first note sequence, the singing pitch acquisition method further includes: aligning and matching the standard note sequence and the first note sequence.

[0010] In an implementation of the first aspect, before acquiring the singing pitch according to the standard note sequence and the first note sequence, the singing pitch acquisition method also includes: acquiring the overall pitch deviation of the standard note sequence and the first note sequence; judging whether there is an octave deviation between the standard note sequence and the first note sequence according to the overall pitch deviation; if there is an octave deviation, eliminating the octave deviation between the standard note sequence and the first note sequence in the pitch dimension.

[0011] In a second aspect, an embodiment of the present application provides a singing pitch acquisition device, which includes: an audio data acquisition module for acquiring singing audio data; a note integration module for processing the singing audio data using a deep learning model to integrate the main notes and ornamental notes of the singing audio data to obtain a first note sequence, wherein the ornamental notes are generated by singing techniques; a singing pitch acquisition module for acquiring singing pitch based on a standard note sequence and the first note sequence.

[0012] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for acquiring singing pitch as described in any one of the first aspects of the present application is implemented.

[0013] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory storing a computer program; and a processor communicatively connected to the memory, for executing the singing pitch acquisition method described in any one of the first aspects of the present application when the computer program is called.

[0014] The singing pitch acquisition method provided in the embodiment of the present application can acquire the singing pitch according to the singing audio data. The process can be automatically completed by an electronic device. With the singing pitch acquisition method, the singer can know the singing pitch status without the need for a vocal teacher or other specialized personnel. Therefore, the singing pitch acquisition method provided in the embodiment of the present application is convenient and fast, and will not cause errors due to human subjective factors.

[0015] In addition, the singing pitch acquisition method provided in the embodiment of the present application uses the ornamental notes generated by singing skills as one of the bases for acquiring singing pitch. Compared with the scheme of acquiring pitch based only on fundamental tone and energy characteristics, the singing pitch acquired in the embodiment of the present application has higher accuracy and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Shown is a schematic diagram of an application scenario of the singing pitch acquisition method described in an embodiment of the present application.

[0017] Figure 2 Shown is a flow chart of the singing pitch acquisition method described in an embodiment of the present application.

[0018] Figure 3A Shown is a flowchart of obtaining a first note sequence in an embodiment of the present application.

[0019] Figure 3B Shown is a structural schematic diagram of the singing transcription model in the embodiment of the present application.

[0020] Figure 3CShown is a structural schematic diagram of the singing skill recognition model in an embodiment of the present application.

[0021] Figure 4 Shown is a flow chart of the training method of the relevant model in the embodiment of the present application.

[0022] Figure 5 Shown is a flow chart of obtaining a singing pitch evaluation structure in an embodiment of the present application.

[0023] Figure 6 Shown is a structural schematic diagram of a singing pitch acquisition device in an embodiment of the present application.

[0024] Figure 7 Shown is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0025] Component number description

[0026] 1 Electronic equipment

[0027] 11 Sound Collection Strong Man

[0028] 111 Microphone

[0029] 112 Sampling Circuit

[0030] 12 Memory

[0031] 13 General Purpose Processor

[0032] 131 CPU

[0033] 132 Neural Network Processor

[0034] 14 Display

[0035] 6 Singing pitch acquisition device

[0036] 61 Audio data acquisition module

[0037] 62 Note Integration Module

[0038] 63 Singing Pitch Acquisition Module

[0039] 7 Electronic devices

[0040] 71 Memory

[0041] 72 Processors

[0042] 73 Display

[0043] Steps S21 to S23

[0044] Steps S31 to S33

[0045] Steps S41 to S43

[0046] Steps S51 to S54 DETAILED DESCRIPTION

[0047] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0048] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0049] In vocal education, intonation practice is a very important aspect. Good intonation is a prerequisite for singing well. In the traditional vocal education process, vocal teachers and other specialists usually judge whether the singer's intonation is accurate. In recent years, with the development of computer music technology and the rise of online music education, some schemes will use related systems and applications to obtain the singer's intonation. These schemes usually perform note recognition and segmentation based on the fundamental timeline and energy during the intonation acquisition process. However, factors such as the singer's pronunciation, singing skills, and the singer's own singing level that causes singing instability will cause the singer's pitch and energy characteristics to be random and unstable. Therefore, even if the accuracy of the fundamental and energy timeline extraction algorithm is high enough, the note sequence obtained by the note recognition and segmentation algorithm based on this will have a large deviation from the actual expression of the singer. The problems caused by this include but are not limited to: the pitch is correct but the fundamental tone and energy characteristics are different in the two cases where the lyrics are sung correctly or incorrectly, which will lead to different recognized notes, missing notes sung with breathy or bubbly sounds, inaccurate pitch calculations due to singing techniques such as glissando or vibrato, and incorrect note segmentation due to unstable volume. These problems further affect the accuracy of the acquired pitch. In addition, most of these technical solutions only provide the final pitch evaluation level or score, but lack the interpretability of the evaluation results.

[0050] At least for the above-mentioned problems, an embodiment of the present application provides a method for obtaining singing pitch. The method for obtaining singing pitch can obtain singing pitch based on singing audio data. The above process can be automatically completed by an electronic device, and basically no human participation is required. With the method for obtaining singing pitch, the singer can know his / her pitch status without the need for specialized personnel such as vocal teachers. Therefore, the method for obtaining singing pitch provided by the embodiment of the present application is convenient and fast, and will not cause errors due to human subjective factors. In addition, the method for obtaining singing pitch provided by the embodiment of the present application uses the ornamental notes generated by singing skills as one of the bases for obtaining singing pitch. Compared with the scheme of obtaining pitch based only on fundamental tone and energy characteristics, the singing pitch obtained by the embodiment of the present application has higher accuracy and interpretability.

[0051] The singing pitch acquisition method provided in the embodiment of the present application can be applied to electronic devices. The electronic devices can include mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) and other terminal devices, and can also be applied to databases, servers and terminal artificial intelligence-based service response systems. The embodiment of the present application does not impose any restrictions on the specific types of electronic devices.

[0052] The following will describe in detail the principles and implementation methods of a method for acquiring singing pitch according to an embodiment of the present application in conjunction with the accompanying drawings, so that those skilled in the art can understand the method for acquiring singing pitch according to an embodiment of the present application without creative work.

[0053] Figure 1 The structure diagram of an electronic device 1 used in the singing pitch acquisition method provided in the embodiment of the present application is shown. Figure 1 As shown, the electronic device 1 includes an audio acquisition device 11 , a memory 12 , at least one general processor 13 and a display 14 .

[0054] The audio acquisition device 11 may include a microphone 111, a sampling circuit 112 and other structures for collecting the singing audio of the singer. The microphone 111 may be arranged inside the electronic device 1 to capture the singing voice of the singer and convert it into an electrical signal. The sampling circuit 112 is used to convert the electrical signal obtained by converting the singing voice into a digital signal. In some possible implementations, the sampling circuit 112 may include an analog-to-digital converter (ADC).

[0055] The memory 12 may include a volatile memory, such as a random access memory (RAM) and a cache. The memory 12 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD). The memory 12 may be used to store data such as singing audio collected by the audio collection device 11. The memory 12 may also be used to store program instructions for the processor to call and execute at least a portion of the singing pitch acquisition method provided in the embodiment of the present application.

[0056] The general processor 13 can be any type of device capable of processing electronic instructions. In the embodiment of the present application, the electronic device 1 may include one or more general processors 13, such as one or two of a central processing unit (CPU) 131 and a neural network processor (NPU) 132. In addition, it may also include one or more of a graphics processor (GPU), a microprocessor, a microcontroller, a main processor, a controller, and an ASIC (Application Specific Integrated Circuit). The general processor 13 is configured to execute various types of digital storage instructions, such as software or firmware programs stored in the memory 15, which enables the electronic device 1 to provide a variety of services. For example, the processor 11 is capable of executing a program or processing data to execute at least a portion of the singing pitch acquisition method provided in the embodiment of the present application.

[0057] The main function of the central processing unit 131 is to parse computer instructions and process data in computer software, realize overall control of the electronic device 1, and control and allocate all hardware resources of the electronic device 1 (such as storage resources, communication resources, I / O interfaces, etc.).

[0058] Neural network processor is a general term for new processors based on neural network algorithms and acceleration. It is specially designed for artificial intelligence and is used to accelerate neural network calculations and solve the problem of low efficiency of traditional chips in neural network calculations.

[0059] It should be noted that the name of the neural network processor does not constitute a limitation of the present application. For example, in other application scenarios, the neural network processor can also be transformed and replaced by other processors with similar functions, such as a tensor processor (Tensor Processing Unit, TPU), a deep learning processor (Deep learning Processing Unit, DPU), and so on.

[0060] The display 14 may specifically include a display screen (display panel). In some implementations, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. In addition, the display 14 may also be a touch panel (touch screen, touch screen), which may include a display screen and a touch-sensitive surface. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the central processor 131 to determine the type of touch event, and then the central processor 131 provides a corresponding visual output on the display device according to the type of touch event.

[0061] Figure 2 The flowchart of the singing pitch acquisition method provided by the embodiment of the present application is shown. The singing pitch acquisition method can be applied to Figure 1 The general purpose processor 13 in the electronic device 1 is shown. Figure 2 As shown, the singing pitch acquisition method provided in the embodiment of the present application includes the following steps S21 to S23.

[0062] S21, obtaining singing audio data. The singing audio data can be obtained by, for example, collecting the singing voice of a singer through the audio collection device 11.

[0063] S22, using a deep learning model to process the singing audio data to integrate the main note and the grace note of the singing audio data to obtain a first note sequence, where the grace note is generated by singing skills. The main note is a note with prominence, independence and stability in the singing audio data. The grace note is a note used to modify the main note, which is usually shorter in duration and higher or lower in pitch than the main note. In the process of vocal singing, the singer's singing skills such as transposition and glissando will generate grace note.

[0064] S23, obtaining singing pitch according to the standard note sequence and the first note sequence. The standard note sequence refers to a note sequence corresponding to a singing score, and the singing score may be, for example, a MIDI score. In some possible implementations, the singing score may be expressed in the same format as the first note sequence to obtain the standard note sequence.

[0065] In one embodiment of the present application, the deep learning model includes a singing transcription model and a singing technique recognition model. Figure 3A , using a deep learning model to process the singing audio data to integrate the main notes and the ornamental notes of the singing audio data to obtain an integrated note sequence includes the following steps S31 to S33.

[0066] S31, using a singing transcription model to process the singing audio data to obtain a second note sequence, where the second note sequence is the singing melody pitch content corresponding to the singing audio data.

[0067] S32, using the singing skill recognition model to process the second note sequence to obtain the positions of the ornament notes and the main note in the second note sequence, and the positions can be represented by position indexes, for example.

[0068] S33, integrating the main note and the ornament note according to the positions of the ornament note and the main note in the second note sequence, so as to obtain a first note sequence.

[0069] Figure 3B Shown is a structural schematic diagram of a singing transcription model in the embodiment of the present application. Figure 3B As shown in the figure, the singing transcription model adopts a structure based on Convolutional Neural Network (CNN) and transformer. Its input is singing audio data, and its output is a sequence of notes that conforms to a specific format. CNN is used to perform data dimensionality reduction and audio feature extraction on singing audio data, and transformer is used to encode and decode the time series of audio features to obtain the correlation between audio features at different time points and obtain the target result.

[0070] Figure 3C The figure shows a structural diagram of the singing skill recognition model in the embodiment of the present application. Figure 3C As shown in the figure, the singing technique recognition model adopts a transformer-based structure, whose input is a note sequence that conforms to a specific format, and whose output is the position index of the grace note and the main note in the note sequence. The singing technique recognition model is used to identify the grace note and the corresponding main note generated by the singing technique, so as to facilitate the post-processing of the grace note.

[0071] In some possible implementations, the specific format may be {note1, note2, ..., noten}, where each note has a structure of {pitch, start_time, end_time}, n is a positive integer, pitch is the note pitch, start_time is the note start time, and end_time is the note end time.

[0072] In one embodiment of the present application, the tonic note and the ornament note are integrated according to their positions in the second note sequence to obtain the first note sequence, which includes: integrating the tonic note and its corresponding ornament note into integrated notes according to their positions in the second note sequence, and each integrated note constitutes the first note sequence. The pitch of the integrated note is the pitch of the tonic note, the start time of the integrated note is the minimum start time of the tonic note and its corresponding ornament note, and the end time of the integrated note is the maximum end time of the tonic note and its corresponding ornament note.

[0073] Figure 4 The flowchart of the training method of the singing transcription model and the singing skill recognition model in the embodiment of the present application is shown. Figure 4 As shown, the training method of the singing transcription model and the singing skill recognition model in the embodiment of the present application includes the following steps S41 to S43.

[0074] S41, obtaining training audio data and its note information labeling results and singing technique labeling results. The singing technique labeling results include the positions of the main note and the ornament note in the note sequence (third note sequence) of the training audio data. The training audio data includes but is not limited to audio data of open source data sets (such as opencpop), audio data directly recorded, or audio data from related websites.

[0075] In some possible implementations, the training audio data may be noise-free, a cappella, single-voice singing audio data. The training audio data may be monophonic audio data in wav format, whose audio sampling rate may be, for example, 16000 Hz or 44100 Hz, and whose bit depth may be, for example, 16 bits.

[0076] In some possible implementations, after the training audio data is acquired, noise reduction processing may be performed on the training audio data to obtain noise-free training audio data.

[0077] In some possible implementations, after the training audio data is acquired, vocal separation may be performed on the training audio data containing the accompaniment to extract the vocal data therein.

[0078] In some possible implementations, audio annotation tools such as praat can be used to annotate the training audio data to obtain its note data annotation results and singing technique annotation results. The audio data annotation results include pitch, start time and end time. In addition, in order to improve the annotation efficiency, in the embodiment of the present application, an open source transcription algorithm can be used to perform preliminary annotation on the training audio data, and annotation verification and modification can be performed based on the preliminary annotation.

[0079] In some possible implementations, based on the characteristics of human vocal singing, for situations where the pitch characteristics are vague due to breathy voice, unvoiced pronunciation, or the use of other singing techniques, the corresponding note information can be marked according to the actual singing expression, so that the singing transcription model can effectively identify such notes.

[0080] In some possible implementations, the training audio data may be cut into segments of shorter duration using an audio cutting algorithm. For example, the training audio data may be cut into segments of less than 15 seconds.

[0081] S42, using the training audio data and the note information labeling results to train the singing transcription model. For example, the singing transcription model can be trained by using methods such as backpropagation (BP), stochastic gradient descent (SGD), and adaptive moment estimation (Adam), but the present application is not limited thereto.

[0082] S43, using the third note sequence and the singing skill annotation result to train the singing skill recognition model. The third note sequence can be obtained by annotation. The singing skill recognition model can be trained by a method similar to the singing transcription model.

[0083] In one embodiment of the present application, the input of the deep learning model is singing audio data, and the output is a first note sequence. In the embodiment of the present application, the deep learning model is trained using the integrated annotation results of the training audio data, so that the deep learning model can directly process the input singing audio data to obtain the first note sequence, thereby reducing the steps of singing skill recognition. Among them, when annotating the training audio data to obtain the integrated annotation results, the pitch of the note at the grace note is annotated as the main pitch.

[0084] In one embodiment of the present application, before using a deep learning model to process the singing audio data, the singing pitch acquisition method may also include: preprocessing the singing audio data so that the data structure and data format of the singing audio data are consistent with the training audio data of the deep learning model.

[0085] In one embodiment of the present application, before using a deep learning model to process the singing audio data, the singing pitch acquisition method may also include: preprocessing the singing audio data to obtain clean pure human voice singing audio data.

[0086] In one embodiment of the present application, before obtaining the singing pitch according to the standard note sequence and the first note sequence, the singing pitch obtaining method may further include: aligning and matching the standard note sequence and the first note sequence. In this way, the pitch deviation of the notes at the corresponding positions can be accurately calculated, so that the singing pitch obtained is more accurate.

[0087] In some possible implementations, DTW can be used to achieve alignment matching of the standard note sequence and the first note sequence. Specifically, the standard note sequence and the first note sequence are used as input, the pitch cosine distance in the two note sequences is used as the edit distance to calculate the distance matrix of the two note sequences, and the Viterbi algorithm is used to calculate the optimal matching path, so as to obtain the alignment matching result of the two note sequences.

[0088] In some other possible implementations, a sequence matching model based on deep learning can be used to achieve alignment and matching of the standard note sequence and the first note sequence. The sequence matching model can be a transformer-based model. When training the matching model, a script can be used to automatically generate simulated matching data as a training data set. The simulated data can use data enhancement methods to generate a variety of actual scenarios that may occur, such as missing sounds, multiple sounds, rhythm changes, and speed changes, so that the trained sequence matching model has better accuracy and robustness.

[0089] See also Figure 5 In one embodiment of the present application, before acquiring the singing pitch according to the standard note sequence and the first note sequence, the singing pitch acquisition method may further include the following steps S51 to S54.

[0090] S51, obtaining the overall pitch deviation between the standard note sequence and the first note sequence.

[0091] S52, judging whether there is an octave deviation between the standard note sequence and the first note sequence according to the overall pitch deviation.

[0092] S53, if there is an octave deviation, execute step S54 after eliminating the octave deviation between the standard note sequence and the first note sequence in the pitch dimension; otherwise, execute step S54.

[0093] S54, obtaining singing pitch evaluation results according to the pitch scoring rules.

[0094] In one embodiment of the present application, the singing pitch acquisition method may further include: displaying the singing pitch to the user in a visual manner so that the user can understand the singer's pitch more intuitively.

[0095] The protection scope of the singing pitch acquisition method provided in the embodiment of the present application is not limited to the execution order of the steps listed in the present embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0096] The embodiment of the present application also provides a singing pitch acquisition device, which can implement the singing pitch acquisition method described in the present application, but the implementation device of the singing pitch acquisition method described in the present application includes but is not limited to the structure of the singing pitch acquisition device listed in the present embodiment. All structural deformations and replacements of the prior art made according to the principles of the present application are included in the protection scope of the present application.

[0097] Figure 6 The structure diagram of the singing pitch acquisition device 6 provided in the embodiment of the present application is shown. Figure 6 As shown, the singing pitch acquisition device 6 includes an audio data acquisition module 61, a note integration module 62 and a singing pitch acquisition module 63. The audio data acquisition module 61 is used to acquire singing audio data. The note integration module 62 is used to process the singing audio data using a deep learning model to integrate the main notes and the ornament notes of the singing audio data to obtain a first note sequence, wherein the ornament notes are generated by singing skills. The singing pitch acquisition module 63 is used to acquire singing pitch according to the standard note sequence and the first note sequence.

[0098] It should be noted that the audio data acquisition module 61, the note integration module 62 and the singing pitch acquisition module 63 in the singing pitch acquisition device 6 provided in the embodiment of the present application are the same as those in the present invention. Figure 2 Steps S21 to S23 in the singing pitch acquisition method shown correspond to each other and will not be described in detail here.

[0099] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the singing pitch acquisition method provided in any embodiment of the present application is implemented.

[0100] Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a tape, a floppy disk, an optical disk, and any combination thereof. The above-mentioned storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disk), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0101] An embodiment of the present application also provides an electronic device. Figure 7 The diagram shows a schematic diagram of the structure of an electronic device 7 in an embodiment of the present application. Figure 7 As shown, in this embodiment, the electronic device 7 includes a memory 71 and a processor 72 .

[0102] The memory 71 is used to store computer programs. In some possible implementations, the memory 71 may include: ROM, RAM, a disk, a USB flash drive, a memory card, or an optical disk, etc., various media that can store program codes.

[0103] In this embodiment, the memory 71 may include a computer system readable medium in the form of a volatile memory, such as a RAM and / or a cache memory. The electronic device 7 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 71 may include at least one program product having a set (e.g., at least one) of program modules, which are configured to perform the functions of the various embodiments of the present application.

[0104] The processor 72 is connected to the memory 71 and is used to execute the computer program stored in the memory 71 so that the electronic device 7 executes the singing pitch acquisition method.

[0105] In some possible implementations, the electronic device 7 provided in the embodiment of the present application may further include a display 73. The display 73 is communicatively connected with the memory 71 and the processor 72, and is used to display a graphical user interface (GUI) related to the singing pitch acquisition method.

[0106] In summary, the singing pitch acquisition method provided in the embodiment of the present application annotates the data set according to the characteristics of human vocal singing, and trains a singing transcription model that conforms to the singing characteristics based on a deep learning algorithm, which effectively improves the accuracy and credibility of the singing transcription results and the pitch evaluation results. In addition, in addition to the pitch dimension, the singing pitch acquisition method identifies and retains information in other dimensions such as rhythm and singing skills, providing a reliable basis for more comprehensive and multi-dimensional pitch evaluation. Furthermore, the singing pitch acquisition method can present the note sequence obtained by singing transcription and the information obtained by the singing skill recognition model to the singer in a visual manner, so that the singer can more intuitively connect his or her own pitch situation and improve the interpretability of the singing pitch evaluation results. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0107] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A singing pitch acquisition method, characterized in that: The singing pitch acquisition method comprises: Get singing audio data; Processing the singing audio data using a deep learning model to integrate the main notes and the ornamental notes of the singing audio data to obtain a first note sequence, wherein the ornamental notes are related to singing skills; The singing pitch is obtained according to the standard note sequence and the first note sequence.

2. The singing pitch acquisition method according to claim 1, characterized in that: The deep learning model includes a singing transcription model and a singing technique recognition model. The deep learning model is used to process the singing audio data to integrate the main notes and the ornament notes of the singing audio data to obtain an integrated note sequence, including: Processing the singing audio data using the singing transcription model to obtain a second note sequence; Processing the second note sequence using the singing skill recognition model to obtain positions of the ornament note and the main note in the second note sequence; The main note and the ornament note are integrated according to their positions in the second note sequence, so as to obtain the first note sequence.

3. The singing pitch acquisition method according to claim 2, characterized in that: Integrating the tonic note and the ornament note according to their positions in the second note sequence to obtain the first note sequence includes: The tonic note and its corresponding ornament note are merged into an integrated note according to their positions in the second note sequence, the pitch of the integrated note is the pitch of the tonic note, the start time of the integrated note is the minimum start time of the tonic note and its corresponding ornament note, the end time of the integrated note is the maximum end time of the tonic note and its corresponding ornament note, and each of the integrated notes constitutes the first note sequence.

4. The singing pitch acquisition method according to claim 2, characterized in that: The training method of the singing transcription model and the singing skill recognition model includes: Acquire training audio data and note information labeling results and singing technique labeling results thereof, wherein the singing technique labeling results include positions of the tonic note and the ornament note in a third note sequence, and the third note sequence is the note sequence of the training audio data; Using the training audio data and the note information labeling results thereof to train the singing transcription model; The singing skill recognition model is trained using the third note sequence and the singing skill labeling result.

5. The singing pitch acquisition method according to claim 1, characterized in that: Before using the deep learning model to process the singing audio data, the singing pitch acquisition method also includes: preprocessing the singing audio data so that the data structure and data format of the singing audio data are consistent with the training audio data of the deep learning model.

6. The singing pitch acquisition method according to claim 1, characterized in that: Before acquiring the singing pitch according to the standard note sequence and the first note sequence, the singing pitch acquisition method further includes: aligning and matching the standard note sequence and the first note sequence.

7. The singing pitch acquisition method according to claim 1, characterized in that: Before acquiring singing pitch according to the standard note sequence and the first note sequence, the singing pitch acquisition method further includes: Obtaining an overall pitch deviation between the standard note sequence and the first note sequence; Determining whether there is an octave deviation between the standard note sequence and the first note sequence according to the overall pitch deviation; If the octave deviation exists, the octave deviation between the standard note sequence and the first note sequence is eliminated in the pitch dimension.

8. A singing pitch acquisition device, characterized in that: The singing pitch acquisition device comprises: An audio data acquisition module, used to acquire singing audio data; a note integration module, configured to process the singing audio data using a deep learning model to integrate the main notes and the ornamental notes of the singing audio data to obtain a first note sequence, wherein the ornamental notes are generated by singing skills; The singing pitch acquisition module is used to acquire the singing pitch according to the standard note sequence and the first note sequence.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the singing pitch acquisition method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The electronic device comprises: A memory storing a computer program; A processor is communicatively connected to the memory, and executes the singing pitch acquisition method described in any one of claims 1 to 7 when calling the computer program.