Automatic accompaniment generation method and device for playing audio characteristics in real time and effector

Through real-time analysis and neural network to judge audio characteristics, and dynamically generate automatic accompaniment, the problem that the existing technology cannot respond to the performer in real time is solved, and personalized accompaniment that interacts with the performer is realized, which is suitable for a variety of music scenes.

CN118968952AActive Publication Date: 2024-11-15CHANGSHA HOTONE AUDIO
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411063335.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-15
Estimated Expiration
2044-08-05

AI Technical Summary

Technical Problem

Existing automatic accompaniment technology cannot analyze and respond to the player's audio characteristics in real time, and cannot simulate the interaction between the accompaniment and the player in the band.

Method used

By obtaining real-time audio sequences, performing energy calculations and signal division, using neural networks to judge audio characteristics, and generating automatic accompaniment in real time, including cepshot analysis and multi-layer neural network training, dynamically adjusting the accompaniment content to respond to the changes of the performer.

Benefits of technology

It realizes personalized accompaniment with real-time interaction with the performer, fills the gap in the interaction between automatic accompaniment and real bands, and provides a richer and expressive music experience, suitable for live performances, music teaching and personal practice.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968952A_ABST
    Figure CN118968952A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of music equipment, and relates to an automatic accompaniment generation method and device for playing audio characteristics in real time and an effector. The method comprises the following steps: acquiring a real-time audio sequence; dividing the audio sequence into a plurality of signal samples, and performing energy calculation on the plurality of signal samples to obtain audio energy of the audio sequence; according to the audio energy, performing playing audio characteristic judgment through a first neural network to obtain real-time accompaniment input; generating an automatic accompaniment in real time according to the real-time accompaniment input; performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence; according to a plurality of maximum value points of the cepstrum sequence, training through a second neural network to obtain a pitch probability; obtaining a polyphonic pitch according to the pitch probability; and according to the audio energy, the pitch probability and the polyphonic pitch, performing playing audio characteristic judgment through a first neural network to obtain real-time accompaniment input. The automatic accompaniment can be generated in real time according to the real-time playing audio characteristics.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of music equipment, and in particular to a method, device and effector for automatically generating accompaniment based on real-time performance audio characteristics. Background Art

[0002] Accompaniment refers to the music part other than the main melody in instrumental performance. Accompaniment can be performed by other instruments or human voice. In instrumental performance, accompaniment is usually used to set off the main melody and make the music more colorful.

[0003] There are many types of accompaniment, which can be divided into the following categories according to their form and function:

[0004] 1) Harmonic accompaniment: Harmonic accompaniment refers to accompaniment using chords or harmonies, which is common in instruments such as piano, guitar, accordion, etc. Harmonic accompaniment can increase the thickness and richness of music, and can also express the emotion and style of music.

[0005] 2) Melodic accompaniment: Melodic accompaniment refers to the use of another melody to accompany the main melody, which is common in instruments such as violin, flute, and saxophone. Melodic accompaniment can form a contrast or counterpoint with the main melody, and can also create musical levels and changes.

[0006] 3) Rhythmic accompaniment: Rhythmic accompaniment refers to the accompaniment that uses repeated rhythmic patterns or drum beats to support the main melody. It is commonly seen in instruments such as drums, bass, and percussion. Rhythmic accompaniment can provide the dynamics and stability of the music, and can also highlight the rhythm and atmosphere of the music.

[0007] 4) Improvisational accompaniment: Improvisational accompaniment refers to the accompaniment that is freely played according to the basic structure and rules of music. It is common in music styles such as jazz, rock, and folk music. Improvisational accompaniment can show the creativity and personality of music, and can also increase the surprise and fun of music.

[0008] When a real band is playing accompaniment, the accompanist will often make real-time changes according to the performance of the performer. For example, if the accompanist feels that the volume of the performer's performance has increased, the accompanist will also adjust the volume of the accompaniment accordingly. Specifically, if the drummer feels that the density of notes played by the guitar has increased, the drummer may choose appropriate drum beats to support the guitarist's performance; generally speaking, if the guitarist's note density is high, the drummer can choose simpler drum beats to maintain the stability and rhythm of the music; the drummer can also increase the dynamics and atmosphere of the drums by changing the speed, strength, and accent of the drum beats. If the bassist can choose the appropriate bass line according to the guitarist's note density to cooperate with the guitarist's performance; generally speaking, if the guitarist's note density is high, the bassist can choose a lower note density to maintain the balance and clarity of the music; the bassist can also increase the expressiveness and changes of the bass by changing the length, strength, and timbre of the notes.

[0009] However, if an instrumentalist practices or performs alone, he can only play the instrument he can play without the help of accompaniment. For this reason, technicians have developed the technology of automatic accompaniment. Automatic accompaniment refers to the use of software or equipment to automatically generate suitable music accompaniment based on the input chords or melody. Automatic accompaniment can help music lovers or learners practice playing or composing songs, and can also increase the fun and diversity of music.

[0010] In the prior art, automatic accompaniment technology is mainly based on preset music patterns or algorithms to generate accompaniment that coordinates with the main melody. These technologies usually include MIDI (Music Digital Interface) programs, music generation software, and systems using artificial intelligence algorithms; some devices or software can automatically generate accompaniments with drums, bass and harmony according to the chords input by the user, based on music theory and preset styles, such as blues, country, funk, etc.; users can also manually adjust parameters such as volume, rhythm, mode, or use loop and transposition functions to practice; usually most devices provide preset optional rhythm patterns and chords, and the rhythm patterns and chords need to be programmed or chord diagrams need to be input in advance, and then the device can play according to the pre-programmed rhythm patterns and chords when playing.

[0011] However, no matter which method is used, these automatic accompaniment devices or software will keep repeating the accompaniment content after determining the specific accompaniment to be generated. They do not support real-time analysis and response to the performance, and have no way to simulate the interaction between the accompanist and the performer in the band. Summary of the invention

[0012] Based on this, it is necessary to provide a method, device and effector for generating automatic accompaniment based on the characteristics of real-time performance audio in order to solve the above technical problems, so as to generate automatic accompaniment in real time according to the characteristics of real-time performance audio.

[0013] The automatic accompaniment generation method based on real-time performance audio characteristics comprises:

[0014] Get real-time audio sequence;

[0015] Dividing the audio sequence into a plurality of signal samples, performing energy calculation on the plurality of signal samples, and obtaining audio energy of the audio sequence;

[0016] According to the audio energy, the first neural network is used to determine the performance audio characteristics to obtain real-time accompaniment input;

[0017] Generate automatic accompaniment in real time based on real-time accompaniment input.

[0018] In one embodiment, it further includes:

[0019] Performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence; training a second neural network based on multiple maximum points of the cepstrum sequence to obtain a pitch probability; and obtaining a polyphonic pitch based on the pitch probability;

[0020] According to the audio energy, the pitch probability and the polyphonic pitch, the performance audio characteristics are judged through a first neural network to obtain real-time accompaniment input.

[0021] In one embodiment, it further includes:

[0022] According to the audio energy, a third neural network is used to determine the probability of the sound start; and according to the sound start probability, the number of the sound start is obtained;

[0023] According to the audio energy, the pitch probability, the polyphonic pitch and the number of note heads, the performance audio characteristics are judged through a first neural network to obtain real-time accompaniment input.

[0024] In one embodiment, the audio sequence is divided into a plurality of signal samples, and energy calculation is performed on the plurality of signal samples to obtain audio energy of the audio sequence, including:

[0025] Dividing the audio sequence into a plurality of signal samples, calculating the sum of squares of the plurality of signal samples, and obtaining sequence energy;

[0026] Using multiple band-dividing filters to divide the audio sequence into multiple signal frequency bands, calculating the energy of each signal frequency band to obtain multiple band-dividing energies;

[0027] The audio energy of the audio sequence is obtained according to the sequence energy and the sub-band energy.

[0028] In one embodiment, according to the audio energy, the performance audio characteristics are judged by a first neural network to obtain real-time accompaniment input, including:

[0029] According to the sequence energy, the band energy, the pitch probability, the polyphonic pitch and the number of note heads, the performance audio characteristics are judged through a first neural network to obtain real-time accompaniment input.

[0030] In one embodiment, performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence includes:

[0031] The audio sequence is subjected to Fourier transform, modulus calculation, logarithm calculation and inverse Fourier transform in sequence to obtain a cepstrum sequence.

[0032] In one embodiment, obtaining a polyphonic pitch according to the pitch probability includes:

[0033] The pitch probability is judged, and the maximum point where the pitch probability meets the preset conditions is taken as a pitch;

[0034] Traverse all the maximum points to obtain the polyphonic pitch.

[0035] In one embodiment, according to the real-time accompaniment input, the automatic accompaniment is generated in real time, including:

[0036] According to the real-time accompaniment input, different repeated sections are produced for each style of accompaniment, and the repeated sections correspond to different performance audio characteristics respectively; and corresponding interludes are produced for each repeated section;

[0037] For the ending measure of the music segment, when it is judged that its performance audio characteristics are inconsistent with the repeated segment belonging to the current paragraph, a transition is used for migration, switching to the corresponding performance audio characteristics, and automatic accompaniment is generated in real time.

[0038] An automatic accompaniment generating device with real-time performance audio characteristics, comprising:

[0039] An acquisition module is used to acquire real-time audio sequences;

[0040] A calculation module, used for dividing the audio sequence into a plurality of signal samples, performing energy calculation on the plurality of signal samples, and obtaining audio energy of the audio sequence;

[0041] A judgment module, used to judge the performance audio characteristics through a first neural network according to the audio energy, and obtain real-time accompaniment input;

[0042] The accompaniment module is used to generate automatic accompaniment in real time according to real-time accompaniment input.

[0043] The effector comprises: an automatic accompaniment generating device for real-time performance audio characteristics, an audio input port, an audio output port and an accompaniment switch;

[0044] The audio input port is connected to a musical instrument to obtain a real-time audio sequence;

[0045] The audio input port, the audio output port and the accompaniment switch are all connected to the automatic accompaniment generating device with real-time performance audio characteristics, so that after receiving the accompaniment generating signal from the accompaniment switch, the automatic accompaniment generating device with real-time performance audio characteristics generates an automatic accompaniment in real time according to the real-time audio sequence input from the audio input port and outputs it through the audio output port;

[0046] The audio input port is also connected to the audio output port so that the real-time audio sequence of the musical instrument can be output through the audio output port.

[0047] The above-mentioned automatic accompaniment generation method, device and effector of real-time performance audio characteristics, after extracting and analyzing the musical characteristics of the performance audio, obtains the judgment of the real-time performance audio characteristics, specifically: sequence energy and band energy represent the intensity of the audio, pitch probability and polyphonic pitch represent the melody trend of the audio, the number of sound heads represents the note density of the audio, and the intensity, note density and melody trend represent the musical characteristics. Therefore, according to the sequence energy, band energy, pitch probability, polyphonic pitch and number of sound heads, the performance audio characteristics are judged, the real-time accompaniment input is obtained, and the automatic accompaniment is generated in real time. This application is not only based on music theory and algorithm to generate accompaniment, but also can perceive and respond to the changes in the performance audio characteristics of the performance in real time, provide a more personalized and expressive music experience, fill the gap in the interaction between automatic accompaniment and real bands and performers, better synchronize with the performer's expression, and have broad application prospects in live performances, music teaching, recording and personal practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic flow chart of a method for generating an automatic accompaniment based on real-time performance audio characteristics in one embodiment;

[0049] Figure 2 is an exemplary schematic diagram of a second neural network in one embodiment;

[0050] Figure 3 is an exemplary schematic diagram of a third neural network in one embodiment;

[0051] Figure 4 is an exemplary schematic diagram of a first neural network in one embodiment;

[0052] Figure 5 is a structural block diagram of an automatic accompaniment generating device for real-time performance audio characteristics in one embodiment;

[0053] Figure 6 FIG. 1 is a schematic diagram of the structure of a guitar effector in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0055] In addition, the descriptions of "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "multiple groups" means at least two groups, such as two groups, three groups, etc., unless otherwise clearly and specifically defined.

[0056] In this application, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0057] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0058] This application provides a method for automatically generating accompaniment based on real-time performance audio characteristics, such as Figure 1 The flowchart shown, in one embodiment, includes:

[0059] Step 102: Acquire a real-time audio sequence.

[0060] In this step, how to obtain a real-time audio sequence belongs to the prior art and will not be described in detail here.

[0061] Step 104: divide the audio sequence into multiple signal samples, perform energy calculation on the multiple signal samples, and obtain audio energy of the audio sequence.

[0062] Specifically: the audio sequence is divided into multiple signal samples, the energy of all signal samples is calculated to obtain the sequence energy; the audio sequence is divided into multiple signal frequency bands, the energy of each signal frequency band is calculated to obtain the sub-band energy; the audio energy of the audio sequence is obtained according to the sequence energy and the multiple sub-band energies.

[0063] Preferably, the method further comprises: performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence; training a second neural network according to a plurality of maximum points of the cepstrum sequence to obtain a pitch probability; and obtaining a polyphonic pitch according to the pitch probability.

[0064] in:

[0065] The audio sequence is subjected to cepstrum analysis to obtain a cepstrum sequence, including: performing Fourier transform, modulus calculation, logarithm calculation and inverse Fourier transform on the audio sequence in sequence to obtain a cepstrum sequence:

[0066] X=F -1 {log|F(x)}

[0067] Where X is the cepstrum sequence, F -1 is the inverse Fourier transform, log is the logarithm, |·| is the modulus, and F(x) is the Fourier transform.

[0068] According to the multiple maximum points of the cepstrum sequence, the second neural network is trained to obtain the pitch probability, including: according to the cepstrum sequence, multiple maximum points are selected as the input of the second neural network, the second neural network is trained, and the output of the second neural network is used as the pitch probability (the pitch probability refers to the probability that the maximum point is the pitch). For example: for an audio sequence of 1024 samples in length, a cepstrum analysis is performed, and the first 20 maximum points of the cepstrum sequence are taken, and the x, y coordinates of the 20 maximum points are combined into a 40*1 vector as the second neural network (such as Figure 2 The RNN recurrent neural network shown in the figure takes 40 feature quantities as input, and passes through two layers of hidden GRU layers with 40 nodes to obtain a 20*1 output vector, which is the output of the second neural network, indicating the probability that these 20 maximum points are pitches respectively; then, after training the second neural network with a large amount of data, a polyphonic detection algorithm can be obtained.

[0069] According to the pitch probability, the polyphonic pitch is obtained, including: judging according to the pitch probability, taking the maximum point whose pitch probability meets the preset condition as a pitch; traversing all the maximum points to obtain the polyphonic pitch. For example, the preset condition is greater than 0.5, that is, each maximum point whose pitch probability is greater than 0.5 is the real polyphonic pitch.

[0070] Further preferably, the method further includes: obtaining the probability of the sound start according to the audio energy by judging through a third neural network; and obtaining the number of the sound start according to the probability of the sound start.

[0071] in:

[0072] The third neural network can be a 1->4->4->1 GRU recurrent neural network, such as Figure 3 shown.

[0073] The judgment is made based on the probability of the note head. When the probability of the note head meets the preset condition (i.e., greater than 0.5), it means that the current is the note head of a sound, otherwise it is not. The number of note heads is further obtained; the number of note heads detected per second is the note density.

[0074] More preferably, the audio sequence is divided into multiple signal samples, and energy calculation is performed on the multiple signal samples to obtain audio energy of the audio sequence, including: dividing the audio sequence into multiple signal samples, calculating the sum of squares of the multiple signal samples to obtain sequence energy; using multiple band filters to divide the audio sequence into multiple signal frequency bands, calculating the energy of each signal frequency band to obtain multiple band energies; and obtaining the audio energy of the audio sequence based on the sequence energy and the band energy.

[0075] in:

[0076]

[0077] Where E is the sequence energy, x i is the i-th signal sample, and N is the length of the audio sequence.

[0078] Several band-splitting filters are used with frequencies of 100, 400, 1000, and 4000 Hz, respectively, to divide the audio sequence into five signal frequency bands. The energy of each signal frequency band is then calculated to obtain five band-splitting energies.

[0079] In this step, firstly, the audio energy of the audio sequence is obtained through the performer's performance audio; further, the real-time polyphonic pitch is obtained; further, the real-time number of sound heads is obtained.

[0080] Step 106, based on the audio energy, the first neural network is used to determine the performance audio characteristics to obtain real-time accompaniment input.

[0081] Preferably, the audio characteristics of the performance are judged by a first neural network based on audio energy to obtain real-time accompaniment input, including: based on audio energy, pitch probability and polyphonic pitch, the audio characteristics of the performance are judged by a first neural network to obtain real-time accompaniment input.

[0082] Further preferably, the audio characteristics of the performance are judged by a first neural network based on audio energy to obtain real-time accompaniment input, including: based on audio energy, pitch probability, polyphonic pitch and the number of note heads, the audio characteristics of the performance are judged by a first neural network to obtain real-time accompaniment input.

[0083] More preferably, the audio characteristics of the performance are judged by the first neural network according to the audio energy to obtain the real-time accompaniment input, including: judging the audio characteristics of the performance by the first neural network according to the sequence energy, the band energy, the pitch probability, the polyphonic pitch and the number of sound heads to obtain the real-time accompaniment input. For example: the sequence energy, the band energy, the three largest pitch probabilities, the three pitches with the largest pitch probability and the number of sound heads in the past 2 seconds are combined into a 13*1 matrix as the first neural network (such as Figure 4 As shown, the input of the neural network 13->13->13->1) is judged by the first neural network to obtain the performance audio characteristics, and the performance audio characteristics are used as the accompaniment input. For the real-time audio sequence, the real-time accompaniment input can be obtained.

[0084] In this step, first, the performance audio characteristics are judged based on the audio energy; further, the performance audio characteristics are judged based on the audio energy, pitch probability and polyphonic pitch; further, the performance audio characteristics are judged based on the audio energy, pitch probability, polyphonic pitch and the number of note heads; further, the performance audio characteristics are judged based on the sequence energy, band energy, pitch probability, polyphonic pitch and the number of note heads.

[0085] It should be noted that the accompaniment input is a number. The closer it is to 1, the higher the performance audio characteristics are, and the closer it is to 0, the calmer the performance audio characteristics are.

[0086] Step 108, generating automatic accompaniment in real time according to the real-time accompaniment input.

[0087] Specifically: based on the real-time accompaniment input, different repeated segments are produced for each style of accompaniment, and the repeated segments correspond to different performance audio characteristics; a corresponding interlude is produced for each repeated segment; for the ending bar of the music segment, when it is judged that its performance audio characteristics are inconsistent with the repeated segment belonging to the current paragraph, an interlude is used to migrate and switch to the corresponding performance audio characteristics to generate automatic accompaniment in real time.

[0088] For example: based on the real-time accompaniment input, for each style of accompaniment, 4 different repetitions are produced, ranging from calm performance audio characteristics to passionate performance audio characteristics, corresponding to 0-0.25, 0.25-0.5, 0.5-0.75, and 0.75-1 of the performance audio characteristics respectively; for each repetition, a corresponding interlude is produced; for the ending bar of each music section (generally 8 bars, or a whole chord cycle), its performance audio characteristics are judged. When the performance audio characteristics are inconsistent with the repetition section belonging to the current paragraph, an interlude is used to migrate the performance audio characteristics, that is, after using a interlude, switch to the corresponding performance audio characteristics to generate automatic accompaniment in real time.

[0089] In this step, the generated accompaniment content is adjusted dynamically in real time according to the performance audio characteristics, so as to realize automatic accompaniment that interacts with the performer in real time.

[0090] The above-mentioned method for generating automatic accompaniment based on the audio characteristics of real-time performance obtains the judgment of the audio characteristics of real-time performance after extracting and analyzing the musical characteristics of the performance audio. Specifically, the sequence energy and band energy represent the intensity of the audio, the pitch probability and polyphonic pitch represent the melody trend of the audio, and the number of sound heads represents the note density of the audio. The intensity, note density and melody trend represent the musical characteristics. Therefore, according to the sequence energy, band energy, pitch probability, polyphonic pitch and number of sound heads, the performance audio characteristics are judged, the real-time accompaniment input is obtained, and the automatic accompaniment is generated in real time. The present application is not only based on music theory and algorithm to generate accompaniment, but also can perceive and respond to the changes in the performance audio characteristics of the performance in real time, provide a more personalized and expressive music experience, fill the gap of the interaction between the automatic accompaniment and the real band and performers, better synchronize with the performer's expression, and has broad application prospects in live performances, music teaching, recording and personal practice.

[0091] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0092] The present application also provides an automatic accompaniment generation device with real-time performance audio characteristics, such as Figure 5As shown, in one embodiment, it includes: an acquisition module 502, a calculation module 504, a judgment module 506 and an accompaniment module 508, wherein:

[0093] An acquisition module 502 is used to acquire a real-time audio sequence;

[0094] A calculation module 504, configured to divide the audio sequence into a plurality of signal samples, perform energy calculation on the plurality of signal samples, and obtain audio energy of the audio sequence;

[0095] A judgment module 506 is used to judge the performance audio characteristics through the first neural network according to the audio energy to obtain real-time accompaniment input;

[0096] The accompaniment module 508 is used to generate automatic accompaniment in real time according to the real-time accompaniment input.

[0097] The specific limitation of the automatic accompaniment generating device for real-time performance audio characteristics can be referred to the limitation of the automatic accompaniment generating method for real-time performance audio characteristics in the above text, which will not be repeated here. Each module in the above-mentioned device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor calls and executes the operations corresponding to the above-mentioned modules.

[0098] The present application also provides an effector, such as Figure 6 As shown, in one embodiment, it includes: an automatic accompaniment generating device with real-time performance audio characteristics, an audio input port, an audio output port and an accompaniment switch.

[0099] The audio input port is connected to the instrument to obtain real-time audio sequences.

[0100] The audio input port, the audio output port and the accompaniment switch are all connected to an automatic accompaniment generating device with real-time performance audio characteristics, so that after receiving the accompaniment generation signal from the accompaniment switch, the automatic accompaniment generating device with real-time performance audio characteristics generates an automatic accompaniment in real time according to the real-time audio sequence input from the audio input port and outputs it through the audio output port.

[0101] The audio input port is also connected to the audio output port so that the real-time audio sequence of the musical instrument can be output through the audio output port.

[0102] For example: Figure 6The real-time accompaniment guitar effects shown in the figure first receives the guitar playing signal (i.e., audio sequence) from the audio input port as an input signal; then, it analyzes the performance audio characteristics according to the input signal, uses the analysis result as the accompaniment input, and controls the built-in accompaniment unit of the effects; finally, the automatic accompaniment generated in real time by the accompaniment unit is mixed with the guitar input signal and outputted from the audio output port together.

[0103] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0104] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating an automatic accompaniment based on real-time performance audio characteristics, characterized in that: include: Get real-time audio sequence; Dividing the audio sequence into a plurality of signal samples, performing energy calculation on the plurality of signal samples, and obtaining audio energy of the audio sequence; According to the audio energy, the first neural network is used to determine the performance audio characteristics to obtain real-time accompaniment input; Generate automatic accompaniment in real time based on real-time accompaniment input.

2. The method for generating automatic accompaniment based on real-time performance audio characteristics according to claim 1, characterized in that: Also includes: Performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence; According to the multiple maximum points of the cepstrum sequence, the pitch probability is obtained by training the second neural network; according to the pitch probability, the polyphonic pitch is obtained; According to the audio energy, the pitch probability and the polyphonic pitch, the performance audio characteristics are judged through a first neural network to obtain real-time accompaniment input.

3. The method for generating an automatic accompaniment based on the real-time performance audio characteristics according to claim 2, characterized in that: Also includes: According to the audio energy, a third neural network is used to determine the probability of the start of the sound; According to the probability of the sound head, the number of the sound head is obtained; According to the audio energy, the pitch probability, the polyphonic pitch and the number of note heads, the performance audio characteristics are judged through a first neural network to obtain real-time accompaniment input.

4. The method for generating automatic accompaniment based on real-time performance audio characteristics according to claim 3, characterized in that: The method further comprises: dividing the audio sequence into a plurality of signal samples, performing energy calculation on the plurality of signal samples, and obtaining audio energy of the audio sequence. Dividing the audio sequence into a plurality of signal samples, calculating the sum of squares of the plurality of signal samples, and obtaining sequence energy; Using multiple band-dividing filters to divide the audio sequence into multiple signal frequency bands, calculating the energy of each signal frequency band to obtain multiple band-dividing energies; The audio energy of the audio sequence is obtained according to the sequence energy and the sub-band energy.

5. The method for generating automatic accompaniment based on real-time performance audio characteristics according to claim 4, characterized in that: According to the audio energy, the performance audio characteristics are judged by a first neural network to obtain real-time accompaniment input, including: According to the sequence energy, the band energy, the pitch probability, the polyphonic pitch and the number of note heads, the performance audio characteristics are judged through a first neural network to obtain real-time accompaniment input.

6. The method for generating automatic accompaniment based on real-time performance audio characteristics according to any one of claims 2 to 5, characterized in that: Performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence, including: The audio sequence is subjected to Fourier transform, modulus calculation, logarithm calculation and inverse Fourier transform in sequence to obtain a cepstrum sequence.

7. The method for automatically generating accompaniment based on the real-time performance audio characteristics according to any one of claims 2 to 5, characterized in that: According to the pitch probability, the polyphonic pitch is obtained, including: The pitch probability is judged, and the maximum point where the pitch probability meets the preset conditions is taken as a pitch; Traverse all the maximum points to obtain the polyphonic pitch.

8. The method for automatically generating accompaniment based on real-time performance audio characteristics according to any one of claims 1 to 5, characterized in that: Generate automatic accompaniment in real time based on real-time accompaniment input, including: According to the real-time accompaniment input, different repeated sections are produced for each style of accompaniment, and the repeated sections correspond to different performance audio characteristics respectively; and corresponding interludes are produced for each repeated section; For the ending measure of the music segment, when it is judged that its performance audio characteristics are inconsistent with the repeated segment belonging to the current paragraph, a transition is used for migration, switching to the corresponding performance audio characteristics, and automatic accompaniment is generated in real time.

9. An automatic accompaniment generating device for real-time performance audio characteristics, characterized in that: include: An acquisition module is used to acquire real-time audio sequences; A calculation module, used for dividing the audio sequence into a plurality of signal samples, performing energy calculation on the plurality of signal samples, and obtaining audio energy of the audio sequence; A judgment module, used to judge the performance audio characteristics through a first neural network according to the audio energy, and obtain real-time accompaniment input; The accompaniment module is used to generate automatic accompaniment in real time according to real-time accompaniment input.

10. An effector, characterized in that: include: An automatic accompaniment generating device for real-time performance audio characteristics, an audio input port, an audio output port, and an accompaniment switch; The audio input port is connected to a musical instrument to obtain a real-time audio sequence; The audio input port, the audio output port and the accompaniment switch are all connected to the automatic accompaniment generating device with real-time performance audio characteristics, so that after receiving the accompaniment generating signal from the accompaniment switch, the automatic accompaniment generating device with real-time performance audio characteristics generates an automatic accompaniment in real time according to the real-time audio sequence input from the audio input port and outputs it through the audio output port; The audio input port is also connected to the audio output port so that the real-time audio sequence of the musical instrument can be output through the audio output port.

Citation Information

Patent Citations

  • Electronic musical instrument, accompaniment sound instruction method, and automatic accompaniment sound generation device

    CN113838446A

  • Electronic musical instrument, electronic musical instrument control method, and program

    CN116057624A

  • Accompaniment generation method and device based on electronic musical instrument, equipment and storage medium

    CN116189636A

  • Automatic musical accompaniment system and musical score tracking system

    JP2007241181A

  • Musical sound processing apparatus and program

    JP2008250008A