Automatic accompaniment generation method, device and effector based on real-time performance audio characteristics

Abstract

The present application belongs to the technical field of music equipment, and relates to a method, device and effector for generating automatic accompaniment based on the audio characteristics of real-time performance. The method includes: obtaining a real-time audio sequence; dividing the audio sequence into multiple signal samples, performing energy calculation on the multiple signal samples, and obtaining the audio energy of the audio sequence; judging the performance audio characteristics through a first neural network based on the audio energy, and obtaining a real-time accompaniment input; generating automatic accompaniment in real time based on the real-time accompaniment input; and further including: performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence; training a second neural network based on multiple maximum points of the cepstrum sequence to obtain a pitch probability; obtaining a polyphonic pitch based on the pitch probability; judging the performance audio characteristics through a first neural network based on the audio energy, pitch probability and polyphonic pitch, and obtaining a real-time accompaniment input. The present application can generate automatic accompaniment in real time based on the audio characteristics of real-time performance.

Description

Technical Field

[0001] The present application relates to the technical field of music equipment, and in particular to a method, device and effector for automatically generating accompaniment based on real-time performance audio characteristics. Background Art

[0002] Accompaniment refers to the musical part of an instrumental performance, in addition to the main melody. Accompaniment can be provided by other instruments or vocals. In instrumental performances, accompaniment is often used to complement the main melody and add richness to the music.

[0003] There are many types of accompaniment, which can be divided into the following categories according to their form and function:

[0004] 1) Harmonic accompaniment: Harmonic accompaniment refers to accompaniment using chords or harmonies, commonly found on instruments such as piano, guitar, and accordion. Harmonic accompaniment can add depth and richness to music, and can also express the emotion and style of the music.

[0005] 2) Melodic accompaniment: Melodic accompaniment is the use of another melody to complement the main melody. It is commonly used on instruments such as violin, flute, and saxophone. Melodic accompaniment can contrast or counterpoint the main melody, or create musical depth and variation.

[0006] 3) Rhythmic accompaniment: Rhythmic accompaniment uses repeated rhythmic patterns or drum beats to support the main melody. It is commonly found on instruments such as drums, bass, and percussion. Rhythmic accompaniment can provide movement and stability to the music, while also highlighting the rhythm and atmosphere of the music.

[0007] 4) Improvisational Accompaniment: Improvisational accompaniment is a free-flowing accompaniment based on the basic structure and rules of music. It is commonly found in musical styles such as jazz, rock, and folk music. Improvisational accompaniment can showcase the creativity and individuality of music, and can also add surprise and interest to the music.

[0008] When a real band plays accompaniment, the accompanist often makes real-time adjustments based on the musician's performance. For example, if the accompanist senses the musician's playing volume increasing, they will adjust the accompaniment accordingly. Specifically, if the drummer senses the guitarist's note density increasing, they may choose appropriate drum beats to support the guitarist's performance. Generally, if the guitarist's note density is high, the drummer may choose simpler beats to maintain the stability and rhythm of the music. The drummer can also increase the dynamics and atmosphere of the drums by varying the speed, strength, and emphasis of the beats. A bassist can choose an appropriate bass line to complement the guitarist's performance based on the guitarist's note density. Generally, if the guitarist's note density is high, the bassist may choose a lower note density to maintain balance and clarity. The bassist can also increase the expressiveness and variety of the bass by varying the length, dynamics, and timbre of the notes.

[0009] However, when an instrumentalist practices or performs alone, they are limited to their own instrument and lack the assistance of accompaniment. This is why researchers have developed automatic accompaniment technology. Automatic accompaniment uses software or devices to automatically generate appropriate musical accompaniment based on input chords or melodies. Automatic accompaniment can help music enthusiasts and learners practice or compose songs, and can also enhance the interest and diversity of music.

[0010] In the existing art, automatic accompaniment technology is mainly based on preset musical patterns or algorithms to generate accompaniment that coordinates with the main melody. These technologies generally include MIDI (Musical Digital Interface) programs, music generation software, and systems that utilize artificial intelligence algorithms. Some devices or software can automatically generate accompaniment with drums, bass, and harmonies based on user-entered chords, music theory, and preset styles, such as blues, country, and funk. Users can also manually adjust parameters such as volume, rhythm, and mode, or use looping and transposition functions for practice. Generally, most devices provide preset optional rhythm patterns and chords, which require pre-programming or inputting chord diagrams before the device can play according to the pre-programmed rhythm patterns and chords.

[0011] However, regardless of the method, these automatic accompaniment devices or software will only repeat the playback of the accompaniment content after determining the specific accompaniment to be generated. They do not support real-time analysis and response to the performance, and there is no way to simulate the interaction between the accompanist and the performer in the band. Summary of the Invention

[0012] Based on this, it is necessary to provide a method, device and effector for generating automatic accompaniment based on the characteristics of real-time performance audio in order to address the above technical problems, so as to generate automatic accompaniment in real time according to the characteristics of real-time performance audio.

[0013] The automatic accompaniment generation method based on real-time performance audio characteristics includes:

[0014] Get real-time audio sequence;

[0015] Dividing the audio sequence into multiple signal samples, performing energy calculation on the multiple signal samples, and obtaining audio energy of the audio sequence;

[0016] Based on the audio energy, a first neural network is used to determine the performance audio characteristics to obtain real-time accompaniment input;

[0017] Generate automatic accompaniment in real time based on real-time accompaniment input.

[0018] In one embodiment, it further includes:

[0019] Performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence; training a second neural network based on multiple maximum points of the cepstrum sequence to obtain pitch probabilities; and obtaining polyphonic pitches based on the pitch probabilities;

[0020] According to the audio energy, the pitch probability and the polyphonic pitch, a performance audio characteristic is judged by a first neural network to obtain a real-time accompaniment input.

[0021] In one embodiment, it further includes:

[0022] According to the audio energy, a third neural network is used to determine the probability of the sound start; and according to the sound start probability, the number of the sound start is determined;

[0023] According to the audio energy, the pitch probability, the polyphonic pitch and the number of note heads, a performance audio characteristic is judged by a first neural network to obtain a real-time accompaniment input.

[0024] In one embodiment, dividing the audio sequence into a plurality of signal samples, and performing energy calculation on the plurality of signal samples to obtain audio energy of the audio sequence includes:

[0025] Dividing the audio sequence into multiple signal samples, calculating the sum of squares of the multiple signal samples, and obtaining sequence energy;

[0026] Using multiple band-dividing filters to divide the audio sequence into multiple signal frequency bands, calculating the energy of each signal frequency band to obtain multiple band-dividing energies;

[0027] The audio energy of the audio sequence is obtained according to the sequence energy and the sub-band energy.

[0028] In one embodiment, based on the audio energy, a first neural network is used to determine the performance audio characteristics to obtain real-time accompaniment input, including:

[0029] According to the sequence energy, the band energy, the pitch probability, the polyphonic pitch and the number of note heads, a performance audio characteristic judgment is performed through a first neural network to obtain a real-time accompaniment input.

[0030] In one embodiment, performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence includes:

[0031] The audio sequence is subjected to Fourier transform, modulus calculation, logarithm calculation and inverse Fourier transform in sequence to obtain a cepstrum sequence.

[0032] In one embodiment, obtaining a polyphonic pitch according to the pitch probability includes:

[0033] Judge based on pitch probability, and take the maximum point where the pitch probability meets the preset conditions as a pitch;

[0034] Traverse all the maximum points to obtain the polyphonic pitch.

[0035] In one embodiment, the automatic accompaniment is generated in real time based on the real-time accompaniment input, including:

[0036] According to the real-time accompaniment input, different repeating sections are produced for each style of accompaniment, and the repeating sections correspond to different performance audio characteristics; and corresponding fills are produced for each repeating section;

[0037] For the ending measure of the music segment, when it is judged that its performance audio characteristics are inconsistent with the repeated segment belonging to the current paragraph, a transition is used to migrate, switch to the corresponding performance audio characteristics, and generate automatic accompaniment in real time.

[0038] An automatic accompaniment generating device with real-time performance audio characteristics, comprising:

[0039] Acquisition module, used to obtain real-time audio sequence;

[0040] a calculation module, configured to divide the audio sequence into a plurality of signal samples, perform energy calculation on the plurality of signal samples, and obtain audio energy of the audio sequence;

[0041] a judgment module, configured to judge the performance audio characteristics through a first neural network based on the audio energy, and obtain real-time accompaniment input;

[0042] The accompaniment module is used to generate automatic accompaniment in real time according to real-time accompaniment input.

[0043] The effector comprises: an automatic accompaniment generating device for real-time performance audio characteristics, an audio input port, an audio output port and an accompaniment switch;

[0044] The audio input port is connected to a musical instrument to obtain a real-time audio sequence;

[0045] The audio input port, the audio output port, and the accompaniment switch are all connected to the automatic accompaniment generating device with real-time performance audio characteristics, so that after receiving the accompaniment generation signal from the accompaniment switch, the automatic accompaniment generating device with real-time performance audio characteristics generates an automatic accompaniment in real time according to the real-time audio sequence input from the audio input port and outputs it through the audio output port;

[0046] The audio input port is also connected to the audio output port so that the real-time audio sequence of the musical instrument can be output from the audio output port.

[0047] The above-mentioned automatic accompaniment generation method, device and effector of real-time performance audio characteristics obtains the judgment of real-time performance audio characteristics after extracting and analyzing the musical characteristics of the performance audio. Specifically: sequence energy and band energy represent the intensity of the audio, pitch probability and polyphonic pitch represent the melody direction of the audio, and the number of note heads represents the note density of the audio. The intensity, note density and melody direction represent the musical characteristics. Therefore, according to the sequence energy, band energy, pitch probability, polyphonic pitch and number of note heads, the performance audio characteristics are judged, the real-time accompaniment input is obtained, and the automatic accompaniment is generated in real time. This application is not only based on music theory and algorithm to generate accompaniment, but can perceive and respond to changes in the performance audio characteristics of the performance in real time, providing a more personalized and expressive music experience, and can fill the gap in the interaction between automatic accompaniment and real bands and performers, and better synchronize with the performer's expression. It has broad application prospects in live performances, music teaching, recording and personal practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 FIG1 is a flow chart of a method for generating an automatic accompaniment based on real-time performance audio characteristics in one embodiment;

[0049] Figure 2 is an exemplary schematic diagram of a second neural network in one embodiment;

[0050] Figure 3 is an exemplary schematic diagram of a third neural network in one embodiment;

[0051] Figure 4 is an exemplary schematic diagram of a first neural network in one embodiment;

[0052] Figure 5 is a structural block diagram of an automatic accompaniment generating device for real-time performance audio characteristics in one embodiment;

[0053] Figure 6 Schematic diagram of the structure of a guitar effector in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for explaining this application and are not intended to limit this application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application without creative work are within the scope of protection of this application.

[0055] In addition, the terms "first," "second," and so on, used in this application are for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "multiple groups" means at least two groups, such as two groups, three groups, and so on, unless otherwise specifically defined.

[0056] In this application, unless otherwise specified or limited, the terms "connect," "fix," etc. should be understood in a broad sense. For example, "fix" can mean a fixed connection, a detachable connection, or an integral connection; it can mean a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean internal communication between two elements or an interaction between two elements, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0057] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0058] This application provides a method for generating an automatic accompaniment of a real-time performance audio characteristic, such as Figure 1 The flowchart shown, in one embodiment, includes:

[0059] Step 102: Acquire a real-time audio sequence.

[0060] In this step, how to obtain a real-time audio sequence belongs to the existing technology and will not be described in detail here.

[0061] Step 104: divide the audio sequence into multiple signal samples, perform energy calculation on the multiple signal samples, and obtain audio energy of the audio sequence.

[0062] Specifically: the audio sequence is divided into multiple signal samples, the energy of all signal samples is calculated to obtain the sequence energy; the audio sequence is divided into multiple signal frequency bands, the energy of each signal frequency band is calculated to obtain the sub-band energy; and the audio energy of the audio sequence is obtained based on the sequence energy and the multiple sub-band energies.

[0063] Preferably, the method further includes: performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence; training a second neural network based on multiple maximum points of the cepstrum sequence to obtain pitch probabilities; and obtaining polyphonic pitches based on the pitch probabilities.

[0064] in:

[0065] Perform cepstrum analysis on the audio sequence to obtain the cepstrum sequence, including: performing Fourier transform, modulus calculation, logarithm calculation and inverse Fourier transform on the audio sequence in sequence to obtain the cepstrum sequence:

[0066] X=F -1 {log|F(x)}

[0067] Where X is the cepstrum sequence, F -1 is the inverse Fourier transform, log is the logarithm, |·| is the modulus, and F(x) is the Fourier transform.

[0068] According to the multiple maximum points of the cepstrum sequence, a second neural network is trained to obtain the pitch probability, including: according to the cepstrum sequence, multiple maximum points are selected as the input of the second neural network, the second neural network is trained, and the output of the second neural network is used as the pitch probability (the pitch probability refers to the probability that the maximum point is the pitch). For example: for an audio sequence with a length of 1024 samples, a cepstrum analysis is performed, and the first 20 maximum points of the cepstrum sequence are taken. The x and y coordinates of these 20 maximum points are combined into a 40*1 vector as the second neural network (such as Figure 2 The RNN recurrent neural network shown in the figure takes as input 40 feature quantities, passes through two layers of hidden GRU layers with 40 nodes, and obtains a 20*1 output vector. This output vector is the output of the second neural network, which indicates the probability that these 20 maximum points are pitches respectively. After training the second neural network with a large amount of data, a polyphonic detection algorithm can be obtained.

[0069] Determining the polyphonic pitch based on the pitch probability involves: determining the pitch probability based on the maximum point where the pitch probability satisfies a preset condition as the pitch; and traversing all the maximum points to obtain the polyphonic pitch. For example, if the preset condition is greater than 0.5, then each maximum point where the pitch probability is greater than 0.5 is a real polyphonic pitch.

[0070] Further preferably, the method further includes: determining, based on the audio energy, by a third neural network, to obtain a probability of a sound start; and obtaining the number of sound starts based on the probability of the sound start.

[0071] in:

[0072] The third neural network can be a 1->4->4->1 GRU cyclic neural network, such as Figure 3 shown.

[0073] The judgment is made based on the probability of the note head. When the probability of the note head meets the preset condition (i.e., greater than 0.5), it means that the current is the note head of a sound. Otherwise, it is not. The number of note heads is further obtained; the number of note heads detected per second is the note density.

[0074] Further preferably, the audio sequence is divided into multiple signal samples, and energy calculation is performed on the multiple signal samples to obtain the audio energy of the audio sequence, including: dividing the audio sequence into multiple signal samples, calculating the sum of squares of the multiple signal samples, and obtaining sequence energy; using multiple band filters to divide the audio sequence into multiple signal frequency bands, calculating the energy of each signal frequency band, and obtaining multiple band energies; and obtaining the audio energy of the audio sequence based on the sequence energy and the band energy.

[0075] in:

[0076]

[0077] Where E is the sequence energy, x i is the i-th signal sample, and N is the length of the audio sequence.

[0078] Several band-splitting filters are used with frequencies of 100, 400, 1000, and 4000 Hz to divide the audio sequence into five signal frequency bands. The energy of each signal frequency band is then calculated to obtain the five band energies.

[0079] In this step, first, the audio energy of the audio sequence is obtained through the performer's performance audio; further, the real-time polyphonic pitch is obtained; further, the real-time number of sound heads is obtained.

[0080] Step 106: Based on the audio energy, the first neural network is used to determine the performance audio characteristics and obtain real-time accompaniment input.

[0081] Preferably, the audio characteristics of the performance are judged by the first neural network based on the audio energy to obtain real-time accompaniment input, including: based on the audio energy, pitch probability and polyphonic pitch, the audio characteristics of the performance are judged by the first neural network to obtain real-time accompaniment input.

[0082] Further preferably, the audio characteristics of the performance are judged by the first neural network based on the audio energy to obtain real-time accompaniment input, including: judging the audio characteristics of the performance by the first neural network based on the audio energy, pitch probability, polyphonic pitch and number of note heads to obtain real-time accompaniment input.

[0083] More preferably, the audio characteristics of the performance are judged by the first neural network according to the audio energy to obtain the real-time accompaniment input, including: judging the audio characteristics of the performance by the first neural network according to the sequence energy, the band energy, the pitch probability, the polyphonic pitch and the number of sound heads to obtain the real-time accompaniment input. For example: the sequence energy, the band energy, the three largest pitch probabilities, the three pitches with the largest pitch probability and the number of sound heads in the past 2 seconds are combined into a 13*1 matrix as the first neural network (such as Figure 4 As shown, the input of the neural network (13->13->13->1) is judged by the first neural network to obtain the performance audio characteristics, and the performance audio characteristics are used as the accompaniment input. For real-time audio sequences, real-time accompaniment input can be obtained.

[0084] In this step, first, the performance audio characteristics are judged based on the audio energy; further, the performance audio characteristics are judged based on the audio energy, pitch probability and polyphonic pitch; further, the performance audio characteristics are judged based on the audio energy, pitch probability, polyphonic pitch and the number of note heads; further, the performance audio characteristics are judged based on the sequence energy, band energy, pitch probability, polyphonic pitch and the number of note heads.

[0085] It should be noted that the accompaniment input is a number. The closer it is to 1, the higher the performance audio characteristics are, and the closer it is to 0, the calmer the performance audio characteristics are.

[0086] Step 108: Generate automatic accompaniment in real time according to the real-time accompaniment input.

[0087] Specifically: based on the real-time accompaniment input, different repeat segments are produced for each style of accompaniment, and the repeat segments correspond to different performance audio characteristics; for each repeat segment, a corresponding interlude is produced; for the ending measure of the music segment, when it is judged that its performance audio characteristics are inconsistent with the repeat segment belonging to the current paragraph, an interlude is used to migrate, switch to the corresponding performance audio characteristics, and generate automatic accompaniment in real time.

[0088] For example: based on the real-time accompaniment input, for each style of accompaniment, 4 different repetitions are produced, ranging from calm performance audio characteristics to passionate performance audio characteristics, corresponding to performance audio characteristics of 0~0.25, 0.25~0.5, 0.5~0.75, and 0.75~1 respectively; for each repetition, a corresponding interlude is produced; for the ending measure of each musical section (generally 8 measures, or a whole chord cycle), its performance audio characteristics are judged. When the performance audio characteristics are inconsistent with the repetition to which the current paragraph belongs, an interlude is used to migrate the performance audio characteristics, that is, after using a interlude, switch to the corresponding performance audio characteristics to generate automatic accompaniment in real time.

[0089] In this step, the generated accompaniment content is dynamically adjusted in real time according to the audio characteristics of the performance, thereby realizing automatic accompaniment that interacts with the performer in real time.

[0090] The above-mentioned method for generating automatic accompaniment based on the audio characteristics of real-time performance obtains the judgment of the audio characteristics of real-time performance after extracting and analyzing the musical characteristics of the performance audio. Specifically, the sequence energy and band energy represent the intensity of the audio, the pitch probability and polyphonic pitch represent the melody direction of the audio, and the number of note heads represents the note density of the audio. The intensity, note density and melody direction represent the musical characteristics. Therefore, according to the sequence energy, band energy, pitch probability, polyphonic pitch and number of note heads, the performance audio characteristics are judged, the real-time accompaniment input is obtained, and the automatic accompaniment is generated in real time. This application is not only based on music theory and algorithm to generate accompaniment, but also can perceive and respond to changes in the performance audio characteristics of the performance in real time, providing a more personalized and expressive music experience, and can fill the gap in the interaction between automatic accompaniment and real bands and performers, and better synchronize with the performer's expression. It has broad application prospects in live performances, music teaching, recording and personal practice.

[0091] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0092] The present application also provides an automatic accompaniment generating device with real-time performance audio characteristics, such as Figure 5As shown, in one embodiment, it includes: an acquisition module 502, a calculation module 504, a judgment module 506 and an accompaniment module 508, wherein:

[0093] An acquisition module 502 is used to acquire a real-time audio sequence;

[0094] A calculation module 504 is configured to divide the audio sequence into a plurality of signal samples, perform energy calculation on the plurality of signal samples, and obtain audio energy of the audio sequence;

[0095] A judgment module 506 is configured to determine the performance audio characteristics based on the audio energy through a first neural network to obtain a real-time accompaniment input;

[0096] The accompaniment module 508 is used to generate automatic accompaniment in real time according to the real-time accompaniment input.

[0097] The specific limitation of the automatic accompaniment generating device for playing audio characteristics in real time can be found in the limitation of the automatic accompaniment generating method for playing audio characteristics in real time above, which will not be repeated here. Each module in the above-mentioned device can be realized in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the corresponding operation of each of the above modules.

[0098] This application also provides an effector, such as Figure 6 As shown, in one embodiment, it includes: an automatic accompaniment generating device with real-time performance audio characteristics, an audio input port, an audio output port and an accompaniment switch.

[0099] The audio input port is connected to the instrument to obtain real-time audio sequences.

[0100] The audio input port, the audio output port and the accompaniment switch are all connected to an automatic accompaniment generating device with real-time performance audio characteristics, so that after receiving the accompaniment generation signal from the accompaniment switch, the automatic accompaniment generating device with real-time performance audio characteristics generates an automatic accompaniment in real time according to the real-time audio sequence input by the audio input port and outputs it through the audio output port.

[0101] The audio input port is also connected to the audio output port so as to output the real-time audio sequence of the musical instrument from the audio output port.

[0102] For example: Figure 6The real-time accompaniment guitar effects pedal shown first receives the guitar playing signal (i.e., audio sequence) as an input signal from the audio input port; then, it analyzes the performance audio characteristics based on the input signal, uses the analysis results as the accompaniment input, and controls the accompaniment unit built into the effects pedal; finally, the automatic accompaniment generated in real time by the accompaniment unit is mixed with the guitar input signal and output together from the audio output port.

[0103] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0104] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for generating an automatic accompaniment based on real-time performance audio characteristics, characterized in that: include: Get real-time audio sequence; Dividing the audio sequence into multiple signal samples, performing energy calculation on the multiple signal samples, and obtaining audio energy of the audio sequence; Based on the audio energy, a first neural network is used to determine the performance audio characteristics to obtain real-time accompaniment input; Generate automatic accompaniment in real time based on real-time accompaniment input; Also includes: Performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence; training a second neural network based on multiple maximum points of the cepstrum sequence to obtain pitch probabilities; and obtaining polyphonic pitches based on the pitch probabilities; According to the audio energy, the pitch probability and the polyphonic pitch, a first neural network is used to determine the performance audio characteristics to obtain a real-time accompaniment input; Also includes: According to the audio energy, a third neural network is used to determine the probability of the sound start; and according to the sound start probability, the number of the sound start is determined; According to the audio energy, the pitch probability, the polyphonic pitch and the number of note heads, a performance audio characteristic is judged by a first neural network to obtain a real-time accompaniment input.

2. The method for generating an automatic accompaniment based on the real-time performance audio characteristics according to claim 1, wherein: Dividing the audio sequence into a plurality of signal samples, and performing energy calculation on the plurality of signal samples to obtain audio energy of the audio sequence, comprising: Dividing the audio sequence into multiple signal samples, calculating the sum of squares of the multiple signal samples, and obtaining sequence energy; Using multiple band-dividing filters to divide the audio sequence into multiple signal frequency bands, calculating the energy of each signal frequency band to obtain multiple band-dividing energies; The audio energy of the audio sequence is obtained according to the sequence energy and the sub-band energy.

3. The method for generating an automatic accompaniment based on the real-time performance audio characteristics according to claim 2, wherein: According to the audio energy, a first neural network is used to determine the performance audio characteristics to obtain real-time accompaniment input, including: According to the sequence energy, the band energy, the pitch probability, the polyphonic pitch and the number of note heads, a performance audio characteristic judgment is performed through a first neural network to obtain a real-time accompaniment input.

4. The method for generating automatic accompaniment based on real-time performance audio characteristics according to any one of claims 1 to 3, characterized in that: Performing cepstrum analysis on the audio sequence to obtain a cepstrum sequence, including: The audio sequence is subjected to Fourier transform, modulus calculation, logarithm calculation and inverse Fourier transform in sequence to obtain a cepstrum sequence.

5. The method for generating automatic accompaniment based on real-time performance audio characteristics according to any one of claims 1 to 3, characterized in that: According to the pitch probability, the polyphonic pitch is obtained, including: Judge based on pitch probability, and take the maximum point where the pitch probability meets the preset conditions as a pitch; Traverse all the maximum points to obtain the polyphonic pitch.

6. The method for generating automatic accompaniment based on real-time performance audio characteristics according to any one of claims 1 to 3, characterized in that: Generate automatic accompaniment in real time based on real-time accompaniment input, including: According to the real-time accompaniment input, different repeating sections are produced for each style of accompaniment, and the repeating sections correspond to different performance audio characteristics; and corresponding fills are produced for each repeating section; For the ending measure of the music segment, when it is judged that its performance audio characteristics are inconsistent with the repeated segment belonging to the current paragraph, a transition is used to migrate, switch to the corresponding performance audio characteristics, and generate automatic accompaniment in real time.

7. An automatic accompaniment generating device with real-time performance audio characteristics, characterized in that: The method for generating automatic accompaniment based on the real-time performance audio characteristics according to any one of claims 1 to 6 comprises: Acquisition module, used to obtain real-time audio sequence; a calculation module, configured to divide the audio sequence into a plurality of signal samples, perform energy calculation on the plurality of signal samples, and obtain audio energy of the audio sequence; a judgment module, configured to judge the performance audio characteristics through a first neural network based on the audio energy, and obtain real-time accompaniment input; The accompaniment module is used to generate automatic accompaniment in real time according to real-time accompaniment input.

8. Effector, characterized in that, The method for generating automatic accompaniment based on the audio characteristics of real-time performance according to any one of claims 1 to 6 comprises: an automatic accompaniment generating device based on the audio characteristics of real-time performance, an audio input port, an audio output port, and an accompaniment switch; The audio input port is connected to a musical instrument to obtain a real-time audio sequence; The audio input port, the audio output port, and the accompaniment switch are all connected to the automatic accompaniment generating device with real-time performance audio characteristics, so that after receiving the accompaniment generation signal from the accompaniment switch, the automatic accompaniment generating device with real-time performance audio characteristics generates an automatic accompaniment in real time according to the real-time audio sequence input from the audio input port and outputs it through the audio output port; The audio input port is also connected to the audio output port so that the real-time audio sequence of the musical instrument can be output from the audio output port.

Images