Musical instrument accompaniment track playing control method and device, equipment and medium

Abstract

The invention relates to the technical field of musical instrument intelligent control, and discloses a musical instrument accompaniment track playing control method, device, equipment and medium, and the method comprises the steps: collecting temporary playing audio data of a user through a sound collection unit in real time or at regular time after receiving an accompaniment track acoustic feature intelligent control instruction sent by the user; performing acoustic feature analysis on the collected temporary playing audio data to obtain multi-dimensional acoustic features, and analyzing playing parameters of an audio playing unit according to the obtained multi-dimensional acoustic features; and after a formal playing start instruction sent by the user is received, the audio playing unit is controlled to play the accompaniment track based on the analyzed playing parameters. The accuracy of accompaniment track playing control and the adaptability of accompaniment track playing matching in the fixed mode are improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a method, device, equipment and medium for controlling the playback of musical instrument accompaniment tracks. Background Art

[0002] With the development of artificial intelligence, sensor technology and audio processing technology, the music field is undergoing unprecedented intelligent changes, providing key support for the transformation of the music field from static pre-recording to dynamic intelligence. In order to make the accompaniment tracks more in line with user needs and improve the user experience, it is necessary to accurately select and control the accompaniment tracks.

[0003] The existing instrument accompaniment track playback control technology adopts a plurality of pre-set fixed modes to control the playback of accompaniment tracks, and the user selects the preset accompaniment tracks according to needs. Therefore, it is unable to cope with real-time performance changes, resulting in insufficient accuracy of the accompaniment track playback control in the fixed mode and insufficient adaptability of the accompaniment track playback matching. Summary of the Invention

[0004] The present invention provides a method, device, equipment and medium for controlling the playback of musical instrument accompaniment tracks, so as to solve the technical problems of insufficient accuracy of accompaniment track playback control in a fixed mode and insufficient adaptability of accompaniment track playback matching.

[0005] In a first aspect, a method for controlling the playback of an instrument accompaniment track is provided, comprising: After receiving the intelligent control instruction of the acoustic characteristics of the accompaniment track from the user, the user's temporary performance audio data is collected in real time or at a fixed time through the sound collection unit; Performing acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyzing the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; After receiving the performance start instruction from the user, the audio playback unit is controlled to play the accompaniment track based on the analyzed playback parameters.

[0006] In a second aspect, a device for controlling the playback of an instrumental accompaniment track is provided, comprising: The temporary performance audio data acquisition module is used to collect the user's temporary performance audio data in real time or at a fixed time through the sound acquisition unit after receiving the accompaniment track acoustic feature intelligent control instruction issued by the user; A playback parameter analysis module is used to perform acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyze the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; The accompaniment track playing module is used to control the audio playing unit to play the accompaniment track based on the analyzed playing parameters after receiving the performance formal start instruction issued by the user.

[0007] In a third aspect, a musical instrument device is provided, comprising a memory, a processor, and an executable program stored in the memory and runnable on the processor, wherein the processor implements the steps of the above-mentioned method for controlling the playback of musical instrument accompaniment tracks when executing the executable program.

[0008] In a fourth aspect, a processor-readable storage medium is provided, wherein the processor-readable storage medium stores an executable program, and when the executable program is executed by the processor, the steps of the above-mentioned instrument accompaniment track playback control method are implemented.

[0009] In the scheme implemented by the above-mentioned instrument accompaniment track playback control method, device, equipment and medium, after receiving the accompaniment track acoustic feature intelligent control instruction issued by the user, the client can collect the user's temporary performance audio data through the sound collection unit in real time or at a fixed time; perform acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyze the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; after receiving the performance formal start instruction issued by the user, control the audio playback unit to play the accompaniment track based on the analyzed playback parameters, and feed the played accompaniment track back to the client. In the present invention, the audio data of the user's temporary performance is obtained in real time by the sound collection unit, and its multi-dimensional acoustic features are analyzed. The system automatically analyzes the performance parameters of the accompaniment track and generates matching playback parameters. Users can fully debug the accompaniment parameters during the trial performance and start the formal playback after confirming that they are correct. This avoids performance interruptions caused by system misjudgment and improves operational reliability, thereby solving the technical problems of insufficient accuracy of accompaniment track playback control and insufficient adaptability of accompaniment track playback matching in fixed modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0011] Figure 12. It is a schematic diagram of an application environment of a method for controlling the playback of musical instrument accompaniment tracks according to an embodiment of the present invention; Figure 2 This is a flow chart of a method for controlling the playback of musical instrument accompaniment tracks according to one embodiment of the present invention; Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S2; Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S2; Figure 5 1 is a schematic structural diagram of a device for controlling the playback of musical instrument accompaniment tracks according to an embodiment of the present invention; Figure 6 is a structural diagram of a musical instrument device according to an embodiment of the present invention; Figure 7 2 is another structural diagram of a musical instrument device in one embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0013] The instrument accompaniment track playback control method provided by the embodiment of the present invention can be applied to Figure 1In an application environment, the client communicates with the server through a network. After receiving the accompaniment track acoustic feature intelligent control instruction issued by the user through the client, the server can collect the user's temporary performance audio data through the sound collection unit in real time or at a fixed time; perform acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyze the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; after receiving the performance formal start instruction issued by the user, control the audio playback unit to play the accompaniment track based on the analyzed playback parameters, and feed back the played accompaniment track to the client. In the present invention, the audio data of the user's temporary performance is obtained in real time through the sound collection unit, its multi-dimensional acoustic features are analyzed, and the audio playback unit is dynamically generated. Personalized playback parameters enable adaptive adjustment of accompaniment parameters through acoustic feature analysis, ensuring real-time synchronization between accompaniment and performance. This is particularly suitable for improvisation, live performances, or teaching scenarios, enabling precise characterization of performance styles and avoiding the one-sidedness of traditional single-dimensional adjustment. Users do not need to be familiar with the accompaniment track parameters in advance; simply using the temporary performance audio can trigger the system to automatically analyze and generate matching playback parameters. Users can fully adjust the accompaniment parameters during the trial stage and only start the actual playback after confirming that they are correct. This avoids performance interruptions caused by system misjudgments and improves operational reliability, thereby resolving the technical issues of insufficient accuracy and adaptability of accompaniment track playback control in fixed modes. The client can be, but is not limited to, various intelligent musical instrument devices (e.g., pianos, electric wind instruments, etc.), personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented as a standalone server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0014] See also Figure 2 As shown, Figure 2 A flow chart of a method for controlling the playback of an instrumental accompaniment track provided by an embodiment of the present invention includes the following steps: S1. After receiving the intelligent control instruction of the acoustic characteristics of the accompaniment track sent by the user, the user's temporary performance audio data is collected in real time or periodically through the sound collection unit.

[0015] In an embodiment of the present invention, the intelligent control instruction of the acoustic characteristics of the accompaniment track can be actively issued by the user through the preset physical controls on the instrument or the preset touch controls on the human-computer interaction operation interface displayed on the instrument display unit, or the instruction can be issued by default after the accompaniment track starts.

[0016] In detail, the sound collection units corresponding to different musical instruments can be the same or different. For example, a piano can be equipped with a dynamic microphone, a condenser microphone, an electromagnetic microphone and other sound collection units, and an electric blowpipe can be equipped with a pickup, an audio collector and other sound collection units, and then the user's temporary performance audio data is collected through the sound collection unit.

[0017] In an embodiment of the present invention, the temporary performance audio data refers to a non-prerecorded audio signal generated by the user through real-time performance of an instrument after receiving an intelligent control instruction for the acoustic characteristics of the accompaniment track.

[0018] In the embodiment of the present invention, the real-time or periodic collection of the user's temporary performance audio data by the sound collection unit includes: Triggering data collection instructions through the sound collection unit; Collecting the analog electrical signal output by the sound collection unit in real time or periodically according to the data collection instruction; The analog electric signal is converted into user's temporary performance audio data.

[0019] In detail, the user actively triggers the physical buttons (such as piano pedals) or touch interface (such as the buttons on the electronic blowphone display) of the instrument to send data collection instructions, or after the accompaniment track is started, the collection function is automatically triggered, and then the analog electrical signal ADC output by the collection unit is converted into digital audio data, and the original signal is pre-processed such as noise reduction and equalization to ensure the accuracy of the collected data.

[0020] For example, if the sound collection unit of a piano is a dynamic microphone or a condenser microphone, it captures the sound waves generated by the vibration of the strings and converts them into analog electrical signals through the diaphragm; if the sound collection unit of an electric blowpipe is a pickup or an audio collector, it directly obtains the electrical signals in the instrument circuit, the electrical signals triggered by the airflow sensor or the keys.

[0021] Furthermore, temporary performance audio data contains rich time domain, frequency domain and spatial domain information, such as the melody and rhythm of the user's improvisation. Therefore, the original audio signal needs to be decomposed into quantifiable and calculable parameters so that it can be understood and processed by the computer system.

[0022] S2. Performing acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyzing the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features.

[0023] In the embodiment of the present invention, the multi-dimensional acoustic features include pitch, audio, beat, etc. in the temporary performance audio data.

[0024] In the embodiment of the present invention, the acoustic feature analysis of the collected temporary performance audio data to obtain multi-dimensional acoustic features includes: Converting the time domain signal of the collected temporary performance audio data into a frequency domain spectrum; extracting the pitch features in the temporary performance audio data according to the frequency domain spectrum; Calculating the short-time energy and zero-crossing rate in the collected temporary performance audio data; determining a volume feature in the temporary performance audio data according to the short-time energy, and determining a rhythm intensity feature in the temporary performance audio data according to the zero-crossing rate; The pitch feature, the volume feature, and the rhythm intensity feature are fused into a multi-dimensional acoustic feature.

[0025] In detail, audio signals are one-dimensional time-domain signals that change over time (such as voltage fluctuations collected by a microphone). However, the pitch, timbre and other characteristics of music are implicit in the frequency components. Therefore, the time-domain signal is divided into short time windows (such as 20-50 milliseconds) through short-time Fourier transform (STFT). The signal in each window is Fourier transformed to decompose it into a superposition of sine waves of different frequencies to obtain a frequency domain spectrum. That is, the horizontal axis of the spectrum is frequency (Hz), and the vertical axis is the amplitude (energy size) of each frequency component, reflecting the frequency composition of the signal. For example, the fundamental frequency of the piano C4 note is 261.6Hz. There will be a significant peak at this frequency point in the spectrum, and its harmonics (such as 523.2Hz, 784.8Hz, etc.) will also show energy distribution at the corresponding frequency points.

[0026] Specifically, the fundamental frequency determines the pitch (such as do / re / mi), the harmonics determine the timbre (such as the difference between piano and violin), and the spectrum provides the data basis for subsequent tone extraction. Through continuous short window transformation, a time-frequency diagram can be generated to observe the changes in pitch over time (such as glissando and vibrato).

[0027] In an embodiment of the present invention, the pitch feature refers to the pitch information contained in the temporary performance audio data, that is, the specific note name or the corresponding MIDI note number (such as 60, 64), which reflects the pitch combination used by the performer in the melody line and is the core element of the musical tonality and melody direction.

[0028] In the embodiment of the present invention, referring to Figure 3 As shown, the step of extracting the pitch features in the temporary performance audio data according to the frequency domain spectrum includes: S31, analyzing the fundamental frequency period of the temporary performance audio data according to the frequency domain spectrum; S32, mapping the fundamental frequency value in the fundamental frequency period to a pitch system in a preset music theory; S33. Determine the pitch characteristics of the temporary performance audio data according to the mapped pitch system.

[0029] In detail, the fundamental frequency with the strongest energy is found from the spectrum, corresponding to the note name in music, such as C4 and G#5, and the correlation of the signal under different time delays is calculated. The inverse of the delay time corresponding to the peak is the fundamental frequency, or the logarithm of the spectrum is taken and then Fourier transform is performed to separate the fundamental frequency and the formant, and the fundamental frequency value is mapped to the standard note name based on the twelve-tone equal temperament corresponding to the pitch system in music theory, that is, the frequency ratio of the fundamental frequency value to the reference tone is calculated, and the number of semitones is determined by logarithmic operation. The note name is determined according to the number of semitones and the octave number. The human auditory threshold for pitch discrimination is about 5~10Hz (low frequency is sensitive, high frequency is slightly worse), so the fundamental frequency value is allowed to be mapped to the same note name within ±5Hz of the theoretical value. The specific note name or the corresponding MIDI note number is determined as the high-pitched information contained in the temporary performance audio data.

[0030] Specifically, short-time energy is a feature that measures the volume. The short-time energy is calculated by summing the squares of the sample values ​​in a short window of the time domain signal, which reflects the energy intensity of the signal and is positively correlated with the volume. It can distinguish between strong performances (such as piano forte) and soft performances (piano). When the volume changes suddenly (such as drum beats), the energy value jumps significantly, and it can detect silent segments and sound segments for audio segmentation or silence suppression. The zero-crossing rate is a measure of the rate of change of the signal. The calculated zero-crossing rate is the number of times the signal passes through the zero axis per unit time, reflecting the complexity of the waveform. The more high-frequency components, the higher the zero-crossing rate, which can distinguish between clear and voiced sounds. The zero-crossing rate of percussion instruments (such as drums) is much higher than that of long notes (such as violin), and fast notes or complex rhythmic patterns correspond to higher zero-crossing rate fluctuations, such as the legato passages of jazz drums.

[0031] Furthermore, feature fusion is performed based on different feature dimensions, and different acoustic features are vector-joined to obtain multi-dimensional acoustic features. The performance style (such as the delicacy of classical music and the intensity of rock music) is then judged by combining pitch jumps (tone), dynamics changes (volume), and note speed (zero-crossing rate).

[0032] Furthermore, the acoustic characteristics of the audio signal are determined by the physical parameters and working status of the playback device. By reverse analyzing the characteristics, the playback parameters of the instrument can be inferred for optimizing sound quality or automated control.

[0033] In embodiments of the present invention, an audio playback unit is a physical device unit that converts electrical signals into sound signals and is the end effector of an audio system. For example, the playback unit structure of a violin is a curved wooden resonance box and F-holes, and the playback unit structure of a classical guitar is a cast iron frame and a wooden soundboard. That is, through physical mechanisms such as electromagnetic induction (moving coil type) and the piezoelectric effect, the electrical parameters of the audio signal (such as voltage and frequency) are converted into air vibrations (sound waves). Its hardware characteristics (such as frequency response range and distortion rate) directly determine the sound quality of the reproduced sound. The playback parameters refer to physical or electronic parameters that control the sound state of the audio playback unit and directly affect the acoustic characteristics of the sound. They include but are not limited to sampling rate, bit rate, number of channels, master volume, volume balance of each frequency band, playback speed, rhythm synchronization signal, reverberation intensity, delay time, start and stop time of each audio track in the audio unit, and volume ratio. In the embodiment of the present invention, referring to Figure 4 As shown, the analysis of the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features includes: S41, associating the multi-dimensional acoustic features with execution parameters in the audio playback unit; S42: determining a playback model of the audio playback unit according to the associated execution parameters and playback structure characteristics of the audio playback units corresponding to different musical instruments; S43: Analyze the playback parameters of the audio playback unit using the playback model.

[0034] In detail, based on the tone, volume and beat parameters in the multidimensional acoustic features, the pitch parameters, volume gain and compressor threshold, playback speed and rhythm synchronization switch in the audio playback unit execution parameters are respectively associated, and the combination of tone and beat can be associated with the arpeggio mode parameters in the audio playback unit execution parameters. For example, if F0=440Hz is detected, the synthesizer oscillator frequency is set to 440Hz; when the volume suddenly increases, the compressor is triggered with a ratio of 4:1 and a threshold of -10dB. When BPM=120 is detected, the accompaniment speed is synchronously set to 120BPM, or arpeggio rhythm is automatically generated when playing fast scales, such as the 16th note mode. The multidimensional acoustic features can be associated with the execution through dynamic thresholds. For example, when the volume RMS exceeds -10dB, the pre-delay time of the reverberation effect is automatically shortened from 20ms to 10ms to enhance the sound clarity.

[0035] Specifically, the audio playback units of different musical instruments respond differently to acoustic characteristics, necessitating the establishment of instrument-specific models. For example, the playback structural characteristics of string instruments are bow friction / plucking and resonance box vibration, the playback structural characteristics of wind instruments are air column resonance and fingering hole control, and the playback structural characteristics of electronic synthesizers are DSP algorithms and digital oscillators. Different playback models are thus constructed. Taking the piano as an example, the physical model is based on the string vibration equation, and the data model collects the acoustic characteristics of the 88 keys at different velocities, such as attack time and harmonic decay rate, and constructs a lookup table. The playback model then receives real-time pitch (F0), volume (keystroke velocity), and beat (legato / staccato mode), and can output the string length simulation parameters corresponding to the pitch, such as L=0.64m for the middle C key, the hammer velocity parameters corresponding to the volume, such as a velocity of 127 corresponding to a velocity of 8m / s, and the pedal control corresponding to the beat, such as the sustain pedal being on for legato and off for staccato.

[0036] In addition, the instrument model is cross-instrument compatible and applicable to similar instruments. For example, all woodwind instruments share the breath-pitch correlation model. For special instruments, such as the celesta, the high-frequency harmonic distribution of its metal plate vibration needs to be modeled separately.

[0037] For example, the user plays the saxophone through the microphone, and the system analyzes its pitch, volume, and beat in real time, and controls the synthesizer to play an adapted piano accompaniment. It detects that the current note F0 is 293.66Hz, D4 tone, and RMS is -15dB, which is judged to be medium-forte and recognized as 4 / 4 beat. The current beat is the second beat of the first measure (non-accented position), and the instrument is identified as saxophone (woodwind). The general model of woodwind instruments is called, and the accompaniment instrument is piano. The piano playback model is called, and chords are generated based on the D4 tone of the saxophone. According to the RMS of the saxophone being -15dB, the piano gain is set to -12dB, the accompaniment volume is slightly lower than the main instrument, synchronized with the 4 / 4 beat, and the bass strength is enhanced (+3dB) at the accented position (first beat). The parameters are then sent to the audio interface to drive the synthesizer to play the accompaniment according to the calculated results.

[0038] S3. After receiving the performance start instruction from the user, the audio playback unit is controlled to play the accompaniment track based on the analyzed playback parameters.

[0039] In an embodiment of the present invention, the formal start instruction of the performance can be actively issued by the user through the preset physical controls on the musical instrument or the preset touch controls on the human-computer interaction operation interface displayed on the musical instrument display unit, or it can be issued by default after a preset time after the temporary performance audio data stops. For example, the accompaniment track will be played by default for the formal performance 5 seconds after the temporary performance ends.

[0040] In an embodiment of the present invention, controlling the audio playback unit to play the accompaniment track based on the analyzed playback parameters includes: The analyzed playback parameters are stored in segments according to a preset time axis, and playback control instructions of the audio playback unit are generated according to the segmented and stored playback parameters; Dividing the playing time periods of the accompaniment tracks according to the structure of the accompaniment tracks, and loading the playing control instructions corresponding to the playing time periods into the register of the audio playing unit; parsing the playback control instructions in the register of the audio playback unit according to the time sequence of the playback time period; The audio playback unit is controlled to play the accompaniment track according to the parsed playback control instruction.

[0041] In detail, according to the structure or beat cycle of the accompaniment track (such as each measure is a segment), the playback parameters are divided into multiple time segments. For example, a song with a 4 / 4 beat and 120BPM has a measure length of 0.5 seconds, which is divided into 32 segments, 8 measures × 4 segments. A key-value pair array or a time series database is used to store the parameters of each segment. Each segment parameter includes a timestamp, pitch, volume, rhythm, etc., and then the stored parameters are converted into control instructions recognizable by the audio playback unit, such as MIDI messages, OSC instructions, and USB audio commands. For example, if the pitch is Cmaj7, then the MIDI chord instructions, notes 60-C, 64-E, and 67-G are triggered simultaneously, then the instructions are serialized into binary or text format (such as JSON, Protobuf) in timeline order to facilitate fast loading into the playback unit register.

[0042] Specifically, audio analysis algorithms such as beat detection and chord change detection are used to automatically identify the structural segments of the accompaniment track, that is, dynamic time warping (DTW) is used to match the structural template of the pre-recorded accompaniment, or a spectral clustering algorithm is used to divide the segments according to the spectral similarity, and then the playback parameters stored in the segments are matched one by one with the accompaniment time periods to form a mapping table of time period → parameter segment. The audio playback unit, such as the sound card or synthesizer, has an independent register area in its memory for storing the control instructions for each time period. For example, an FPGA audio processing board is provided with 32 register groups, each group corresponding to the parameters of a time period. The corresponding parameters can be loaded into the register 50ms before the start of the current time period, or all time period parameters can be stored in the register in advance. , the system generates a synchronization signal through the hardware clock or software clock. When the system clock reaches the start time of a certain time period, it triggers the register read operation and parses the specific playback parameters in the playback control instruction. The audio stream parameters are adjusted according to the parsed instructions, that is, the synthesizer generates the corresponding pitch by changing the oscillator frequency for tone control, the digital attenuator adjusts the signal amplitude for volume control, and the clock divider generates a trigger signal corresponding to BPM for rhythm control. During the playback process, if a new control instruction is received (such as the user temporarily adjusts the volume), the current execution process is immediately interrupted, the new parameters are loaded and re-parsed. If the instruction for a certain time period is not loaded in time, the parameters of the previous section are automatically used until the end of the current section to avoid silence or erroneous sound.

[0043] In an embodiment of the present invention, the step of generating a playback control instruction of the audio playback unit according to the segmented and stored playback parameters includes: Split the playback parameters after segmented storage into independent parameters corresponding to each track in the accompaniment track; Structuring the independent parameters to obtain structured parameters; The structured parameters are converted into playback control instructions of the audio playback unit according to a preset instruction template.

[0044] In detail, the overall playback parameters of each time period are split into independent parameters for each track, forming a three-level structure of time period → track → parameter, and a nested data structure is used to store the track parameters. A command template is defined for the parameter type of each track, and the mapping relationship between the parameter and the playback unit control interface is clarified. For example, if the track type is a guitar track, the parameter type is a chord pitch, the command format is [TRK=01, CMD=SET_PITCH, VAL={pitch}], and the hardware control interface is the oscillator frequency register; if the track type is a drum track, the parameter type is a rhythm pattern, the command format is [TRK=02, CMD=SET_RHYTHM, VAL={pattern}], and the hardware control interface is the rhythm generator mode selection; if the track type is a bass track, the parameter type is a low-frequency gain, the command format is [TRK=03, CMD=SET_EQ, BAND=LOW, GAIN={gain}], the hardware control interface is the equalizer parameter register, where TRK is the track number, CMD is the command type, VAL is the parameter value, SET_PITCH means setting the pitch, and PITCH is the specific pitch value.

[0045] Specifically, the track parameters of each time period are traversed and corresponding instructions are generated according to the template. For example, when generating guitar track control instructions, the track parameters are pitch=Cmaj7, volume=-12dB, and reverb=0.8. Then the guitar track control instructions are [TRK=01,CMD=SET_PITCH, VAL=Cmaj7], [TRK=01, CMD=SET_VOLUME, VAL=-12dB], and [TRK=01, CMD=SET_REVERB, VAL=0.8s], where pitch is the pitch value, volume is the volume value, and reverb is the reverb value. Multiple instructions for the same track are sorted by priority, such as pitch adjustment takes precedence over volume adjustment to avoid conflicts. An execution time point is assigned to each instruction, usually the start time of the time period, to ensure that multi-track instructions are triggered synchronously.

[0046] In addition, if the parameters of the same audio track in adjacent time periods do not change, the instructions are merged or marked as using the previous value to reduce register load. For example, if a guitar track has a Cmaj7 chord in time periods 0-15 seconds and 15-30 seconds, only the SET_PITCH instruction is generated in the first time period and omitted in subsequent time periods. It is also possible to merge multiple related parameters of the same audio track into a compound instruction, such as an instruction to adjust the volume and effects at the same time, to reduce the number of instructions.

[0047] For example, in a digital audio workstation, an electronic dance music is produced, which includes three tracks: drum set, synth bass, and pad sound. The parameters of each track need to be switched according to the paragraph. The prelude is 0-30 seconds, drum track, rhythm type = 4 / 4 beat quarter note kick drum, volume = -15dB; bass track: single note C3, volume = -20dB, low frequency gain + 6dB; pad track: long note Cmaj7, volume = -18dB, reverberation time 2.0s, drum track instructions are [TRK=01, CMD=SET_RHYTHM,VAL=quarter_kick], [TRK=01, CMD=SET_VOLUME, VAL=-15dB]; bass track instructions: [TRK=02, CMD=SET_NOTE, VAL=C3], [TRK=02, CMD=SET_EQ, BAND=LOW, GAIN=+6dB]; pad The track instructions are [TRK=03, CMD=SET_CHORD,VAL=Cmaj7], [TRK=03, CMD=SET_REVERB,VAL=2.0s]. The instructions of each track are loaded into the corresponding register respectively and executed in parallel during playback to achieve the fade-in of the prelude and the synchronous start of each track.

[0048] Preferably, in order to achieve accurate matching of the accompaniment track and the performer, the method further comprises: Acquire and store the user's historical performance audio data and historical accompaniment track information; After receiving an accompaniment track intelligent recommendation instruction from a user, the system obtains the user's historical performance audio data and historical accompaniment track information, performs acoustic feature analysis on the obtained historical performance audio data to obtain multi-dimensional acoustic features, and performs genre feature analysis on the obtained historical accompaniment track information to obtain multi-dimensional genre features; Analyzing, based on a predetermined mapping relationship between optional accompaniment tracks and standard multidimensional acoustic features, a first list of optional accompaniment tracks corresponding to standard multidimensional acoustic features having an approximate distance to the obtained multidimensional acoustic features less than a first preset threshold; Analyzing, based on a predetermined mapping relationship between optional accompaniment tracks and standard multi-dimensional music style features, a second list of optional accompaniment tracks corresponding to standard multi-dimensional music style features whose approximate distance to the obtained multi-dimensional music style features is less than a second preset threshold; If the first optional accompaniment track list and the second optional accompaniment track list have common accompaniment tracks, the common accompaniment track will be pushed to the user as the selected accompaniment track for selection and playback, or, if the first optional accompaniment track list and the second optional accompaniment track list do not have common accompaniment tracks, all the optional accompaniment tracks in the first optional accompaniment track list and the second optional accompaniment track list will be pushed to the user for selection and playback, or, if the first optional accompaniment track list and the second optional accompaniment track list do not have common accompaniment tracks, the first optional accompaniment track in the first optional accompaniment track list corresponding to the standard multidimensional acoustic feature with the smallest approximate distance to the obtained multidimensional acoustic feature and / or the second optional accompaniment track in the second optional accompaniment track list corresponding to the standard multidimensional musical style feature with the smallest approximate distance to the obtained multidimensional musical style feature will be pushed to the user for selection and playback.

[0049] In detail, a memory can be configured on the musical instrument. This storage method is suitable for the case where the user of the musical instrument is a single fixed user; the acquired audio data and accompaniment track information can also be uploaded to a cloud server for storage. When uploaded to the cloud server for storage, the user's account can be mapped and stored with the target user's historical audio data and accompaniment track information. This storage method does not require the user of the musical instrument to be a single fixed user. As long as the user logs in to the cloud server account through the communication unit configured on the musical instrument, the musical instrument can obtain the corresponding historical performance audio data and accompaniment track information, where the historical performance audio data refers to the audio files and related parameter information recorded or generated by the user in the past performances, such as singing and playing musical instruments, reflecting the user's actual performance; the historical accompaniment track information includes the track name, track lyricist and composer, etc.

[0050] Specifically, the intelligent recommendation instruction for accompaniment tracks can be actively issued by the user through the preset physical controls on the instrument or the preset touch controls on the human-computer interaction operation interface displayed on the instrument display unit, or the instruction can be issued by default after the instrument is powered on. The acoustic characteristics of the performance data can be analyzed, and the pitch characteristics, rhythm characteristics, dynamic characteristics and timbre characteristics of the performance data can be analyzed through an acoustic feature algorithm, where the acoustic feature algorithm includes but is not limited to a short-time energy analysis algorithm and a fast Fourier transform algorithm. The historical accompaniment track information is analyzed for its style characteristics to obtain multi-dimensional style characteristics. The multi-dimensional style characteristics include melody, style (classical, modern, etc.) and other characteristics. Melody refers to the sequence of high and low, long and short pitches in music. Style is the comprehensive characteristics of music in terms of overall style, creative techniques, cultural background, etc., covering dimensions such as the era background, regional characteristics, performance form, aesthetic orientation, etc. of music. It is an important symbol to distinguish different music types. For example, classical style includes Baroque music, classical music, and romantic music; modern style includes pop music, rock music, electronic music, etc.

[0051] Furthermore, the acoustic feature vector of the user's performance is compared with the standard acoustic feature library of optional accompaniments, and the Euclidean distance or cosine similarity is calculated. If the Euclidean distance is less than the first preset threshold, the acoustic features are considered to be similar, such as high pitch and rhythm matching. The style features are vectorized and the Hamming distance or Jaccard coefficient is calculated. If the Jaccard coefficient is greater than the second preset threshold, the styles are considered to be similar, such as both are popular lyrical styles. The accompaniments whose distance from the user's performance acoustic features is less than the first threshold are screened to construct a first optional list, and the accompaniments whose distance from the user's historical preferred style features is less than the second threshold are screened to construct a second optional list. If there are common tracks, the intersection is taken as the first recommendation, and both acoustics and style are matched. If there are no common tracks but there is a possibility of intersection, the two lists are merged and sorted by distance, and tracks with similar dual features are displayed first. If there is no intersection at all, the top 3 tracks with the smallest distance in the two lists are taken respectively, and the matching dimensions are marked, such as best acoustics and best style, and the selected accompaniment tracks are recommended to the user for playing.

[0052] Preferably, in order to avoid a large deviation between the historical performance data and the live performance data due to objective conditions, thereby affecting the on-site adaptability of the accompaniment track, the method further includes: Collecting background audio data of the performance area through a sound collection unit; Perform acoustic feature analysis on the collected background audio data to obtain background sound labels; Determining a third optional accompaniment track list corresponding to the obtained background sound tag according to a predetermined mapping relationship between the optional accompaniment tracks and the applicable background sound tags; If the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have common accompaniment tracks, the common accompaniment track will be pushed to the user as the selected accompaniment track for selection and playback, or, if the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have no common accompaniment tracks, all the optional accompaniment tracks in the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list will be pushed to the user for selection and playback, or, if the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have no common accompaniment tracks, the third optional accompaniment track in the third optional accompaniment track list corresponding to the standard multidimensional acoustic feature with the smallest approximate distance to the obtained multidimensional acoustic feature and / or the fourth optional accompaniment track in the third optional accompaniment track list corresponding to the standard multidimensional musical style feature with the smallest approximate distance to the obtained multidimensional musical style feature will be pushed to the user for selection and playback.

[0053] In detail, objective conditions include changes in physical condition and huge changes in the external performance environment, for example, professional internal performance space and open and noisy outdoor performance space, etc., and the on-site adaptability of the accompaniment repertoire is analyzed based on the objective conditions, that is, the background audio data of the performance area is collected by the sound collection unit, and the background audio data includes environmental noise, and then the collected background audio data is subjected to acoustic feature analysis to obtain background sound labels, and the background sound labels include quiet environment labels, low noise environment labels, medium noise environment labels, noisy environment labels, etc., among which the quiet environment label refers to an environment with extremely low background noise and almost no interference Sound is suitable for scenarios that require high concentration or listening to details, such as a bedroom late at night; the low-noise environment label refers to an environment with slight background noise but does not affect normal conversation or listening, and is a comfortable and acceptable environment, such as a quiet cafe (off-peak hours); the medium-noise environment label refers to an environment with obvious perceptible background noise, which interferes with concentration or listening to a certain extent, but the main sounds (such as human voices and musical instruments) can still be identified, such as streets during rush hour and inside shopping malls; the noisy environment label refers to an environment with strong and complex background noise, which seriously interferes with normal listening or conversation, and requires a higher volume to communicate, such as at concerts, construction sites, and subway cars.

[0054] Specifically, the machine learning model analyzes historical usage scenarios, automatically generates mapping relationships, and calculates the semantic similarity between background sound labels and preset labels. The accompaniment tracks with a similarity greater than a threshold are screened, and the candidate tracks are sorted by matching degree to generate a third optional list. If there are three common tracks, the three intersection tracks are directly selected as the recommendation results because they meet the three conditions of acoustic adaptation, style preference, and environment matching at the same time, and have the highest priority; if there is no intersection in the three, but there is at least one track that belongs to the intersection of any two lists, then all the list tracks are merged, and after deduplication, they are sorted and displayed by matching degree, with pairwise intersection tracks being displayed first, followed by single list tracks; if the three lists have no overlap at all (that is, no track belongs to two or three lists at the same time), the tracks closest to the acoustic characteristics and style characteristics are selected from the third list respectively as environment-priority recommendations, so that the selected accompaniment tracks are recommended to the user for playback.

[0055] It can be seen that through real-time or timed acquisition, the system can dynamically obtain the real-time audio of the user's performance, avoiding the limitations of static preset parameters; by analyzing multi-dimensional acoustic characteristics, the multi-dimensional acoustic characteristics cover dimensions such as frequency, amplitude, rhythm, harmonics, and dynamic range, ensuring that the analysis results are comprehensive and accurate, and the playback parameters are directly related to the volume, rhythm, timbre and other attributes of the accompaniment, so that the accompaniment can be matched with the user's performance in real time, creating an immersive ensemble experience.

[0056] It can be seen that in the above scheme, the audio data of the user's temporary performance is obtained in real time through the sound collection unit, its multi-dimensional acoustic characteristics are analyzed, and the personalized playback parameters of the audio playback unit are dynamically generated, so that the accompaniment parameters are adaptively adjusted through the acoustic characteristic analysis to ensure real-time synchronization of the accompaniment and the performance, which is especially suitable for impromptu performances, live performances or teaching scenarios, to achieve accurate portrayal of the performance style, and avoid the one-sidedness of traditional single-dimensional adjustments; the user does not need to be familiar with the accompaniment track parameters in advance, only needs to use the temporary performance audio to trigger the system to automatically analyze and generate matching playback parameters. The user can fully debug the accompaniment parameters in the trial stage, and start the formal playback after confirming that they are correct, avoiding performance interruptions caused by system misjudgment and improving operational reliability, thereby solving the technical problems of insufficient accuracy of accompaniment track playback control in fixed mode and insufficient adaptability of accompaniment track playback matching.

[0057] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0058] In one embodiment, a device for controlling the playing of musical instrument accompaniment tracks is provided, which corresponds to the method for controlling the playing of musical instrument accompaniment tracks in the above embodiment. Figure 5 As shown, the instrument accompaniment track playback control device includes a temporary performance audio data acquisition module 101, a playback parameter analysis module 102 and an accompaniment track playback module 103. The functional modules are described in detail as follows: The temporary performance audio data collection module 101 is used to collect the user's temporary performance audio data in real time or at a fixed time through the sound collection unit after receiving the accompaniment track acoustic feature intelligent control instruction issued by the user; The playback parameter analysis module 102 is used to perform acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyze the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; The accompaniment music playing module 103 is used to control the audio playing unit to play the accompaniment music based on the analyzed playing parameters after receiving the performance formal start instruction issued by the user.

[0059] In one embodiment, the playback parameter analysis module 102, when performing acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, is configured to: Converting the time domain signal of the collected temporary performance audio data into a frequency domain spectrum; extracting the pitch features in the temporary performance audio data according to the frequency domain spectrum; Calculating the short-time energy and zero-crossing rate in the collected temporary performance audio data; determining a volume feature in the temporary performance audio data according to the short-time energy, and determining a rhythm intensity feature in the temporary performance audio data according to the zero-crossing rate; The pitch feature, the volume feature, and the rhythm intensity feature are fused into a multi-dimensional acoustic feature.

[0060] In one embodiment, the playback parameter analysis module 102, when extracting the pitch features in the temporary performance audio data based on the frequency domain spectrum, is further configured to: Analyzing the fundamental frequency period of the temporary performance audio data according to the frequency domain spectrum; Mapping the fundamental frequency value in the fundamental frequency period to a pitch system in a preset music theory; The pitch characteristics of the temporary performance audio data are determined according to the mapped pitch system.

[0061] In one embodiment, the playback parameter analysis module 102, when analyzing the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features, is further configured to: Associating the multidimensional acoustic features with execution parameters in the audio playback unit; Determining a playback model of the audio playback unit according to the associated execution parameters and playback structure characteristics of the audio playback units corresponding to different musical instruments; The playback model is used to analyze playback parameters of the audio playback unit.

[0062] In one embodiment, the accompaniment track playing module 103, when controlling the audio playing unit to play the accompaniment track based on the analyzed playing parameters, is configured to: The analyzed playback parameters are stored in segments according to a preset time axis, and playback control instructions of the audio playback unit are generated according to the segmented and stored playback parameters; Dividing the playing time periods of the accompaniment tracks according to the structure of the accompaniment tracks, and loading the playing control instructions corresponding to the playing time periods into the register of the audio playing unit; parsing the playback control instructions in the register of the audio playback unit according to the time sequence of the playback time period; The audio playback unit is controlled to play the accompaniment track according to the parsed playback control instruction.

[0063] In one embodiment, when performing instrument accompaniment track playback control, it is also used to: Acquire and store the user's historical performance audio data and historical accompaniment track information; After receiving an accompaniment track intelligent recommendation instruction from a user, the system obtains the user's historical performance audio data and historical accompaniment track information, performs acoustic feature analysis on the obtained historical performance audio data to obtain multi-dimensional acoustic features, and performs genre feature analysis on the obtained historical accompaniment track information to obtain multi-dimensional genre features; Analyzing, based on a predetermined mapping relationship between optional accompaniment tracks and standard multidimensional acoustic features, a first list of optional accompaniment tracks corresponding to standard multidimensional acoustic features having an approximate distance to the obtained multidimensional acoustic features less than a first preset threshold; Analyzing, based on a predetermined mapping relationship between optional accompaniment tracks and standard multi-dimensional music style features, a second list of optional accompaniment tracks corresponding to standard multi-dimensional music style features whose approximate distance to the obtained multi-dimensional music style features is less than a second preset threshold; If the first optional accompaniment track list and the second optional accompaniment track list have common accompaniment tracks, the common accompaniment track will be pushed to the user as the selected accompaniment track for selection and playback, or, if the first optional accompaniment track list and the second optional accompaniment track list do not have common accompaniment tracks, all the optional accompaniment tracks in the first optional accompaniment track list and the second optional accompaniment track list will be pushed to the user for selection and playback, or, if the first optional accompaniment track list and the second optional accompaniment track list do not have common accompaniment tracks, the first optional accompaniment track in the first optional accompaniment track list corresponding to the standard multidimensional acoustic feature with the smallest approximate distance to the obtained multidimensional acoustic feature and / or the second optional accompaniment track in the second optional accompaniment track list corresponding to the standard multidimensional musical style feature with the smallest approximate distance to the obtained multidimensional musical style feature will be pushed to the user for selection and playback.

[0064] In one embodiment, when performing instrument accompaniment track playback control, it is also used to: Collecting background audio data of the performance area through a sound collection unit; Perform acoustic feature analysis on the collected background audio data to obtain background sound labels; Determining a third optional accompaniment track list corresponding to the obtained background sound tag according to a predetermined mapping relationship between the optional accompaniment tracks and the applicable background sound tags; If the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have common accompaniment tracks, the common accompaniment track will be pushed to the user as the selected accompaniment track for selection and playback, or, if the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have no common accompaniment tracks, all the optional accompaniment tracks in the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list will be pushed to the user for selection and playback, or, if the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have no common accompaniment tracks, the third optional accompaniment track in the third optional accompaniment track list corresponding to the standard multidimensional acoustic feature with the smallest approximate distance to the obtained multidimensional acoustic feature and / or the fourth optional accompaniment track in the third optional accompaniment track list corresponding to the standard multidimensional musical style feature with the smallest approximate distance to the obtained multidimensional musical style feature will be pushed to the user for selection and playback.

[0065] The present invention provides a device for controlling the playback of musical instrument accompaniment tracks, which obtains audio data of a user's temporary performance in real time through a sound acquisition unit, analyzes its multi-dimensional acoustic characteristics, and dynamically generates personalized playback parameters of an audio playback unit, thereby realizing adaptive adjustment of the accompaniment parameters through acoustic characteristic analysis, ensuring real-time synchronization of the accompaniment and the performance, and being particularly suitable for impromptu performances, live performances or teaching scenarios, achieving accurate portrayal of the performance style, and avoiding the one-sidedness of traditional single-dimensional adjustment; the user does not need to be familiar with the accompaniment track parameters in advance, and only needs to trigger the system to automatically analyze and generate matching playback parameters through the temporary performance audio; the user can fully debug the accompaniment parameters in the trial performance stage, and start the formal playback after confirming that they are correct, thereby avoiding performance interruptions caused by system misjudgment and improving operational reliability, thereby solving the technical problems of insufficient accuracy of accompaniment track playback control in a fixed mode and insufficient adaptability of accompaniment track playback matching.

[0066] The specific definition of the instrument accompaniment track playback control device can be found in the definition of the instrument accompaniment track playback control method above, which will not be repeated here. The various modules in the above-mentioned instrument accompaniment track playback control device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the musical instrument device in the form of hardware, or can be stored in the memory of the musical instrument device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0067] In one embodiment, a musical instrument device is provided. The musical instrument device may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The musical instrument device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the musical instrument device is used to provide computing and control capabilities. The memory of the musical instrument device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, an executable program and a database. The internal memory provides an environment for the operation of the operating system and the executable program in the non-volatile storage medium. The network interface of the musical instrument device is used to communicate with an external client via a network connection. When the executable program is executed by the processor, it realizes the functions or steps on the server side of a method for controlling the playback of an instrument accompaniment track.

[0068] In one embodiment, a musical instrument device is provided. The musical instrument device may be a client, and its internal structure diagram may be as follows: Figure 7 As shown. The musical instrument device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the musical instrument device is used to provide computing and control capabilities. The memory of the musical instrument device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and an executable program. The internal memory provides an environment for the operation of the operating system and the executable program in the non-volatile storage medium. The network interface of the musical instrument device is used to communicate with an external server via a network connection. When the executable program is executed by the processor, it realizes the functions or steps on the client side of a method for controlling the playback of musical instrument accompaniment tracks.

[0069] In one embodiment, a musical instrument device is provided, including a memory, a processor, and an executable program stored in the memory and executable on the processor. When the processor executes the executable program, the following steps are implemented: After receiving the intelligent control instruction of the acoustic characteristics of the accompaniment track from the user, the user's temporary performance audio data is collected in real time or at a fixed time through the sound collection unit; Performing acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyzing the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; After receiving the performance start instruction from the user, the audio playback unit is controlled to play the accompaniment track based on the analyzed playback parameters.

[0070] In one embodiment, a processor-readable storage medium is provided, on which an executable program is stored. When the executable program is executed by a processor, the following steps are implemented: After receiving the intelligent control instruction of the acoustic characteristics of the accompaniment track from the user, the user's temporary performance audio data is collected in real time or at a fixed time through the sound collection unit; Performing acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyzing the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; After receiving the performance start instruction from the user, the audio playback unit is controlled to play the accompaniment track based on the analyzed playback parameters.

[0071] It should be noted that the above functions or steps that can be implemented by the processor-readable storage medium or musical instrument device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0072] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through an executable program. The executable program can be stored in a non-volatile processor-readable storage medium. When the executable program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0073] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0074] It should be noted that if software tools or components other than those of our company appear in the embodiments of this application, they are only used for illustration and do not represent actual use.

[0075] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for controlling the playback of musical instrument accompaniment tracks, characterized in that: include: After receiving the intelligent control instruction of the acoustic characteristics of the accompaniment track from the user, the user's temporary performance audio data is collected in real time or at a fixed time through the sound collection unit; Performing acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyzing the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; After receiving the performance start instruction from the user, the audio playback unit is controlled to play the accompaniment track based on the analyzed playback parameters.

2. The method for controlling the playing of musical instrument accompaniment tracks according to claim 1, wherein: The acoustic feature analysis of the collected temporary performance audio data to obtain multi-dimensional acoustic features includes: Converting the time domain signal of the collected temporary performance audio data into a frequency domain spectrum; extracting the pitch features in the temporary performance audio data according to the frequency domain spectrum; Calculating the short-time energy and zero-crossing rate in the collected temporary performance audio data; determining a volume feature in the temporary performance audio data according to the short-time energy, and determining a rhythm intensity feature in the temporary performance audio data according to the zero-crossing rate; The pitch feature, the volume feature, and the rhythm intensity feature are fused into a multi-dimensional acoustic feature.

3. The instrument accompaniment track playback control method according to claim 2, wherein: The step of extracting the tone features from the temporary performance audio data according to the frequency domain spectrum comprises: Analyzing the fundamental frequency period of the temporary performance audio data according to the frequency domain spectrum; Mapping the fundamental frequency value in the fundamental frequency period to a pitch system in a preset music theory; The pitch characteristics of the temporary performance audio data are determined according to the mapped pitch system.

4. The method for controlling the playing of musical instrument accompaniment tracks according to claim 1, wherein: Analyzing the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features includes: Associating the multidimensional acoustic features with execution parameters in the audio playback unit; Determining a playback model of the audio playback unit according to the associated execution parameters and playback structure characteristics of the audio playback units corresponding to different musical instruments; The playback model is used to analyze playback parameters of the audio playback unit.

5. The method for controlling the playing of musical instrument accompaniment tracks according to claim 1, wherein: The controlling the audio playback unit to play the accompaniment track based on the analyzed playback parameters includes: The analyzed playback parameters are stored in segments according to a preset time axis, and playback control instructions of the audio playback unit are generated according to the segmented and stored playback parameters; Dividing the playing time periods of the accompaniment tracks according to the structure of the accompaniment tracks, and loading the playing control instructions corresponding to the playing time periods into the register of the audio playing unit; parsing the playback control instructions in the register of the audio playback unit according to the time sequence of the playback time period; The audio playback unit is controlled to play the accompaniment track according to the parsed playback control instruction.

6. The method for controlling the playing of musical instrument accompaniment tracks according to claim 1, wherein: The method further comprises: Acquire and store the user's historical performance audio data and historical accompaniment track information; After receiving an accompaniment track intelligent recommendation instruction from a user, the system obtains the user's historical performance audio data and historical accompaniment track information, performs acoustic feature analysis on the obtained historical performance audio data to obtain multi-dimensional acoustic features, and performs genre feature analysis on the obtained historical accompaniment track information to obtain multi-dimensional genre features; Analyzing, based on a predetermined mapping relationship between optional accompaniment tracks and standard multidimensional acoustic features, a first list of optional accompaniment tracks corresponding to standard multidimensional acoustic features having an approximate distance to the obtained multidimensional acoustic features less than a first preset threshold; Analyzing, based on a predetermined mapping relationship between optional accompaniment tracks and standard multi-dimensional music style features, a second list of optional accompaniment tracks corresponding to standard multi-dimensional music style features whose approximate distance to the obtained multi-dimensional music style features is less than a second preset threshold; If the first optional accompaniment track list and the second optional accompaniment track list have common accompaniment tracks, the common accompaniment track will be pushed to the user as the selected accompaniment track for selection and playback, or, if the first optional accompaniment track list and the second optional accompaniment track list do not have common accompaniment tracks, all the optional accompaniment tracks in the first optional accompaniment track list and the second optional accompaniment track list will be pushed to the user for selection and playback, or, if the first optional accompaniment track list and the second optional accompaniment track list do not have common accompaniment tracks, the first optional accompaniment track in the first optional accompaniment track list corresponding to the standard multidimensional acoustic feature with the smallest approximate distance to the obtained multidimensional acoustic feature and / or the second optional accompaniment track in the second optional accompaniment track list corresponding to the standard multidimensional musical style feature with the smallest approximate distance to the obtained multidimensional musical style feature will be pushed to the user for selection and playback.

7. The method for controlling the playing of musical instrument accompaniment tracks according to claim 6, wherein: The method further comprises: Collecting background audio data of the performance area through a sound collection unit; Perform acoustic feature analysis on the collected background audio data to obtain background sound labels; Determining a third optional accompaniment track list corresponding to the obtained background sound tag according to a predetermined mapping relationship between the optional accompaniment tracks and the applicable background sound tags; If the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have common accompaniment tracks, the common accompaniment track will be pushed to the user as the selected accompaniment track for selection and playback, or, if the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have no common accompaniment tracks, all the optional accompaniment tracks in the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list will be pushed to the user for selection and playback, or, if the first optional accompaniment track list, the second optional accompaniment track list and the third optional accompaniment track list have no common accompaniment tracks, the third optional accompaniment track in the third optional accompaniment track list corresponding to the standard multidimensional acoustic feature with the smallest approximate distance to the obtained multidimensional acoustic feature and / or the fourth optional accompaniment track in the third optional accompaniment track list corresponding to the standard multidimensional musical style feature with the smallest approximate distance to the obtained multidimensional musical style feature will be pushed to the user for selection and playback.

8. A device for controlling the playback of musical instrument accompaniment tracks, characterized in that: include: The temporary performance audio data acquisition module is used to collect the user's temporary performance audio data in real time or at a fixed time through the sound acquisition unit after receiving the accompaniment track acoustic feature intelligent control instruction issued by the user; A playback parameter analysis module is used to perform acoustic feature analysis on the collected temporary performance audio data to obtain multi-dimensional acoustic features, and analyze the playback parameters of the audio playback unit based on the obtained multi-dimensional acoustic features; The accompaniment track playing module is used to control the audio playing unit to play the accompaniment track based on the analyzed playing parameters after receiving the performance formal start instruction issued by the user.

9. A musical instrument device comprising a memory, a processor, and an executable program stored in the memory and executable on the processor, wherein: When the processor executes the executable program, the instrument accompaniment track playback control method according to any one of claims 1 to 7 is implemented.

10. A processor-readable storage medium storing an executable program, characterized in that: When the executable program is executed by the processor, the instrument accompaniment track playback control method according to any one of claims 1 to 7 is implemented.

Images