Visual sound analysis method for vocal music teaching

Abstract

The invention discloses a sound visualization analysis method for vocal music teaching, and particularly relates to the field of music education, comprising sensor setting, data acquisition, sound data analysis, multi-dimensional correlation analysis and visualization. Through accurate sound and physiological data collection and in combination with multi-dimensional sound data analysis and correlation analysis, the vocal music skill and the sounding state of a singer can be comprehensively and deeply evaluated, the analysis result is intuitive and easy to understand through the real-time data visualization function, the singer and a teacher can quickly understand and make adjustment conveniently, and in addition, the analysis efficiency is improved. The method can also predict the change trend of sound indexes, provides a scientific basis for personalized teaching and training, and is beneficial to improving the vocal music teaching effect and the performance level of singers.

Description

Technical Field

[0001] The present invention relates to the technical field of music education, and more specifically, to a sound visualization analysis method used in vocal music teaching. Background Art

[0002] Vocal music teaching, as an important branch of music education, aims to improve students' voice control ability, singing skills and musical expression through systematic training and guidance. With the continuous development of music education and the continuous advancement of science and technology, the field of vocal music teaching is also constantly exploring and applying new technical means to improve teaching effectiveness and meet students' personalized needs.

[0003] In current vocal music teaching, existing technical means mainly focus on the analysis and processing of sound signals. Specifically, these technologies usually collect students' singing voices through professional audio recording equipment, and then use audio analysis software to perform detailed analysis and processing of the sound signals. This process includes sound signal filtering, noise reduction, spectrum analysis, pitch detection and other links, aiming to extract key parameters related to singing performance, such as volume, pitch, timbre, resonance effect, etc. Based on these analysis results, teachers can provide targeted guidance and improvement to students' singing skills.

[0004] Although the existing vocal teaching technologies have improved the objectivity and accuracy of teaching to a certain extent, there are still some obvious shortcomings. First, these technologies mainly focus on the analysis of sound signals, but ignore the physiological factors closely related to sound production, such as the movement state of the neck and throat muscles, which limits the in-depth understanding and comprehensive evaluation of students' singing skills. Secondly, the analysis results of existing technologies are usually presented in the form of numbers and charts, which are difficult to understand for students with non-music backgrounds, and are not conducive to improving teaching effectiveness. In addition, existing technologies lack real-time feedback and personalized guidance during students' singing, making it difficult to meet students' diverse learning needs. Therefore, the field of vocal teaching still needs to explore more comprehensive, intuitive and easy-to-understand teaching methods and technical means. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a sound visualization analysis method for vocal music teaching, which solves the problems raised in the above-mentioned background technology through the following scheme.

[0006] To achieve the above object, the present invention provides the following technical solution: a sound visualization analysis method for vocal music teaching, comprising: Step 1: Sensor setup: including microphone array and fiber optic sensors of neck and laryngeal muscles for sound collection and physiological data collection; Step 2: Data collection: used to collect sound data and physiological data; Step 3: Sound data analysis: used to analyze the sound data collected in step 2 to obtain dynamic range and timbre change data, resonance and vocal efficiency data, and skill and adaptability data; Step 4: Multidimensional correlation analysis: By introducing a machine learning algorithm and combining it with time series analysis, the correlation between the sound data and the neck and throat muscle movement data is comprehensively analyzed; Step 5: Visualization: This includes real-time data dashboards and historical data and forecast trend comparison charts to convert data into an intuitive form for presentation.

[0007] Preferably, the sound collection uses a multi-directional microphone array, which is arranged in a semicircular track around the singer, about 0.8-1.2 meters away from the singer, and is equipped with a professional-grade audio interface to convert the sound signal into a digital signal and transmit it to the computer.

[0008] Preferably, the physiological data is collected by wrapping optical fiber sensors around the muscle groups in the neck and throat to sense muscle deformation and tension changes.

[0009] Preferably, the optical fiber sensor is a special optical fiber sensor made based on the fiber Bragg grating principle. The core component is a section of specially treated optical fiber. Refractive index modulation areas are periodically distributed in the core to form a Bragg grating. The outer layer of the optical fiber sensor is made of biocompatible material that fits human skin to ensure that it does not cause irritation or discomfort to the singer's skin during long-term wearing. The internal optical fiber is encapsulated.

[0010] Preferably, the sound data is acquired by continuously collecting sound signals at a sampling rate of 48kHz and 24-bit quantization accuracy through a microphone array when the singer performs vocal exercises or sings different pieces of music. During the acquisition process, the sound signals are segmented and marked according to different stages of the singing for subsequent targeted analysis.

[0011] Preferably, the physiological data is recorded by fiber optic sensors of the neck and laryngeal muscles at a sampling rate of 1000 Hz to record muscle deformation and tension data.

[0012] Preferably, the dynamic range and timbre change data include the volume attenuation ratio from the lowest note to the highest note, the timbre brightness index, the timbre consistency when spanning the range, and the vibrato rate and amplitude ratio; the resonance and vocal efficiency data include the head cavity resonance ratio, the degree of vocal cord closure during vocalization, the breath flow and pitch stability coefficient, and the flexibility of resonance cavity conversion; the skill and adaptability data include the accuracy of ornamentation, the rhythm flexibility index, the sound intensity control accuracy, and the sound fatigue index.

[0013] Preferably, the method for analyzing the volume attenuation ratio from the lowest sound to the highest sound is specifically as follows: performing time domain analysis on the sound signal, identifying the starting and ending positions of the lowest sound and the highest sound, calculating the root mean square value of the volume between the two, and calculating the ratio by the formula The volume attenuation ratio is obtained, where is the RMS value of the lowest volume, is the root mean square value of the volume of the highest sound. The specific analysis method of the timbre brightness index is as follows: the sound signal is converted into the frequency domain using Fourier transform, and weights are assigned to different frequency bands according to the human ear's perception characteristics of sounds of different frequencies. Calculate the timbre brightness index, where is the weight of the kth frequency band, is the energy proportion of the frequency band, m represents the number of divided frequency bands, and the specific method for analyzing the consistency of timbre when the range spans is as follows: at the key transition point of the range span, the Mel frequency cepstrum coefficient feature vector of the sound is extracted, and the cosine similarity between adjacent feature vectors is calculated using the formula The timbre consistency index is obtained, where and are the i-th MFCC coefficients of two adjacent transition points, n is the dimension of the MFCC coefficient, and the specific analysis method of the vibrato rate and amplitude ratio is as follows: the instantaneous frequency of the sound signal is extracted by Hilbert transform, the vibrato part is detected, the number of frequency fluctuations per unit time is calculated to obtain the vibrato rate, and the ratio of the maximum amplitude of the frequency fluctuation to the average frequency is taken as the vibrato amplitude ratio, that is, , , Indicates the vibrato rate, Indicates the number of frequency fluctuations, Indicates time, represents the amplitude ratio, Indicates the maximum frequency fluctuation amplitude, Represents the average frequency.

[0014] Preferably, the head cavity resonance ratio analysis method is specifically as follows: based on the resonance peak analysis of the sound signal, the frequency and intensity of the resonance peak are determined by using the linear prediction coding technology, and the energy of the head cavity resonance resonance peak is compared with the total sound energy, and the formula is used. Calculate the head cavity resonance ratio, Indicates the energy of the resonance peak related to the head cavity resonance, Represents the total energy of the sound signal. The specific analysis method of the vocal cord closure during phonation is as follows: Analyze the harmonic structure of the sound signal, calculate the amplitude ratio and phase relationship between the harmonic and the fundamental wave, and use the empirical formula to estimate the vocal cord closure , and represents the amplitude and phase of the jth harmonic, and represents the amplitude and phase of the fundamental wave, and represents the empirical coefficient, p represents the number of harmonics, and the specific analysis method of the breath flow and pitch stability coefficient is as follows: combining the breath flow data collected by the respiratory sensor and the pitch change of the sound signal, calculate the ratio of the standard deviation of the pitch to the average breath flow, and the formula is: , represents the standard deviation of pitch, The analysis method of resonance cavity conversion flexibility is as follows: by monitoring the rapid change of sound signal energy in different frequency bands and the migration of resonance peaks, the change rate of resonance peak frequency and intensity in adjacent time periods is calculated as an indicator of resonance cavity conversion flexibility. The formula is , and are the resonance peak frequency and intensity at the tth moment, q represents the total number of time periods analyzed, Indicates the flexibility of resonance cavity conversion. represents the resonance peak frequency at time t+1, Represents the intensity at time t+1.

[0015] Preferably, the analysis method of the accuracy of the ornaments is as follows: the ornaments of the performance are compared with the ornaments in the standard score, and the errors in pitch, duration and start time between the two are calculated using the formula Calculate the accuracy index, , as well as They represent the pitch, duration and start time of the standard ornament. , as well as They represent the pitch, duration and start time of the actual singing grace note. , as well as Respectively represent the maximum values ​​of pitch, duration and start time of all ornament parameters. , as well as They represent the minimum values ​​of pitch, duration and start time of all ornamentation parameters respectively, s represents the number of ornamentations, and the analysis method of rhythm flexibility index is as follows: the actual rhythm of the performance is determined by using the beat tracking algorithm, and compared with the standard rhythm, and the statistics of rhythm deviation within a certain time window are calculated. Calculate the rhythm flexibility index, The variance of the rhythm deviation is expressed in the following way: the analysis method of the sound intensity control accuracy is as follows: the dynamic range of the sound signal is analyzed and divided into multiple levels, and the accuracy and delicacy of the conversion of the intensity between different levels in the actual singing is calculated. calculate, Indicates whether the u-th intensity conversion is accurate, 1 for accurate and 0 for inaccurate, v represents the total number of intensity conversions. The specific analysis method of the sound fatigue index is: monitor the long-term change trend of the sound signal, including the drift of the fundamental frequency and the attenuation of the harmonics, and establish a sound fatigue model using the formula Calculate the voice fatigue index, represents the change in fundamental frequency, and Respectively represent the changes in the w-th harmonic amplitude and phase, represents the amplitude of the wth harmonic at the initial moment, , as well as is the model coefficient, and x is the total number of harmonics.

[0016] Preferably, the machine learning algorithm sets the sound feature vector as , the neck and laryngeal muscle movement feature vector is , the prediction function obtained through training is , represents the weight corresponding to the sound feature, represents the weight corresponding to the neck and throat muscle movement characteristics, b is the bias term, n1 represents the number of elements in the sound feature vector, s i1 represents the i1th feature in the sound feature vector, k1 represents the number of elements in the neck and throat muscle movement feature vector, m j1 Represents the j1th feature in the neck and laryngeal muscle movement feature vector.

[0017] Preferably, the time series analysis is based on the sound fatigue index F t and any parameter of the neck and throat muscle movement to establish an ARIMA (p, d, q) model. Let F t After d differences, , build the model , represents the free regression coefficient, represents the moving average, represents the white noise term, p1 represents the autoregressive order, i2 is used to traverse each lag order from 1 to p1 in the autoregressive part, q1 represents the moving average order, j2 is used to traverse each lag order from 1 to q1 in the moving average part, and t1 represents the current moment.

[0018] Preferably, the real-time data dashboard displays the indicators output by step 3 and step 4 in a visual manner, each indicator is represented by a circular dial, and the pointer points to the current numerical position in real time. The dial is divided into different color areas according to the set threshold range, green indicates the normal range, yellow indicates approaching the warning range, and red indicates exceeding the warning range.

[0019] Preferably, the historical data and predicted trend comparison chart draws a comprehensive chart to display the singer's historical data and the predicted trend of the future by the multidimensional correlation model. The horizontal axis of the chart represents time, and the vertical axis represents the values ​​of various indicators. Lines of different colors are used to represent the changing trends of historical data and the future trends predicted by the model. The blue line represents the actual data changes of the sound resonance effect index in the past month, and the red line represents the development trend of the indicator in the next two weeks based on the model prediction. Labels and annotations are added to the chart to facilitate users to understand the changes in the data. For abnormal situations that may occur in the predicted trend, early warning annotations are made in advance. When it is predicted that the voice fatigue index will exceed the warning threshold in the next week, it is marked with a yellow triangle on the chart, and a prompt "Please note that the voice fatigue may reach a dangerous level in the next week, please adjust the training plan" is displayed, and a detailed analysis is performed in a graphical way, specifically, a tree diagram is used to display the contribution of each factor in the sound characteristics and neck and throat muscle movement parameters to the abnormal indicators, and different colors and line thicknesses are used to indicate the degree of influence. Clicking each factor in the tree diagram can also display detailed historical data.

[0020] Technical effects and advantages of the present invention: The present invention realizes the precise collection of sound signals and physiological data by setting up a multi-directional microphone array and fiber optic sensors for the neck and throat muscles. The microphone array can capture the singer's voice in all directions, while the fiber optic sensors can accurately sense the deformation and tension of the neck and throat muscles, providing a high-quality data basis for subsequent analysis. In the data collection stage, the system continuously records sound and physiological data at a high sampling rate, and marks them in segments according to different stages of singing. At the same time, it also records metadata such as the song name and singer information, which is convenient for the subsequent classification management and in-depth analysis of data. This step ensures the comprehensiveness and accuracy of the data. Sound data analysis is the core link of the method. By performing multi-dimensional analysis on the collected sound data, including dynamic range, timbre changes, resonance effects, vocal efficiency, vocal skills and adaptability, the system can accurately evaluate the singer's voice. The system can also analyze the singers’ vocal level and existing problems, which provides a scientific basis for subsequent personalized teaching and training. By introducing machine learning algorithms and time series analysis, the relationship between sound characteristics and neck and throat muscle movements is deeply explored, which can not only reveal the mystery of the vocal mechanism, but also predict the possible changing trends of sound indicators in the future. Based on these analysis results, teachers can formulate personalized training plans for singers to improve their vocal skills and performance level. The analysis results are presented in an intuitive and easy-to-understand way. Through the real-time data dashboard and the comparison chart of historical data and predicted trends, singers and teachers can clearly see the values ​​and changing trends of various indicators. At the same time, the system can also provide early warning and detailed analysis windows to help singers find problems in time and adjust training plans. This visual presentation method makes the analysis results more intuitive and easy to understand, which is convenient for application in teaching and training practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] refer to Figure 1 A sound visualization analysis method for vocal music teaching is shown, and the specific steps include: Step 1: Sensor setup: This includes the setup of microphone arrays and fiber optic sensors for neck and laryngeal muscles for sound collection and physiological data acquisition.

[0024] The sound collection uses a multi-directional microphone array, which is arranged in a semicircular track around the singer, about 0.8-1.2 meters away from the singer, and is equipped with a professional-grade audio interface to convert the sound signal into a digital signal and transmit it to the computer.

[0025] The physiological data is collected by wrapping optical fiber sensors around the muscle groups of the neck and larynx to sense muscle deformation and tension changes.

[0026] The fiber optic sensor is specifically a special fiber optic sensor made based on the fiber Bragg grating principle. The core component is a specially treated optical fiber with periodically distributed refractive index modulation areas in the core to form a Bragg grating. The outer layer of the fiber optic sensor is made of biocompatible material that fits human skin to ensure that it does not cause irritation or discomfort to the singer's skin during long-term wearing. The internal optical fiber is encapsulated to ensure high sensitivity to tiny muscle deformations and tension changes, and has good flexibility to adapt to the complex movements of the neck and throat muscles.

[0027] The fiber optic sensor has significant advantages over traditional electromyographic sensors. Because it is based on optical signal transmission, it is minimally affected by electromagnetic interference and can work stably in complex electronic equipment environments. Moreover, it can measure the dynamic changes of muscles with extremely high precision, and can be accurate to deformation at the micron level. For example, when the laryngeal muscles produce extremely subtle contractions or relaxations during the phonation process, the Bragg grating period in the fiber optic sensor will change accordingly, causing the wavelength of the reflected light to change. Through high-precision wavelength demodulation equipment, this wavelength change can be accurately converted into muscle deformation and tension data, providing extremely reliable and accurate data support for subsequent analysis of muscle movement during phonation.

[0028] Step 2: Data collection: used to collect sound data and physiological data.

[0029] The sound data is obtained by continuously collecting sound signals at a sampling rate of 48kHz and 24-bit quantization accuracy using a microphone array when the singer is practicing vocals or singing different pieces of music. During the collection process, the sound signals are segmented and marked according to different stages of the singing for subsequent targeted analysis.

[0030] The physiological data are recorded by fiber optic sensors of the neck and laryngeal muscles at a sampling rate of 1000 Hz to record muscle deformation and tension data.

[0031] The step 2 also records metadata such as the name of the song, singer information, and performance time, making it easier to classify and manage the data.

[0032] Step 3: Sound data analysis: used to analyze the sound data collected in step 2 to obtain dynamic range and timbre change data, resonance and vocal efficiency data, and skill and adaptability data.

[0033] The dynamic range and timbre change data include the volume attenuation ratio from the lowest note to the highest note, the timbre brightness index, the timbre consistency when spanning the range, and the vibrato rate and amplitude ratio; the resonance and vocal efficiency data include the head cavity resonance ratio, the degree of vocal cord closure during vocalization, the breath flow and pitch stability coefficient, and the flexibility of resonance cavity conversion; the skill and adaptability data include the accuracy of ornamentation, the rhythm flexibility index, the sound intensity control accuracy, and the sound fatigue index.

[0034] The specific method for analyzing the volume attenuation ratio from the lowest sound to the highest sound is: perform time domain analysis on the sound signal, identify the starting and ending positions of the lowest sound and the highest sound, calculate the root mean square value of the volume between the two, and use the formula The volume attenuation ratio is obtained, where is the RMS value of the lowest volume, is the root mean square value of the volume of the highest sound. The specific analysis method of the timbre brightness index is as follows: the sound signal is converted into the frequency domain using Fourier transform, and weights are assigned to different frequency bands according to the human ear's perception characteristics of sounds of different frequencies. Calculate the timbre brightness index, where is the weight of the kth frequency band, is the energy proportion of the frequency band, m represents the number of divided frequency bands, and the specific method for analyzing the consistency of timbre when the range spans is as follows: at the key transition point of the range span, the Mel frequency cepstrum coefficient feature vector of the sound is extracted, and the cosine similarity between adjacent feature vectors is calculated using the formula The timbre consistency index is obtained, where and are the i-th MFCC coefficients of two adjacent transition points, n is the dimension of the MFCC coefficient, and the specific analysis method of the vibrato rate and amplitude ratio is as follows: the instantaneous frequency of the sound signal is extracted by Hilbert transform, the vibrato part is detected, the number of frequency fluctuations per unit time is calculated to obtain the vibrato rate, and the ratio of the maximum amplitude of the frequency fluctuation to the average frequency is taken as the vibrato amplitude ratio, that is, , , Indicates the vibrato rate, Indicates the number of frequency fluctuations, Indicates time, represents the amplitude ratio, Indicates the maximum frequency fluctuation amplitude, Represents the average frequency.

[0035] The head cavity resonance ratio analysis method is specifically as follows: based on the resonance peak analysis of the sound signal, the frequency and intensity of the resonance peak are determined by using the linear prediction coding technology, and the energy of the head cavity resonance resonance peak is compared with the total sound energy, and the formula is used. Calculate the head cavity resonance ratio, Indicates the energy of the resonance peak related to the head cavity resonance, Represents the total energy of the sound signal. The specific analysis method of the vocal cord closure during phonation is as follows: Analyze the harmonic structure of the sound signal, calculate the amplitude ratio and phase relationship between the harmonic and the fundamental wave, and use the empirical formula to estimate the vocal cord closure , and represents the amplitude and phase of the jth harmonic, and represents the amplitude and phase of the fundamental wave, and represents the empirical coefficient, p represents the number of harmonics, and the specific analysis method of the breath flow and pitch stability coefficient is as follows: combining the breath flow data collected by the respiratory sensor and the pitch change of the sound signal, calculate the ratio of the standard deviation of the pitch to the average breath flow, and the formula is: , represents the standard deviation of pitch, The analysis method of resonance cavity conversion flexibility is as follows: by monitoring the rapid change of sound signal energy in different frequency bands and the migration of resonance peaks, the change rate of resonance peak frequency and intensity in adjacent time periods is calculated as an indicator of resonance cavity conversion flexibility. The formula is , and are the resonance peak frequency and intensity at the tth moment, q represents the total number of time periods analyzed, Indicates the flexibility of resonance cavity conversion. represents the resonance peak frequency at time t+1, Represents the intensity at time t+1.

[0036] The accuracy analysis method of the ornaments is as follows: the ornaments of the performance are compared with the ornaments in the standard score, and the errors in pitch, duration and start time between the two are calculated using the formula Calculate the accuracy index, , as well as They represent the pitch, duration and start time of the standard ornament. , as well as They represent the pitch, duration and start time of the actual singing grace note. , as well as Respectively represent the maximum values ​​of pitch, duration and start time of all ornament parameters. , as well as They represent the minimum values ​​of pitch, duration and start time of all ornamentation parameters respectively, s represents the number of ornamentations, and the analysis method of rhythm flexibility index is as follows: the actual rhythm of the performance is determined by using the beat tracking algorithm, and compared with the standard rhythm, and the statistics of rhythm deviation within a certain time window are calculated. Calculate the rhythm flexibility index, The variance of the rhythm deviation is expressed in the following way: the analysis method of the sound intensity control accuracy is as follows: the dynamic range of the sound signal is analyzed and divided into multiple levels, and the accuracy and delicacy of the conversion of the intensity between different levels in the actual singing is calculated. calculate, Indicates whether the u-th intensity conversion is accurate, 1 for accurate and 0 for inaccurate, v represents the total number of intensity conversions. The specific analysis method of the sound fatigue index is: monitor the long-term change trend of the sound signal, including the drift of the fundamental frequency and the attenuation of the harmonics, and establish a sound fatigue model using the formula Calculate the voice fatigue index, represents the change in fundamental frequency, and Respectively represent the changes in the w-th harmonic amplitude and phase, represents the amplitude of the wth harmonic at the initial moment, , as well as is the model coefficient, and x is the total number of harmonics.

[0037] Step 4: Multidimensional correlation analysis: By introducing a combination of machine learning algorithms and time series analysis, the correlation between sound data and neck and throat muscle movement data is comprehensively analyzed.

[0038] The machine learning algorithm assumes that the sound feature vector is , the neck and laryngeal muscle movement feature vector is , the prediction function obtained through training is , represents the weight corresponding to the sound feature, represents the weight corresponding to the neck and throat muscle movement characteristics, b is the bias term, n1 represents the number of elements in the sound feature vector, s i1 represents the i1th feature in the sound feature vector, k1 represents the number of elements in the neck and throat muscle movement feature vector, m j1 Represents the j1th feature in the neck and laryngeal muscle movement feature vector.

[0039] The machine learning algorithm is used to predict changes in sound indicators such as the vibrato rate and amplitude ratio of the sound under specific muscle movement states.

[0040] The time series analysis was based on the sound fatigue index F t and any parameter of the neck and throat muscle movement to establish an ARIMA (p, d, q) model. Let F t After d differences, , build the model , represents the free regression coefficient, represents the moving average, represents the white noise term, p1 represents the autoregressive order, i2 is used to traverse each lag order from 1 to p1 in the autoregressive part, q1 represents the moving average order, j2 is used to traverse each lag order from 1 to q1 in the moving average part, and t1 represents the current moment.

[0041] The time series analysis analyzes the relationship between the two in the time dimension through the ARIMA (p, d, q) model, and predicts the possible development of voice fatigue at several future time points as the average tension of the laryngeal muscles continues to change.

[0042] Step 5: Visualization: This includes real-time data dashboards and historical data and forecast trend comparison charts to convert data into an intuitive form for presentation.

[0043] The real-time data dashboard displays the indicators output from step 3 and step 4 in a visual manner. Each indicator is represented by a circular dial, and the pointer points to the current numerical position in real time. The dial is divided into different color areas according to the set threshold range. Green indicates the normal range, yellow indicates approaching the warning range, and red indicates exceeding the warning range.

[0044] The historical data and predicted trend comparison chart draws a comprehensive chart to show the singer's historical data and the predicted trend of the future by the multi-dimensional correlation model. The horizontal axis of the chart represents time, and the vertical axis represents the values ​​of various indicators. Lines of different colors are used to represent the changing trends of historical data and the future trends predicted by the model. The blue line represents the actual data changes of the sound resonance effect indicator in the past month, and the red line represents the development trend of the indicator in the next two weeks based on the model prediction. Labels and annotations are added to the chart to facilitate users to understand the changes in the data. For abnormal situations that may occur in the predicted trend, early warning annotations are made in advance. When it is predicted that the voice fatigue index will exceed the warning threshold in the next week, it is marked with a yellow triangle on the chart, and a prompt "Please note that the voice fatigue may reach a dangerous level in the next week, please adjust the training plan" is prompted, and a detailed analysis is performed in a graphical way. Specifically, a tree diagram is used to show the contribution of each factor in the sound characteristics and neck and throat muscle movement parameters to the abnormal indicators, and different colors and line thicknesses are used to indicate the degree of influence. Clicking each factor in the tree diagram can also display detailed historical data.

[0045] The present invention realizes the precise collection of sound signals and physiological data by setting up a multi-directional microphone array and fiber optic sensors for the neck and throat muscles. The microphone array can capture the singer's voice in all directions, while the fiber optic sensors can accurately sense the deformation and tension of the neck and throat muscles, providing a high-quality data basis for subsequent analysis. In the data collection stage, the system continuously records sound and physiological data at a high sampling rate, and marks them in segments according to different stages of singing. At the same time, it also records metadata such as the song name and singer information, which is convenient for the subsequent classification management and in-depth analysis of data. This step ensures the comprehensiveness and accuracy of the data. Sound data analysis is the core link of the method. By performing multi-dimensional analysis on the collected sound data, including dynamic range, timbre changes, resonance effects, vocal efficiency, vocal skills and adaptability, the system can accurately evaluate the singer's voice. The system can also analyze the singers’ vocal level and existing problems, which provides a scientific basis for subsequent personalized teaching and training. By introducing machine learning algorithms and time series analysis, the relationship between sound characteristics and neck and throat muscle movements is deeply explored, which can not only reveal the mystery of the vocal mechanism, but also predict the possible changing trends of sound indicators in the future. Based on these analysis results, teachers can formulate personalized training plans for singers to improve their vocal skills and performance level. The analysis results are presented in an intuitive and easy-to-understand way. Through the real-time data dashboard and the comparison chart of historical data and predicted trends, singers and teachers can clearly see the values ​​and changing trends of various indicators. At the same time, the system can also provide early warning and detailed analysis windows to help singers find problems in time and adjust training plans. This visual presentation method makes the analysis results more intuitive and easy to understand, which is convenient for application in teaching and training practice.

[0046] Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other; Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A sound visualization analysis method for vocal music teaching, characterized in that: include: Step 1: Sensor setup: including microphone array and fiber optic sensors of neck and laryngeal muscles for sound collection and physiological data collection; Step 2: Data collection: used to collect sound data and physiological data; Step 3: Sound data analysis: used to analyze the sound data collected in step 2 to obtain dynamic range and timbre change data, resonance and vocal efficiency data, and skill and adaptability data; Step 4: Multidimensional correlation analysis: By introducing a machine learning algorithm and combining it with time series analysis, the correlation between the sound data and the neck and throat muscle movement data is comprehensively analyzed; Step 5: Visualization: This includes real-time data dashboards and historical data and forecast trend comparison charts to convert data into an intuitive form for presentation.

2. The sound visualization analysis method for vocal music teaching according to claim 1, characterized in that: The sound collection uses a multi-directional microphone array, which is arranged in a semicircular track around the singer, about 0.8-1.2 meters away from the singer, and is equipped with a professional-grade audio interface to convert the sound signal into a digital signal and transmit it to the computer; the physiological data collection uses a fiber optic sensor wrapped around the muscle groups of the neck and throat to sense muscle deformation and tension changes; The sound data is collected by the microphone array at a sampling rate of 48kHz and 24-bit quantization accuracy when the singer is practicing vocal music or singing different pieces of music. During the collection process, segmented marking is performed according to different stages of singing for subsequent targeted analysis; the physiological data is recorded by the neck and throat muscle fiber optic sensors at a sampling rate of 1000Hz to record muscle deformation and tension data; The dynamic range and timbre change data include the volume attenuation ratio from the lowest note to the highest note, the timbre brightness index, the timbre consistency when spanning the range, and the vibrato rate and amplitude ratio; the resonance and vocal efficiency data include the head cavity resonance ratio, the degree of vocal cord closure during vocalization, the breath flow and pitch stability coefficient, and the flexibility of resonance cavity conversion; the skill and adaptability data include the accuracy of ornamentation, the rhythm flexibility index, the sound intensity control accuracy, and the sound fatigue index.

3. The sound visualization analysis method for vocal music teaching according to claim 2, characterized in that: The specific method for analyzing the volume attenuation ratio from the lowest sound to the highest sound is: perform time domain analysis on the sound signal, identify the starting and ending positions of the lowest sound and the highest sound, calculate the root mean square value of the volume between the two, and use the formula The volume attenuation ratio is obtained, where is the RMS value of the lowest volume, is the root mean square value of the volume of the highest sound. The specific analysis method of the timbre brightness index is as follows: the sound signal is converted into the frequency domain using Fourier transform, and weights are assigned to different frequency bands according to the human ear's perception characteristics of sounds of different frequencies. Calculate the timbre brightness index, where is the weight of the kth frequency band, is the energy proportion of the frequency band, m represents the number of divided frequency bands, and the specific method for analyzing the consistency of timbre when the range spans is as follows: at the key transition point of the range span, the Mel frequency cepstrum coefficient feature vector of the sound is extracted, and the cosine similarity between adjacent feature vectors is calculated using the formula The timbre consistency index is obtained, where and are the i-th MFCC coefficients of two adjacent transition points, n is the dimension of the MFCC coefficient, and the specific analysis method of the vibrato rate and amplitude ratio is as follows: the instantaneous frequency of the sound signal is extracted by Hilbert transform, the vibrato part is detected, the number of frequency fluctuations per unit time is calculated to obtain the vibrato rate, and the ratio of the maximum amplitude of the frequency fluctuation to the average frequency is taken as the vibrato amplitude ratio, that is, , , Indicates the vibrato rate, Indicates the number of frequency fluctuations, Indicates time, represents the amplitude ratio, Indicates the maximum frequency fluctuation amplitude, Represents the average frequency.

4. The sound visualization analysis method for vocal music teaching according to claim 2, characterized in that: The head cavity resonance ratio analysis method is specifically as follows: based on the resonance peak analysis of the sound signal, the frequency and intensity of the resonance peak are determined by using the linear prediction coding technology, and the energy of the head cavity resonance resonance peak is compared with the total sound energy, and the formula is used. Calculate the head cavity resonance ratio, Indicates the energy of the resonance peak related to the head cavity resonance, Represents the total energy of the sound signal. The specific analysis method of the vocal cord closure during phonation is as follows: Analyze the harmonic structure of the sound signal, calculate the amplitude ratio and phase relationship between the harmonic and the fundamental wave, and use the empirical formula to estimate the vocal cord closure , and represents the amplitude and phase of the jth harmonic, and represents the amplitude and phase of the fundamental wave, and represents the empirical coefficient, p represents the number of harmonics, and the specific analysis method of the breath flow and pitch stability coefficient is as follows: combining the breath flow data collected by the respiratory sensor and the pitch change of the sound signal, calculate the ratio of the standard deviation of the pitch to the average breath flow, and the formula is: , represents the standard deviation of pitch, The analysis method of resonance cavity conversion flexibility is as follows: by monitoring the rapid change of sound signal energy in different frequency bands and the migration of resonance peaks, the change rate of resonance peak frequency and intensity in adjacent time periods is calculated as an indicator of resonance cavity conversion flexibility. The formula is , and are the resonance peak frequency and intensity at the tth moment, q represents the total number of time periods analyzed, Indicates the flexibility of resonance cavity conversion. represents the resonance peak frequency at time t+1, Represents the intensity at time t+1.

5. The sound visualization analysis method for vocal music teaching according to claim 2, characterized in that: The accuracy analysis method of the ornaments is as follows: the ornaments of the performance are compared with the ornaments in the standard score, and the errors in pitch, duration and start time between the two are calculated using the formula Calculate the accuracy index, , as well as They represent the pitch, duration and start time of the standard ornament. , as well as They represent the pitch, duration and start time of the actual singing grace note. , as well as Respectively represent the maximum values ​​of pitch, duration and start time of all ornament parameters. , as well as They represent the minimum values ​​of pitch, duration and start time of all ornamentation parameters respectively, s represents the number of ornamentations, and the analysis method of rhythm flexibility index is as follows: the actual rhythm of the performance is determined by using the beat tracking algorithm, and compared with the standard rhythm, and the statistics of rhythm deviation within a certain time window are calculated. Calculate the rhythm flexibility index, The variance of the rhythm deviation is expressed in the following way: the analysis method of the sound intensity control accuracy is as follows: the dynamic range of the sound signal is analyzed and divided into multiple levels, and the accuracy and delicacy of the conversion of the intensity between different levels in the actual singing is calculated. calculate, Indicates whether the u-th intensity conversion is accurate, 1 for accurate and 0 for inaccurate, v represents the total number of intensity conversions. The specific analysis method of the sound fatigue index is: monitor the long-term change trend of the sound signal, including the drift of the fundamental frequency and the attenuation of the harmonics, and establish a sound fatigue model using the formula Calculate the voice fatigue index, represents the change in fundamental frequency, and Respectively represent the changes in the w-th harmonic amplitude and phase, represents the amplitude of the wth harmonic at the initial moment, , as well as is the model coefficient, and x is the total number of harmonics.

6. The sound visualization analysis method for vocal music teaching according to claim 1, characterized in that: The machine learning algorithm assumes that the sound feature vector is , the neck and laryngeal muscle movement feature vector is , the prediction function obtained through training is , represents the weight corresponding to the sound feature, represents the weight corresponding to the neck and throat muscle movement characteristics, b is the bias term, n1 represents the number of elements in the sound feature vector, s i1 represents the i1th feature in the sound feature vector, k1 represents the number of elements in the neck and throat muscle movement feature vector, m j1 Represents the j1th feature in the neck and laryngeal muscle movement feature vector.

7. The sound visualization analysis method for vocal music teaching according to claim 1, characterized in that: The time series analysis was based on the sound fatigue index F t and any parameter of the neck and throat muscle movement to establish an ARIMA (p, d, q) model. Let F t After d differences, , build the model , represents the free regression coefficient, represents the moving average, represents the white noise term, p1 represents the autoregressive order, i2 is used to traverse each lag order from 1 to p1 in the autoregressive part, q1 represents the moving average order, j2 is used to traverse each lag order from 1 to q1 in the moving average part, and t1 represents the current moment.

8. The sound visualization analysis method for vocal music teaching according to claim 1, characterized in that: The real-time data dashboard displays the indicators output from step 3 and step 4 in a visual manner. Each indicator is represented by a circular dial, and the pointer points to the current numerical position in real time. The dial is divided into different color areas according to the set threshold range. Green indicates the normal range, yellow indicates approaching the warning range, and red indicates exceeding the warning range.

9. The sound visualization analysis method for vocal music teaching according to claim 1, characterized in that: The historical data and predicted trend comparison chart draws a comprehensive chart to show the singer's historical data and the predicted trend of the future by the multi-dimensional correlation model. The horizontal axis of the chart represents time, and the vertical axis represents the values ​​of various indicators. Lines of different colors are used to represent the changing trends of historical data and the future trends predicted by the model. The blue line represents the actual data changes of the sound resonance effect indicator in the past month, and the red line represents the development trend of the indicator in the next two weeks based on the model prediction. Labels and annotations are added to the chart to facilitate users to understand the changes in the data. For abnormal situations that may occur in the predicted trend, early warning annotations are made in advance. When it is predicted that the voice fatigue index will exceed the warning threshold in the next week, it is marked with a yellow triangle on the chart, and the prompt "Please note that the voice fatigue may reach a dangerous level in the next week, please adjust the training plan" is used. A detailed analysis is carried out in a graphical way, specifically, a tree diagram is used to show the contribution of each factor in the sound characteristics and neck and throat muscle movement parameters to the abnormal indicators, and different colors and line thicknesses are used to indicate the degree of influence. Clicking each factor in the tree diagram can also display detailed historical data.

Images