Sounding position determination method and device and vocal music practice system

Abstract

The invention provides a sounding position determination method and device and a vocal music practice system. The method comprises the following steps: acquiring a voice signal and a physiological signal of a sounding person; performing sound source positioning on the voice signal, and determining a first sound production position; determining a second sound production position by applying the voice signal and the physiological signal based on a sound production position determination model; and determining the sounding position of the sounding person based on the first sounding position and the second sounding position. According to the sound production position determination method and device and the vocal music practice system provided by the invention, sound source localization is performed on the voice signal, a small number of special sound production positions can be accurately detected, and meanwhile, the sound production position determination model can accurately detect the sound production position of a new sound production person based on voice, so that the sound production efficiency is improved. Therefore, the sounding position of the sounding person can be further accurately obtained by combining the first sounding position and the second sounding position.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for determining a vocalization position and a vocal training system. Background Art

[0002] The vocal position refers to the position where the airflow is blocked and the sound is produced when pronouncing. It also refers to the dominant resonance position that is activated. The vocal position can be further divided into different resonance areas, such as the head cavity, nasal cavity, pharyngeal cavity, oral cavity, laryngeal cavity and chest cavity. The vocal position has a certain degree of flexibility and adjustability. When singing or speaking, the vocal position can be adjusted as needed to obtain different sound quality and timbre.

[0003] At present, the vocal position is mostly determined by the subjective judgment of experienced personnel, that is, by observing the pitch, timbre and facial muscles of the vocalizer, the vocal position of the practitioner is guessed. However, this method mostly relies on personal experience and subjective feelings, so the determined vocal position is prone to deviation. Summary of the invention

[0004] The present invention provides a method and device for determining a vocalization position and a vocal training system, so as to solve the defects in the prior art.

[0005] The present invention provides a method for determining a sound source position, comprising the following steps: Acquire the voice signal and physiological signal of the speaker; Performing sound source localization on the speech signal to determine a first sound source position; Based on the sound location determination model, applying the speech signal and the physiological signal to determine a second sound location; The sound position of the person making the sound is determined based on the first sound position and the second sound position.

[0006] According to a method for determining a sound source position provided by the present invention, the method of applying the voice signal and the physiological signal to determine a second sound source position includes: Extracting speech features from the speech signal; Extracting physiological features from the physiological signals; The second sound emission position is determined based on the speech feature and the physiological feature.

[0007] According to a method for determining a sound position provided by the present invention, determining the second sound position based on the speech feature and the physiological feature includes: concatenating the speech feature and the physiological feature; The second sound emission position is determined based on the concatenated features.

[0008] According to a method for determining a sound position provided by the present invention, the speech signal is collected by a microphone array; The performing sound source localization on the speech signal to determine the first sound source position includes: Based on the signal difference between the microphones in the microphone array, the speech signal collected by each microphone is enhanced to obtain the target signal of each microphone; Based on the target signals of each microphone, sound source localization is performed to determine the first sound emission position.

[0009] According to a method for determining a sound position provided by the present invention, the sound position determination model is selected from a model library based on the attribute characteristics of the person making the sound, and the model library stores sound position determination models corresponding to different attribute characteristics.

[0010] According to a method for determining a sound source position provided by the present invention, the physiological signal includes a body activity signal and / or a temperature distribution signal.

[0011] According to a method for determining a sound position provided by the present invention, the method further comprises: Performing pitch detection on the speech signal to obtain a pitch detection result; Based on the pitch detection result and the vocalization position of the vocalizer, the vocalization accuracy of the vocalizer is determined.

[0012] The present invention also provides a device for determining a sound source position, comprising the following modules: An acquisition unit, used to acquire a voice signal and a physiological signal of a speaker; A positioning unit, used to perform sound source positioning on the speech signal to determine a first sound source position; A prediction unit, configured to determine a second sounding position based on a sounding position determination model and applying the speech signal and the physiological signal; A determination unit is used to determine the sound position of the person making the sound based on the first sound position and the second sound position.

[0013] The present invention also provides a vocal music practice system, comprising: A microphone array, a camera, an infrared sensor, and a device for determining the location of a sound as described above; The microphone array is used to collect the voice signal, the camera is used to collect the body activity signal in the physiological signal, and the infrared sensor is used to collect the temperature distribution signal in the physiological signal.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-described methods for determining a sound source position is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for determining the sound position as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for determining a sound source position as described above is implemented.

[0017] The vocal position determination method, device and vocal practice system provided by the present invention can accurately detect a few special vocal positions by locating the sound source of the speech signal. At the same time, the vocal position determination model can accurately detect the vocal position of a new vocal person based on the speech, so that the vocal position of the vocal person can be further accurately obtained by combining the first vocal position and the second vocal position. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of the method for determining the sound position provided by the present invention.

[0020] Figure 2 It is a flowchart diagram of the implementation of step 130 in the method for determining the sound source position provided by the present invention.

[0021] Figure 3 It is a flowchart diagram of the implementation of step 133 in the method for determining the sound source position provided by the present invention.

[0022] Figure 4 It is a flowchart diagram of the implementation of step 120 in the method for determining the sound source position provided by the present invention.

[0023] Figure 5 It is a flow chart of another method for determining a sound source position provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the device for determining the sound position provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] In the field of vocal training, the vocal teacher's intuition is mainly used to determine the student's vocal position, and the accuracy of the vocalization is determined based on the vocal position. However, vocal teachers rely on auditory senses to evaluate the student's vocal position and sound quality. This process is deeply influenced by the teacher's personal experience and subjective emotions, which can easily lead to non-objectivity in the evaluation results. Since different vocal teachers may hold different evaluation criteria and preferences, the same student's vocal performance may receive completely different feedback from different teachers. In addition, given the differences in individual physiological structures (such as cavity size and morphology), the vocalization methods and results also show diverse characteristics. However, the current vocal training strategies often follow standardized frameworks and models, which are difficult to accurately meet the personalized needs of each student, thus limiting the optimization of teaching effects. In vocal teaching, the professional terms and theoretical concepts used by teachers are often more profound and abstract, which poses a challenge to understanding for beginners. In addition, the lack of intuitive and concrete explanation and display tools makes it difficult for students to effectively identify and correct specific problems in their own vocalization process. In addition, hiring senior vocal teachers requires a high economic cost, which constitutes a significant burden for students with limited economic conditions. At the same time, vocal music teacher resources are relatively scarce, and the fixed teaching time and location limit the accessibility and flexibility of vocal music training services, making it difficult to meet the diverse needs of students.

[0028] In this regard, the present invention provides a method for determining the vocal position. By integrating multiple sensors (sound sensors, visual sensors and thermal imaging sensors), the method can capture and record the student's vocal process in real time, and conduct an objective and quantitative analysis thereof, which helps to reduce the deviation caused by the subjective judgment of vocal teachers and provide vocal learners with more interesting, convenient and personalized training.

[0029] Figure 1 is a flow chart of the method for determining the sound position provided by the present invention, such as Figure 1 As shown, the method includes step 110 , step 120 , step 130 and step 140 .

[0030] Step 110: Acquire the voice signal and physiological signal of the speaker.

[0031] Here, the person making the sound is the person whose sound position needs to be determined. The voice signal refers to the sound signal emitted by the person making the sound, which can be generated by the vibration of the vocal cords and the resonance of the sound position (i.e., the resonance cavity, such as the head cavity, nasal cavity, pharyngeal cavity, oral cavity, laryngeal cavity, chest cavity, etc.). The voice signal can usually be obtained through a microphone or other acoustic sensors.

[0032] Physiological signals refer to body signals that are directly related to the vocalization process. Physiological signals reflect physiological activities during the vocalization process, such as the movement of the vocal cords, muscle activity, and the generation of airflow. Physiological signals may include body activity signals, temperature distribution signals, and the like. Among them, body activity signals can be obtained through a camera, which is used to characterize the changes in the movements of specific parts of the speaker during the vocalization process, including oral expansion or contraction, muscle changes, and the like. Temperature distribution signals can be obtained through an infrared sensor, which is used to characterize the temperature distribution of different parts of the speaker during the vocalization process. For example, when the vocalization position is the throat cavity, the contraction of the laryngeal muscles during the vocalization process will cause the temperature to rise, and the capillary blood flow will accelerate.

[0033] Step 120: localize the sound source of the speech signal to determine the first sound source position.

[0034] Specifically, speech signals have unique spectral characteristics, which are affected by the sound source (such as human vocal cords, laryngeal cavity, nasal cavity, oral cavity, etc.) and the resonance cavity. Through sound source localization, the spectral characteristics of speech signals can be analyzed to infer the characteristics of the sound source and the type of resonance cavity.

[0035] Exemplarily, in a plurality of microphone arrays, the time for a voice signal to arrive at different microphones will be delayed to varying degrees, i.e., the arrival time difference. By analyzing the arrival time difference, the source direction of the voice signal can be determined, and further combined with the acoustic characteristics of the resonance cavity, the specific resonance cavity position, i.e., the first sound emission position, can be inferred. For example, the resonance cavity has a specific resonance frequency, and when the frequency of the voice signal matches these resonance frequencies, a resonance phenomenon occurs, which enhances the voice signal. Through sound source localization, the resonance frequency of the voice signal can be analyzed to determine the type of the resonance cavity, and then locate the first sound emission position.

[0036] Before the sound source of the speech signal is localized, the speech signal may be preprocessed, such as denoising, filtering, enhancement, etc., to improve the clarity and accuracy of the speech signal.

[0037] Step 130: Based on the sound location determination model, the speech signal and the physiological signal are applied to determine the second sound location.

[0038] Specifically, speech signals are used to characterize the characteristics of the vocalization process from an acoustic level. For example, the fundamental frequency is the frequency of the vocal cord vibration, which reflects the speaker's pitch. The formant is caused by the filtering effect of the vocal tract shape on the sound. Different vocal tract shapes will produce different formant characteristics, thus determining the timbre of the sound.

[0039] Physiological signals are used to characterize the characteristics of the vocalization process from a physiological level. For example, during the vocalization process, the states of the vocal cords, body, lips, muscles, etc. may change, and the changes corresponding to different vocalization positions are different, that is, different vocalization positions correspond to different physiological signals.

[0040] It can be seen that speech signals and physiological signals have different focuses when reflecting the vocalization process. Combining them can complement each other and improve the accuracy of determining the vocalization position.

[0041] Among them, before determining the second sound position, a sound position determination model can be constructed based on a large amount of training data (including sample speech signals, sample physiological signals and sample sound position labels). The model can be a machine learning model (such as a support vector machine, a neural network, etc.) or a deep learning model (such as a convolutional neural network, a recurrent neural network, etc.).

[0042] Step 140: Determine the sound position of the person making the sound based on the first sound position and the second sound position.

[0043] Specifically, the first sound position is determined after the sound source of the speech signal is localized, that is, the first sound position mainly relies on the acoustic characteristics of the speech signal and the corresponding acoustic standards for positioning, but the acoustic information may be affected by factors such as environmental noise and propagation medium, resulting in errors in the positioning results, and each person's physiological structure characteristics (such as vocal tract length, resonance cavity shape, etc.) are different, which may lead to limited applicability of acoustic standards among individuals.

[0044] In addition, the second sound position is determined by applying the voice signal and the physiological signal through the sound position determination model. Since the physiological signal contains the physiological structure characteristics of the person making the sound, the second sound position can be accurately obtained by combining the voice signal and the physiological signal, combining the acoustic characteristics and the individual differences of the person making the sound.

[0045] Furthermore, the vocal position determination model can adapt to different individual differences through training and learning, that is, the vocal position determination model has learning ability, can identify and learn new acoustic characteristics and individual differences, and can then continuously adapt to new changes and accurately obtain the second vocal position. Compared with the first vocal position being limited by acoustic standards, the use of the vocal position determination model can dynamically predict the vocal positions of different individuals.

[0046] In addition, the vocal position determination model can be pre-trained through sample speech signals, sample physiological signals and sample vocal position labels. Since the sample speech signals, sample physiological signals and sample vocal position labels obtain generalized feature information, and it is impossible to summarize the feature information corresponding to a few special vocal positions, the vocal position determination model cannot accurately detect a few special vocal positions. The acoustic standards can include standards corresponding to a few special vocal positions, and then the few special vocal positions can be accurately detected based on the acoustic standards.

[0047] The method for determining the sound position provided in the embodiment of the present invention can accurately detect a few special sound positions by locating the sound source of the speech signal. At the same time, the sound position determination model can accurately detect the sound position of a new speaker based on the speech, so that the sound position of the speaker can be further accurately obtained by combining the first sound position and the second sound position.

[0048] Based on the above embodiments, Figure 2 is a flow chart of an implementation of step 130 in the method for determining the sound source position provided by the present invention, such as Figure 2 As shown, step 130 uses the voice signal and the physiological signal to determine the second sound position, including: Step 131: extract speech features from the speech signal.

[0049] Specifically, speech features are parameters extracted from speech signals that can reflect the acoustic characteristics of the speaker. Speech features may include fundamental frequency, formant, spectral characteristics, cepstrum characteristics, etc. Optionally, the speech signal may be converted to the Bark domain, and speech features may be extracted from the converted speech signal. When extracting speech features of speech signals, linear predictive coding (LPC) features, (Mel-Frequency Cepstral Coefficients, MFCC) features, perceptual linear prediction (PLP) features, etc. may be used.

[0050] Step 132: extract physiological features from physiological signals.

[0051] Specifically, physiological characteristics refer to parameters extracted from physiological signals that can reflect the physiological state of the person making the sound. Physiological characteristics may include respiratory rate, respiratory depth, muscle tension, etc. Respiratory rate can be monitored by a respiratory sensor; muscle tension can be monitored by a muscle activity sensor.

[0052] Step 133: Determine a second sounding position based on the speech characteristics and the physiological characteristics.

[0053] Specifically, the speech features and physiological features reflect the information of the vocalization process from the acoustic level and the physiological level respectively. When determining the second vocalization position, a multi-feature fusion method can be used, that is, the speech features and physiological features are fused and processed, and the correlation and complementarity between them are used to improve the accuracy and reliability of the analysis, so that the influence of various factors in the vocalization process can be comprehensively considered, so as to more accurately determine the second vocalization position.

[0054] Based on any of the above embodiments, Figure 3 is a flow chart of an implementation of step 133 in the method for determining the sound position provided by the present invention, such as Figure 3 As shown, step 133 determines the second sounding position based on the voice feature and the physiological feature, including: Step 1331, concatenating speech features and physiological features; Step 1332: determine the second sound position based on the spliced ​​features.

[0055] Specifically, the speech features and the physiological features reflect the information of the vocalization process from the acoustic level and the physiological level respectively. After the speech features and the physiological features are concatenated, the concatenated features include the acoustic information and the physiological information of the vocalization process.

[0056] Among them, considering that different features may have different dimensions and value ranges, the speech features and physiological features can be standardized before splicing to ensure that they have the same weight and influence in the spliced ​​feature vector.

[0057] In addition, when splicing speech features and physiological features, they can be spliced ​​in a certain order (such as physiological features are spliced ​​after speech features) to obtain spliced ​​features, and the second sound position is determined based on the acoustic information and physiological information contained in the spliced ​​features.

[0058] Based on any of the above embodiments, the speech signal is collected by a microphone array, and the microphone array is an array composed of multiple microphones and is used to collect speech signals from different directions.

[0059] Figure 4 is a flow chart of an implementation of step 120 in the method for determining the sound position provided by the present invention, such as Figure 4 As shown, step 120 performs sound source localization on the speech signal to determine the first sound source position, including: Step 121: Based on the signal difference between the microphones in the microphone array, the speech signal collected by each microphone is enhanced to obtain the target signal of each microphone; Step 122: localize the sound source based on the target signal of each microphone to determine the first sound source position.

[0060] Specifically, a microphone array consists of multiple microphones that can be arranged in a certain geometric structure and can simultaneously collect voice signals from different directions. Due to the different distances between different microphones and the sound source, as well as the influence of factors such as environmental noise and reflection, the signals collected by each microphone will be different, that is, signal differences. Signal differences can include time differences (the time it takes for sound waves to reach different microphones is different), phase differences (the phase shift generated by sound waves on different microphones) and intensity differences (the signal strength generated by sound waves on different microphones is different).

[0061] Based on the above signal differences, signal processing technology can be used to enhance the speech signals collected by each microphone. Enhancement methods usually include noise reduction, echo removal, signal filtering, etc., aiming to improve the quality and recognizability of speech signals so that the target signal has a higher signal-to-noise ratio and clarity.

[0062] In addition, during the sound generation process, different sound generation locations generate sound waves with different frequencies and characteristics, which are affected by environmental factors during propagation, such as reflection, diffraction, etc. However, since the microphone array can capture the subtle differences in these sound waves, the sound generation location can be determined by analyzing the differences between the target signals.

[0063] For example, laryngeal voice usually has a higher frequency and a shorter wavelength, while chest voice has a lower frequency and a longer wavelength. Different sound generation locations will produce different characteristics such as time difference, phase difference, and intensity difference during the propagation of sound waves. By measuring these characteristics and combining the geometric layout of the microphone array and signal processing technology, the first sound generation location can be accurately determined.

[0064] Based on any of the above embodiments, the sound location determination model is selected from a model library based on the attribute characteristics of the person making the sound, and the model library stores sound location determination models corresponding to different attribute characteristics.

[0065] Specifically, attribute features refer to factors that can describe or distinguish the individual characteristics of different speakers, and attribute features may include age, gender, height, weight, etc. Considering that speakers with different attribute features may also have differences in physiological signals and voice signals generated by using the same voice position for voice, the embodiment of the present invention matches the corresponding voice position determination model based on the attribute features of the speaker, thereby achieving personalized matching for speakers with different attribute features, avoiding the problem of errors in positioning results caused by ignoring individual differences.

[0066] Optionally, a pre-trained model can be obtained based on a large number of sample speech signals, sample physiological signals and sample sound position labels, and then, based on the pre-trained model, the pre-trained model can be fine-tuned using speech signals, physiological signals and sound position labels of speakers with different attribute characteristics to obtain a sound position determination model with corresponding attribute characteristics.

[0067] It can be seen that in the process of selecting the sound location determination model in the embodiment of the present invention, a model matching the sound source is selected from the model library according to the attribute characteristics of the sound source person. These models are trained and optimized to more accurately determine the sound location for different sound sources.

[0068] Based on any of the above embodiments, the physiological signal includes a body activity signal and / or a temperature distribution signal.

[0069] Specifically, body activity signals refer to movement signals of specific parts of the speaker (such as ears, chin, lips, oral cavity, etc.) during the vocalization process, such as the expansion or contraction of the oral cavity. Body activity signals can be obtained through visual sensors, which continuously collect changes in specific parts of the speaker and obtain body activity signals based on the changes. For example, during the vocalization process, the speaker's muscles are detected to be changing.

[0070] The temperature distribution signal refers to the temperature information of different parts of the speaker during the vocalization process. The temperature distribution signal can be obtained by an infrared sensor. For example, when the vocalization position is the throat cavity, the contraction of the throat muscles during the vocalization process will cause the temperature to rise, and the capillary blood flow will accelerate. Among them, the temperature distribution signal can be a temperature distribution map of the speaker during the vocalization process, and the temperature distribution map is used to characterize the distribution of the body surface temperature of the speaker.

[0071] Based on any of the above embodiments, the voice position of the person making the voice is determined, and then the following step is further included: Performing pitch detection on the speech signal to obtain a pitch detection result; Based on the pitch detection result and the vocalization position of the vocalizer, the vocalization accuracy of the vocalizer is determined.

[0072] Specifically, the pitch detection result refers to the degree of consistency with the standard pitch obtained after the frequency analysis of the voice signal of the vocal person. Specifically, the pitch detection result measures the degree of match between the sound frequency emitted by the vocal person and the fixed frequency of each note in the music. If the notes sung by the vocal person are exactly the same as the standard pitch, then the pitch detection result is perfect. In other words, the pitch detection result is used to measure the accuracy of the pitch.

[0073] Vocal accuracy includes not only the accuracy of pitch, but also the correctness of the vocal position. Vocal accuracy is an important indicator for evaluating the overall performance of vocal personnel. It requires vocal personnel to not only achieve accurate pitch, but also ensure the correct vocal position when singing or playing.

[0074] Therefore, based on the pitch detection result, the accuracy of the voice speaker's pitch can be evaluated, and based on the voice speaker's voice position, whether the voice speaker's voice position is correct can be evaluated. Combining the pitch detection result and the voice speaker's position can accurately determine the voice speaker's voice accuracy.

[0075] For example, after obtaining the vocalization accuracy, a vocalization accuracy report can be generated, and based on the analysis results of the vocalization accuracy report, specific aspects that need to be improved can be determined. For example, if the intonation detection result shows that there is a large pitch deviation, then improving the intonation accuracy will be taken as an improvement goal. Similarly, if the vocalization position is incorrect or the sound quality is poor, correcting the occurrence position will be taken as an improvement goal.

[0076] Based on any of the above embodiments, Figure 5 is a flow chart of another method for determining a sound source position provided by the present invention, such as Figure 5 As shown, the method includes: S1. Data collection: The sound of the speaker and the surrounding environment is acquired through the microphone array to obtain the original signal S.

[0077] The visual sensor collects data of specific parts of the speaker, and based on the data of specific parts (such as ears, chins, and lips, etc.), constructs a vocal cavity model of the speaker. The speaker in this model is in a silent state, which is recorded as the static state M. _static In the subsequent working mode, the visual sensor continuously collects the changes in specific parts of the speaker during the vocalization process, such as muscle expansion or contraction, and records the collected change state as M _dynamic The difference between the two is recorded as ΔM (i.e. the body activity signal in the physiological signal), which represents the muscle changes of the speaker when speaking.

[0078] In addition, the temperature distribution map T of the speaker (i.e., the temperature distribution signal in the physiological signal) can be obtained through the infrared sensor. The temperature distribution map is used to characterize the temperature information of various parts of the speaker during the vocalization process.

[0079] S2. Determine the sound position: First, the original signal S is subjected to noise reduction processing, mainly including reverberation and noise reduction. The speech signal obtained at this time is S _denoise , assuming there are 6 microphones, the corresponding 6 speech signals are recorded as [S _1 ;S _2;S _3 ;S _4 ;S _5 ;S _6 ]. The above 6 speech signals are processed by beamforming to separate the external noise in the speech signals from the speech signals, and obtain the noise-reduced speech signal B1.

[0080] After obtaining the de-noised speech signal B1, the speech signal B1 is transformed into the frequency domain through Fourier transform to obtain the frequency domain signals [F1,…,F6] corresponding to the six microphones. According to the characteristics of the annular array, beamforming is performed again, and the RTF-GSC and GEVD strategies are used to divide the frequency domain signal into six regions, corresponding to six vocal positions (i.e., head cavity, nasal cavity, pharyngeal cavity, oral cavity, laryngeal cavity and chest cavity). The partition result of the frequency domain signal can be regarded as the first vocal position, and the first vocal position R1 can be represented by the sound region indicator vector V[a,b,c,d,e,f], where a+b+c+d+e+f=1, and each value represents the probability of the sound region corresponding to the vocal position appearing.

[0081] In addition, the frequency domain signal [F1, ..., F6], the body activity signal ΔM, and the temperature distribution signal T are input into the sound position determination model. The sound position determination model converts the frequency domain signal to the Bark domain, the visual signal ΔM is first convolved and then flattened into one dimension, and the temperature signal T is also first convolved and then flattened into one dimension. Finally, the feature vectors of the three modes are spliced ​​to obtain the final splicing feature Feature [B1, B2, B3, B4, B5, B6, M1, T1]. Based on the splicing feature, the second sound position is obtained. The second sound position R2 can be the sound range indication vector V _final [a,b,c,d,e,f], for example [0,0,0,0,1,0] represents the oral resonance area.

[0082] After obtaining the first sound emission position R1 and the second sound emission position R2, the weight coefficient can be used to obtain the final sound emission position R=R1×W1+R2×W2, where W1 and W2 can be iterative parameters or model training parameters.

[0083] S3. Voice accuracy: The vocal position of the speaker can be projected onto the vocal cavity model of the speaker, and the voice signal B1 of the speaker is analyzed for pitch. The vocal accuracy of the speaker is determined based on the pitch detection result and the vocal position.

[0084] The sound location determination device provided by the present invention is described below. The sound location determination device described below and the sound location determination method described above can be referenced to each other.

[0085] Based on any of the above embodiments, Figure 6 is a schematic diagram of the structure of the device for determining the sound position provided by the present invention. Figure 6 As shown, the device comprises: An acquisition unit 610 is used to acquire a voice signal and a physiological signal of a speaker; A positioning unit 620, configured to perform sound source positioning on the speech signal and determine a first sound source position; A prediction unit 630, configured to determine a second sounding position based on a sounding position determination model and applying a speech signal and a physiological signal; The determination unit 640 is used to determine the sound position of the person making the sound based on the first sound position and the second sound position.

[0086] Based on any of the above embodiments, applying the voice signal and the physiological signal to determine the second sound emission position includes: Extract speech features from speech signals; Extract physiological features from physiological signals; Based on the voice features and the physiological features, a second vocalization position is determined.

[0087] Based on any of the above embodiments, determining the second sound emission position based on the speech feature and the physiological feature includes: splicing phonetic features and physiological features; The second sound position is determined based on the concatenated features.

[0088] Based on any of the above embodiments, the speech signal is collected by a microphone array; Performing sound source localization on the speech signal to determine the first sound source position includes: Based on the signal difference between the microphones in the microphone array, the speech signal collected by each microphone is enhanced to obtain the target signal of each microphone; Based on the target signals of each microphone, sound source localization is performed to determine the first sound emission position.

[0089] Based on any of the above embodiments, the sound location determination model is selected from a model library based on the attribute characteristics of the person making the sound, and the model library stores sound location determination models corresponding to different attribute characteristics.

[0090] Based on any of the above embodiments, the physiological signal includes a body activity signal and / or a temperature distribution signal.

[0091] Based on any of the above embodiments, the voice position of the person making the voice is determined, and then the following step is further included: Performing pitch detection on the speech signal to obtain a pitch detection result; Based on the pitch detection result and the vocalization position of the vocalizer, the vocalization accuracy of the vocalizer is determined.

[0092] Based on any of the above embodiments, the present invention further provides a vocal practice system, comprising: A microphone array, a camera, an infrared sensor, and a device for determining a sound position as described in any of the above embodiments; The microphone array is used to collect voice signals, the camera is used to collect body activity signals in physiological signals, and the infrared sensor is used to collect temperature distribution signals in physiological signals.

[0093] Optionally, the vocal practice system can be head-mounted, that is, the vocal practice system can be worn on the head of the vocal person. The vocal practice system can be provided with a microphone array to collect voice signals, and small cameras can be arranged on both sides and the back of the head-mounted device to collect the head shape and neck data of the vocal person, and then build a vocal cavity model based on the collected data. In addition, the small camera continuously collects body activity signals during the vocalization process. Furthermore, the vocal practice system can also be provided with an infrared sensor to collect temperature distribution signals in physiological signals.

[0094] Based on the voice signals, body activity signals and temperature distribution signals collected by the system, the vocal position of the person speaking can be accurately determined, which greatly improves the practice experience of vocal practitioners and lowers the threshold for scientific vocalization.

[0095] Figure 7 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the method for determining the sound position, which includes: obtaining a voice signal and a physiological signal of a person who makes a sound; locating the sound source of the voice signal to determine a first sound position; based on a sound position determination model, applying the voice signal and the physiological signal to determine a second sound position; and determining the sound position of the person who makes the sound based on the first sound position and the second sound position.

[0096] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0097] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the sound position determination method provided by the above-mentioned methods, which includes: obtaining a voice signal and a physiological signal of a person who makes a sound; locating the sound source of the voice signal to determine a first sound position; based on a sound position determination model, applying the voice signal and the physiological signal to determine a second sound position; and determining the sound position of the person who makes the sound based on the first sound position and the second sound position.

[0098] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the sound position determination method provided by the above-mentioned methods, the method comprising: acquiring a voice signal and a physiological signal of a person making a sound; locating the sound source of the voice signal to determine a first sound position; based on a sound position determination model, applying the voice signal and the physiological signal to determine a second sound position; and determining the sound position of the person making the sound based on the first sound position and the second sound position.

[0099] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining a sound source position, characterized in that: include: Acquire the voice signal and physiological signal of the speaker; Performing sound source localization on the speech signal to determine a first sound source position; Based on the sound location determination model, applying the speech signal and the physiological signal to determine a second sound location; The sound position of the person making the sound is determined based on the first sound position and the second sound position.

2. The method for determining the sound position according to claim 1, characterized in that: The applying the voice signal and the physiological signal to determine the second sound emission position includes: Extracting speech features from the speech signal; Extracting physiological features from the physiological signals; The second sound emission position is determined based on the speech feature and the physiological feature.

3. The method for determining the sound position according to claim 2, characterized in that: The determining the second sound emission position based on the speech feature and the physiological feature includes: concatenating the speech feature and the physiological feature; The second sound emission position is determined based on the concatenated features.

4. The method for determining the sound source position according to any one of claims 1 to 3, characterized in that: The speech signal is collected by a microphone array; The performing sound source localization on the speech signal to determine the first sound source position includes: Based on the signal difference between the microphones in the microphone array, the speech signal collected by each microphone is enhanced to obtain the target signal of each microphone; Based on the target signals of each microphone, sound source localization is performed to determine the first sound emission position.

5. The method for determining the sound source position according to any one of claims 1 to 3, characterized in that: The sound position determination model is selected from a model library based on the attribute characteristics of the person making the sound, and the model library stores sound position determination models corresponding to different attribute characteristics.

6. The method for determining the sound source position according to any one of claims 1 to 3, characterized in that: The physiological signal includes a body activity signal and / or a temperature distribution signal.

7. The method for determining the sound source position according to any one of claims 1 to 3, characterized in that: The step of determining the sound position of the sounding person further includes: Performing pitch detection on the speech signal to obtain a pitch detection result; Based on the pitch detection result and the vocalization position of the vocalizer, the vocalization accuracy of the vocalizer is determined.

8. A device for determining a sound source position, characterized in that: include: An acquisition unit, used to acquire a voice signal and a physiological signal of a speaker; A positioning unit, used to perform sound source positioning on the speech signal to determine a first sound source position; A prediction unit, configured to determine a second sounding position based on a sounding position determination model and applying the speech signal and the physiological signal; A determination unit is used to determine the sound position of the person making the sound based on the first sound position and the second sound position.

9. A vocal training system, characterized in that: include: A microphone array, a camera, an infrared sensor, and a device for determining a sound position as claimed in claim 8; The microphone array is used to collect the voice signal, the camera is used to collect the body activity signal in the physiological signal, and the infrared sensor is used to collect the temperature distribution signal in the physiological signal.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for determining the sound source position according to any one of claims 1 to 7 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining the sound source position according to any one of claims 1 to 7 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining the sound source position according to any one of claims 1 to 7 is implemented.

Images