A speech recognition method and system for rail transit customer service robot

Abstract

The present invention relates to the field of speech recognition technology, and specifically to a speech recognition method and system for a rail transit customer service robot, the method comprising: processing a speech signal to obtain a time series spectrum diagram and a time domain waveform diagram; determining a first reference frequency and an attribution interval of the first reference frequency according to the speech frequency at different times in the time series spectrum diagram; determining a main trend index of the attribution interval and an adjacent trend index of a time series sub-interval according to the signal amplitude at different times in the time domain waveform diagram; determining the occlusion frequency of the speech signal by using the signal amplitude at different times in the time series sub-interval, the adjacent trend index and the main trend index; determining the frequency interval of the occlusion frequency in the time series spectrum diagram, setting the spectrum value of the frequency interval to a preset value, and obtaining enhanced speech data; performing speech recognition on the enhanced speech data. In this way, the present invention improves the accuracy of the speech recognition results of the rail transit customer service robot.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speech recognition method and system for a rail transit customer service robot. Background Art

[0002] Rail transit customer service robots are intelligent robots used in public transportation systems such as subways, light rails, and high-speed rails. They use technologies such as speech recognition, natural language processing, and artificial intelligence to interact with passengers, answer questions raised by passengers, and provide customer services such as ticket purchases, transfers, and route inquiries. The accuracy of the speech recognition of rail transit customer service robots can directly affect the passenger's service experience and travel efficiency. Highly accurate speech recognition can quickly and accurately understand passengers' inquiries and provide key information such as route inquiries, ticket information, and train schedules.

[0003] In some scenarios, since passengers come from different regions, their dialects and language habits are different, and some passengers' expressions differ from standard Mandarin, which affects the accuracy of the rail transit customer service robot's voice recognition. Therefore, due to the diversity of passengers' accents and the high noise in the rail transit environment, the rail transit customer service robot's voice recognition environment is relatively complex, and the rail transit customer service machine may find it difficult to accurately recognize the voices of passengers with different accents. The rail transit customer service machine's voice recognition has great limitations, resulting in low accuracy of the rail transit customer service robot's voice recognition results. Summary of the invention

[0004] In order to solve the technical problem that the accuracy of the speech recognition results of the rail transit customer service robot is low, the purpose of the present invention is to provide a speech recognition method and system for the rail transit customer service robot. The technical solution adopted is as follows:

[0005] In the first aspect, an embodiment of the present invention provides a speech recognition method for a rail transit customer service robot, comprising: processing the speech signal of a user and the rail transit customer service robot during the conversation to obtain a time series spectrum diagram and a time domain waveform diagram; determining a first reference frequency and an attribution interval of the first reference frequency according to the speech frequency at different times in the time series spectrum diagram; determining a main trend index of the attribution interval and an adjacent trend index of a time series sub-interval within the attribution interval according to the signal amplitude at different times in the time domain waveform diagram, the main trend index representing the possibility that the attribution interval belongs to a keyword field; determining the occlusion frequency of the speech signal by using the signal amplitude at different times of the time series sub-interval, the adjacent trend index of the time series sub-interval, and the main trend index of the attribution interval to which the time series sub-interval belongs; determining the frequency interval of the occlusion frequency in the time series spectrum diagram, setting the spectrum value of the frequency interval to a preset value to occlude the speech frequency within the frequency interval, and obtaining enhanced speech data; performing speech recognition on the enhanced speech data.

[0006] Optionally, determining the first reference frequency and the interval to which the first reference frequency belongs based on the speech frequencies at different moments in the time series spectrum graph includes: obtaining the maximum speech frequency and the minimum speech frequency from the time series spectrum graph, and determining the interval between the maximum speech frequency and the minimum speech frequency as the frequency coverage interval; dividing the frequency coverage interval into multiple sub-intervals; taking the average value of the speech frequency at each moment in the sub-interval as the first reference frequency; determining the frame to which the first reference frequency belongs based on the main frequency of any frame in the time series spectrum graph and the first reference frequency of each sub-interval; and determining the time series interval consisting of time-series continuous frames to which the first reference frequency belongs as the interval to which the first reference frequency belongs.

[0007] Optionally, determining the main trend index of the belonging interval and the adjacent trend index of the time series sub-interval within the belonging interval according to the signal amplitude at different moments of the time domain waveform graph includes: determining the fluctuation change index at different moments according to the signal amplitude at each moment of the time domain waveform graph, the fluctuation change index being the difference between the maximum value of the adjacent moment adjacent to the current moment and the signal amplitude at the moment after the adjacent moment; clustering the fluctuation change indexes of all moments within the belonging interval to obtain multiple first clusters; determining the main trend index of the belonging interval according to the first number of moments included in the first cluster and the average index of the fluctuation change index of each moment in the first cluster; determining the time series interval composed of continuous moments belonging to the same cluster within the belonging interval as the time series sub-interval, and determining the mean of the signal amplitude of all moments in the time series sub-interval as the characteristic amplitude of the time series sub-interval; determining the adjacent trend index of the current time series sub-interval using the first characteristic amplitude of the current time series sub-interval within the belonging interval, the second characteristic amplitude of the previous first time series sub-interval adjacent to the current time series sub-interval, the first central moment of the current time series sub-interval and the second central moment of the first time series sub-interval.

[0008] Optionally, determining the main trend index of the belonging interval based on the first number of moments included in the first cluster and the average index of the fluctuation change index at each moment in the first cluster includes: processing the first number of each first cluster using a probability distribution calculation function to obtain a processing result; calculating the first product of the processing result of each first cluster and the average index of the first cluster, and superimposing each first product to obtain the main trend index of the belonging interval.

[0009] Optionally, determining the proximity trend index of the current time subinterval using the first characteristic amplitude of the current time subinterval within the belonging interval, the second characteristic amplitude of the previous first time subinterval adjacent to the current time subinterval, the first center moment of the current time subinterval and the second center moment of the first time subinterval includes: calculating the first difference between the first characteristic amplitude and the second characteristic amplitude, and the second difference between the first center moment and the second center moment; determining a first ratio of the first difference to the second difference as the proximity trend index of the current time subinterval.

[0010] Optionally, the occlusion frequency of the speech signal is determined by using the signal amplitude at different moments of the timing subinterval, the proximity trend index of the timing subinterval and the main trend index of the belonging interval to which the timing subinterval belongs, including: determining the characteristic amplitude of each timing subinterval within the belonging interval by using the signal amplitude at different moments of each timing subinterval; determining the trend of the belonging interval according to the characteristic amplitude of each timing subspace, the proximity trend index of each timing subinterval and the main trend index of the belonging interval to which the timing subinterval belongs; and determining the occlusion frequency of the speech signal by using the trend of the belonging interval.

[0011] Optionally, determining the trend of the belonging interval based on the characteristic amplitude of each time series subspace, the adjacent trend index of each time series subinterval, and the main trend index of the belonging interval to which the time series subinterval belongs includes: determining the first characteristic amplitude of the current time series subinterval, the previous first time series subinterval adjacent to the current time series subinterval, the third characteristic amplitude of at least one second time series subinterval before the first time series subinterval, the adjacent trend index of the second time series subinterval, the first center moment of the current time series subinterval, and the third center moment of the second time series subinterval; determining the revised trend index of the current time series subinterval based on the third characteristic amplitude, the adjacent trend index of the second time series subinterval, the first characteristic amplitude, the first center moment, the third center moment, and the main trend index of the belonging interval; determining the trend of the belonging interval based on the second number of each time series subinterval in the belonging interval, the main trend index, and the revised trend index of each time series subinterval in the belonging interval.

[0012] Optionally, using the trend of the belonging interval to determine the occlusion frequency of the speech signal includes: determining the merging probability of the current belonging interval based on the minimum value of the trends among the current belonging interval and other belonging intervals, and the trend of the current belonging interval; determining the keyword character interval based on the merging probability of each belonging interval, wherein the keyword character interval includes multiple continuous belonging intervals; for any belonging interval in the keyword character interval, taking the duration of the keyword character interval in which any belonging interval is located as the retention index of any belonging interval; determining the second reference frequency of all belonging intervals in each keyword character interval; for any second reference frequency, taking the sum of the retention indexes of all belonging intervals of the second reference frequency as the retention degree of the second reference frequency; clustering the retention degrees of all second reference frequencies to obtain multiple second clusters; calculating the average retention degree in each second cluster; and determining the average value of the second reference frequency in the second cluster with the smallest average retention degree as the occlusion frequency.

[0013] Optionally, determining the revised trend index of the current time series sub-interval based on the third characteristic amplitude, the adjacent trend index of the second time series sub-interval, the first characteristic amplitude, the first central moment, the third central moment and the main trend index of the belonging interval includes: calculating the third difference between the first characteristic amplitude and each third characteristic amplitude, and the fourth difference between the first central moment and each third central moment; calculating the second ratio of each third difference to each fourth difference; calculating the absolute value of the fifth difference between each third characteristic amplitude and the first characteristic amplitude, and selecting the maximum value from the absolute values ​​of each fifth difference; calculating the maximum value using a probability distribution calculation function to obtain a calculation result; determining the second product between each second ratio and the calculation result, and superimposing each second product to obtain the revised trend index of the current time series sub-interval.

[0014] In a second aspect, an embodiment of the present invention provides a speech recognition system for a rail transit customer service robot, comprising: a processor and a memory; wherein the memory is used to store a computer program that can be run on the processor; and the processor is used to execute the program stored in the memory to implement the steps of the speech recognition method for a rail transit customer service robot as mentioned in the first aspect.

[0015] The present invention has the following beneficial effects: firstly, the voice signal of the user and the rail transit customer service robot during the conversation is processed to obtain a time series spectrum diagram and a time domain waveform diagram; then, the first reference frequency and the belonging interval of the first reference frequency are determined according to the voice frequency at different times in the time series spectrum diagram; then, the main trend index of the belonging interval and the adjacent trend index of the time series sub-interval in the belonging interval are determined according to the signal amplitude at different times in the time domain waveform diagram, and the main trend index represents the possibility that the belonging interval belongs to the keyword field; secondly, the signal amplitude at different times of the time series sub-interval, the adjacent trend index of the time series sub-interval and the main trend index of the belonging interval to which the time series sub-interval belongs are used to determine the occlusion frequency of the voice signal; and the frequency interval of the occlusion frequency in the time series spectrum diagram is determined, and the spectrum value of the frequency interval is set to a preset value to occlude the voice frequency within the frequency interval to obtain enhanced voice data; finally, voice recognition is performed on the enhanced voice data.

[0016] In this way, the embodiment of the present invention can determine the main trend index of the belonging interval of each reference frequency according to the time-series spectrum diagram of the voice signal. The main trend index can ensure the possibility that the belonging interval belongs to the keyword, so as to determine the occlusion frequency of the voice signal according to the main trend index and the adjacent trend index of each time-series sub-interval in the belonging interval. The voice signal within the spectrum interval where the occlusion frequency is located may be non-keyword fields such as noise, modal particles and accent words. Therefore, the voice frequency within the frequency interval of the occlusion frequency is occluded to obtain enhanced voice data. The enhanced voice data removes non-keyword data such as noise other than keywords. When the enhanced voice data is subjected to voice recognition, the influence of noise and other factors on the recognition results is avoided, the limitations of the voice recognition of the rail transit customer service machine are reduced, and the accuracy of the voice recognition results of the rail transit customer service robot is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flow chart of a speech recognition method for a rail transit customer service robot provided by one embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a time domain waveform diagram provided by an embodiment of the present invention;

[0020] Figure 3A schematic diagram of a time series spectrum diagram provided by an embodiment of the present invention;

[0021] Figure 4 A schematic structural diagram of a speech recognition system for a rail transit customer service robot provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the speech recognition method and system for a rail transit customer service robot proposed in accordance with the present invention, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0023] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0024] The following is a detailed description of a speech recognition method and system for a rail transit customer service robot provided by the present invention in conjunction with the accompanying drawings.

[0025] Embodiment 1:

[0026] See also Figure 1 , which shows a flow chart of a speech recognition method for a rail transit customer service robot provided by an embodiment of the present invention, including:

[0027] S101, processing the voice signal of the user during the conversation with the rail transit customer service robot to obtain a time series spectrum diagram and a time domain waveform diagram.

[0028] Specifically, during the conversation between the rail transit customer service robot and the user, the user's voice is sampled at a fixed sampling rate of 16kHz to obtain a voice signal. The voice signal is then processed. The processing process is as follows: the voice signal is divided into several frames, each of which can be 25ms, and a short-time Fourier transform is applied to each frame to convert the time domain signal into a frequency domain signal to generate a time series spectrum diagram. The voice signal is converted into a time domain waveform diagram. For example, Figure 2 A schematic diagram of a time domain waveform diagram provided by an embodiment of the present invention, Figure 3A schematic diagram of a time series spectrum diagram provided for an embodiment of the present invention. Generally, when a user utters a keyword, the loudness of the voice will have a natural fluctuation. At the beginning of a keyword, the vibration of the vocal cords gradually increases, causing the signal amplitude of each interval in the time domain waveform diagram to gradually increase from a small value. When the keyword is pronounced, the signal amplitude will gradually decrease. This trend reflects the natural process of the user's voice from the beginning to the end. Figure 3 The time series spectrum in the figure reflects the frequency change of the speech signal. Figure 3 It can be seen that it contains the maximum speech frequency and the minimum speech frequency.

[0029] S102, determining a first reference frequency and an attribution interval of the first reference frequency according to speech frequencies at different moments in the time series spectrum diagram.

[0030] Specifically, when the rail transit customer service robot provides assistance to users, it mainly matches relevant information from the knowledge base containing real-time train data to provide assistance to users. When recognizing the user's voice, it mainly extracts keywords from the user's voice. The keywords are matched with the information in the knowledge base, so the premise is to accurately extract keywords from the user's voice. In the process of extracting keywords, in order to address the problems of user regional dialects and language habits, data enhancement technology is currently used to simulate different speech loss scenarios by blocking some frequency intervals in the time series spectrogram, thereby enhancing the model's robustness to dialects and accents. When blocking the time series spectrogram, it is necessary to suppress the frequency intervals containing noise and non-keywords, retain the frequencies containing keywords, and improve the accuracy of keyword extraction.

[0031] Further, when determining the first reference frequency and the interval to which the first reference frequency belongs, as an optional embodiment of the present invention, firstly, the maximum speech frequency and the minimum speech frequency are obtained from the time series spectrum diagram, and the interval between the maximum speech frequency and the minimum speech frequency is determined as the frequency coverage interval; then the frequency coverage interval is divided into multiple sub-intervals; then the average value of the speech frequency at each moment in the sub-interval is taken as the first reference frequency; secondly, based on the main frequency of any frame in the time series spectrum diagram and the first reference frequency of each sub-interval, the frame to which the first reference frequency belongs is determined; finally, the time series interval composed of the time-series continuous frames to which the first reference frequency belongs is determined as the interval to which the first reference frequency belongs.

[0032] Specifically, the number of sub-intervals into which the frequency coverage interval is divided can be determined according to actual conditions, and in the embodiment of the present invention, the value is 10. The average frequency of each sub-interval is obtained respectively, and it is recorded as a first reference frequency. Since the timing information is lost in the time series spectrum diagram, it is impossible to perform time series audio signal analysis based on the time series spectrum diagram, so the first reference frequencies in the time series spectrum diagram are corresponded in time series, and then the keywords can be further judged based on the change law of the signal amplitude of the interval corresponding to each first reference frequency.

[0033] Among them, the process of obtaining the main frequency of any frame in the time series spectrum diagram is as follows: obtain the spectrum diagram of each frame, the horizontal axis is the frequency value, and the vertical axis is the energy value. For any frequency value in the spectrum diagram of any frame, the frequency degree of the frequency value is calculated by the formula: softmax (energy value corresponding to the frequency value) × the frequency value, and then the frequency degrees of all frequency values ​​in the frame are accumulated to obtain the main frequency of the frame. Further, in the time series spectrum diagram, for any frame, the absolute value of the difference between the main frequency of the frame and each first reference frequency is calculated respectively, and the first reference frequency with the smallest absolute value of the difference is obtained, and the frame is recorded as the belonging frame of the first reference frequency with the smallest absolute value of the difference. For any first reference frequency, the time series interval composed of the belonging frames of the first reference frequency that are continuous in time series is recorded as an belonging interval of the first reference frequency.

[0034] S103, determining the main trend index of the belonging interval and the adjacent trend index of the time series sub-interval within the belonging interval according to the signal amplitude at different times of the time domain waveform diagram.

[0035] Among them, the main trend index represents the possibility that the belonging interval belongs to the keyword field.

[0036] Specifically, when a user pronounces a keyword, the loudness of the sound will naturally fluctuate. At the beginning of a keyword, the vibration of the vocal cords gradually increases, causing the signal amplitude of the waveform in the time domain waveform graph to gradually increase from a small value. When the keyword pronunciation ends, the signal amplitude will gradually decrease. This trend reflects the natural process of speech from utterance to end. Therefore, within a belonging interval, if the waveform in the time domain waveform graph has a trend, the stronger the trend, it means that the passenger is speaking.

[0037] Further, when determining the main trend index of the belonging interval and the adjacent trend index of the time series sub-interval within the belonging interval, as an optional embodiment of the present invention, first, the fluctuation change index at different moments is determined according to the signal amplitude at each moment of the time domain waveform diagram, and the fluctuation change index is the difference between the maximum value of the adjacent moment adjacent to the current moment and the signal amplitude at the next moment after the adjacent moment; then the fluctuation change indexes of all moments in the belonging interval are clustered to obtain multiple first clusters; then the main trend index of the belonging interval is determined according to the first number of moments included in the first cluster and the average index of the fluctuation change index of each moment in the first cluster; secondly, the time series interval composed of continuous moments belonging to the same cluster within the belonging interval is determined as the time series sub-interval, and the mean value of the signal amplitude of all moments in the time series sub-interval is determined as the characteristic amplitude of the time series sub-interval; finally, the adjacent trend index of the current time series sub-interval is determined using the first characteristic amplitude of the current time series sub-interval within the belonging interval, the second characteristic amplitude of the previous first time series sub-interval adjacent to the current time series sub-interval, the first central moment of the current time series sub-interval and the second central moment of the first time series sub-interval.

[0038] Specifically, for any moment, in the time domain waveform diagram, the maximum value adjacent to the moment is obtained, and the maximum value is used as the fluctuation degree of the moment. The moment after the moment in the time sequence with a different fluctuation degree from the moment is obtained, recorded as the rear moment of the moment, and the difference between the fluctuation degree of the moment and the fluctuation degree of the rear moment is recorded as the fluctuation change index of the moment. In the trend calculation process, although the overall amplitude change of the keyword usually shows a trend, the amplitude of the phoneme is often small, or even close to zero. This will cause the amplitude to drop or low value briefly in the process of gradually increasing or decreasing, which is contrary to the overall trend, so it is necessary to reduce the weight of these data in the trend calculation process. Therefore, the embodiment of the present invention clusters the fluctuation change index of all moments in each belonging interval, wherein the clustering algorithm can adopt a density-based clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN). Since the time for the amplitude drop caused by the phoneme is relatively short, the longer the cluster occupies in an belonging interval, the more it belongs to the main change trend of this belonging interval.

[0039] Furthermore, when determining the main trend index of the belonging interval, as an optional embodiment of the present invention, the first quantity of each first cluster is processed using a probability distribution calculation function to obtain a processing result; the first product of the processing result of each first cluster and the average index of the first cluster is calculated, and the first products are superimposed to obtain the main trend index of the belonging interval.

[0040] Specifically, the embodiment of the present invention is based on the As an example, the following formula is used to calculate the The main trend index of the belonging interval:

[0041]

[0042] In the above formula, For the The main trend index of the belonging interval. For the The number of the first cluster in the belonging interval. For the The first The first number of moments contained in the first cluster. For the The first The average index of the fluctuation change index at all moments in the first cluster. Evaluates functions for probability distributions.

[0043] Furthermore, after obtaining the main trend index of a single attribution interval, in order to judge the trend within the single attribution interval, it is necessary to judge whether each sub-interval within the single attribution interval conforms to the main trend. In any attribution interval, the time series interval composed of consecutive moments belonging to the same first cluster is recorded as a time series sub-interval, and for any time series sub-interval, the mean of the signal amplitudes at all moments in the time series sub-interval is recorded as the characteristic amplitude of the time series sub-interval. When determining the proximity trend index of the current time series sub-interval, as an optional embodiment of the present invention, the first difference between the first characteristic amplitude and the second characteristic amplitude, and the second difference between the first center moment and the second center moment are calculated; the first ratio of the first difference to the second difference is determined as the proximity trend index of the current time series sub-interval.

[0044] Specifically, the present invention is based on the The first Taking the current time series sub-interval as an example, the following formula is used to calculate the approaching trend index of the current time series sub-interval:

[0045]

[0046] In the above formula, For the The first The approaching trend index of a time series subinterval. For the The first The first characteristic amplitude of a time series subinterval. For the The first The second characteristic amplitude of a time series sub-interval is the second characteristic amplitude of the previous first time series sub-interval adjacent to the current time series sub-interval. For the The first The first central moment of a time series subinterval. For the The first The second central moment of a time subinterval.

[0047] S104, determining the occlusion frequency of the speech signal by using the signal amplitudes at different times of the time series subinterval, the proximity trend index of the time series subinterval, and the main trend index of the belonging interval to which the time series subinterval belongs.

[0048] Specifically, when determining the occlusion frequency of a speech signal, the trend of each belonging interval can be used to determine it. First, the characteristic amplitude of each time subinterval within the belonging interval is determined using the signal amplitude at different times of each time subinterval; then, the trend of the belonging interval is determined based on the characteristic amplitude of each time subspace, the adjacent trend index of each time subinterval, and the main trend index of the belonging interval to which the time subinterval belongs; finally, the occlusion frequency of the speech signal is determined using the trend of the belonging interval.

[0049] Further, when determining the trend of the attribution interval, as an optional embodiment of the present invention, the first characteristic amplitude of the current time sub-interval, the previous first time sub-interval adjacent to the current time sub-interval, the third characteristic amplitude of at least one second time sub-interval before the first time sub-interval, the proximity trend index of the second time sub-interval, the first center moment of the current time sub-interval and the third center moment of the second time sub-interval are determined; the modified trend index of the current time sub-interval is determined based on the third characteristic amplitude, the proximity trend index of the second time sub-interval, the first characteristic amplitude, the first center moment, the third center moment and the main trend index of the attribution interval; the trend of the attribution interval is determined based on the second number of each time sub-interval in the attribution interval, the main trend index and the modified trend index of each time sub-interval in the attribution interval.

[0050] Specifically, due to the decrease in signal amplitude caused by the phoneme, the proximity trend index of the time series interval adjacent to the time series sub-interval where the phoneme is located is corrected. When the proximity trend index of the time series sub-interval deviates greatly from the main trend index of the belonging interval, it is likely to be affected by the phoneme. Therefore, after ignoring the time series sub-interval affected by the phoneme, if the proximity trend index of the time series sub-interval before the time series sub-interval affected by the phoneme is similar to the main trend index of the belonging interval, it indicates that the time series sub-interval conforms to the trend.

[0051] Furthermore, when determining the corrected trend index of the current time series sub-interval, the embodiment of the present invention first calculates the third difference between the first characteristic amplitude and each third characteristic amplitude, and the fourth difference between the first center moment and each third center moment; then calculates the second ratio of each third difference to each fourth difference; then calculates the absolute value of the fifth difference between each third characteristic amplitude and the first characteristic amplitude, and selects the maximum value from the absolute values ​​of each fifth difference; secondly, calculates the maximum value using the probability distribution calculation function to obtain a calculation result; finally, determines the second product between each second ratio and the calculation result, and superimposes each second product to obtain the corrected trend index of the current time series sub-interval.

[0052] Specifically, the embodiment of the present invention is based on the The first Taking the current time series sub-interval as an example, the following formula is used to calculate the modified trend index of the current time series sub-interval:

[0053]

[0054] In the above formula, For the The first Modified trend index for a time series subinterval. For the The first The first characteristic amplitude of a time series subinterval. For the The first The first time subinterval before The characteristic amplitude of the first time sub-interval, that is, the third characteristic amplitude of at least one second time sub-interval before the first time sub-interval, the first time sub-interval is The previous adjacent timing subinterval of a timing subinterval. For the The first The first central moment of a time series subinterval. For the The first The first time subinterval before The second central moment of a time subinterval. For the The first The first time subinterval before The approaching trend index of a time series subinterval. For the The main trend index of the belonging interval. Indicates that from The first The time sub-interval to The maximum value of the absolute values ​​of the differences between the third characteristic amplitude and the first characteristic amplitude of all the time series sub-intervals between the time series sub-intervals is selected, that is, the maximum value is selected from the absolute values ​​of the fifth differences. Evaluates functions for probability distributions. Indicates The first The number of second timing subintervals preceding the adjacent first timing subinterval of the timing subintervals.

[0055] Further, when determining the trend of the attribution interval, as an optional embodiment of the present invention, the absolute value of the sixth difference between the modified trend index and the main trend index of each time sub-interval is first calculated; the third ratio of the superposition value of the absolute values ​​of each sixth difference and the second quantity is calculated; and the absolute value of the third ratio and the main trend index is determined to be the trend of the attribution interval.

[0056] Specifically, the embodiment of the present invention is based on the Taking the belonging interval as an example, the trend of the belonging interval is calculated using the following formula:

[0057]

[0058] In the above formula, For the The trend of the belonging interval. For the A second number of timing subintervals in the belonging interval. For the The first Modified trend index for a time series subinterval. For the The main trend index of the belonging interval.

[0059] Furthermore, after obtaining the trend of each attribution interval, the higher the trend of the attribution interval, the more likely it is that it belongs to the utterance time interval of the keyword. Since the data enhancement technology simulates different speech loss scenarios by blocking the time series spectrum graph, it is necessary to determine the frequency range containing the keyword. Since a single keyword in the keyword will contain multiple attribution intervals during the pronunciation process, the time series sub-intervals with higher trends are clustered, that is, the attribution intervals corresponding to a single keyword will be clustered in a continuous time series sub-interval, so the trend of the attribution intervals in the continuous time series sub-intervals is relatively high, so multiple continuous attribution intervals that may belong to the keyword are merged to determine the frequency range of the keyword. When merging attribution intervals, if the trends of the attribution intervals in the neighborhood of a single attribution interval are relatively high, the greater the possibility that the attribution interval and the attribution interval of the neighborhood are the same keyword, the greater the probability of merging.

[0060] Further, when using the trend of the belonging interval to determine the occlusion frequency of the speech signal, as an optional embodiment of the present invention, first, according to the minimum value of the trend between the current belonging interval and other belonging intervals, and the trend of the current belonging interval, the merging probability of the current belonging interval is determined; then, according to the merging probability of each belonging interval, the keyword character interval is determined, and the keyword character interval includes multiple continuous belonging intervals; secondly, for any belonging interval in the keyword character interval, the duration of the keyword character interval in which any belonging interval is located is used as the retention index of any belonging interval; then the second reference frequency of all the belonging intervals in each keyword character interval is determined; for any second reference frequency, the sum of the retention indexes of all the belonging intervals of the second reference frequency is used as the retention degree of the second reference frequency; then the retention degrees of all the second reference frequencies are clustered to obtain multiple second clusters; and the average retention degree in each second cluster is calculated; finally, the average value of the second reference frequency in the second cluster with the smallest average retention degree is determined as the occlusion frequency.

[0061] Specifically, the embodiment of the present invention is based on the Taking the current belonging interval as an example, the following formula is used to calculate the merging probability of the current belonging interval:

[0062]

[0063] In the above formula, For the The combined probability of belonging intervals. is the number of belonging intervals. For the The belonging interval and The minimum trend between the belonging intervals. For the The trend of the belonging interval. is the normalization function, which is used to Normalize.

[0064] Furthermore, when determining the keyword character interval according to the merging probability of each belonging interval, the merging probability of all belonging intervals is clustered using the K-means algorithm. Since the entire speech part is divided into keywords and non-keywords, K=2 is taken in the embodiment of the present invention to obtain two clusters, and the cluster with the highest average merging probability is recorded as the merging class, and the interval composed of continuous belonging intervals in the merging class is recorded as a keyword character interval.

[0065] Furthermore, in the process of communication between the user and the rail transit customer service robot, there may be more modal particles, which contain less information, so the keywords are further screened. The keyword is often the part of the voice that conveys the main information, and is usually pronounced more clearly and slowly. When expressing the keyword, the speaker will intentionally or unintentionally slow down the speech speed to make it easier for the rail transit customer service robot to understand. This means that the pronunciation time of the keyword is slightly longer and the speech speed is slower, which is reflected in the longer length of the keyword character interval at the keyword. Therefore, when obtaining the occlusion frequency, first obtain the second reference frequency of all the belonging intervals in each keyword character interval, and the second reference frequency belonging to the belonging interval can be the average of all the first reference frequencies in the belonging interval. Then for any belonging interval, the time length of the keyword character interval in the belonging interval is recorded as the retention index of the belonging interval, and for any second reference frequency, the sum of the retention indexes of all the belonging intervals of the second reference frequency is recorded as the retention degree of the second reference frequency. Since the pronunciation time of the keyword is slightly longer and the speech speed is slower, it forms a clear difference in retention degree from non-keywords. Therefore, the K-means algorithm is used to cluster the retention degrees of all second reference frequencies, K=2 is set, and the average value of the second reference frequency in a cluster with the smallest average retention degree is recorded as the occlusion frequency.

[0066] S105, determining a frequency interval of the masking frequency in the time series spectrum diagram, setting the spectrum value of the frequency interval to a preset value to mask the speech frequency within the frequency interval, and obtaining enhanced speech data.

[0067] Specifically, the masking frequency is a frequency where a non-keyword is located and needs to be suppressed. The preset value can be set to 0. The frequency interval corresponding to each masking frequency is obtained, and the spectrum values ​​of these frequency intervals are set to 0 on the time series spectrum diagram, thereby obtaining enhanced speech data.

[0068] S106, performing speech recognition on the enhanced speech data.

[0069] Specifically, speech recognition is performed on the enhanced speech data to obtain keywords. In addition, the speech recognition results can be input into a trained convolutional neural network (CNN), matching items consistent with the speech recognition results from the knowledge base, and generating reasonable responses or performing related operations. Among them, when training the CNN model, the training set mainly comes from the interaction of actual passengers in the rail transit system, collecting voice samples, and annotating specific keywords for each voice sample. For example, when a passenger says keywords such as "ticket inquiry" and "timetable", these keywords need to be used as labels. The loss function used is the cross entropy loss function. Furthermore, the type of the CNN network can be a VGG-like model.

[0070] The embodiment of the present invention can determine the main trend index of the belonging interval of each reference frequency according to the time-series spectrum diagram of the voice signal. The main trend index can ensure the possibility that the belonging interval belongs to the keyword, so as to determine the occlusion frequency of the voice signal according to the main trend index and the adjacent trend index of each time-series sub-interval in the belonging interval. The voice signal within the frequency spectrum interval where the occlusion frequency is located may be non-keyword fields such as noise, modal particles and accent words. Therefore, the voice frequency within the frequency interval of the occlusion frequency is occluded to obtain enhanced voice data. The enhanced voice data removes non-keyword data such as noise other than keywords. When the enhanced voice data is subjected to voice recognition, the influence of noise and other factors on the recognition result is avoided, the limitation of the voice recognition of the rail transit customer service machine is reduced, and the accuracy of the voice recognition result of the rail transit customer service robot is improved.

[0071] Embodiment 2:

[0072] Corresponding to the speech recognition method for a rail transit customer service robot provided in the above embodiment, based on the same technical concept, an embodiment of the present invention further provides a speech recognition system for a rail transit customer service robot, and the speech recognition system for a rail transit customer service robot is used to execute the speech recognition method for a rail transit customer service robot. Figure 4 A schematic diagram of a structure of a speech recognition system for a rail transit customer service robot provided by an embodiment of the present invention is shown in FIG. Figure 4 The speech recognition system for the rail transit customer service robot may have relatively large differences due to different configurations or performances, and may include one or more processors 401 and memory 402, the memory 402 is used to store computer programs that can be run on the processor 401, and the processor 401 is used to execute the program stored in the memory 402 to achieve the above Figure 1The memory 402 may be a temporary storage or a permanent storage. The application stored in the memory 402 may include one or more modules (not shown in the figure), each of which may include a series of computer executable instructions for the speech recognition system of the rail transit customer service robot.

[0073] Furthermore, the processor 401 can be configured to communicate with the memory 402 to execute a series of computer executable instructions in the memory 402 on the speech recognition system for the rail transit customer service robot. The speech recognition system for the rail transit customer service robot can also include one or more power supplies 403, one or more wired or wireless network interfaces 404, one or more input and output interfaces 405, and one or more keyboards 406.

[0074] Specifically in this embodiment, the speech recognition system for the rail transit customer service robot includes a processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate with each other through the bus; the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to achieve the above Figure 1 The various steps in the method embodiment are similar to those in the method embodiment, and have the beneficial effects of the above method embodiments. To avoid repetition, the embodiments of the present invention will not be described again here.

[0075] It should be noted that the speech recognition system for a rail transit customer service robot provided in an embodiment of the present invention and the speech recognition method for a rail transit customer service robot provided in an embodiment of the present invention are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned speech recognition method for a rail transit customer service robot, and has the same or similar beneficial effects, and the repeated parts will not be repeated.

[0076] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0077] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A speech recognition method for a rail transit customer service robot, characterized in that: The speech recognition method for a rail transit customer service robot comprises: The voice signals between the user and the rail transit customer service robot during the conversation are processed to obtain the time series spectrum diagram and time domain waveform diagram; Determine a first reference frequency and an attribution interval of the first reference frequency according to the speech frequencies at different moments in the time series spectrogram; Determine the main trend index of the belonging interval and the adjacent trend index of the time sequence subinterval within the belonging interval according to the signal amplitude at different times of the time domain waveform diagram, wherein the main trend index indicates the possibility that the belonging interval belongs to the keyword field; Determine the occlusion frequency of the speech signal by using the signal amplitude at different times of the time series subinterval, the proximity trend index of the time series subinterval, and the main trend index of the belonging interval to which the time series subinterval belongs; Determine a frequency interval of the masking frequency in the time series spectrum diagram, set the spectrum value of the frequency interval to a preset value to mask the speech frequency within the frequency interval, and obtain enhanced speech data; Performing speech recognition on the enhanced speech data; The determining of the main trend index of the attribution interval and the adjacent trend index of the time sequence subinterval within the attribution interval according to the signal amplitudes at different times of the time domain waveform diagram comprises: Determine the fluctuation change index at different moments according to the signal amplitude at each moment of the time domain waveform diagram, wherein the fluctuation change index is the difference between the maximum value at an adjacent moment adjacent to the current moment and the signal amplitude at a moment after the adjacent moment; Clustering the fluctuation change indexes at all times within the belonging interval to obtain a plurality of first clusters; Determine the main trend index of the belonging interval according to the first number of moments included in the first cluster and the average index of the fluctuation change index of each moment in the first cluster; Determine a time series interval formed by continuous moments belonging to the same cluster within the belonging interval as a time series sub-interval, and determine the mean of the signal amplitudes of all moments within the time series sub-interval as the characteristic amplitude of the time series sub-interval; Determine the approaching trend index of the current time series subinterval by using the first characteristic amplitude of the current time series subinterval within the belonging interval, the second characteristic amplitude of the previous first time series subinterval adjacent to the current time series subinterval, the first center moment of the current time series subinterval, and the second center moment of the first time series subinterval; Determining the main trend index of the attribution interval according to the first number of moments included in the first cluster and the average index of the fluctuation change index of each moment in the first cluster includes: Using a probability distribution calculation function to process the first quantity of each of the first clusters to obtain a processing result; Calculating a first product of the processing result of each of the first clusters and the average index of the first cluster, and superimposing each of the first products to obtain a main trend index of the attribution interval; The method of determining the approaching trend index of the current time series subinterval by using the first characteristic amplitude of the current time series subinterval within the belonging interval, the second characteristic amplitude of the previous first time series subinterval adjacent to the current time series subinterval, the first central moment of the current time series subinterval, and the second central moment of the first time series subinterval comprises: Calculating a first difference between the first characteristic amplitude and the second characteristic amplitude, and a second difference between the first center moment and the second center moment; Determine a first ratio of the first difference to the second difference as an approaching trend index of the current time series subinterval; Determining the occlusion frequency of the speech signal by using the signal amplitude at different times of the time sequence subinterval, the proximity trend index of the time sequence subinterval, and the main trend index of the belonging interval to which the time sequence subinterval belongs includes: Determine the characteristic amplitude of each time sequence sub-interval by using the signal amplitude at different times of each time sequence sub-interval within the belonging interval; Determine the trend of the belonging interval according to the characteristic amplitude of each of the time series subintervals, the adjacent trend index of each of the time series subintervals, and the main trend index of the belonging interval to which each of the time series subintervals belongs; The occlusion frequency of the speech signal is determined by utilizing the trend of the attribution interval.

2. The speech recognition method for a rail transit customer service robot according to claim 1, characterized in that: The determining of the first reference frequency and the attribution interval of the first reference frequency according to the speech frequencies at different moments in the time series spectrogram comprises: Acquire a maximum voice frequency and a minimum voice frequency from the time series spectrum graph, and determine an interval between the maximum voice frequency and the minimum voice frequency as a frequency coverage interval; Dividing the frequency coverage interval into a plurality of sub-intervals; Taking the average value of the speech frequency at each moment in the subinterval as the first reference frequency; Determine the frame to which the first reference frequency belongs according to the main frequency of any frame in the time series spectrum diagram and the first reference frequency of each sub-interval; Determine a time interval consisting of time-series consecutive belonging frames of the first reference frequency as the belonging interval of the first reference frequency.

3. The speech recognition method for a rail transit customer service robot according to claim 1, characterized in that: Determining the trend of the belonging interval according to the characteristic amplitude of each of the time series subintervals, the adjacent trend index of each of the time series subintervals, and the main trend index of the belonging interval to which each of the time series subintervals belongs includes: Determine a first characteristic amplitude of a current time series subinterval, a previous first time series subinterval adjacent to the current time series subinterval, a third characteristic amplitude of at least one second time series subinterval before the first time series subinterval, an approaching trend index of the second time series subinterval, a first central moment of the current time series subinterval, and a third central moment of the second time series subinterval; Determine a modified trend index of the current time series subinterval according to the third characteristic amplitude, the approaching trend index of the second time series subinterval, the first characteristic amplitude, the first central moment, the third central moment, and the main trend index of the attribution interval; The trend of the attribution interval is determined according to the second number of each of the time series sub-intervals in the attribution interval, the main trend index, and the modified trend index of each of the time series sub-intervals in the attribution interval.

4. The speech recognition method for a rail transit customer service robot according to claim 1, characterized in that: The determining the occlusion frequency of the speech signal by using the trend of the belonging interval includes: Determining a merging probability of the current belonging interval according to a minimum value of the trend between the current belonging interval and other belonging intervals and the trend of the current belonging interval; Determine a keyword character interval according to the merging probability of each of the belonging intervals, wherein the keyword character interval includes a plurality of continuous belonging intervals; For any attribution interval within the keyword character interval, the duration of the keyword character interval in which the any attribution interval is located is used as a retention index of the any attribution interval; Determine the second reference frequency of all the belonging intervals in each of the keyword character intervals; For any second reference frequency, taking the sum of the retention indexes of all belonging intervals of the second reference frequency as the retention degree of the second reference frequency; Clustering the retention degrees of all second reference frequencies to obtain multiple second clusters; calculating the average degree of retention within each of the second clusters; The average value of the second reference frequencies in the second cluster with the smallest average preservation degree is determined as the occlusion frequency.

5. The speech recognition method for a rail transit customer service robot according to claim 3, characterized in that: Determining the modified trend index of the current time series subinterval according to the third characteristic amplitude, the approaching trend index of the second time series subinterval, the first characteristic amplitude, the first central moment, the third central moment, and the main trend index of the attribution interval includes: Calculating a third difference between the first characteristic amplitude and each of the third characteristic amplitudes, and a fourth difference between the first center moment and each of the third center moments; Calculating a second ratio of each of the third differences to each of the fourth differences; Calculate the absolute value of the fifth difference between each of the third characteristic amplitudes and the first characteristic amplitude, and select the maximum value from the absolute values ​​of the fifth difference; Calculating the maximum value using a probability distribution calculation function to obtain a calculation result; A second product between each of the second ratios and the calculation result is determined, and each of the second products is superimposed to obtain a modified trend index of the current time series sub-interval.

6. A speech recognition system for a rail transit customer service robot, characterized in that: The speech recognition system for a rail transit customer service robot comprises: a processor and a memory; wherein the memory is used to store a computer program that can be run on the processor; and the processor is used to execute the program stored in the memory to implement the steps of the speech recognition method for a rail transit customer service robot as described in any one of claims 1-5.

Images