User interaction method based on smart television

Abstract

The invention relates to the technical field of intelligent televisions, and discloses a user interaction method based on an intelligent television, which comprises a data acquisition module, a data processing module, a data analysis module, an authentication module and a control module. According to the user interaction method based on the smart television, the volume matching degree, the speech speed matching degree, the tone matching degree and the matching degree score are compared with corresponding threshold values, and when any two numerical values of the volume matching degree, the speech speed matching degree and the tone matching degree are larger than the corresponding threshold values and the matching degree score is larger than the matching degree score threshold value, the user interaction is performed. The authentication module judges that the user authentication is successful, and when any two numerical values of the volume matching degree, the speed matching degree and the tone matching degree are smaller than corresponding threshold values or the matching degree score is smaller than a matching degree score threshold value, the authentication module judges that the user authentication is unsuccessful, so that voice recognition authentication can be performed on a specific user; and confusion of television control caused by voice control of multiple persons is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of smart televisions, and in particular to a user interaction method based on smart televisions. Background Art

[0002] Smart TV is a new product formed based on the impact of the Internet wave. Its purpose is to bring users a more convenient experience. It has become a trend in television. Smart TV breaks the constraints of remote controls on traditional TVs and realizes four major functions: take away viewing, classified viewing, multi-screen viewing and anytime viewing, which has promoted the development of smart TV to a new height. With the rapid development of smart TVs, the number of smart TV users is increasing. Unlike traditional digital TVs, smart TVs have added various new functions such as Internet browsing, sending and receiving emails, TV shopping, distance learning, telemedicine, stock trading, information consultation, etc. These new functions require more and more user interaction. Traditional digital TVs rely on the interaction of infrared remote controls. With the increasing number of TV channels, the disadvantages of traditional TV remote control methods for turning on, turning off and changing channels are becoming more and more obvious. Smart TVs with voice control, gesture control and even face recognition control have already entered people's homes. Many brands at home and abroad already have smart voice-controlled TVs.

[0003] However, current smart TVs still have certain shortcomings in terms of voice-controlled channel selection functions. For example, when multiple people are present at the same time, the TV will recognize the voices of multiple people and be controlled by multiple people, which can easily cause TV control confusion, making it difficult for consumers to watch normally, affecting the viewing experience, and failing to create a good interactive experience for users. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the deficiencies in the prior art, the present invention provides a user interaction method based on a smart television, which has the advantages of authenticating the voice of a specific user through a voice authentication module, thereby avoiding being controlled by multiple people during the voice control process, causing confusion in television control, and solving the above-mentioned problems.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: a user interaction method based on a smart TV, comprising the following steps:

[0008] S1, the data acquisition module collects the user's voice through the built-in microphone, and the data acquisition module extracts the real-time audio signal Xh and the real-time syllable number Yjs from the collected user's voice. The data acquisition module sends the collected data to the data processing module through the network;

[0009] S2, the data processing module calculates the volume Yl according to the real-time audio signal Xh, the data processing module calculates the speaking speed Ys according to the real-time syllable number Yjs, the data processing module is preset with an autocorrelation function Zxhs, the data processing module calculates the tone Yd according to the real-time audio signal Xh and the autocorrelation function Zxhs, the data processing module pre-processes the real-time audio signal Xh to obtain the feature vector Tzxl, the data processing module calculates the volume matching degree Ylpp, the speaking speed matching degree Yspp and the tone matching degree Ydpp according to the volume Yl, the speaking speed Ys and the tone Yd respectively, the data processing module is preset with a target feature vector Tzmb, the data processing module calculates the matching score Ppdf according to the feature vector Tzxl and the target feature vector Tzmb, and the data processing module sends these data to the data analysis module through the network;

[0010] S3. The volume matching threshold Ylpp is preset in the data analysis module. yz , speech speed matching threshold Yspp yz , pitch matching threshold Ydpp yz And the matching score threshold Ppdf yz The data analysis module compares the volume matching degree Ylpp, speech speed matching degree Yspp, pitch matching degree Ydpp and matching degree score Ppdf with the volume matching degree threshold Ylpp yz , speech speed matching threshold Yspp yz , pitch matching threshold Ydpp yz And the matching score threshold Ppdf yz After comparison, the data analysis module sends the comparison result to the authentication module through the network;

[0011] S4: When any two values ​​of the volume matching degree Ylpp, the speech speed matching degree Yspp and the pitch matching degree Ydpp are greater than the corresponding threshold value and the matching score Ppdf is greater than the matching score threshold Ppdf yz The authentication module determines that the user authentication is successful when any two values ​​of the volume matching degree Ylpp, the speech speed matching degree Yspp, and the pitch matching degree Ydpp are less than the corresponding threshold or the matching score Ppdf is less than the matching score threshold Ppdf yz When the authentication module determines that the user authentication is unsuccessful;

[0012] S5. The authentication module sends the authentication result to the control module through the network. When the user authentication is successful, the user can issue corresponding voice commands to the TV through the control module. When the user authentication is unsuccessful, the user cannot issue corresponding voice commands to the TV through the control module.

[0013] Preferably, the characteristic vector Tzx1 is expressed as {Tzx11, Tzx12, ..., Tzx1n}, the target feature vector Tzmb is expressed as {Tzmb1, Tzmb2, ..., Tzmb n}.

[0014] Preferably, the volume Y1 algorithm expression is as follows:

[0015]

[0016] In the formula, Yl represents volume, Xh i represents the audio signal value of the i-th sample, and N represents the total number of samples of the signal.

[0017] Preferably, the speech rate Ys algorithm expression is as follows:

[0018]

[0019] In the formula, Ys represents the speaking rate, Yjs represents the real-time number of syllables, and Cxsj represents the duration.

[0020] Preferably, the autocorrelation function Zxhs(τ) formula is as follows:

[0021]

[0022] In the formula, Zxhs(τ) represents the autocorrelation value of the signal at a lag of τ, and Xh i Indicates the audio signal value of the i-th sample, Xh (i+τ) Represents the audio signal value of the (i+τ)th sample, N represents the total number of samples of the signal, τ represents the delay, is the independent variable of the autocorrelation function, represents the time offset of the signal, and finds the peak value according to the autocorrelation function, which is recorded as Zxhs max , peak value Zxhs max The corresponding hysteresis value τ is the basic period T.

[0023] Preferably, the tone Yd algorithm expression is as follows:

[0024]

[0025] In the formula, Yd represents the pitch and T represents the basic period.

[0026] Preferably, the volume matching degree Ylpp algorithm expression is as follows:

[0027]

[0028] In the formula, Ylpp represents the volume matching degree, Yl represents the volume, Ylck represents the volume reference value, and Ylcy represents the volume matching degree. max Indicates the maximum value of volume change;

[0029] The speech speed matching degree Yspp algorithm expression is as follows:

[0030]

[0031] In the formula, Ylpp represents the speech speed matching degree, Ys represents the speech speed, Ysck represents the speech speed reference value, and Ysbh represents the speech speed matching degree. max Indicates the maximum value of speech rate change;

[0032] The pitch matching degree Ydpp algorithm expression is as follows:

[0033]

[0034] In the formula, Ydpp represents the pitch matching degree, Yd represents the pitch, Ydck represents the pitch reference value, and Ydbh represents the pitch matching degree. max Indicates the maximum value of pitch variation.

[0035] Preferably, the matching score Ppdf algorithm expression is as follows:

[0036]

[0037] In the formula, Ppdf represents the matching score, Tzxl i represents the i-th eigenvector, Tzmb i represents the i-th target feature vector, i=1 means starting from Tzxl1*Tzmb1, n means calculating till the nth data, i.e. calculating till Tzxl n *Tzmb n So far, n means there are n data in the data set, and the calculated result is the dot product of the feature vector and the target feature vector. It means that the squares of all eigenvectors are added together, and the result is the square root of the sum. It means that the squares of all target feature vectors are added together, and the result is the square root of the sum.

[0038] Preferably, the data analysis module compares the volume matching degree Ylpp, the speech speed matching degree Yspp, the pitch matching degree Ydpp and the matching score Ppdf with the volume matching degree threshold Ylpp respectively. yz , speech speed matching threshold Yspp yz , pitch matching threshold Ydpp yz And the matching score threshold Ppdf yzThe data analysis module sends the comparison result to the authentication module through the network. When any two values ​​of the volume matching degree Ylpp, the speech speed matching degree Yspp and the pitch matching degree Ydpp are greater than the corresponding threshold value and the matching score Ppdf is greater than the matching score threshold Ppdf yz When , the authentication module determines that the user authentication is successful;

[0039] When any two values ​​of volume matching Ylpp, speech speed matching Yspp and pitch matching Ydpp are less than the corresponding threshold or the matching score Ppdf is less than the matching score threshold Ppdf yz , the authentication module determines that the user authentication is unsuccessful.

[0040] Preferably, the authentication module sends the authentication result to the control module through the network. When the user authentication is successful, the user can issue corresponding voice commands to the TV through the control module. When the user authentication is unsuccessful, the user cannot issue corresponding voice commands to the TV through the control module.

[0041] Compared with the prior art, the present invention provides a user interaction method based on a smart TV, which has the following beneficial effects:

[0042] The present invention obtains the volume matching degree, speech speed matching degree, tone matching degree and matching degree score by calculation, and then compares the volume matching degree, speech speed matching degree, tone matching degree and matching degree score with corresponding thresholds respectively; when any two values ​​of the volume matching degree, speech speed matching degree and tone matching degree are greater than the corresponding threshold and the matching degree score is greater than the matching degree score threshold, the authentication module determines that the user authentication is successful; when any two values ​​of the volume matching degree, speech speed matching degree and tone matching degree are less than the corresponding threshold or the matching degree score is less than the matching degree score threshold, the authentication module determines that the user authentication is unsuccessful, so that voice recognition authentication can be performed on a specific user, thereby preventing a television from being voice-controlled by multiple people and causing confusion in television control. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the steps of the method of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1, a user interaction method based on a smart TV, comprising the following steps:

[0046] S1, the data acquisition module collects the user's voice through the built-in microphone, and the data acquisition module extracts the real-time audio signal Xh and the real-time syllable number Yjs from the collected user's voice. The data acquisition module sends the collected data to the data processing module through the network;

[0047] S2, the data processing module calculates the volume Yl according to the real-time audio signal Xh, and the volume Yl algorithm expression is as follows:

[0048]

[0049] In the formula, Yl represents volume, Xh i represents the audio signal value of the i-th sample, N represents the total number of signal samples, volume Yl is also called loudness or intensity, which is a basic attribute of sound, corresponding to people's auditory perception of the size of the sound. The data processing module calculates the volume matching degree Ylpp according to the volume Yl. The volume matching degree Ylpp algorithm expression is as follows:

[0050]

[0051] In the formula, Ylpp represents the volume matching degree, Yl represents the volume, Ylck represents the volume reference value, and Ylcy represents the volume matching degree. max Indicates the maximum value of volume change. The volume reference values ​​here are Ylck and Ylcy. max The maximum values ​​of volume changes are all known numbers;

[0052] The data processing module calculates the speech rate Ys according to the real-time syllable number Yjs. The speech rate Ys algorithm expression is as follows:

[0053]

[0054] In the formula, Ys represents the speaking speed, Yjs represents the number of real-time syllables, and Cxsj represents the duration. The speaking speed Ys is the speaking speed, which is a factor that cannot be ignored. It not only affects feature extraction and frame synchronization analysis, but also has an important impact on the accuracy of the acoustic model and language model. The data processing module calculates the speaking speed matching degree Yspp according to the speaking speed Ys. The speaking speed matching degree Yspp algorithm expression is as follows:

[0055]

[0056] In the formula, Ylpp represents the speech speed matching degree, Ys represents the speech speed, Ysck represents the speech speed reference value, and Ysbh represents the speech speed matching degree. max Indicates the maximum value of speech speed change, where Ysck is the speech speed reference value, Ysbh is maxThe maximum values ​​of speech rate changes are all known numbers;

[0057] The data processing module is preset with an autocorrelation function Zxhs, and the formula of the autocorrelation function Zxhs(τ) is as follows:

[0058]

[0059] In the formula, Zxhs(τ) represents the autocorrelation value of the signal at a lag of τ, and Xh i Indicates the audio signal value of the i-th sample, Xh (i+τ) Represents the audio signal value of the (i+τ)th sample, N represents the total number of samples of the signal, τ represents the delay, is the independent variable of the autocorrelation function, represents the time offset of the signal, and finds the peak value according to the autocorrelation function, which is recorded as Zxhs max , peak value Zxhs max The corresponding lag value τ is the basic period T. The data processing module calculates the pitch Yd according to the basic period T and the real-time audio signal Xh. The pitch Yd algorithm expression is as follows:

[0060]

[0061] In the formula, Yd represents the tone, T represents the basic period, and the tone Yd is also called the pitch, which is mainly related to the frequency of the sound wave. The height of the tone Yd directly determines the auditory perception of the sound. The sound wave with a high frequency sounds higher in pitch, while the sound wave with a low frequency sounds lower in pitch. The data processing module calculates the tone matching degree Ydpp according to the tone Yd. The tone matching degree Ydpp algorithm expression is as follows:

[0062]

[0063] In the formula, Ydpp represents the pitch matching degree, Yd represents the pitch, Ydck represents the pitch reference value, and Ydbh represents the pitch matching degree. max Indicates the maximum value of the pitch change, here the pitch reference value Ydck, the maximum value of the pitch change Ydbh max All are known numbers;

[0064] The target feature vector Tzmb is preset in the data processing module. The data processing module calculates the matching score Ppdf according to the feature vector Tzxl and the target feature vector Tzmb, where the expressions of the feature vector Tzxl and the target feature vector Tzmb are {Tzxl1, Tzxl2, ..., Tzxl n}, {Tzmb1, Tzmb2, ..., Tzmb n}, the feature vector Tzxl and the target feature vector Tzmb are both fixed-length numerical arrays used to represent a certain aspect of the data. Each element usually represents a dimension of the feature. The matching score Ppdf algorithm expression is as follows:

[0065]

[0066] In the formula, Ppdf represents the matching score, Tzxl i represents the i-th eigenvector, Tzmb i represents the i-th target feature vector, ∑ i n ＝1 (Tzxl i *Tzmb i ) where i=1 means starting from Tzxl1*Tzmb1, and n means calculating until the nth data, i.e. calculating until Tzxl n *Tzmb n So far, n means there are n data in the data set, and the calculated result is the dot product of the feature vector and the target feature vector. It means that the squares of all eigenvectors are added together, and the result is the square root of the sum. It means that the squares of all target feature vectors are added together, and the result is the square root of the sum. In user identification, the matching score Ppdf is an indicator for evaluating the similarity between the input speech and the target speaker model. The matching score Ppdf is a key indicator for evaluating the similarity between the input speech and the known template. Its main function is to judge and identify the content of the pronunciation by quantifying the similarity between the input speech and the reference template.

[0067] In audio analysis and processing tasks, pitch Yd, volume Yl and speech speed Ys are important features. In speech recognition systems, the matching degree of pitch Yd, volume Yl and speech speed Ys has an important impact on recognition accuracy. Therefore, it is necessary to use pitch Yd, volume Yl and speech speed Ys to complete user authentication.

[0068] The data processing module sends these data to the data analysis module through the network;

[0069] S3. The volume matching threshold Ylpp is preset in the data analysis module. yz , speech speed matching threshold Yspp yz , pitch matching threshold Ydpp yz And the matching score threshold Ppdf yzThe data analysis module compares the volume matching degree Ylpp, speech speed matching degree Yspp, pitch matching degree Ydpp and matching degree score Ppdf with the volume matching degree threshold Ylpp yz , speech speed matching threshold Yspp yz , pitch matching threshold Ydpp yz And the matching score threshold Ppdf yz After comparison, the data analysis module sends the comparison result to the authentication module through the network;

[0070] S4: When any two values ​​of the volume matching degree Ylpp, the speech speed matching degree Yspp and the pitch matching degree Ydpp are greater than the corresponding threshold value and the matching score Ppdf is greater than the matching score threshold Ppdf yz The authentication module determines that the user authentication is successful when any two values ​​of the volume matching degree Ylpp, the speech speed matching degree Yspp, and the pitch matching degree Ydpp are less than the corresponding threshold or the matching score Ppdf is less than the matching score threshold Ppdf yz When the authentication module determines that the user authentication is unsuccessful;

[0071] S5. The authentication module sends the authentication result to the control module through the network. When the user authentication is successful, the user can issue corresponding voice commands to the TV through the control module. When the user authentication is unsuccessful, the user cannot issue corresponding voice commands to the TV through the control module.

[0072] The system calculates the volume matching, speech speed matching, pitch matching and matching score, and then compares the volume matching, speech speed matching, pitch matching and matching score with the corresponding thresholds respectively. When any two values ​​of the volume matching, speech speed matching and pitch matching are greater than the corresponding thresholds and the matching score is greater than the matching score threshold, the authentication module determines that the user authentication is successful. When any two values ​​of the volume matching, speech speed matching and pitch matching are less than the corresponding threshold or the matching score is less than the matching score threshold, the authentication module determines that the user authentication is unsuccessful. In this way, voice recognition authentication can be performed on specific users to avoid the TV being controlled by the voices of multiple people, causing confusion in TV control.

[0073] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A user interaction method based on a smart TV, characterized in that: The following steps are involved: S1, the data acquisition module collects the user's voice through the built-in microphone, and the data acquisition module extracts the real-time audio signal Xh and the real-time syllable number Yjs from the collected user's voice. The data acquisition module sends the collected data to the data processing module through the network; S2, the data processing module calculates the volume Yl according to the real-time audio signal Xh, the data processing module calculates the speaking speed Ys according to the real-time syllable number Yjs, the data processing module is preset with an autocorrelation function Zxhs, the data processing module calculates the tone Yd according to the real-time audio signal Xh and the autocorrelation function Zxhs, the data processing module pre-processes the real-time audio signal Xh to obtain the feature vector Tzxl, the data processing module calculates the volume matching degree Ylpp, the speaking speed matching degree Yspp and the tone matching degree Ydpp according to the volume Yl, the speaking speed Ys and the tone Yd respectively, the data processing module is preset with a target feature vector Tzmb, the data processing module calculates the matching score Ppdf according to the feature vector Tzxl and the target feature vector Tzmb, and the data processing module sends these data to the data analysis module through the network; S3. The volume matching threshold Ylpp is preset in the data analysis module. yz , speech speed matching threshold Yspp yz , pitch matching threshold Ydpp yz And the matching score threshold Ppdf yz The data analysis module compares the volume matching degree Ylpp, speech speed matching degree Yspp, pitch matching degree Ydpp and matching degree score Ppdf with the volume matching degree threshold Ylpp yz , speech speed matching threshold Yspp yz , pitch matching threshold Ydpp yz And the matching score threshold Ppdf yz After comparison, the data analysis module sends the comparison result to the authentication module through the network; S4: When any two values ​​of the volume matching degree Ylpp, the speech speed matching degree Yspp and the pitch matching degree Ydpp are greater than the corresponding threshold value and the matching score Ppdf is greater than the matching score threshold Ppdf yz The authentication module determines that the user authentication is successful when any two values ​​of the volume matching degree Ylpp, the speech speed matching degree Yspp, and the pitch matching degree Ydpp are less than the corresponding threshold or the matching score Ppdf is less than the matching score threshold Ppdf yz When the authentication module determines that the user authentication is unsuccessful; S5. The authentication module sends the authentication result to the control module through the network. When the user authentication is successful, the user can issue corresponding voice commands to the TV through the control module. When the user authentication is unsuccessful, the user cannot issue corresponding voice commands to the TV through the control module.

2. A user interaction method based on a smart TV according to claim 1, characterized in that: The characteristic vector Tzx1 is expressed as {Tzx11, Tzx12, ..., Tzx1 n }, the target feature vector Tzmb is expressed as {Tzmb1, Tzmb2, ..., Tzmb n }.

3. A user interaction method based on a smart TV according to claim 2, characterized in that: The volume Y1 algorithm expression is as follows: In the formula, Yl represents volume, Xh i represents the audio signal value of the i-th sample, and N represents the total number of samples of the signal.

4. The user interaction method based on a smart TV according to claim 3, characterized in that: The algorithm expression of the speech rate Ys is as follows: In the formula, Ys represents the speaking rate, Yjs represents the real-time number of syllables, and Cxsj represents the duration.

5. A user interaction method based on a smart TV according to claim 4, characterized in that: The autocorrelation function Zxhs(τ) formula is as follows: In the formula, Zxhs(τ) represents the autocorrelation value of the signal at a lag of τ, and Xh i Indicates the audio signal value of the i-th sample, Xh (i+τ) Represents the audio signal value of the (i+τ)th sample, N represents the total number of samples of the signal, τ represents the delay, is the independent variable of the autocorrelation function, represents the time offset of the signal, and finds the peak value according to the autocorrelation function, which is recorded as Zxhs max , peak value Zxhs max The corresponding hysteresis value τ is the basic period T.

6. A user interaction method based on a smart TV according to claim 5, characterized in that: The tone Yd algorithm expression is as follows: In the formula, Yd represents the pitch and T represents the basic period.

7. A user interaction method based on a smart TV according to claim 6, characterized in that: The volume matching degree Ylpp algorithm expression is as follows: In the formula, Ylpp represents the volume matching degree, Yl represents the volume, Ylck represents the volume reference value, and Ylcy represents the volume matching degree. max Indicates the maximum value of volume change; The speech speed matching degree Yspp algorithm expression is as follows: In the formula, Ylpp represents the speech speed matching degree, Ys represents the speech speed, Ysck represents the speech speed reference value, and Ysbh represents the speech speed matching degree. max Indicates the maximum value of speech rate change; The pitch matching degree Ydpp algorithm expression is as follows: In the formula, Ydpp represents the pitch matching degree, Yd represents the pitch, Ydck represents the pitch reference value, and Ydbh represents the pitch matching degree. max Indicates the maximum value of pitch variation.

8. The user interaction method based on a smart TV according to claim 7, characterized in that: The matching score Ppdf algorithm expression is as follows: In the formula, Ppdf represents the matching score, Tzxl i represents the i-th eigenvector, Tzmb i represents the i-th target feature vector, i=1 means starting from Tzxl1*Tzmb1, n means calculating till the nth data, i.e. calculating till Tzxl n *Tzmb n So far, n means there are n data in the data set, and the calculated result is the dot product of the feature vector and the target feature vector. It means that the squares of all eigenvectors are added together, and the result is the square root of the sum. It means that the squares of all target feature vectors are added together, and the result is the square root of the sum.

9. The user interaction method based on a smart TV according to claim 8, characterized in that: The data analysis module compares the volume matching degree Ylpp, the speech speed matching degree Yspp, the pitch matching degree Ydpp and the matching degree score Ppdf with the volume matching degree threshold Ylpp respectively. yz , speech speed matching threshold Yspp yz , pitch matching threshold Ydpp yz And the matching score threshold Ppdf yz The data analysis module sends the comparison result to the authentication module through the network. When any two values ​​of the volume matching degree Ylpp, the speech speed matching degree Yspp and the pitch matching degree Ydpp are greater than the corresponding threshold value and the matching score Ppdf is greater than the matching score threshold Ppdf yz When , the authentication module determines that the user authentication is successful; When any two values ​​of volume matching Ylpp, speech speed matching Yspp and pitch matching Ydpp are less than the corresponding threshold or the matching score Ppdf is less than the matching score threshold Ppdf yz , the authentication module determines that the user authentication is unsuccessful.

10. The user interaction method based on a smart TV according to claim 9, characterized in that: The authentication module sends the authentication result to the control module through the network. When the user authentication is successful, the user can issue corresponding voice commands to the TV through the control module. When the user authentication is unsuccessful, the user cannot issue corresponding voice commands to the TV through the control module.

Images