Audio and video playing control method and device for intelligent voice television

Abstract

The invention is suitable for the technical field of intelligent televisions, and provides an audio and video playing control method and device for an intelligent voice television, and the method comprises the steps: receiving initial voice instructions of a plurality of users, and enabling the users and the initial voice instructions to be in one-to-one correspondence; historical playing data and user categories of all the users are obtained, and user playing evaluation parameters of all the users are determined according to the historical playing data and the user categories; performing denoising processing on each initial voice instruction to obtain a plurality of target voice instructions; performing voice recognition on each target voice instruction by adopting a pre-trained target voice recognition model to obtain a plurality of target recognition results; determining a target audio / video according to the user playing evaluation parameter and the target recognition result, and playing the target audio / video; the problem of intelligent voice television instruction execution errors caused by inaccurate voice recognition results in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart televisions, and in particular to an audio and video playback control method and device for a smart voice television. Background Art

[0002] With the development of artificial intelligence technology, especially the advancement of natural language processing and speech recognition technology, smart voice TV has become an important part of modern home entertainment. Smart voice TV combines advanced speech recognition technology and Internet connection functions, allowing users to control various functions of the TV through simple voice commands, including but not limited to channel switching, content search, playback control, etc., greatly improving the user experience.

[0003] However, existing smart voice TVs still face some challenges in audio and video playback, such as errors in command execution due to inaccurate voice recognition results. Although voice recognition technology is quite mature, it is still affected by factors such as environmental noise and the speaker's accent, which may lead to deviations in recognition results. Summary of the invention

[0004] The present invention provides an audio and video playback control method and device for an intelligent voice television, so as to solve the problem of intelligent voice television command execution error caused by inaccurate voice recognition results in the prior art.

[0005] The audio and video playback control method of the intelligent voice television provided by the present invention comprises:

[0006] Receiving initial voice commands from multiple users, wherein there is a one-to-one correspondence between the users and the initial voice commands;

[0007] Performing denoising processing on each of the initial voice commands respectively to obtain multiple target voice commands;

[0008] Using a pre-trained target speech recognition model to perform speech recognition on each of the target speech instructions to obtain multiple target recognition results;

[0009] Respectively obtaining historical playback data and user categories of each of the users, and determining user playback evaluation parameters of each user according to the historical playback data and the user category;

[0010] The target audio and video is determined according to the user playback evaluation parameter and the target recognition result, and the target audio and video is played.

[0011] Optionally, the method further includes: using a pre-trained target speech recognition model to perform speech recognition on each of the target speech instructions to obtain multiple target recognition results;

[0012] Acquire sample speech data, annotate text information of the sample speech data, and obtain an annotation result of the sample speech data;

[0013] Forming a first sample data set according to the sample speech data and the labeling result of the sample speech data;

[0014] An initial speech recognition model is constructed, and the initial speech recognition model is trained using the first sample data set to obtain the target speech recognition model.

[0015] Optionally, forming a first sample data set according to the sample speech data and the labeling result of the sample speech data further includes:

[0016] Acquire associated words of the new audio and video, and determine the semantic similarity between the annotation result of the sample speech data and the associated words;

[0017] If the word meaning similarity is less than the word meaning similarity threshold, acquiring the voice data of the associated word and the text annotation result of the associated word to form a second sample data set;

[0018] The second sample data set is merged into the first sample data set.

[0019] Optionally, forming a first sample data set according to the sample speech data and the labeling result of the sample speech data further includes:

[0020] Extracting the accent features of the user and the accent features of the sample voice data, and obtaining feature similarity between the accent features of the user and the accent features of the sample voice data;

[0021] If the feature similarity is less than the feature similarity threshold, acquiring the target accent speech data;

[0022] annotating text information of the target accent speech data to obtain an annotation result of the target accent speech data, and forming a third sample data set according to the target accent speech data and the annotation result of the target accent speech data;

[0023] The third sample data set is merged into the first sample data set.

[0024] Optionally, determining the target audio and video according to the user playback evaluation parameter and the target recognition result includes:

[0025] If there is inconsistency between the audio and video to be played corresponding to each target recognition result, the permission type of each audio and video to be played is obtained respectively, wherein the target recognition result and the video to be played are in a one-to-one correspondence;

[0026] Obtaining noise evaluation parameters of each target recognition result respectively, and determining target playback evaluation parameters of each to-be-played video according to the noise evaluation parameters and the permission type;

[0027] The target audio and video is determined according to the user playback evaluation parameter and the target playback evaluation parameter.

[0028] Optionally, determining a target playback evaluation parameter of each to-be-played video according to the noise evaluation parameter and the permission type includes:

[0029] If the permission type is a permissioned type, obtaining a type evaluation parameter of the permissioned type, and determining a target playback evaluation parameter of each to-be-played video according to the noise evaluation parameter and the type evaluation parameter;

[0030] If the permission type is a no permission type, a target playback evaluation parameter of each video to be played is determined according to the noise evaluation parameter.

[0031] Optionally, determining the user playback evaluation parameter of each user according to the historical playback data and the user category further includes:

[0032] Obtaining historical playback evaluation parameters of the historical playback data;

[0033] If the user category is a registered category, then obtaining a category evaluation parameter of the registered category, and obtaining a user playback evaluation parameter of each user according to the category evaluation parameter and the historical playback evaluation parameter;

[0034] If the user category is a non-registered category, the historical play evaluation parameter is determined as the user play evaluation parameter of each user.

[0035] The present invention also provides an audio and video playback control device for an intelligent voice television, comprising:

[0036] A voice receiving module, used to receive initial voice commands from multiple users, wherein there is a one-to-one correspondence between the users and the initial voice commands;

[0037] A voice denoising module, used to perform denoising processing on each of the initial voice instructions respectively to obtain multiple target voice instructions;

[0038] A speech recognition module, used to respectively use a pre-trained target speech recognition model to perform speech recognition on each of the target speech instructions to obtain multiple target recognition results;

[0039] A user parameter determination module, used to respectively obtain the historical playback data and user category of each of the users, and determine the user playback evaluation parameter of each user according to the historical playback data and the user category;

[0040] The audio and video playing module is used to determine the target audio and video according to the user playing evaluation parameter and the target recognition result, and play the target audio and video.

[0041] The present invention also provides an electronic device, comprising: a processor and a memory;

[0042] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the electronic device executes the audio and video playback control method of the smart voice TV.

[0043] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the audio and video playback control method of the intelligent voice television as described above is implemented.

[0044] Beneficial effects of the present invention: The audio and video playback control method of the intelligent voice TV in the present invention first receives the initial voice commands of multiple users; secondly, denoises each initial voice command to obtain multiple target voice commands; then uses the pre-trained target voice recognition model to perform voice recognition on each target voice command to obtain multiple target recognition results; again, obtains the historical playback data and user category of each user, and determines the user playback evaluation parameters of each user according to the historical playback data and user category; finally, determines the target audio and video according to the user playback evaluation parameters and the target recognition results, and plays the target audio and video; the present invention denoises the initial voice command and uses the pre-trained target voice recognition model to perform voice recognition on each target voice command, so that the voice recognition result is more accurate, thereby solving the problem of intelligent voice TV command execution error caused by inaccurate voice recognition results in the prior art. The present invention also determines the target audio and video by the user playback evaluation parameters and the target recognition results, so that the played audio and video are more in line with user needs, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 It is a flowchart of an audio and video playback control method of an intelligent voice television in an embodiment of the present invention;

[0047] Figure 2It is a flow chart of the audio and video playback control device of the intelligent voice television in an embodiment of the present invention;

[0048] Figure 3 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0050] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0051] In order to clearly understand the technical solution of the present application, the solution of the prior art is first introduced in detail.

[0052] With the development of artificial intelligence technology, especially the advancement of natural language processing and speech recognition technology, smart voice TV has become an important part of modern home entertainment. Smart voice TV combines advanced speech recognition technology and Internet connection functions, allowing users to control various functions of the TV through simple voice commands, including but not limited to channel switching, content search, playback control, etc., greatly improving the user experience.

[0053] However, existing smart voice TVs still face some challenges in audio and video playback, such as errors in command execution due to inaccurate voice recognition results. Although voice recognition technology is quite mature, it is still affected by factors such as environmental noise and the speaker's accent, which may lead to deviations in recognition results. In addition, when smart voice TVs receive voice commands from multiple different users at the same time within a period of time, it is unreasonable to determine which user's command is executed.

[0054] Therefore, in view of the command execution errors caused by inaccurate voice recognition results in the prior art, the inventors found in their research that by denoising the initial voice commands and using a pre-trained target voice recognition model to perform voice recognition on each target voice command, the voice recognition results are more accurate, thereby solving the problems of smart voice TV command execution errors caused by inaccurate voice recognition results in the prior art. In view of the problem of unreasonable execution of commands after receiving voice commands from multiple different users, the target audio and video are determined through user playback evaluation parameters and target recognition results, so that the played audio and video are more in line with user needs, thereby improving the user experience.

[0055] Therefore, the inventors have proposed a technical solution of the embodiment of the present invention based on the above creative discovery. In order to illustrate the technical solution of the present invention, a specific embodiment is used for illustration below.

[0056] In this embodiment, the smart voice TV is a TV product that combines smart TV functions with voice recognition technology. It can not only receive traditional TV signals, but also has Internet connection capabilities and supports control and interaction through voice commands. The smart voice TV can capture the user's voice commands through a built-in microphone array and convert them into text commands, and supports searching and playing specific audio and video content through the converted text commands. In addition, the smart voice TV also provides an intuitive user interface that allows users to easily interact with the TV. It usually includes a graphical user interface (GUI) and supports touch screen or gesture control.

[0057] Figure 1 It is a flow chart of an audio and video playback control method of an intelligent voice television provided in one embodiment of the present invention.

[0058] like Figure 1 As shown, the audio and video playback control method of the smart voice TV includes steps S110-S150:

[0059] S110, receiving initial voice commands from multiple users;

[0060] S120, performing denoising processing on each of the initial voice commands to obtain a plurality of target voice commands;

[0061] S130, respectively using a pre-trained target speech recognition model to perform speech recognition on each target speech instruction to obtain a plurality of target recognition results;

[0062] S140, respectively obtaining historical play data and user categories of each user, and determining user play evaluation parameters of each user according to the historical play data and user categories;

[0063] S150, determining the target audio and video according to the user's playback evaluation parameter and the target recognition result, and playing the target audio and video.

[0064] In step S110 of the present embodiment, the initial voice command is a voice command for controlling audio and video playback. One user corresponds to one initial voice command. If the same user issues multiple initial voice commands within a period of time, the latest initial voice command shall prevail.

[0065] In step S120 of this embodiment, the implementation process of denoising the initial voice instructions includes: (1) Data preprocessing: The received voice instructions are uniformly converted into a 16kHz WAV format. Silence detection technology is used to remove silent segments in voice instructions. (2) Noise estimation: Background noise is estimated in silent segments before or after the start of speech. The noise characteristics are estimated using a recursive averaging method. (3) Frequency domain denoising: The audio signal is subjected to a fast Fourier transform (FFT) and converted to the frequency domain. A spectral subtraction algorithm is applied to remove noise. The estimated noise characteristics are used to adjust the filter parameters to reduce the noise component. The denoised frequency domain signal is inversely transformed back to the time domain. (4) Feature extraction: MFCC features are extracted from the denoised audio signal for subsequent speech recognition. (5) Post-processing: Dynamic range compression is performed on the denoised audio signal to improve the clarity of the speech. By denoising the initial voice command to obtain the target voice command, and performing voice recognition on the target voice command to obtain the target recognition result, the accuracy of the target recognition result is improved, thereby solving the problem of smart voice TV command execution error caused by inaccurate voice recognition results in the prior art.

[0066] In addition, the pre-trained target speech denoising model can also be used to denoise the initial speech instruction to obtain the target speech instruction. The process of obtaining the target speech denoising model includes but is not limited to: obtaining the speech data to be denoised, denoising the speech data to be denoised to obtain denoised speech data, forming a denoised sample data set according to the speech data to be denoised and the denoised speech data, and dividing the denoised sample data set into a denoised training data set and a denoised test data set; using the denoised training data set to train the initial speech denoising model to obtain a denoising model for speech denoising; inputting the denoised test data set into the denoising model to obtain a denoising test result; obtaining the error between the denoised speech data and the denoising test result according to the cross entropy loss function to obtain a first error, and using the first error back propagation to update the denoising model to obtain the target speech denoising model. Specifically, the initial speech denoising model includes but is not limited to a recurrent neural network, wherein the recurrent neural network can be specifically a bidirectional long short-term memory network. A pre-trained target speech denoising model is used to obtain a target voice command, and speech recognition is performed on the target voice command to obtain a target recognition result, thereby improving the accuracy of the target recognition result, and further solving the problem of smart voice TV command execution error caused by inaccurate speech recognition results in the prior art.

[0067] In step S130 of the present embodiment, when a pre-trained target speech recognition model is used to perform speech recognition on each target speech instruction to obtain multiple target recognition results, it is necessary to obtain a target speech recognition model. The process of obtaining the target speech recognition model includes but is not limited to: obtaining sample speech data, annotating the text information of the sample speech data, and obtaining the annotation results of the sample speech data; forming a first sample data set based on the sample speech data and the annotation results of the sample speech data; constructing an initial speech recognition model, and using the first sample data set to train the initial speech recognition model to obtain a target speech recognition model. The initial speech recognition model includes but is not limited to a convolutional neural network model. The sample speech data includes but is not limited to speech data in the audio and video fields.

[0068] Specifically, using the first sample data set to train the initial speech recognition model to obtain a target speech recognition model includes but is not limited to: dividing the first sample data set into a first training data set and a first test data set; using the first training data set to train the initial speech recognition model to obtain a recognition model for speech recognition; inputting the first test data set into the recognition model to obtain a speech test result; obtaining the error between the first annotation result and the speech test result according to the cross entropy loss function to obtain a second error, and using the second error back propagation to update the recognition model to obtain a target speech recognition model.

[0069] In one embodiment, it is considered that the target voice recognition result may be inaccurate for the target voice command due to the release of new audio and video or the increase of audio and video played by smart voice TV. Therefore, a first sample data set is formed according to the sample voice data and the annotation results of the sample voice data, and then it also includes: obtaining the associated words of the new audio and video, and determining the semantic similarity between the annotation results of the sample voice data and the associated words; if the semantic similarity is less than the semantic similarity threshold, the voice data of the associated words and the text annotation results of the associated words are obtained to form a second sample data set; the second sample data set is merged into the first sample data set. Among them, the new audio and video include but are not limited to the audio and video released in the latest period of time or the audio and video that can be searched by the smart voice TV, and the associated words of the new audio and video include but are not limited to the name of the new audio and video, such as the name of a movie. The semantic similarity can be calculated by a pre-trained semantic similarity calculation model, and the semantic similarity threshold can be determined according to the needs of the target user, for example, it can be 95%, 96%. By merging the second sample data set into the first sample data set, and using the merged first sample data set to train the initial speech recognition model, the accuracy of the recognition result of the target speech recognition model is improved.

[0070] In one embodiment, considering that different accents may cause the target speech recognition result to be inaccurate for the target speech instruction. A first sample data set is formed according to the sample speech data and the annotation result of the sample speech data, and then it also includes: extracting the accent features of the user and the accent features of the sample speech data, and obtaining the feature similarity of the accent features of the user and the accent features of the sample speech data; if the feature similarity is less than the feature similarity threshold, obtaining the target accent speech data; annotating the text information of the target accent speech data to obtain the annotation result of the target accent speech data, and forming a third sample data set according to the target accent speech data and the annotation result of the target accent speech data; merging the third sample data set into the first sample data set. If the feature similarity between the user's accent feature and the accent feature of the sample voice data is less than the feature similarity threshold, the user is determined as a target accent user. The target accent voice data acquisition method includes but is not limited to acquiring initial accent voice data and extracting the accent features of the initial accent voice data, acquiring the feature similarity between the accent features of the initial accent voice data and the accent features of the target accent user, and if the feature similarity between the accent features of the initial accent voice data and the accent features of the target accent user is greater than the accent similarity threshold, the initial accent voice data is determined as the target accent voice data, that is, the feature similarity between the accent features of the target accent voice data and the accent features of the target accent user is greater than the accent similarity threshold. The accent similarity threshold is set according to the needs of the target user. The accent feature extraction process can be extracted using a pre-trained accent feature model. The feature similarity can be calculated using a pre-trained feature similarity calculation model, and the feature similarity threshold can be determined according to the needs of the target user, for example, it can be 95% or 96%. By merging the third sample data set into the first sample data set and using the merged first sample data set to train the initial speech recognition model, the accuracy of the recognition result of the target speech recognition model is improved.

[0071] In step S140 of this embodiment, the historical playback data includes the number of historical playbacks and the historical playback duration, the number of historical playbacks includes but is not limited to the total number of historical playbacks, and the historical playback duration includes but is not limited to the total historical playback duration. The user playback evaluation parameter is the priority of the current user attribute data to the current intelligent voice TV playing audio and video, wherein the user attribute data includes the user's historical playback data and user category data. Determining the user playback evaluation parameters of each user according to the historical playback data and the user category includes: obtaining the historical playback evaluation parameters of the historical playback data; if the user category is a registered category, obtaining the category evaluation parameters of the registered category, and obtaining the user playback evaluation parameters of each user according to the category evaluation parameters and the historical playback evaluation parameters; if the user category is a non-registered category, determining the historical playback evaluation parameters as the user playback evaluation parameters of each user. Among them, the historical playback evaluation parameters are the degree of influence of the current user's historical playback data on the priority of the current intelligent voice TV playing audio and video. Obtaining the user category of each user includes but is not limited to obtaining the voiceprint data of each user, and judging whether the voiceprint data of each user is the voiceprint data of a registered user. If so, the user category of the user is a registered category, and if not, the user category of the user is a non-registered category. The category evaluation parameter of the registered category can be set according to user needs, such as 50, 60, 70, etc. According to the category evaluation parameter and the historical playback evaluation parameter, obtaining the user playback evaluation parameter of each user includes adding the category evaluation parameter to the historical playback parameter to obtain the user playback evaluation parameter.

[0072] Specifically, obtaining the historical playback evaluation parameters of the historical playback data includes: obtaining the number evaluation parameters of each historical playback number respectively; obtaining the duration evaluation parameters of each of the historical playback durations respectively; weighting the number evaluation parameters and the duration evaluation parameters to obtain multiple historical playback evaluation parameters. Obtaining the number evaluation parameters of each historical playback number includes but is not limited to setting the evaluation scores corresponding to each playback number in advance, and the percentage system can be used to determine the evaluation scores corresponding to the playback number as the number evaluation parameters of each historical playback number, and the evaluation scores corresponding to each playback number can be set according to the needs of the target users. Obtaining the duration evaluation parameters of each historical playback duration includes but is not limited to setting the evaluation scores corresponding to each playback duration in advance, and the percentage system can be used to determine the evaluation scores corresponding to the playback duration as the duration evaluation parameters of each historical playback duration, and the evaluation scores corresponding to each playback duration can be set according to the needs of the target users. The number evaluation parameter and the duration evaluation parameter are weighted to obtain multiple historical playback evaluation parameters, including setting the number weight of the number evaluation parameter and the duration weight of the duration evaluation parameter, obtaining the product of the number weight and the number evaluation parameter to obtain the number product, obtaining the product of the duration evaluation parameter and the duration weight to obtain the duration product, and adding the number product and the duration product to obtain the historical playback evaluation parameter. The number weight and duration weight can be set according to user needs

[0073] In step S150 of this embodiment, considering that the noise data contained in each initial voice command may be inconsistent, and the difference in noise data will affect the accuracy of the voice recognition result, in order to eliminate the influence of the noise data in the initial voice recognition command on the target voice recognition, the target audio and video is determined according to the user playback evaluation parameter and the target recognition result, including: if there is inconsistency in the audio and video to be played corresponding to each target recognition result, the permission type of each audio and video to be played is obtained respectively, wherein the target recognition result and the video to be played are in a one-to-one correspondence; the noise evaluation parameters of each target recognition result are obtained respectively, and the target playback evaluation parameters of each video to be played are determined according to the noise evaluation parameters and the permission type; the target audio and video is determined according to the user playback evaluation parameter and the target playback evaluation parameter. Among them, the target playback evaluation parameter is the priority of the current user's voice command to the current smart voice TV to play audio and video. By determining the permission type of audio and video, and determining the target audio and video according to the permission type, the user is satisfied with the priority of watching the audio and video with permission, thereby improving the user experience.

[0074] In one embodiment, obtaining the noise evaluation parameter of the target recognition result includes obtaining the initial voice recognition instruction corresponding to the target recognition result, obtaining the environmental noise data in the initial voice instruction data, performing graded processing on the environmental noise data, obtaining the noise level of the environmental noise data, and determining the noise evaluation parameter according to the noise level. The evaluation score corresponding to each noise level is set in advance, and the evaluation score corresponding to the noise level is determined as the noise evaluation parameter. The evaluation score corresponding to each noise level can be determined according to user needs, for example, a percentage system can be used. The noise level can be set according to user needs, for example, it can include three levels: high, medium, and low. The higher the noise level, the higher the sound decibel. Considering that the higher the noise level, the stronger the interference, the higher the noise level, the lower the evaluation score. The sound decibel corresponding to the high noise level is in the first decibel range, the sound decibel corresponding to the medium noise level is in the second decibel range, and the sound decibel corresponding to the low noise level is in the third decibel range. The decibel values ​​of the first decibel range, the second decibel range, and the third decibel range can be set according to the needs of the target user. The lower limit value of the third decibel is greater than the upper limit value of the second decibel range, and the lower limit value of the second decibel range is greater than the upper limit value of the first decibel range.

[0075] In one embodiment, the target playback evaluation parameters of each video to be played are determined according to the noise evaluation parameter and the permission type, including: if the permission type is a permission type, the type evaluation parameter of the permission type is obtained, and the target playback evaluation parameters of each video to be played are determined according to the noise evaluation parameter and the type evaluation parameter; if the permission type is an unauthorized type, the target playback evaluation parameters of each video to be played are determined according to the noise evaluation parameter. The target playback evaluation parameter is the priority of the current user attribute data and the current user's voice command for the current smart voice TV to play audio and video. The type evaluation parameter of the permission type can be set according to user needs, for example, 50, 60, 70, etc. Determining the target playback evaluation parameters of each video to be played according to the noise evaluation parameter and the type evaluation parameter includes adding the noise evaluation parameter and the type evaluation parameter to obtain the target playback evaluation parameter. For the smart voice TV receiving voice commands of multiple different users at the same time within a period of time, there is also an unreasonable situation in which the command of which user is executed. By comprehensively considering the user attribute data (user playback evaluation parameter) and the user's voice command (target playback evaluation parameter), the determined target audio and video is more in line with the user, thereby improving the user experience.

[0076] Based on the same inventive concept as the method in the embodiment of the audio and video playback control method of the intelligent voice TV, accordingly, this embodiment also provides an audio and video playback control device for an intelligent voice TV.

[0077] Figure 2A schematic diagram of the flow of the audio and video playback control device of the intelligent voice television provided by the present invention.

[0078] like Figure 2 As shown, the device 2 includes: 21 a speech receiving module, 22 a speech denoising module, 23 a speech recognition module, 24 a user parameter determination module and 25 an audio and video playback module.

[0079] A voice receiving module, used to receive initial voice commands from multiple users, wherein there is a one-to-one correspondence between the users and the initial voice commands;

[0080] A voice denoising module, used to perform denoising processing on each of the initial voice instructions respectively to obtain multiple target voice instructions;

[0081] A speech recognition module, used to respectively use a pre-trained target speech recognition model to perform speech recognition on each of the target speech instructions to obtain multiple target recognition results;

[0082] A user parameter determination module, used to respectively obtain the historical playback data and user category of each of the users, and determine the user playback evaluation parameter of each user according to the historical playback data and the user category;

[0083] The audio and video playback module is used to determine the target audio and video according to the user playback evaluation parameter and the target recognition result, and play the target audio and video, wherein the voice receiving module, the voice denoising module, the voice recognition module, the user parameter determination module, and the audio and video playback module are connected.

[0084] In some exemplary embodiments, the audio and video playback control device of the intelligent voice television further includes:

[0085] A first annotation module is used to obtain sample speech data, annotate text information of the sample speech data, and obtain an annotation result of the sample speech data;

[0086] A first forming module, used for forming a first sample data set according to the sample speech data and the labeling result of the sample speech data;

[0087] The model building module is used to build an initial speech recognition model and use the first sample data set to train the initial speech recognition model to obtain a target speech recognition model.

[0088] In some exemplary embodiments, the audio and video playback control device of the intelligent voice television further includes:

[0089] A word meaning similarity module, used to obtain associated words of new audio and video, and determine the similarity between the annotation result of the sample speech data and the word meaning of the associated words;

[0090] A second forming module is used for acquiring the voice data of the associated word and the text annotation result of the associated word to form a second sample data set if the word meaning similarity is less than the word meaning similarity threshold;

[0091] The first merging module is used to merge the second sample data set into the first sample data set.

[0092] In some exemplary embodiments, the audio and video playback control device of the intelligent voice television further includes:

[0093] A feature similarity module, used to extract the accent features of the user and the accent features of the sample voice data, and obtain the feature similarity between the accent features of the user and the accent features of the sample voice data;

[0094] An accent data module, configured to obtain target accent speech data if the feature similarity is less than a feature similarity threshold;

[0095] annotating text information of the target accent speech data to obtain an annotation result of the target accent speech data, and forming a third sample data set according to the target accent speech data and the annotation result of the target accent speech data;

[0096] The third sample data set is merged into the first sample data set.

[0097] In some exemplary embodiments, the audio and video playback module includes:

[0098] A permission determination unit, for respectively obtaining permission types of the audio and video to be played if there is inconsistency between the audio and video to be played corresponding to each target recognition result, wherein the target recognition result and the video to be played are in a one-to-one correspondence;

[0099] A target parameter unit, used to obtain noise evaluation parameters of each target recognition result respectively, and determine a target playback evaluation parameter of each to-be-played video according to the noise evaluation parameter and the permission type;

[0100] The target audio and video unit is used to determine the target audio and video according to the user playback evaluation parameter and the target playback evaluation parameter.

[0101] In some exemplary embodiments, the target parameter unit includes:

[0102] A first target parameter subunit, configured to obtain a type evaluation parameter of the permission type if the permission type is a permission type, and determine a target playback evaluation parameter of each to-be-played video according to the noise evaluation parameter and the type evaluation parameter;

[0103] The second target parameter subunit is used to determine the target playback evaluation parameter of each to-be-played video according to the noise evaluation parameter if the permission type is a no-permission type.

[0104] In some exemplary embodiments, the user parameter determination module includes:

[0105] A historical parameter unit, used to obtain historical playback evaluation parameters of the historical playback data;

[0106] A first user parameter unit, configured to obtain a category evaluation parameter of the registered category if the user category is a registered category, and obtain a user playback evaluation parameter of each user according to the category evaluation parameter and the historical playback evaluation parameter;

[0107] The second user parameter unit is configured to determine the historical playback evaluation parameter as the user playback evaluation parameter of each user if the user category is a non-registered category.

[0108] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0109] In one embodiment, see Figure 3 This embodiment also provides an electronic device 300, including a memory 301, a processor 302, and a computer program stored in the memory and executable on the processor, wherein the processor 302 implements the steps of the method described in any of the above embodiments when executing the computer program.

[0110] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0111] The electronic device provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run the computer program so that the electronic device executes each step of the above method.

[0112] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0113] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0114] In the above embodiments, the description refers to "this embodiment", "an embodiment", "another embodiment", "in some exemplary embodiments" or "other embodiments" to indicate that the specific features, structures or characteristics described in conjunction with the embodiments are included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment", "an embodiment", "another embodiment" do not necessarily all refer to the same embodiment.

[0115] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0116] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0117] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0118] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0119] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for controlling audio and video playback of an intelligent voice television, characterized in that: The method comprises: Receiving initial voice commands from multiple users, wherein there is a one-to-one correspondence between the users and the initial voice commands; Performing denoising processing on each of the initial voice commands respectively to obtain multiple target voice commands; Using a pre-trained target speech recognition model to perform speech recognition on each of the target speech instructions to obtain multiple target recognition results; Respectively obtaining historical playback data and user categories of each of the users, and determining user playback evaluation parameters of each user according to the historical playback data and the user category; The target audio and video is determined according to the user playback evaluation parameter and the target recognition result, and the target audio and video is played.

2. The method for controlling audio and video playback of an intelligent voice television according to claim 1, characterized in that: The method further comprises: using a pre-trained target speech recognition model to perform speech recognition on each of the target speech instructions to obtain a plurality of target recognition results; Acquire sample speech data, annotate text information of the sample speech data, and obtain an annotation result of the sample speech data; Forming a first sample data set according to the sample speech data and the labeling result of the sample speech data; An initial speech recognition model is constructed, and the initial speech recognition model is trained using the first sample data set to obtain the target speech recognition model.

3. The method for controlling audio and video playback of an intelligent voice television according to claim 2, characterized in that: The step of forming a first sample data set according to the sample speech data and the labeling result of the sample speech data further includes: Acquire associated words of the new audio and video, and determine the semantic similarity between the annotation result of the sample speech data and the associated words; If the word meaning similarity is less than the word meaning similarity threshold, acquiring the voice data of the associated word and the text annotation result of the associated word to form a second sample data set; The second sample data set is merged into the first sample data set.

4. The method for controlling audio and video playback of an intelligent voice television according to claim 2, characterized in that: The step of forming a first sample data set according to the sample speech data and the labeling result of the sample speech data further includes: Extracting the accent features of the user and the accent features of the sample voice data, and obtaining feature similarity between the accent features of the user and the accent features of the sample voice data; If the feature similarity is less than the feature similarity threshold, acquiring the target accent speech data; annotating text information of the target accent speech data to obtain an annotation result of the target accent speech data, and forming a third sample data set according to the target accent speech data and the annotation result of the target accent speech data; The third sample data set is merged into the first sample data set.

5. The method for controlling audio and video playback of an intelligent voice television according to claim 1, characterized in that: The determining the target audio and video according to the user playback evaluation parameter and the target recognition result includes: If there is inconsistency between the audio and video to be played corresponding to each target recognition result, the permission type of each audio and video to be played is obtained respectively, wherein the target recognition result and the video to be played are in a one-to-one correspondence; Obtaining noise evaluation parameters of each target recognition result respectively, and determining target playback evaluation parameters of each to-be-played video according to the noise evaluation parameters and the permission type; The target audio and video is determined according to the user playback evaluation parameter and the target playback evaluation parameter.

6. The method for controlling audio and video playback of an intelligent voice television according to claim 5, characterized in that: The step of determining a target playback evaluation parameter for each video to be played according to the noise evaluation parameter and the permission type includes: If the permission type is a permissioned type, obtaining a type evaluation parameter of the permissioned type, and determining a target playback evaluation parameter of each to-be-played video according to the noise evaluation parameter and the type evaluation parameter; If the permission type is a no permission type, a target playback evaluation parameter of each video to be played is determined according to the noise evaluation parameter.

7. The method for controlling audio and video playback of an intelligent voice television according to claim 1, characterized in that: The determining of the user playback evaluation parameter of each user according to the historical playback data and the user category includes: Obtaining historical playback evaluation parameters of the historical playback data; If the user category is a registered category, then obtaining a category evaluation parameter of the registered category, and obtaining a user playback evaluation parameter of each user according to the category evaluation parameter and the historical playback evaluation parameter; If the user category is a non-registered category, the historical play evaluation parameter is determined as the user play evaluation parameter of each user.

8. An audio and video playback control device for an intelligent voice television, characterized in that: The device comprises: A voice receiving module, used to receive initial voice commands from multiple users, wherein there is a one-to-one correspondence between the users and the initial voice commands; A voice denoising module, used to perform denoising processing on each of the initial voice instructions respectively to obtain multiple target voice instructions; A speech recognition module, used to respectively use a pre-trained target speech recognition model to perform speech recognition on each of the target speech instructions to obtain multiple target recognition results; A user parameter determination module, used to respectively obtain the historical playback data and user category of each of the users, and determine the user playback evaluation parameter of each user according to the historical playback data and the user category; The audio and video playback module is used to determine the target audio and video according to the user playback evaluation parameter and the target recognition result, and play the target audio and video.

9. An electronic device, characterized in that: including a processor, a memory and a communication bus; The communication bus is used to connect the processor and the memory; The processor is configured to execute the computer program stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is used to make a computer execute the method according to any one of claims 1 to 7.

Images