Information processing method and device

Abstract

The present application provides an information processing method, including: obtaining a first audio segment in first audio information; determining the start time and end time of the first audio segment, and determining predicted text information corresponding to the first audio segment; determining second text information corresponding to the first audio segment in first text information corresponding to the first audio information based on the start time and end time of the first audio segment; replacing the second text information with the predicted text information; the present application also provides an information processing device.

Description

Technical Field

[0001] The present application relates to the field of voice information processing technology, and in particular to an information processing method and device. Background Art

[0002] In Automatic Speech Recognition (ASR) technology, the recognition accuracy of proper nouns in a certain field is lower than that of general nouns, resulting in errors in speech recognition results and affecting user experience. Therefore, there is an urgent need to improve the recognition accuracy of proper nouns in speech recognition technology. Summary of the Invention

[0003] The present application provides an information processing method and device to at least solve the above technical problems existing in the prior art.

[0004] On one hand, the present application provides an information processing method, comprising:

[0005] Obtaining a first audio segment from the first audio information;

[0006] Determining a start time and an end time of the first audio segment, and determining predicted text information corresponding to the first audio segment;

[0007] determining, according to the start time and end time of the first audio segment, second text information corresponding to the first audio segment in the first text information corresponding to the first audio information;

[0008] The second text information is replaced by the predicted text information.

[0009] In the above solution, the predicted text information is determined based on a text model, and the method further includes:

[0010] Get the first sentence sample;

[0011] Determining a text prediction label corresponding to the first sentence sample based on the text model;

[0012] Determining a difference between a text sample label corresponding to the first sentence sample and the text prediction label;

[0013] Based on the difference between the text sample label and the text prediction label, the model parameters of the text model are adjusted.

[0014] In the above solution, the text sample label corresponds to at least one of the first sentence samples with different emotions, different tones, different intonations or different speaking speeds.

[0015] In the above solution, determining the predicted text information corresponding to the first audio segment includes:

[0016] determining a text model for recognizing the first audio segment based on an application and / or language environment in which the first audio information was acquired;

[0017] Based on the text model, predicted text information corresponding to the first audio segment is identified.

[0018] In the above solution, obtaining the first audio segment in the first audio information includes:

[0019] determining a second time before the first time based on a first time at which the first audio information is currently received and a time threshold;

[0020] An audio segment in the first audio information whose start time is the second time and whose end time is the first time is determined to be the first audio segment.

[0021] In the above solution, obtaining the first audio segment in the first audio information includes:

[0022] When it is identified that the first audio information includes a target sentence type, determining a second time before the first time based on a first time when an audio segment corresponding to the target sentence type ends and a time threshold;

[0023] determining, in the first audio information, an audio segment with a start time of the second time and an end time of the first time as the first audio segment;

[0024] The target sentence type includes at least: a sentence type with a target text structure, a sentence type including a target keyword, or a sentence type including a target keyword.

[0025] In the above solution, the target sentence type is determined based on a sentence type model, and before identifying that the first audio information includes the target sentence type, the method further includes:

[0026] Obtain a second sentence sample;

[0027] Determining a sentence type prediction label corresponding to the second sentence sample based on the sentence type model;

[0028] determining a difference between a sentence type sample label corresponding to the second sentence sample and the sentence type prediction label;

[0029] The model parameters of the sentence type model are adjusted based on the difference between the sentence type prediction label and the sentence type sample label.

[0030] In the above solution, determining, based on the start time and end time of the first audio segment, the second text information corresponding to the first audio segment in each first text information corresponding to the first audio information includes:

[0031] Determining time information of an audio segment corresponding to each text in the first text information in the first audio information;

[0032] Searching the time information for the start time and the end time of the first audio segment;

[0033] Determine second text information corresponding to the start time and end time of the first audio segment.

[0034] In the above solution, after replacing the second text information with the predicted text information, the method further includes:

[0035] comparing the time similarity and phoneme similarity of each of the first text information including the first audio information and the predicted text information;

[0036] determining the first text information including the predicted text information and having the highest weighted value of time similarity and phoneme similarity as the third text information corresponding to the first audio information;

[0037] The third text information is output.

[0038] Another aspect of the present application provides an information processing device, the device comprising:

[0039] an acquiring unit, configured to acquire a first audio segment from the first audio information;

[0040] a processing unit, configured to determine a start time and an end time of the first audio segment, and determine predicted text information corresponding to the first audio segment; and determine, based on the start time and the end time of the first audio segment, second text information corresponding to the first audio segment in the first text information corresponding to the first audio information;

[0041] A replacing unit is configured to replace the second text information with the predicted text information.

[0042] Through the information processing method provided in the embodiment of the present application, a first audio segment in a first audio information is obtained; the start time and end time of the first audio segment are determined, and the predicted text information corresponding to the first audio segment is determined; second text information corresponding to the first audio segment in the first text information corresponding to the first audio information is determined based on the start time and end time of the first audio segment; the second text information is replaced with the predicted text information; speech recognition can be performed on the first audio segment corresponding to the proper noun to obtain predicted text information, and the position of the proper noun in the speech recognition result is determined based on the start time and end time of the first audio segment. By replacing the second text information with the predicted text information, the recognition accuracy of the proper noun in the speech recognition technology is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 An optional flow chart of the information processing method provided in an embodiment of the present application is shown;

[0044] Figure 2 Another optional flow chart of the information processing method provided in the embodiment of the present application is shown;

[0045] Figure 3 Another optional flow chart of the information processing method provided in the embodiment of the present application is shown;

[0046] Figure 4 Another optional flow chart of the information processing method provided in the embodiment of the present application is shown;

[0047] Figure 5 An optional structural diagram of an information processing device provided in an embodiment of the present application is shown;

[0048] Figure 6 A schematic diagram of the hardware structure of the information processing device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0049] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0050] In the application process of ASR, for specific fields, some technologies need to be adopted to improve the recognition accuracy of some proper nouns in the field, otherwise the recognition effect will be poor.

[0051] For example, the speech "Shanghai Baozhen Middle Road" can be easily recognized as "Shanghai Baozheng Middle Road".

[0052] The names of some of the company's newly released products are also prone to misidentification without some technical processing, such as "Lenovo Yoga", "thinkplus Pocket", "motorola edge s", "Motorola edge s", "razr 5G", "Lenovo ThinkBook", "Lemon K12 Pro", etc.

[0053] To improve the recognition accuracy of proper nouns, two main techniques are currently commonly used. The first involves expanding the language model (LM), augmenting the original large speech model with a small amount of text from a new domain. The second involves modifying the code in the recognition decoder using hotword technology, biasing the recognition results towards words in a hotword list. The drawback of both approaches is that they require extensive manual effort, including modifications to the dictionary, LM, and decoder.

[0054] Therefore, in response to the defects in speech recognition in the related art, the present application provides an information processing method that can overcome some or all of the shortcomings of the existing technology.

[0055] Figure 1 An optional flow chart of the information processing method provided in an embodiment of the present application is shown, and will be explained according to each step.

[0056] Step S101: Acquire a first audio segment in first audio information.

[0057] In some embodiments, an information processing device (hereinafter referred to as the device) obtains a first audio segment in first audio information.

[0058] In specific implementation, the device can determine the second time before the first time based on the first time of currently receiving the first audio information and the time threshold; and determine that the audio segment in the first audio information whose starting time is the second time and whose ending time is the first time is the first audio segment.

[0059] Alternatively, in a specific implementation, the device can identify that the first audio information includes a target sentence type, and based on the first time at which the audio segment of the target sentence type ends and a time threshold, determine a second time before the first time; determine that the audio segment in the first audio information whose start time is the second time and whose end time is the first time is the first audio segment; wherein the target sentence type includes at least: a sentence type with a target text structure, a sentence type including a target keyword, or a sentence type including a target keyword.

[0060] Step S102: determining the start time and end time of the first audio segment, and determining predicted text information corresponding to the first audio segment.

[0061] In some embodiments, the device determines a start time and an end time of the first audio segment in the first audio information.

[0062] In some embodiments, the apparatus may determine the predicted text information corresponding to the first audio segment based on a text model. Optionally, the apparatus inputs the first audio segment into the text model, and the output of the text model is the predicted text information corresponding to the first audio segment.

[0063] In a specific implementation, the apparatus may train the text model before step S101; optionally, the apparatus may obtain a first sentence sample; determine a text prediction label corresponding to the first sentence sample based on the text model; determine the difference between the text sample label corresponding to the first sentence sample and the text prediction label; and adjust model parameters of the text model based on the difference between the text sample label and the text prediction label, thereby implementing text model training.

[0064] Step S103 : determining, according to the start time and the end time of the first audio segment, second text information corresponding to the first audio segment in the first text information corresponding to the first audio information.

[0065] In some embodiments, the device determines the time information of the audio segment corresponding to each text in each first text information in the first audio information; searches for the start time and end time of the first audio segment in the time information; and determines the second text information corresponding to the start time and end time of the first audio segment.

[0066] Step S104: Replace the second text information with the predicted text information.

[0067] In some embodiments, the apparatus replaces the second text information with the predictive text information.

[0068] In some optional embodiments, the device may further compare the time similarity and phoneme similarity of each of the first text information including the predicted text information in the first audio information; determine the first text information having the highest weighted value of time similarity and phoneme similarity of the first text information including the predicted text information as the third text information corresponding to the first audio information; and output the third text information.

[0069] In this way, through the information processing method provided in the embodiment of the present application, by obtaining the first audio segment in the first audio information, and then determining the predicted text information corresponding to the first audio segment, the first audio segment is processed separately to ensure the recognition accuracy of the first audio segment; then, based on the start time and end time of the first audio segment, the second text information corresponding to the first audio segment in the first text information corresponding to the first audio information is determined; the second text information is replaced with the predicted text information; so as to improve the recognition accuracy of the first audio information. In addition, in speech recognition technology, as the user inputs voice, the output recognition text information needs to change according to the user's latest voice input content, and the final recognition result will not be output until the user's voice input is completed. In the information processing method provided in the embodiment of the present application, after determining the predicted text information corresponding to the first audio segment, it can assist in determining the text information corresponding to other audio segments, thereby reducing the amount of calculation and improving speech recognition efficiency.

[0070] Figure 2 Another optional flow chart of the information processing method provided in an embodiment of the present application is shown, and will be explained according to each step.

[0071] Step S201: training a text model.

[0072] In some embodiments, the information processing device obtains at least one first sentence sample corresponding to a text sample label with different emotions, different tones, different intonations, or different speech rates. Based on the text model, the information processing device determines a text prediction label corresponding to the first sentence sample; determines a difference between the text sample label corresponding to the first sentence sample and the text prediction label; and adjusts model parameters of the text model based on the difference between the text sample label and the text prediction label.

[0073] In some optional embodiments, the device may further obtain at least one third sentence sample corresponding to a text sample label with a different emotion, tone, intonation, or speaking speed. Based on the text model after adjusting the model parameters, a text test label corresponding to the third sentence sample is determined; a difference between the text sample label corresponding to the third sentence sample and the text test label is determined; and based on the difference between the text sample label corresponding to the third sentence sample and the text test label, the model parameters of the text model are adjusted. In this manner, the text model is tested.

[0074] Step S202: Acquire a first audio segment in the first audio information.

[0075] In some embodiments, the device can determine a second time before the first time based on the first time of currently receiving the first audio information and a time threshold; and determine that the audio segment in the first audio information whose starting time is the second time and whose ending time is the first time is the first audio segment.

[0076] In some optional embodiments, the apparatus may further determine, based on the first time, the time threshold, and the first period, at least one first time and at least one second time before the at least one first time, and determine at least one first audio segment based on the at least one first time and the second time corresponding to the at least one first time.

[0077] Step S203: Determine the predicted text information corresponding to the first audio segment.

[0078] In some embodiments, the apparatus may determine the predicted text information corresponding to the first audio segment based on a text model. Optionally, the apparatus inputs the first audio segment into the text model, and the output of the text model is the predicted text information corresponding to the first audio segment.

[0079] Step S204: Determine, according to the start time and the end time of the first audio segment, the second text information corresponding to the first audio segment in the first text information corresponding to the first audio information.

[0080] In some embodiments, the device determines the start time and end time of the first audio segment in the first audio information, and the device determines the time information of the audio segment corresponding to each text in each first text information in the first audio information; searches for the start time and end time of the first audio segment in the time information; and determines the second text information corresponding to the start time and end time of the first audio segment.

[0081] Step S205: Replace the second text information with the predicted text information.

[0082] In some embodiments, the apparatus replaces the second text information with the predictive text information.

[0083] Step S206: output the third text information.

[0084] In some optional embodiments, the device compares the time similarity and phoneme similarity of each of the first text information including the predicted text information in the first audio information; determines the first text information with the highest weighted value of time similarity and phoneme similarity of the first text information including the predicted text information as the third text information corresponding to the first audio information; and outputs the third text information.

[0085] In this way, through the information processing method provided in the embodiment of the present application, by obtaining the first audio segment in the first audio information, and then determining the predicted text information corresponding to the first audio segment, the first audio segment is processed separately to ensure the recognition accuracy of the first audio segment; then, based on the start time and end time of the first audio segment, the second text information corresponding to the first audio segment in the first text information corresponding to the first audio information is determined; the second text information is replaced with the predicted text information; so as to improve the recognition accuracy of the first audio information. In addition, in speech recognition technology, as the user inputs voice, the output recognition text information needs to change according to the user's latest voice input content, and the final recognition result will not be output until the user's voice input is completed. In the information processing method provided in the embodiment of the present application, after determining the predicted text information corresponding to the first audio segment, it can assist in determining the text information corresponding to other audio segments, thereby reducing the amount of calculation and improving speech recognition efficiency.

[0086] Figure 3 Another optional flow chart of the information processing method provided in an embodiment of the present application is shown, and will be explained according to each step.

[0087] Step S301: training a text model.

[0088] In some embodiments, the information processing device obtains at least one first sentence sample corresponding to a text sample label with different emotions, different tones, different intonations, or different speech rates. Based on the text model, the information processing device determines a text prediction label corresponding to the first sentence sample; determines a difference between the text sample label corresponding to the first sentence sample and the text prediction label; and adjusts model parameters of the text model based on the difference between the text sample label and the text prediction label.

[0089] In some optional embodiments, the device may further obtain at least one third sentence sample corresponding to a text sample label with a different emotion, tone, intonation, or speaking speed. Based on the text model after adjusting the model parameters, a text test label corresponding to the third sentence sample is determined; a difference between the text sample label corresponding to the third sentence sample and the text test label is determined; and based on the difference between the text sample label corresponding to the third sentence sample and the text test label, the model parameters of the text model are adjusted. In this manner, the text model is tested.

[0090] Step S302: training a sentence type model.

[0091] In some embodiments, the device obtains a second sentence sample; determines a sentence type prediction label corresponding to the second sentence sample based on the sentence type model; determines the difference between the sentence type sample label corresponding to the second sentence sample and the sentence type prediction label; and adjusts the model parameters of the sentence type model based on the difference between the sentence type prediction label and the sentence type sample label.

[0092] Step S303: Acquire a first audio segment in the first audio information.

[0093] In some embodiments, when the device identifies that the first audio information includes a target sentence type based on the sentence type model, it determines a second time before the first time based on the first time at which the audio segment of the target sentence type ends and a time threshold; and determines that the audio segment in the first audio information whose start time is the second time and whose end time is the first time is the first audio segment; wherein the target sentence type includes at least: a sentence type with a target text structure, a sentence type including a target keyword, or a sentence type including a target keyword.

[0094] In some optional embodiments, the device identifies, based on the sentence type model, that the first audio information includes target keywords such as "road", "street", and "station", or identifies that the first audio information includes target keywords such as "airport", "bus", "internet cafe", "supermarket", "square", and "bank", or identifies that the first audio information has a sentence type with a target text structure of "AABB" or "ABB". It should be noted that in the embodiments of the present application, the target keywords "road", "street", and "station", the target keywords "airport", "bus", "internet cafe", "supermarket", "square", and "bank", and the target text structures "AABB" and "ABB" are merely used as examples to illustrate the embodiments of the present application and are not intended to limit the present application.

[0095] During specific implementation, when the device receives the first audio information as "Check how to get to XX Square", the device recognizes the target keyword "square" at the 2nd second of the first audio information, then confirms that the first time is the 2nd second, and based on the first time and the time threshold (such as 0.5 seconds), determines that the second time is the 1.5th second, and determines that the audio segment corresponding to the 1.5th second to the 2nd second in the first audio information is the first audio segment.

[0096] Step S304: Determine predicted text information corresponding to the first audio segment.

[0097] In some embodiments, the apparatus determines the predicted text information corresponding to the first audio segment based on a text model. Optionally, the apparatus inputs the first audio segment into the text model, and the output of the text model is the predicted text information corresponding to the first audio segment.

[0098] Step S305 : Determine, according to the start time and the end time of the first audio segment, the second text information corresponding to the first audio segment in the first text information corresponding to the first audio information.

[0099] In some embodiments, the device determines the start time and end time of the first audio segment in the first audio information, and the device determines the time information of the audio segment corresponding to each text in each first text information in the first audio information; searches for the start time and end time of the first audio segment in the time information; and determines the second text information corresponding to the start time and end time of the first audio segment.

[0100] Step S306: Replace the second text information with the predicted text information.

[0101] In some embodiments, the apparatus replaces the second text information with the predictive text information.

[0102] Step S307: output the third text information.

[0103] In some optional embodiments, the device compares the time similarity and phoneme similarity of each of the first text information including the predicted text information in the first audio information; determines the first text information with the highest weighted value of time similarity and phoneme similarity of the first text information including the predicted text information as the third text information corresponding to the first audio information; and outputs the third text information.

[0104] In this way, through the information processing method provided by the embodiment of the present application, when the target sentence type is identified based on the sentence type model, the first audio segment in the first audio information is obtained, and then the predicted text information corresponding to the first audio segment is determined, so as to realize separate processing of the first audio segment and ensure the recognition accuracy of the first audio segment; then, based on the start time and end time of the first audio segment, the second text information corresponding to the first audio segment in the first text information corresponding to the first audio information is determined; the second text information is replaced with the predicted text information; so as to improve the recognition accuracy of the first audio information. In addition, in speech recognition technology, as the user inputs voice, the output recognition text information needs to change according to the user's latest voice input content, and the final recognition result will not be output until the user's voice input is completed. In the information processing method provided by the embodiment of the present application, after determining the predicted text information corresponding to the first audio segment, it can assist in determining the text information corresponding to other audio segments, thereby reducing the amount of calculation and improving speech recognition efficiency.

[0105] In some optional embodiments, before step S101, step S201, and step S301, the apparatus may further determine a text model for identifying the first audio segment.

[0106] In a specific implementation, the device determines a text model for identifying the first audio segment based on an application and / or language environment that collects the first audio information; and identifies predicted text information corresponding to the first audio segment based on the text model.

[0107] Accordingly, the device can also train text models based on different applications and / or language environments.

[0108] For example, if the first audio information is collected by a map navigation application, the device may recognize the first audio segment based on the text model corresponding to the map navigation application. This avoids the problem of excessive computational overhead caused by using the same text model regardless of the application and / or language environment.

[0109] Figure 4 Another optional flow chart of the information processing method provided in the embodiment of the present application is shown, and will be explained according to each step.

[0110] Step S401: determine a first sentence sample.

[0111] In some embodiments, the information processing device obtains at least one of the first sentence samples with different emotions, different tones, different intonations, or different speaking speeds corresponding to the text sample labels.

[0112] In a specific implementation, the device can synthesize training corpora (first sentence samples) of different speaker styles by adjusting speaker embedding (Speaker Embedding) in Text To Speech (TTS).

[0113] Step S402: training a text model.

[0114] In some embodiments, the device determines a text prediction label corresponding to the first sentence sample based on the text model; determines the difference between the text sample label corresponding to the first sentence sample and the text prediction label; and adjusts the model parameters of the text model based on the difference between the text sample label and the text prediction label.

[0115] In specific implementation, the device can train different text models based on different text prediction labels (i.e., different text prediction labels correspond to different text models), and the output of the text model is "yes text prediction label" or "not text prediction label"; the device can also train the same text model based on different text prediction labels (i.e., different text prediction labels correspond to the same text model), and the output of the text model is "yes XX text prediction label" or "not text prediction label" (i.e., the number of categories of the text model is the number of text prediction labels + 1).

[0116] Step S403: Acquire a first audio segment in the first audio information.

[0117] In some embodiments, the device can determine a second time before the first time based on the first time of currently receiving the first audio information and a time threshold; and determine that the audio segment in the first audio information whose starting time is the second time and whose ending time is the first time is the first audio segment.

[0118] In some optional embodiments, the apparatus may further determine, based on the first time, the time threshold, and the first period, at least one first time and at least one second time before the at least one first time, and determine at least one first audio segment based on the at least one first time and the second time corresponding to the at least one first time.

[0119] In a specific implementation, the apparatus forms a first audio segment by tracing back a time threshold (eg, 1.5 seconds) from a first time from the first audio information received in real time.

[0120] For example, the device may backtrack 1.5 seconds (time threshold) from the 3rd second (first time) of the first audio information to obtain the audio segment between the 1.5th second (second time) and the 3rd second as the first audio segment. The device may also backtrack 1.5 seconds (time threshold) from the 3.1st second of the first audio information to obtain the audio segment between the 1.6th second and the 3.1st second as the first audio segment.

[0121] Step S404: Determine predicted text information corresponding to the first audio segment.

[0122] In some embodiments, the apparatus may determine the predicted text information corresponding to the first audio segment based on a text model. Optionally, the apparatus inputs the first audio segment into the text model, and the output of the text model is the predicted text information corresponding to the first audio segment.

[0123] Step S405 : Determine, according to the start time and the end time of the first audio segment, the second text information corresponding to the first audio segment in the first text information corresponding to the first audio information.

[0124] In some embodiments, when the device identifies the predicted text information corresponding to the first audio segment, the device determines the start time and end time of the first audio segment in the first audio information, and the device determines the time information of the audio segment corresponding to each text in each first text information in the first audio information; searches for the start time and end time of the first audio segment in the time information; and determines the second text information corresponding to the start time and end time of the first audio segment.

[0125] Step S406: Replace the second text information with the predicted text information.

[0126] In some embodiments, the apparatus replaces the second text information with the predictive text information.

[0127] Step S407: output the third text information.

[0128] In some optional embodiments, the device compares the time similarity and phoneme similarity of each of the first text information including the predicted text information in the first audio information; determines the first text information with the highest weighted value of time similarity and phoneme similarity of the first text information including the predicted text information as the third text information corresponding to the first audio information; and outputs the third text information.

[0129] In some optional embodiments, in the process of converting the first voice information into text information, there will be multiple first text information in the n-best list; based on the time similarity and phoneme similarity of each first text information; the first text information with the highest weighted value of time similarity and phoneme similarity of the first text information including the predicted text information is determined to be the third text information.

[0130] In this way, the information processing method provided in the embodiment of the present application can improve the recognition performance of ASR in specific fields; there is no need to make any modifications to the ASR system; the training data preparation, training and other processes of the text model basically do not require any manual intervention and can be implemented automatically; when the new ASR usage field is changed, the activation word model can be retrained. The method of improving ASR performance through the information processing method provided in the embodiment of the present application is highly scalable.

[0131] Figure 5 An optional structural diagram of the information processing device provided in an embodiment of the present application is shown, and will be explained according to each part.

[0132] In some embodiments, the information processing device 500 includes: an acquisition unit 501 , a processing unit 502 , and a replacement unit 503 .

[0133] The acquiring unit 501 is configured to acquire a first audio segment from the first audio information;

[0134] The processing unit 502 is configured to determine the start time and end time of the first audio segment, and determine the predicted text information corresponding to the first audio segment; and determine, based on the start time and end time of the first audio segment, second text information corresponding to the first audio segment in the first text information corresponding to the first audio information;

[0135] The replacing unit 503 is configured to replace the second text information with the predicted text information.

[0136] The processing unit 502 is further configured to obtain a first sentence sample; determine a text prediction label corresponding to the first sentence sample based on the text model; determine a difference between the text sample label corresponding to the first sentence sample and the text prediction label; and adjust model parameters of the text model based on the difference between the text sample label and the text prediction label.

[0137] In some embodiments, the text sample label corresponds to at least one of the first sentence samples with different emotions, different tones, different intonations, or different speaking speeds.

[0138] The processing unit 502 is specifically configured to determine a text model for identifying the first audio segment based on an application and / or language environment that collected the first audio information; and identify predicted text information corresponding to the first audio segment based on the text model.

[0139] The acquiring unit 501 is specifically configured to determine a second time before the first time based on a first time at which the first audio information is currently received and a time threshold;

[0140] An audio segment in the first audio information whose start time is the second time and whose end time is the first time is determined to be the first audio segment.

[0141] The acquiring unit 501 is specifically configured to, when identifying that the first audio information includes a target sentence type, determine a second time before the first time based on a first time at which an audio segment corresponding to the target sentence type ends and a time threshold;

[0142] determining, in the first audio information, an audio segment with a start time of the second time and an end time of the first time as the first audio segment;

[0143] The target sentence type includes at least: a sentence type with a target text structure, a sentence type including a target keyword, or a sentence type including a target keyword.

[0144] The processing unit 502 is also used to obtain a second sentence sample; determine the sentence type prediction label corresponding to the second sentence sample based on the sentence type model; determine the difference between the sentence type sample label corresponding to the second sentence sample and the sentence type prediction label; and adjust the model parameters of the sentence type model based on the difference between the sentence type prediction label and the sentence type sample label.

[0145] The processing unit 502 is specifically configured to determine time information of an audio segment corresponding to each text in the first text information in the first audio information;

[0146] Searching the time information for the start time and the end time of the first audio segment;

[0147] Determine second text information corresponding to the start time and end time of the first audio segment.

[0148] The processing unit 502 is further configured to compare the time similarity and phoneme similarity of each of the first text information including the predicted text information in the first audio information; determine the first text information having the highest weighted value of time similarity and phoneme similarity of the first text information including the predicted text information as the third text information corresponding to the first audio information; and output the third text information.

[0149] In some optional embodiments, the information processing device 500 can be set in the cloud or on a terminal device.

[0150] Figure 6 700 is a schematic diagram of the hardware structure of the information processing device provided in the embodiment of the present application. The information processing device 700 includes: at least one processor 701, a memory 702 and at least one network unit 704. The various components in the information processing device 700 are coupled together through a bus system 705. It can be understood that the bus system 705 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 705 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus system 705.

[0151] It is understood that the memory 702 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be ROM, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disk, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk or a magnetic tape. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The memory 702 described in the embodiments of the present application is intended to include, but is not limited to, these and any other suitable types of memory.

[0152] The memory 702 in the embodiment of the present application is used to store various types of data to support the operation of the information processing device 700. Examples of such data include any computer program for operating on the information processing device 700, such as the application 722. The program for implementing the method of the embodiment of the present application may be included in the application 722.

[0153] The method disclosed in the embodiment of the present application can be applied to the processor 701 or implemented by the processor 701. The processor 701 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the method can be completed by an integrated logic circuit of the hardware in the processor 701 or by instructions in the form of software. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 701 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 702. The processor 701 reads the information in the memory 702 and completes the steps of the aforementioned method in combination with its hardware.

[0154] In an exemplary embodiment, the information processing device 700 can be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, MCUs, MPUs, or other electronic components to execute the aforementioned method.

[0155] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0156] The computer program product may be written in any combination of one or more programming languages to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0157] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0158] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0159] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0160] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0161] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0162] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0163] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An information processing method, comprising: Obtaining a first audio segment from the first audio information; The first audio segment is an audio segment corresponding to text information that needs to be replaced; The obtaining of the first audio segment from the first audio information includes: identifying a target sentence type in the first audio information, and determining a second time before the first time based on a first end time of the audio segment corresponding to the target sentence type and a time threshold; determining an audio segment in the first audio information having a start time of the second time and an end time of the first time as the first audio segment; wherein the target sentence type includes at least: a sentence type having a target text structure, a sentence type including a target keyword, or a sentence type including a target keyword; Determining a start time and an end time of the first audio segment, and determining predicted text information corresponding to the first audio segment; Determining the predicted text information corresponding to the first audio segment includes: determining a text model for identifying the first audio segment based on an application and / or language environment in which the first audio segment is acquired; and identifying the predicted text information corresponding to the first audio segment based on the text model. determining, according to the start time and end time of the first audio segment, second text information corresponding to the first audio segment in the first text information corresponding to the first audio information; The second text information is replaced by the predicted text information.

2. The method according to claim 1, wherein the predicted text information is determined based on a text model, the method further comprising: Get the first sentence sample; Determining a text prediction label corresponding to the first sentence sample based on the text model; Determining a difference between a text sample label corresponding to the first sentence sample and the text prediction label; Based on the difference between the text sample label and the text prediction label, the model parameters of the text model are adjusted.

3. The method according to claim 2, The text sample label corresponds to at least one of the first sentence samples with different emotions, different tones, different intonations or different speaking speeds.

4. The method according to claim 1, wherein obtaining the first audio segment in the first audio information comprises: determining a second time before the first time based on a first time at which the first audio information is currently received and a time threshold; An audio segment in the first audio information whose start time is the second time and whose end time is the first time is determined to be the first audio segment.

5. The method according to claim 1, wherein the target sentence type is determined based on a sentence type model, and before identifying that the first audio information includes the target sentence type, the method further comprises: Obtain a second sentence sample; Determining a sentence type prediction label corresponding to the second sentence sample based on the sentence type model; determining a difference between a sentence type sample label corresponding to the second sentence sample and the sentence type prediction label; The model parameters of the sentence type model are adjusted based on the difference between the sentence type prediction label and the sentence type sample label.

6. The method according to claim 1, wherein determining, based on the start time and end time of the first audio segment, the second text information corresponding to the first audio segment in each first text information corresponding to the first audio information comprises: Determining time information of an audio segment corresponding to each text in the first text information in the first audio information; Searching the time information for the start time and the end time of the first audio segment; Determine second text information corresponding to the start time and end time of the first audio segment.

7. The method according to claim 1, after replacing the second text information with the predicted text information, the method further comprises: comparing the time similarity and phoneme similarity of each of the first text information including the first audio information and the predicted text information; determining the first text information including the predicted text information and having the highest weighted value of time similarity and phoneme similarity as the third text information corresponding to the first audio information; The third text information is output.

8. An information processing device, comprising: an acquiring unit, configured to acquire a first audio segment from the first audio information; The first audio segment is an audio segment corresponding to text information that needs to be replaced; The obtaining of the first audio segment from the first audio information includes: identifying a target sentence type in the first audio information, and determining a second time before the first time based on a first end time of the audio segment corresponding to the target sentence type and a time threshold; determining an audio segment in the first audio information having a start time of the second time and an end time of the first time as the first audio segment; wherein the target sentence type includes at least: a sentence type having a target text structure, a sentence type including a target keyword, or a sentence type including a target keyword; a processing unit, configured to determine a start time and an end time of the first audio segment, and determine predicted text information corresponding to the first audio segment; and determine, based on the start time and the end time of the first audio segment, second text information corresponding to the first audio segment in the first text information corresponding to the first audio information; Determining the predicted text information corresponding to the first audio segment includes: determining a text model for identifying the first audio segment based on an application and / or language environment in which the first audio segment is acquired; and identifying the predicted text information corresponding to the first audio segment based on the text model. A replacing unit is configured to replace the second text information with the predicted text information.

Images