Speech playback method and apparatus, and electronic device and storage medium

Abstract

A speech playback method and apparatus, and an electronic device and a storage medium. The method comprises: performing speech recognition on a speech segment to be played, so as to obtain initial text data corresponding to said speech segment, and time information corresponding to the initial text data (S110); on the basis of the initial text data and the time information, obtaining initial structured information (S120); if a modification operation on the initial text data is detected, determining modification information corresponding to the initial text data (S130); on the basis of the modification information, updating content of a label of a modification state corresponding to each byte in the initial structured information, so as to obtain target structured information (S140); and on the basis of the target structured information, performing speech playback on said speech segment, so as to ensure aligned playback of modified text and the speech segment (S150). The speech playback method can improve the reliability of speech playback.

Description

Voice playing method and device, electronic device, and storage medium

[0001] Cross-reference to Related Applications

[0002] This application claims priority to and the benefit of the filing date of the patent application No. 202410724499.4 filed with the China National Intellectual Property Office on June 5, 2024, which is hereby incorporated by reference in its entirety for all purposes. TECHNICAL FIELD

[0003] The present application relates to the field of voice data processing, and more particularly, to a voice playing method and device, an electronic device, and a storage medium. BACKGROUND

[0004] With the development of science and technology, electronic devices are applied more and more widely, such as electronic devices are applied to conference recording, voice-to-text conversion, and the like. Since the voice recognition technology has the problem of incorrect recognition of conversation content, in the related art, a scheme of repairing the original conference recognition content is introduced. Based on this, in the related art, there is a problem that the alignment between the repaired text and the original voice segment is inaccurate, and the voice playing is unreliable.

[0005] SUMMARY

[0006] In view of the above problems, the present application provides a voice playing method and device, an electronic device, and a storage medium, which can set a structured label for the voice recognition result of a voice segment to be played, and update the structured label based on modification information corresponding to a modification operation on the text of voice recognition when the modification operation is detected, and play the voice segment to be played based on the updated structured label, thereby improving the accuracy of the alignment between the modified text and the voice segment, improving the reliability of the voice playing, and improving the user experience.

[0007] In a first aspect, an embodiment of the present application provides a voice playing method, which comprises: performing voice recognition on a to-be-played voice segment to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information comprises timestamps corresponding to each byte in the initial text data; obtaining initial structured information based on the initial text data and the time information, wherein the initial structured information comprises labels of initial states corresponding to the each byte and a label of a modification state being empty, the initial states comprising initial playing order, initial content, initial start time and initial end time, and the modification state comprising modified playing order, modified content, modified start time and modified end time; if a modification operation on the initial text data is detected, determining modification information corresponding to the initial text data; updating content of the label of the modification state corresponding to the each byte in the initial structured information based on the modification information to obtain target structured information; and performing voice playing on the to-be-played voice segment according to the target structured information.

[0008] In a second aspect, an embodiment of the present application provides a voice playing device, which comprises: a voice segment automatic recognition module, an identification result structured module, a modification information determination module, a structured information updating module and a voice playing module. The voice segment automatic recognition module is configured to perform voice recognition on a to-be-played voice segment to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information comprises timestamps corresponding to each byte in the initial text data. The identification result structured module is configured to obtain initial structured information based on the initial text data and the time information, wherein the initial structured information comprises labels of initial states corresponding to the each byte and a label of a modification state being empty, the initial states comprising initial playing order, initial content, initial start time and initial end time, and the modification state comprising modified playing order, modified content, modified start time and modified end time. The modification information determination module is configured to, if a modification operation on the initial text data is detected, determine modification information corresponding to the initial text data. The structured information updating module is configured to update content of the label of the modification state corresponding to the each byte in the initial structured information based on the modification information to obtain target structured information. The voice playing module is configured to perform voice playing on the to-be-played voice segment according to the target structured information.

[0009] In a third aspect, an embodiment of the present application provides an electronic device, which comprises a memory and a processor, the memory is coupled to the processor, and the memory stores instructions, when the instructions are executed by the processor, the processor performs the above method.

[0010] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores program codes. The program codes can be invoked by a processor to execute the above method.

[0011] The voice playing method and device, the electronic device and the storage medium provided by the embodiments of the present application can improve the accuracy of the alignment of the modified text and the voice segment, improve the reliability of the voice playing, and improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0013] FIG. 1 shows a flowchart of a voice playing method according to an embodiment of the present application;

[0014] FIG. 2 shows a flowchart of a voice playing method according to an embodiment of the present application;

[0015] FIG. 3 shows a flowchart of a voice playing method according to an embodiment of the present application;

[0016] FIG. 4 shows a flowchart of a voice playing method according to an embodiment of the present application;

[0017] Fig. 5 shows a module block diagram of a voice playing device according to an embodiment of the present application;

[0018] Fig. 6 shows a block diagram of an electronic device for performing a voice playing method according to an embodiment of the present application;

[0019] Fig. 7 shows a storage unit for storing or carrying program codes for implementing a voice playing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0021] With the development of science and technology, electronic devices are applied more and more widely and have more and more functions. In current intelligent dialogue systems, due to the limitations of speech recognition technology, the recognized conversation content has a certain degree of error. In order to solve this problem, a method of repairing the original conference recognition content to improve the speech recognition accuracy and semantic consistency is proposed. However, in the process of voice playing, due to the inaccurate alignment between the repaired text and the original voice segment, the voice playing has the problem of low reliability.

[0022] In view of the above problems, the inventors have found, after long-term research, and proposed the voice playing method, device, electronic device and storage medium provided in the embodiments of the present application. The voice playing method, device, electronic device and storage medium provided in the embodiments of the present application set a structured label for the result of speech recognition of a to-be-played voice segment, and update the structured label based on modification information corresponding to a modification operation on the text of the voice recognition when the modification operation is detected, and play the to-be-played voice segment based on the updated structured label, thereby improving the accuracy of the alignment between the modified text and the voice segment, improving the reliability of the voice playing, and improving the user experience. The specific voice playing method is described in detail in subsequent embodiments.

[0023] Please refer to Fig. 1, which shows a flowchart of a voice playing method according to an embodiment of the present application. In specific embodiments, the voice playing method can be applied to a voice playing device 200 as shown in Fig. 5 and an electronic device 100 (Fig. 6) configured with the voice playing device 200. In the following, the specific process of the present embodiment will be described taking the electronic device as an example. It can be understood that the electronic device to which the present embodiment is applied can include a smart phone, a tablet computer, a smart television, a smart wearable device, etc., which are not limited herein. The voice playing method can specifically include the following steps, which will be described in detail with reference to Fig. 1.

[0024] Step S110: performing speech recognition on the to-be-played voice segment to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information includes time stamps corresponding to each byte in the initial text data.

[0025] In some embodiments, the to-be-played voice segment can be pre-set in the electronic device, and the electronic device can also obtain the to-be-played voice segment from an associated cloud or electronic device through wireless communication technology (such as WiFi, Bluetooth, zigbee, etc.).

[0026] As an implementable manner, the electronic device includes a sound pickup device, and the electronic device can receive a sound pickup instruction input by a user. Accordingly, the electronic device can control the sound pickup device to perform sound collection in response to the sound pickup instruction input by the user, and accordingly, the electronic device can obtain the sound collected by the sound pickup device and determine the sound as the to-be-played voice segment. Wherein, the electronic device can also perform filtering, noise reduction, etc. on the sound collected by the sound pickup device, extract the audio of the human voice included in the sound, and determine the audio including the human voice as the to-be-played voice segment.

[0027] In some embodiments, after the electronic device obtains the to-be-played voice segment, the electronic device can perform speech recognition on the to-be-played voice segment to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data. Wherein, the time information corresponding to the initial text data can include time stamps corresponding to each byte in the initial text data.

[0028] Wherein, the electronic device can have an automatic speech recognition (ASR) function, based on which the electronic device can perform text recognition on the to-be-played voice segment and determine the time stamp corresponding to each byte included in the text based on the ASR function. Wherein, the electronic device can convert the to-be-played voice segment into text by analyzing the to-be-played voice segment based on the ASR function to obtain initial text data corresponding to the to-be-played voice segment, and the electronic device can also obtain the time stamp corresponding to each character included in the initial text data. Wherein, the time stamp corresponding to each character can represent the time point or time period when each character appears in the to-be-played voice segment.

[0029] The electronic device can be preconfigured with an acoustic model (e.g., a Hidden Markov Model (HMM), a Gaussian Mixture Model (GMM), a Deep Neural Network (DNN), etc.) and a language model (e.g., a NNLM, a RNNLM, a word2vec, a GloVe, an ELMo, etc.). Accordingly, the electronic device can implement an ASR function based on the acoustic model and the language model, so that the electronic device can perform speech recognition processing on the to-be-played voice segment using the acoustic model and the language model to obtain accurate recognized text and time information, thereby obtaining word-level time alignment information between the to-be-played voice segment and the initial text data.

[0030] In step S120, initial structured information is obtained based on the initial text data and the time information. The initial structured information includes labels of initial states corresponding to each byte in the initial text data and a label of a modification state being empty. The initial state includes an initial play order, initial content, an initial start time, and an initial end time. The modification state includes a modified play order, modified content, a modified start time, and a modified end time.

[0031] In some embodiments, after the electronic device obtains the initial text data corresponding to the to-be-played voice segment and the time information corresponding to the initial text, the electronic device can obtain initial structured information based on the initial text data and the time information. The initial structured information can include labels of initial states corresponding to each byte in the initial text data and a label of a modification state being empty. The initial state can include an initial play order, initial content, an initial start time, and an initial end time. The modification state can include a modified play order, modified content, a modified start time, and a modified end time. The label of the modification state being empty can be understood as a case where the recognized text is not modified.

[0032] The initial play order in the label of the initial state corresponding to each byte in the initial text data can be understood as an index of a play order corresponding to each byte in the initial text data. The initial content can be understood as the content of the byte. The initial start time can be understood as a start time of the byte in the to-be-played voice segment. The initial end time can be understood as an end time of the byte in the to-be-played voice segment. The modified play order in the label of the modification state corresponding to each byte in the initial text data can be understood as an index of a modified order corresponding to each byte in the initial text data after modification of the initial text data. The modified content can be understood as the content of the byte after modification. The modified start time can be understood as a start time of the byte in the to-be-played voice segment after modification of the initial text data. The modified end time can be understood as an end time of the byte in the to-be-played voice segment after modification of the initial text data.

[0033] The initial structured information can be understood as recording the results of text recognition of the speech segment to be played and the time recognition of the words included in the text at the character level, thereby establishing a character-level association between the spoken content and the speech segment, and improving the accuracy of speech and text alignment playback.

[0034] In some implementations, the audio segment to be played may correspond to at least one initial speaker. Accordingly, after obtaining the audio segment to be played, the electronic device may also perform voice segmentation on the audio segment to determine the speaking content of different speakers at different time periods. For example, the electronic device may perform voiceprint recognition on the audio segment to be played to obtain the audio segments corresponding to at least one initial speaker and the speaking time corresponding to at least one initial speaker.

[0035] The electronic device can be pre-configured with a voiceprint recognition model (e.g., GMM, d-vector, x-vector, ResNet, etc.). This model can determine whether the initial speaker is the same person by comparing the voiceprint information of two input speech segments. It distinguishes different speakers based on their acoustic features, thus performing voice segmentation. Based on this, the electronic device can input the speech segment to be played into the voiceprint recognition model, obtain the voiceprint information corresponding to at least one initial speaker, and cluster the voiceprint information of at least one initial speaker in the speech segment to be played to obtain the speech segments corresponding to at least one initial speaker and the speaking time corresponding to at least one initial speaker.

[0036] As one feasible approach, the electronic device can simultaneously perform speech recognition on the audio segment to be played, obtaining the initial text data corresponding to the audio segment and the time information corresponding to the initial text data, and simultaneously perform voiceprint recognition on the audio segment to be played, obtaining the audio segments corresponding to at least one initial speaker and the speaking time corresponding to at least one initial speaker. Correspondingly, in the process of obtaining initial structured information based on the initial text data and the time information corresponding to the initial text data, the electronic device can also obtain initial structured information based on the audio segments corresponding to at least one initial speaker, the speaking time corresponding to at least one initial speaker, the initial text data, and the time information. Accordingly, the initial state label corresponding to each byte in the initial structured information may include the label of the initial speaker corresponding to each byte; similarly, the modified state label corresponding to each byte may also include the label of the modified speaker corresponding to each byte.

[0037] The initial speaker corresponding to each byte can be understood as the initial speaker corresponding to the byte in the to-be-played voice segment, and the modified speaker corresponding to each byte can be understood as the modified speaker corresponding to the byte in the to-be-played voice segment after modification of the initial text data.

[0038] The initial structured information can be understood as recording the results of text recognition of the to-be-played voice segment, time recognition of the words included in the text, and voice segmentation of the to-be-played voice segment according to the character level. The electronic device can determine the speaking content of different speakers in different time periods by combining the time information returned by the ASR recognition and the time information of the voice segmentation. By combining the recognition result of the ASR with the result of the voice segmentation, the speaking content of different speakers in a specific time period and the corresponding voice segment time can be accurately determined, thereby realizing accurate alignment of the text and the voice. The electronic device performs data alignment through time mapping to ensure accurate association between the speaking content and the voice segment, thereby improving the accuracy of voice and text alignment and playback.

[0039] For example, the initial structured information can include the order index (which can be understood as the initial playback order in this embodiment), content (which can be understood as the initial content in this embodiment), start and end times (which can be understood as the initial start time and the initial end time in this embodiment), speaker information (which can be understood as the initial speaker in this embodiment), modified speaker information (which can be understood as the modified speaker in this embodiment), and state (which can be understood as the modification operation corresponding to the initial text data in this embodiment), modified text content (which can be understood as the modified content in this embodiment), and other label content.

[0040] For example, the code content of the initial structured information in the JSON format is as follows:

[0041] It can be understood that each recognized word in the initial structured information has its index, content, start and end times, speaker information, and state, which facilitates subsequent alignment and playback of the modified text and the voice segment based on the initial text data. The state can include a None state without modification or a Modified state with modification. The Modified state can include an add state, a sub state, or a del state.

[0042] Step S130: If a modification operation for the initial text data is detected, determining the modification information corresponding to the initial text data.

[0043] In some embodiments, after obtaining the initial text data corresponding to the to-be-played voice segment, the electronic device can output target prompt information, which can include the initial text data and can be used to prompt the result of the text recognition of the to-be-played voice segment. Accordingly, if the electronic device detects modified text data or a modification instruction input based on the target prompt information, it can be determined that a modification operation for the initial text data is detected.

[0044] Accordingly, if the electronic device detects a modification operation for the initial text data, it can determine the modification information corresponding to the initial text data. The modification information can include modified text data or the modification instruction.

[0045] As an implementable way, the modification instruction can carry a modification position corresponding to the initial text data, or a corresponding byte modification operation, or a speaker modification corresponding to a byte. Accordingly, the electronic device can obtain the modified text data corresponding to the modification operation based on the modification instruction.

[0046] After obtaining the modified text data, the electronic device can process the modified text data and the initial text data based on the edit distance algorithm to obtain modification information corresponding to the initial text data, such as byte deletion, byte replacement, byte addition, speaker information change, etc.

[0047] Step S140: Update the content of the label of the modification state of each byte in the initial structured information based on the modification information to obtain target structured information.

[0048] In some embodiments, after obtaining the modification information corresponding to the initial text data, the electronic device can update the content of the label of the modification state of each byte in the initial structured information based on the modification information to obtain target structured information.

[0049] The electronic device can determine the modified state of each byte in the initial text data based on the modification information corresponding to the initial text data, such as unmodified, deleted, replaced, added, etc., and update the content of the label of the modification state of each byte in the initial structured information based on the modified text data and the modified state of each byte to obtain target structured information, thereby realizing accurate alignment of the repaired text and the to-be-played voice segment, improving the user experience of voice playback, and improving the resource utilization rate of voice playback.

[0050] In some embodiments, the modification information can further include modification information of the speaker, and the modification state corresponding to each byte in the initial structured information can further include a label corresponding to the modified speaker. Accordingly, in the process of updating the label content of the modification state corresponding to each byte in the initial structured information to obtain the target structured information based on the modification information, the label content of the modified speaker corresponding to each byte in the initial structured information can be updated to obtain the target structured information, thereby improving the reliability of voice playback.

[0051] Step S150: voice playing the to-be-played voice segment according to the target structured information.

[0052] In some embodiments, after the electronic device obtains the target structured information corresponding to the to-be-played voice segment, the electronic device can voice play the to-be-played voice segment according to the target structured information. For example, the electronic device can play the to-be-played voice segment based on the label content of the modification state corresponding to each byte in the target structured information, such as playing the to-be-played voice segment based on the modification play order corresponding to each byte, playing the to-be-played voice segment based on the modification start time and the modification end time corresponding to each byte, and the like. Thus, the text after modification and the corresponding voice segment are accurately aligned at the character level, and the reliability of voice playing is improved.

[0053] As an implementable manner, in the process of voice playing the to-be-played voice segment according to the target structured information, the electronic device can detect the label of the modification state corresponding to each byte. If it is detected that there is a fourth byte with an empty label of the modification state in the target structured information, the electronic device can voice play the to-be-played voice segment according to the label of the initial state corresponding to the fourth byte. The fourth byte with an empty label of the modification state can be understood as a byte that is not modified.

[0054] As another implementable manner, in the process of voice playing the to-be-played voice segment according to the target structured information, if it is detected that there is a fifth byte with a non-empty label of the modification state in the target structured information, the electronic device can voice play the to-be-played voice segment according to the label of the modification state corresponding to the fifth byte. The fifth byte with a non-empty label of the modification state can be understood as a byte that is modified.

[0055] Wherein, it can be understood that the electronic device can play in time sequence according to the playing order of each byte in the target structured information when playing the text alignment of the to-be-played voice segment, wherein, if the label of the modification state corresponding to the byte is detected as non-empty, the to-be-played voice segment is played in priority according to the label content of the modification state, and if the label of the modification state corresponding to the byte is detected as empty, the to-be-played voice segment is played according to the label content of the initial state, so as to play the alignment of voice and text based on the modified content, improve the reliability of voice playing, and improve the experience of users.

[0056] In some embodiments, the electronic device can also receive a user input specifying playing instruction, wherein the specifying playing instruction can carry a specified position, and the specifying playing instruction can be used to instruct the electronic device to play the to-be-played voice segment based on the specified position and the target structured information. Wherein, the specified position can be understood as a specified time period in the to-be-played voice segment, or can be understood as a specified text content in the initial text data, which is not limited here; so that the electronic device plays the alignment of the repaired text and the to-be-played voice segment, provides users with more efficient and reliable intelligent conversation experience, improves the accuracy of voice playing, and improves the experience of users.

[0057] In addition, it should be noted that the target structured information can include the initial state label corresponding to each byte in the initial text data, and can also include the modification state label corresponding to each byte in the modified initial text data. Based on this, the electronic device can receive a user input reset instruction, and the reset instruction can be used to instruct the electronic device to restore the initial structured information corresponding to the to-be-played voice segment. Accordingly, the electronic device can obtain the initial structured information corresponding to the to-be-played voice segment based on the initial state label corresponding to each byte in the target structured information in response to the reset instruction, improve the accuracy and experience of users for text repair, and structure the corresponding relationship between text and voice segment according to the byte level, improve the accuracy of the alignment of the modified text and the voice segment, improve the reliability of voice playing, and improve the experience of users.

[0058] The voice playing method provided in an embodiment of the present application comprises the following steps: performing voice recognition on a to-be-played voice segment to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information comprises time stamps corresponding to each byte in the initial text data; obtaining initial structured information based on the initial text data and the time information, wherein the initial structured information comprises labels of initial states of each byte and a label of a modification state being empty, the initial states comprise initial playing order, initial content, initial start time and initial end time, and the modification state comprises modified playing order, modified content, modified start time and modified end time; if a modification operation for the initial text data is detected, determining modification information corresponding to the initial text data; updating content of the label of the modification state of each byte in the initial structured information based on the modification information to obtain target structured information; and playing the to-be-played voice segment according to the target structured information, so that the labels of the structure are set according to the result of voice recognition on the to-be-played voice segment, and if a modification operation for the text of voice recognition is detected, the labels of the structure are updated based on modification information corresponding to the modification operation, and the to-be-played voice segment is played based on the updated labels of the structure, thereby improving the accuracy of the modification text and the voice segment alignment, improving the reliability of voice playing, and improving the user experience.

[0059] Referring to FIG. 2, FIG. 2 shows a flowchart of a voice playing method provided in an embodiment of the present application. The method is applied to the electronic device described above, and the following will be described in detail with respect to the flowchart shown in FIG. 2. The voice playing method can specifically comprise the following steps:

[0060] Step S210: performing voice recognition on a to-be-played voice segment to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information comprises time stamps corresponding to each byte in the initial text data.

[0061] Step S220: obtaining initial structured information based on the initial text data and the time information, wherein the initial structured information comprises labels of initial states of each byte and a label of a modification state being empty, the initial states comprise initial playing order, initial content, initial start time and initial end time, and the modification state comprises modified playing order, modified content, modified start time and modified end time.

[0062] Step S230: if a modification operation for the initial text data is detected, determining modification information corresponding to the initial text data.

[0063] The specific description of steps S210-S230 can refer to the description of steps S110-S130, which will not be repeated here.

[0064] Step S240: If the modification operation on the initial text data is a content addition operation, a first position corresponding to the added byte in the initial text data is determined, and a byte adjacent to the first position in the initial text data is determined as a first byte.

[0065] In some embodiments, after the electronic device detects the modification operation on the initial text data, the modification information corresponding to the initial text data can be determined according to the edit distance algorithm. Wherein, the electronic device can obtain the modified initial text data based on the modification operation, and determine the modified initial text data as the modified text data; accordingly, the electronic device can process the modified text data and the initial text data based on the edit distance algorithm, and obtain the modification operation corresponding to the initial text data, such as deletion, replacement, addition, etc.

[0066] As an implementable manner, if the electronic device detects that the modification operation on the initial text data is a content addition operation, the first position corresponding to the added byte in the initial text data can be determined, and the byte adjacent to the first position in the initial text data can be determined as the first byte.

[0067] Wherein, the number of first bytes can include one or two, which is not limited here. For example, the single word index corresponding to the initial text data is 1-5; accordingly, it is detected according to the edit distance algorithm that the modification operation on the initial text data is a content addition operation, and the first position corresponding to the added byte in the initial text data is between position 4 and position 5. Accordingly, the bytes adjacent to the first position in the initial text data can be determined as the first byte, that is, position 4 and position 5 are determined as the first byte.

[0068] Step S250: updating the content of the label of the modification state of each byte in the initial structured information according to the first position and the first byte, to obtain the target structured information.

[0069] In some embodiments, after the electronic device obtains the first position corresponding to the added byte and the first byte in the initial text data, the content of the label of the modification state of each byte in the initial structured information can be updated according to the first position and the first byte, to obtain the target structured information.

[0070] The electronic device can add a first structure in the initial structured information according to the first position, and set the content of the label of the initial state corresponding to the first structure as empty. Accordingly, the electronic device can set the content of the label of the modified content corresponding to the first structure according to the added byte; and the electronic device can also set the content of the label of the non-modified content in the modified state corresponding to the first structure according to the first byte, to obtain the target structured information. The label of the non-modified content in the modified state can include a label of a modified start time, a label of a modified end time, and a label of a modified playing order, etc.

[0071] As an implementable manner, the process in which the electronic device sets the content of the label of the non-modified content in the modified state corresponding to the first structure according to the first byte can include obtaining a target time length according to the content of the label of the initial start time corresponding to the first byte and the content of the label of the initial end time corresponding to the first byte, and setting the content of the label of the modified start time corresponding to the first structure and the content of the label of the modified end time corresponding to the first structure according to the target time length, and updating the content of the label of the modified start time corresponding to the first byte and the content of the label of the modified end time corresponding to the first byte.

[0072] In view of the fact that there is a non-single-character playing situation in the process in which the electronic device plays the to-be-played voice segment, based on this, the electronic device can divide the target time length equally to the first byte and the added byte, so as to set the content of the label of the modified start time corresponding to the first structure and the content of the label of the modified end time corresponding to the first structure, and update the content of the label of the modified start time corresponding to the first byte and the content of the label of the modified end time corresponding to the first byte, thereby improving the reliability of voice playing.

[0073] As an implementable manner, the process in which the electronic device sets the content of the label of the non-modified content in the modified state corresponding to the first structure according to the first byte can also include determining a first byte located at a previous position of the first position in the initial text data as a second byte, and setting the content of the label of the modified playing order corresponding to the first structure and the content of the label of the modified playing order corresponding to a byte located after the second byte in the initial text data according to the content of the label of the initial playing order corresponding to the second byte, to obtain the target structured information.

[0074] The electronic device can set the content of the label of the modified playing order corresponding to the first structure as one more than the initial playing order corresponding to the second byte, and set the content of the label of the modified playing order corresponding to each byte located after the second byte in the initial text data as one more than the initial playing order corresponding to the byte.

[0075] For example, the single word index corresponding to the initial text data is 1-5; accordingly, it is detected according to the edit distance algorithm that the modification operation for the initial text data is a content addition operation, and it is determined that the first position corresponding to the added byte in the initial text data is between the 4th position and the 5th position. Accordingly, the byte adjacent to the first position in the initial text data can be determined as the first byte, that is, the 4th position and the 5th position are determined as the first byte. Accordingly, the electronic device can increase the first structure in the initial structured information based on the first position, and can set the content of the label of the initial state corresponding to the first structure as empty, and can set the label content of the modified content corresponding to the first structure as the content of the added byte, and can obtain the target time length according to the initial start time of the 4th position and the initial end time of the 5th position, and divide the target time length equally to the 4th position, the first position and the 5th position. For example, the content of the initial start time corresponding to the byte located at the 4th position is obtained, and the initial start time content is added by one third of the target time length to obtain the modified end time corresponding to the byte located at the 4th position, and the modified end time corresponding to the byte located at the 4th position is taken as the modified start time corresponding to the byte located at the first position, and the modified start time corresponding to the byte located at the first position is added by one third of the target time length to obtain the modified end time corresponding to the byte located at the first position, and the modified end time corresponding to the byte located at the first position is taken as the modified initial time corresponding to the byte located at the 5th position.

[0076] Among them, the electronic device can also set the content of the label of the modified start time corresponding to the first position and the content of the label of the modified end time corresponding to the first structure, and update the content of the label of the modified start time and the content of the label of the modified end time corresponding to the 4th position and the 5th position.

[0077] Optionally, the electronic device can also determine the label content corresponding to the added byte according to the modification operation and set it in the first structure. Accordingly, the electronic device can also set the state corresponding to the first structure as "add". Accordingly, the electronic device can determine the first byte located at the previous position of the first position in the initial text data as the second byte, and determine the initial playback order corresponding to the second byte as the start order of the first structure, and set the label content of the modified playback order corresponding to the first structure and the label content of the modified playback order corresponding to the byte located after the second byte in the initial text data according to the start order, thereby obtaining the target structured information.

[0078] Step S260: performing voice playback on the to-be-played voice segment according to the target structured information.

[0079] Among them, the specific description of step S260 can refer to the description of step S150 in the preceding, which will not be described in detail here.

[0080] Compared with the voice playing method shown in FIG. 1, the voice playing method provided by the embodiment of the present application can further determine a first position corresponding to an added byte in the initial text data if the modification operation on the initial text data is a content adding operation, and determine a first byte adjacent to the first position in the initial text data; update the content of the label of the modification state corresponding to each byte in the initial structured information according to the first position and the first byte, to obtain target structured information, and then set the structured label according to the result of voice recognition on the to-be-played voice segment, and in the case that a content adding operation on the text of voice recognition is detected, increase a structure in the initial structured information and set the content of the label corresponding to the structure, and update the content of the label corresponding to each byte in the initial structured information, so that the to-be-played voice segment is played based on the updated structured label, thereby improving the accuracy of the modification text and the voice segment alignment, improving the reliability of voice playing, and improving the user experience.

[0081] Please refer to FIG. 3, which shows a flowchart of the voice playing method provided by the embodiment of the present application. The method is applied to the electronic device described above, and the flowchart shown in FIG. 3 will be described in detail as follows. The voice playing method can specifically include the following steps:

[0082] Step S310: performing voice recognition on a to-be-played voice segment, to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information includes time stamps corresponding to each byte in the initial text data.

[0083] Step S320: obtaining initial structured information based on the initial text data and the time information, wherein the initial structured information includes labels of initial states corresponding to the each byte and labels of modification states being empty, the initial states include initial playing order, initial content, initial start time and initial end time, and the modification states include modified playing order, modified content, modified start time and modified end time.

[0084] Step S330: if a modification operation on the initial text data is detected, determining modification information corresponding to the initial text data.

[0085] The specific description of steps S310-S330 can be found in the description of steps S110-S130 above, which will not be repeated here.

[0086] Step S340: if the modification operation on the initial text data is a content replacing operation, determining a second position corresponding to a replaced byte in the initial text data.

[0087] In some embodiments, after detecting the modification operation on the initial text data, the electronic device can determine the modification information corresponding to the initial text data according to the edit distance algorithm. Specifically, the electronic device can obtain the modified initial text data based on the modification operation, and determine the modified initial text data as the modified text data. Then, the electronic device can process the modified text data and the initial text data based on the edit distance algorithm, and obtain the modification operation corresponding to the initial text data.

[0088] As an implementable manner, if the electronic device detects that the modification operation on the initial text data is a content replacement operation, the second position corresponding to the replacement byte in the initial text data can be determined. It can be understood that replacement means that the original byte content is replaced, and at this time, the original byte content in the initial structured information already exists in the corresponding structure. Based on this, in the case of content replacement operation, the initial structured information does not need to add a structure, and the content included in the label of the initial state of the byte located at the second position in the initial text data can remain unchanged.

[0089] Step S350: updating the content of the label of the modification content corresponding to the byte located at the second position in the initial text data according to the replacement byte, to obtain the target structured information.

[0090] In some embodiments, after determining the second position, the electronic device can update the content of the label of the modification content corresponding to the byte located at the second position in the initial text data according to the replacement byte, to obtain the target structured information.

[0091] Alternatively, the electronic device can also determine the content of the label of the modified speaker corresponding to the replacement byte according to the modification operation, and update the content of the label of the modified speaker corresponding to the byte located at the second position. Correspondingly, the electronic device can also set the state corresponding to the replacement byte as “replacement sub”.

[0092] Step S360: performing voice playing on the to-be-played voice segment according to the target structured information.

[0093] For specific description of step S360, please refer to the description of step S150 in the foregoing, which will not be described in detail here.

[0094] Compared with the voice playing method shown in FIG. 1, the voice playing method provided by the embodiment of the present application can further determine a second position corresponding to a replacement byte in the initial text data if the modification operation on the initial text data is a content replacement operation; update the content of the label of the modified content of the byte at the second position in the initial text data according to the replacement byte, to obtain target structured information; and then set the structured label according to the result of voice recognition on the to-be-played voice segment, and in the case where a content replacement operation on the text of voice recognition is detected, determine the position of the added byte in the initial structured information and update the content of the label of the modified content of the byte at the position, so that the to-be-played voice segment is played based on the updated structured label, the accuracy of the alignment of the modified text and the voice segment is improved, the reliability of voice playing is improved, and the experience of the user is improved.

[0095] Referring to FIG. 4, FIG. 4 shows a flowchart of a voice playing method provided by an embodiment of the present application. The method is applied to the electronic device described above, and the flowchart shown in FIG. 4 will be described in detail below. The voice playing method can specifically include the following steps:

[0096] Step S410: performing voice recognition on a to-be-played voice segment, to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information includes time stamps corresponding to each byte in the initial text data.

[0097] Step S420: obtaining initial structured information based on the initial text data and the time information, wherein the initial structured information includes labels of initial states of the each byte and a label of a modification state being empty, and the initial state includes an initial playing order, an initial content, an initial start time and an initial end time, and the modification state includes a modified playing order, a modified content, a modified start time and a modified end time.

[0098] Step S430: if a modification operation on the initial text data is detected, determining modification information corresponding to the initial text data.

[0099] For specific descriptions of steps S410-S430, refer to the descriptions of steps S110-S130 in the foregoing, which will not be repeated here.

[0100] Step S440: if the modification operation on the initial text data is a content deletion operation, determining a third position corresponding to a deletion byte in the initial text data, and determining a byte at a position in front of the third position in the initial text data as a third byte.

[0101] In some embodiments, after detecting the modification operation on the initial text data, the electronic device can determine the modification information corresponding to the initial text data according to the edit distance algorithm. Specifically, the electronic device can obtain the modified initial text data based on the modification operation, and determine the modified initial text data as the modified text data. Then, the electronic device can process the modified text data and the initial text data based on the edit distance algorithm, and obtain the modification operation corresponding to the initial text data.

[0102] As an implementable manner, if the electronic device detects that the modification operation on the initial text data is a content deletion operation, the electronic device can determine a third position corresponding to the deletion byte in the initial text data, and determine a third byte located in front of the third position in the initial text data as the third byte.

[0103] It should be understood that the content deletion operation on the initial text data can be understood as deleting the recognized content, for example, deleting the recognized noise. Accordingly, the electronic device can keep the content of the label corresponding to the initial state of the deletion byte in the initial structured information unchanged, so as to facilitate the user to reset the target structured information to the initial structured information, improve the flexibility of text modification, and improve the user experience.

[0104] Step S450: setting the content of the label corresponding to the modification content of the byte located at the third position as empty, setting the content of the label corresponding to the modification start time of the byte located at the third position as empty, and setting the content of the label corresponding to the modification end time of the byte located at the third position as the content of the label corresponding to the initial end time of the third byte, to obtain the target structured information.

[0105] In some embodiments, after determining the third position corresponding to the deletion byte in the initial text data and the third byte located in front of the third position in the initial text data, the electronic device can set the content of the label corresponding to the modification content of the byte located at the third position in the initial structured information as empty, set the content of the label corresponding to the modification start time of the byte located at the third position as empty, and set the content of the label corresponding to the modification end time of the byte located at the third position as the content of the label corresponding to the initial end time of the third byte, to obtain the target structured information.

[0106] It should be understood that the content of the label corresponding to the modification start time of the byte located at the third position is set as empty, and the content of the label corresponding to the initial end time of the third byte is assigned to the content of the label corresponding to the modification end time of the byte located at the third position, so as to avoid modifying the playing time of the other bytes except the deletion byte in the initial text data, and reduce the power consumption of the voice and text alignment playing.

[0107] Optionally, the electronic device can also determine the content of the label corresponding to the modified speaker of the deletion byte according to the modification operation and set it in the initial structured information. Accordingly, the electronic device can also set the state corresponding to the deletion byte as "deletion del", thereby obtaining the target structured information.

[0108] Step S460: performing voice playing on the to-be-played voice segment according to the target structured information.

[0109] For specific description of step S460, please refer to the description of step S150 in the foregoing, which will not be described in detail here.

[0110] Compared with the voice playing method shown in FIG. 1, the voice playing method provided by the embodiment of the present application can further determine a third position corresponding to a deletion byte in the initial text data if the modification operation on the initial text data is a content deletion operation, and determine a third byte as a byte located at a position in front of the third position in the initial text data; set the content of the label of the modified content corresponding to the byte at the third position as empty, set the content of the label of the modified start time corresponding to the byte at the third position as empty, and set the content of the label of the modified end time corresponding to the byte at the third position as the content of the label of the initial end time corresponding to the third byte, thereby obtaining the target structured information. Then, the voice playing method sets the structured label according to the result of voice recognition on the to-be-played voice segment, and updates the content of the label of the modified state corresponding to the deletion byte in the initial text data in the case of detecting a content deletion operation on the text of voice recognition, so as to play the to-be-played voice segment based on the updated structured label, thereby improving the accuracy of the modification text and the voice segment alignment, improving the reliability of voice playing, and improving the experience of users.

[0111] Please refer to FIG. 5, which shows a module block diagram of a voice playing device provided by an embodiment of the present application. The voice playing device 200 is applied to the electronic device described above. The voice playing device 200 shown in FIG. 5 will be described in detail as follows. The voice playing device 200 includes a voice segment automatic recognition module 210, a recognition result structured module 220, a modification information determination module 230, a structured information updating module 240, and a voice playing module 250, wherein:

[0112] The voice segment automatic recognition module 210 is configured to perform voice recognition on a to-be-played voice segment, obtain initial text data corresponding to the to-be-played voice segment, and obtain time information corresponding to the initial text data, wherein the time information includes a timestamp corresponding to each byte in the initial text data.

[0113] The recognition result structuring module 220 is configured to obtain initial structured information based on the initial text data and the time information, wherein the initial structured information includes labels of initial states corresponding to the bytes and a label of a modification state being empty, the initial states include an initial playing order, initial content, an initial start time and an initial end time, and the modification state includes a modified playing order, modified content, a modified start time and a modified end time.

[0114] The modification information determining module 230 is configured to determine modification information corresponding to the initial text data if a modification operation on the initial text data is detected.

[0115] The structured information updating module 240 is configured to update content of the label of the modification state corresponding to the bytes in the initial structured information based on the modification information, and obtain target structured information.

[0116] The voice playing module 250 is configured to play the voice segment to be played according to the target structured information.

[0117] Further, the structured information updating module 240 can include a content addition position determining unit and an addition operation updating unit.

[0118] The content addition position determining unit is configured to determine a first position of an added byte corresponding to the initial text data if the modification operation on the initial text data is a content addition operation, and determine a first byte adjacent to the first position in the initial text data.

[0119] The addition operation updating unit is configured to update content of the label of the modification state corresponding to the bytes in the initial structured information according to the first position and the first byte, and obtain the target structured information.

[0120] Further, the addition operation updating unit can include a structure body adding unit and a structure body modification state setting unit.

[0121] The structure body adding unit is configured to add a first structure body in the initial structured information according to the first position, and set a label of an initial state corresponding to the first structure body to be empty.

[0122] The structure body modification state setting unit is configured to set content of a label of modified content corresponding to the first structure body according to the added byte, and set content of a label of non-modified content in the modification state corresponding to the first structure body according to the first byte, and obtain the target structured information.

[0123] Further, the structure modification state setting unit can comprise: a target duration obtaining unit, an addition operation time updating unit, a second byte determining unit and a play order updating unit, wherein:

[0124] The target duration obtaining unit is configured to obtain a target duration according to the content of the label of the initial start time corresponding to the first byte and the content of the label of the initial end time corresponding to the first byte.

[0125] The addition operation time updating unit is configured to set the content of the label of the modified start time corresponding to the first structure and the content of the label of the modified end time corresponding to the first structure according to the target duration, and update the content of the label of the modified start time corresponding to the first byte and the content of the label of the modified end time corresponding to the first byte.

[0126] The second byte determining unit is configured to determine a first byte located at a position in front of the first position in the initial text data as a second byte.

[0127] The play order updating unit is configured to set the content of the label of the modified play order corresponding to the first structure and the content of the label of the modified play order corresponding to the byte located behind the second byte in the initial text data according to the content of the label of the initial play order corresponding to the second byte, and obtain the target structured information.

[0128] Further, the structured information updating module 240 can comprise: a content replacement position determining unit and a replacement operation updating unit, wherein:

[0129] The content replacement position determining unit is configured to, if the modification operation for the initial text data is a content replacement operation, determine a second position in the initial text data corresponding to a replacement byte.

[0130] The replacement operation updating unit is configured to update the content of the label of the modified content corresponding to the byte located at the second position in the initial text data according to the replacement byte, and obtain the target structured information.

[0131] Further, the structured information updating module 240 can comprise: a content deletion position determining unit and a deletion operation updating unit, wherein:

[0132] The content deletion position determining unit is configured to, if the modification operation for the initial text data is a content deletion operation, determine a third position in the initial text data corresponding to a deletion byte, and determine a third byte located at a position in front of the third position in the initial text data as a third byte.

[0133] The deletion operation updating unit sets the content of the label of the modification content corresponding to the byte at the third position as null, sets the content of the label of the modification start time corresponding to the byte at the third position as null, and sets the content of the label of the modification end time corresponding to the byte at the third position as the content of the label of the initial end time corresponding to the third byte, to obtain the target structured information.

[0134] Further, the voice playing module 250 can include a first playing unit or a second playing unit, wherein:

[0135] The first playing unit is configured to, if it is detected that there is a fourth byte with a null label of the corresponding modification state in the target structured information, play the voice segment to be played according to the label of the initial state corresponding to the fourth byte.

[0136] The second playing unit is configured to, if it is detected that there is a fifth byte with a non-null label of the corresponding modification state in the target structured information, play the voice segment to be played according to the label of the modification state corresponding to the fifth byte.

[0137] Further, the voice segment to be played corresponds to at least one initial speaker, and the recognition result structuring module 220 can include a voiceprint recognition unit and a recognition result structuring subunit, wherein:

[0138] The voiceprint recognition unit is configured to perform voiceprint recognition on the voice segment to be played, to obtain a voice segment corresponding to each of the at least one initial speaker and a speaking time corresponding to each of the at least one initial speaker.

[0139] The recognition result structuring subunit is configured to obtain the initial structured information based on the voice segment corresponding to each of the at least one initial speaker, the speaking time corresponding to each of the at least one initial speaker, the initial text data, and the time information, wherein the label of the initial state corresponding to each byte in the initial structured information includes a label of an initial speaker corresponding to each byte.

[0140] Further, the modification information includes information of a modification speaker, and the modification state includes a modification speaker, and the structured information updating module 240 can include a structured information updating subunit, wherein:

[0141] The structured information updating subunit is configured to update the content of the label of the modification speaker corresponding to each byte in the initial structured information based on the information of the modification speaker, to obtain the target structured information.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and module can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0143] In several embodiments provided in the present application, the coupling between the modules can be electrical, mechanical or other forms of coupling.

[0144] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically independently, or two or more modules can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0145] Please refer to FIG. 6, which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 can be a smart phone, a tablet computer, an electronic book or other electronic device capable of running an application program. The electronic device 100 in the present application can include one or more of the following components: a processor 110, a memory 120 and one or more application programs, wherein the one or more application programs can be stored in the memory 120 and configured to be executed by the one or more processors 110, and the one or more programs are configured to perform the method as described in the foregoing method embodiments.

[0146] The processor 110 can include one or more processing cores. The processor 110 connects various parts in the entire electronic device 100 through various interfaces and lines, performs various functions of the electronic device 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be realized in the form of at least one of digital signal processing, field programmable gate array and programmable logic array. The processor 110 can integrate a combination of one or several of central processing unit (CPU), graphics processing unit (GPU) and modem. Among them, the CPU mainly processes operating systems, user interfaces and application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 110, but realized by a separate communication chip.

[0147] The memory 120 can include random access memory and can also include read only memory. The memory 120 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing each of the methods described below, and the like. The data storage area can also store data created by the electronic device 100 in use (such as a phone book, audio and video data, chat record data), and the like.

[0148] Referring to FIG. 7, a structural block diagram of a computer readable storage medium according to an embodiment of the present application is shown. The computer readable medium 300 stores program codes therein, which can be invoked by a processor to execute the methods described in the above method embodiments.

[0149] The computer readable storage medium 300 can be an electronic storage such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer readable storage medium 300 includes a non-transitory computer readable medium. The computer readable storage medium 300 has a storage space for program codes 310 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program codes 310 can be compressed in an appropriate form, for example.

[0150] To sum up, the voice playing method and device, the electronic device, and the storage medium provided by the embodiments of the present application set a structured label for the result of voice recognition of a voice segment to be played, and in the case where a modification operation on the text of voice recognition is detected, update the structured label based on modification information corresponding to the modification operation, and play the voice segment to be played based on the updated structured label, which improves the accuracy of aligning the modified text and the voice segment, improves the reliability of voice playing, and improves the user experience.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features therein; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A voice play method characterized by, The method comprises: performing speech recognition on a to-be-played voice segment to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information comprises timestamps corresponding to each byte in the initial text data; obtaining initial structured information based on the initial text data and the time information, wherein the initial structured information comprises labels of initial states of the each byte and a label of a modification state being empty, the initial states comprising initial playing order, initial content, initial start time and initial end time, and the modification state comprising modified playing order, modified content, modified start time and modified end time; if a modification operation on the initial text data is detected, determining modification information corresponding to the initial text data; updating content of the label of the modification state corresponding to the each byte in the initial structured information based on the modification information to obtain target structured information; performing speech playing on the to-be-played voice segment according to the target structured information; wherein the updating of the content of the label of the modification state corresponding to the each byte in the initial structured information based on the modification information to obtain target structured information comprises: if the modification operation on the initial text data is a content addition operation, determining a first position corresponding to an added byte in the initial text data, and determining a first byte adjacent to the first position in the initial text data; updating the content of the label of the modification state corresponding to the each byte in the initial structured information according to the first position and the first byte to obtain the target structured information.

2. The method of claim 1, wherein, The updating of the content of the label of the modification state corresponding to the each byte in the initial structured information according to the first position and the first byte to obtain the target structured information comprises: adding a first structure to the initial structured information according to the first position, and setting a label of an initial state corresponding to the first structure to be empty; setting content of a label of modified content corresponding to the first structure according to the added byte, and setting content of a label of non-modified content in a modified state corresponding to the first structure according to the first byte to obtain the target structured information.

3. The method of claim 2, wherein, The setting of the content of the label of the non-modified content in the modified state corresponding to the first structure according to the first byte to obtain the target structured information comprises: obtaining a target time length according to content of a label of an initial start time corresponding to the first byte and content of a label of an initial end time corresponding to the first byte; setting content of a label of a modified start time corresponding to the first structure and content of a label of a modified end time corresponding to the first structure according to the target time length, and updating content of a label of a modified start time corresponding to the first byte and content of a label of a modified end time corresponding to the first byte; determining a second byte in the initial text data located in a position preceding the first position as the first byte; According to content of the tag of the initial playing order corresponding to the second byte, content of the tag of the modified playing order corresponding to the first structure and content of the tag of the modified playing order corresponding to the byte located after the second byte in the initial text data are set, and the target structured information is obtained.

4. The method of claim 1, wherein, The updating of the content of the tag of the modification state corresponding to each byte in the initial structured information based on the modification information to obtain the target structured information comprises: If the modification operation on the initial text data is a content replacement operation, a second position corresponding to a replacement byte in the initial text data is determined; According to the replacement byte, content of the tag of the modified content corresponding to the byte located at the second position in the initial text data is updated, and the target structured information is obtained.

5. The method of claim 1, wherein, The updating of the content of the tag of the modification state corresponding to each byte in the initial structured information based on the modification information to obtain the target structured information comprises: If the modification operation on the initial text data is a content deletion operation, a third position corresponding to a deletion byte in the initial text data is determined, and a third byte located at a position before the third position in the initial text data is determined; Content of the tag of the modified content corresponding to the byte located at the third position is set as empty, content of the tag of the modified start time corresponding to the byte located at the third position is set as empty, and content of the tag of the modified end time corresponding to the byte located at the third position is set as content of the tag of the initial end time corresponding to the third byte, and the target structured information is obtained.

6. The method of claim 1, wherein, The voice playing of the to-be-played voice segment according to the target structured information comprises: If it is detected that there is a fourth byte with empty tag of the modification state in the target structured information, the to-be-played voice segment is voice played according to the tag of the initial state corresponding to the fourth byte; or If it is detected that there is a fifth byte with non-empty tag of the modification state in the target structured information, the to-be-played voice segment is voice played according to the tag of the modification state corresponding to the fifth byte.

7. The method according to any one of claims 1 to 6, characterized in that, The to-be-played voice segment corresponds to at least one initial speaker, and the initial structured information is obtained based on the initial text data and the time information, comprising: Voiceprint recognition is performed on the to-be-played voice segment to obtain voice segments corresponding to the at least one initial speaker respectively and speaking times corresponding to the at least one initial speaker respectively; The initial structured information is obtained based on the voice segments corresponding to the at least one initial speaker respectively, the speaking times corresponding to the at least one initial speaker respectively, the initial text data and the time information, wherein the tag of the initial state corresponding to each byte in the initial structured information comprises a tag of an initial speaker corresponding to each byte.

8. The method of claim 7, wherein, The modification information includes information of a modified speaker, and the modification state includes the modified speaker. The updating of the label of the modification state corresponding to each byte in the initial structured information based on the modification information includes: Updating the label of the modification speaker corresponding to each byte in the initial structured information based on the information of the modified speaker, to obtain the target structured information.

9. A speech playback apparatus, characterized by comprising: The device includes: An automatic speech recognition module configured to perform speech recognition on a to-be-played voice segment, to obtain initial text data corresponding to the to-be-played voice segment and time information corresponding to the initial text data, wherein the time information includes a timestamp corresponding to each byte in the initial text data; An initial structured information obtaining module configured to obtain initial structured information based on the initial text data and the time information, wherein the initial structured information includes a label of an initial state corresponding to each byte and a label of a modification state being empty, the initial state includes an initial play order, initial content, an initial start time, and an initial end time, and the modification state includes a modified play order, modified content, a modified start time, and a modified end time; A modification information determining module configured to determine modification information corresponding to the initial text data if a modification operation on the initial text data is detected; An initial structured information updating module configured to update the label of the modification state corresponding to each byte in the initial structured information based on the modification information, to obtain target structured information; wherein the initial structured information updating module includes a content addition position determining unit and an addition operation updating unit; the content addition position determining unit is configured to determine a first position of an added byte in the initial text data and determine a first byte adjacent to the first position in the initial text data if the modification operation on the initial text data is a content addition operation; and the addition operation updating unit is configured to update the label of the modification state corresponding to each byte in the initial structured information based on the first position and the first byte, to obtain the target structured information; A voice play module configured to perform voice play on the to-be-played voice segment based on the target structured information. One or more processors; 10. An electronic device, comprising: Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the method of any one of claims 1-8. The computer-readable storage medium stores program code, and the program code can be invoked and executed by a processor to perform the method of any one of claims 1-8. ​ 11. A computer readable storage medium, characterized in that, ​

Images