Audio processing method and device, electronic equipment and storage medium
By obtaining audio text data and using a large language model to divide paragraphs and outputting general ideas, the problem of searching in the minutes of the long meeting was solved and the efficiency of user review was improved.
Patent Information
- Application Number
- CN202411093763.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-31
AI Technical Summary
Long meeting minutes make it difficult for users to quickly find or reverse locate the details of the meeting content, resulting in tedious search work in lengthy meeting recordings or documents.
By obtaining the text data corresponding to the audio, using a pre-trained large language model, the target prompt word and text data are input to obtain the content division results, and the general idea of each paragraph is output to help users quickly locate the content of specific paragraphs.
It significantly improves the efficiency of users reviewing the content of the conference and quickly locates specific paragraph content in the target audio through the general program.
Smart Images
Figure CN119226502A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of audio processing technology, and specifically relates to an audio processing method, device, electronic device and storage medium. Background Art
[0002] With the rapid rise of big language models in the field of artificial intelligence, the conference software industry has also ushered in a disruptive innovation. Traditional meeting records are limited to automatic transcription of voice content, but now these software have evolved to the point where they can use advanced big language models to automatically organize meeting content into detailed meeting minutes.
[0003] However, as meetings get longer, meeting minutes can become very long, which becomes a challenge when users need to quickly find or reverse locate meeting details. The original meeting documents may be too long to directly locate key information, causing users to have to perform tedious searches in lengthy meeting recordings or documents when they need to review specific discussion points or decisive moments. Summary of the invention
[0004] In view of the above problems, the present application proposes an audio processing method, device, electronic device and storage medium to improve the above problems.
[0005] In a first aspect, an embodiment of the present application provides an audio processing method, the method comprising: obtaining text data corresponding to a target audio, the text data being data obtained by splicing the text content according to the index of each character in the text content corresponding to the target audio; inputting a target prompt word and the text data into a pre-trained large language model, and obtaining a content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea outline of the content of each paragraph.
[0006] In a second aspect, an embodiment of the present application provides an audio processing device, comprising: a data acquisition unit, used to acquire text data corresponding to a target audio, wherein the text data is data obtained by splicing the text content according to the index of each word in the text content corresponding to the target audio; a result output unit, used to input a target prompt word and the text data into a pre-trained large language model, and acquire a content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea outline of the content of each paragraph.
[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.
[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is run.
[0009] The embodiments of the present application provide an audio processing method, device, electronic device and storage medium. First, the text data corresponding to the target audio is obtained, and the text data is the data obtained by splicing the text content according to the index of each character in the text content corresponding to the target audio. Then, the target prompt word and the text data are input into a pre-trained large language model to obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea outline of each paragraph content. Through the above method, users can quickly locate the specific paragraph content in the target audio through the main idea outline, which significantly improves the user's review efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0011] Figure 1 A schematic diagram showing an application scenario of an audio processing method proposed in an embodiment of the present application;
[0012] Figure 2 A schematic diagram showing an application scenario of an audio processing method proposed in an embodiment of the present application;
[0013] Figure 3 A flowchart of an audio processing method proposed in an embodiment of the present application is shown;
[0014] Figure 4 A flowchart of an audio processing method proposed in another embodiment of the present application is shown;
[0015] Figure 5 A flowchart of an audio processing method proposed in another embodiment of the present application is shown;
[0016] Figure 6 A structural block diagram of an audio processing device proposed in an embodiment of the present application is shown;
[0017] Figure 7 A structural block diagram of an audio processing device proposed in an embodiment of the present application is shown;
[0018] Figure 8 A structural block diagram of an electronic device for executing an audio processing method according to an embodiment of the present application is shown;
[0019] Fig. 9 A storage unit in an embodiment of the present application is shown for storing or carrying program codes for implementing the audio processing method according to an embodiment of the present application. DETAILED DESCRIPTION
[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0021] With the rapid rise of big language models in the field of artificial intelligence, the conference software industry has also ushered in a disruptive innovation. Traditional meeting records are limited to automatic transcription of voice content, but now these software have evolved to the point where they can use advanced big language models to automatically organize meeting content into detailed meeting minutes.
[0022] However, as meetings get longer, meeting minutes can become very long, which becomes a challenge when users need to quickly find or reverse locate meeting details. The original meeting documents may be too long to directly locate key information, causing users to have to perform tedious searches in lengthy meeting recordings or documents when they need to review specific discussion points or decisive moments.
[0023] Therefore, the inventors proposed the audio processing method, device, electronic device and storage medium in the present application. First, the text data corresponding to the target audio is obtained, and the text data is the data obtained by splicing the text content according to the index of each character in the text content corresponding to the target audio, and then the target prompt word and the text data are input into a pre-trained large language model to obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea outline of each paragraph content. Through the above method, users can quickly locate the specific paragraph content in the target audio through the main idea outline, which significantly improves the user's review efficiency.
[0024] The following is an introduction to the application environment of the audio processing method provided by the present invention:
[0025] In the embodiment of the present application, the audio processing method provided can be executed by an electronic device. In this way, all steps in the audio processing method provided by the embodiment of the present application can be executed by the electronic device. For example, Figure 1 As shown, the processor of the electronic device 100 can obtain text data corresponding to the target audio, and the text data is data obtained by splicing the text content according to the index of each character in the text content corresponding to the target audio; the target prompt word and the text data are input into a pre-trained large language model to obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea of each paragraph content.
[0026] Furthermore, the audio processing method provided in the embodiment of the present application can also be executed by a server (cloud). Correspondingly, in this mode executed by the server, the target audio can be obtained by the electronic device, and the target audio can be synchronously sent to the server, and then the server obtains the text data corresponding to the target audio, and the text data is the data obtained by splicing the text content according to the index of each character in the text content corresponding to the target audio; the target prompt word and the text data are input into the pre-trained large language model, and the content division result corresponding to the target audio output by the large language model is obtained, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the general idea, and output the general idea of each paragraph content.
[0027] In addition, the electronic device and the server may also perform the steps in collaboration. In this manner, some steps of the audio processing method provided in the embodiment of the present application are performed by the electronic device, while other steps are performed by the server.
[0028] For example, Figure 2 As shown, the electronic device 100 can execute the audio processing method including: obtaining text data corresponding to the target audio, wherein the text data is data obtained by splicing the text content according to the index of each character in the text content corresponding to the target audio; and then the server 200 executes inputting the target prompt word and the text data into a pre-trained large language model, and obtaining the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea outline of each paragraph content.
[0029] It should be noted that in this method of collaborative execution by the electronic device and the server, the steps respectively executed by the electronic device and the server are not limited to the methods introduced in the above examples. In actual applications, the steps respectively executed by the electronic device and the server can be dynamically adjusted according to actual conditions.
[0030] It should be noted that the electronic device 100 can be Figure 1 and Figure 2 In addition to the smartphone shown in the figure, it can also be a car device, a wearable device, a tablet computer, a laptop computer, a smart speaker, etc. The server 120 can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers.
[0031] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0032] See also Figure 3 , an audio processing method provided in an embodiment of the present application is applied to Figure 1 or Figure 2 The electronic device or server shown, the method comprising:
[0033] Step S110: Acquire text data corresponding to the target audio, wherein the text data is data obtained by concatenating the text content corresponding to the target audio according to the index of each character in the text content.
[0034] In an embodiment of the present application, the target audio may be the acquired audio that needs to be voice recognized. The target audio may be audio data collected by an audio acquisition device. The text content corresponding to the target audio is the text content obtained after voice recognition of the target audio, and the text content may include all characters corresponding to the target audio, where the characters may include text, punctuation marks, etc. The index of each character refers to the order in which each character appears in the target audio. For example, if the text content obtained after voice recognition of the target audio is "Hello!", then the index of "you" is 1, indicating that the character appears first in the target audio; the index of "good" is 2, indicating that the character appears second in the target audio; the index of "!" is 3, indicating that the character appears third in the target audio.
[0035] According to the index of each character in the text content corresponding to the target audio, the data obtained after splicing the text content can be understood as text data with or without line breaks obtained by splicing all recognized characters in the order in which each character appears in the target audio. Among them, when the text data is text data without line breaks, the text data can be a whole paragraph of text.
[0036] In the embodiment of the present application, the audio acquisition device can be a microphone, a recording device or other device that can collect voice data, which is not specifically limited here. The target audio collected in different application scenarios is different. For example, in a meeting scenario, the target audio collected by the audio acquisition device can be conference conversation voice data, meeting minutes, etc.; in a shopping mall scenario, the target audio collected by the audio acquisition device can be a conversation voice data between a salesperson and a customer; for example, in a class scenario, the target audio collected by the audio acquisition device can be a question-and-answer conversation voice data between a student and a teacher, etc. Optionally, in different application scenarios, the target audio can be voice data of different users. For example, in an interview process, the target audio can be the voice data of the interviewer, or the voice data of the interviewee; for example, in a video watching scenario, the target audio can be the voice data of the user watching the video, or the voice data of the user playing the video, or the voice data of the character in the video.
[0037] As a method, in response to a data acquisition instruction, the target audio is collected through an audio collection device. The data acquisition instruction is an instruction for instructing an electronic device to collect voice data. The data acquisition instruction may be an instruction sent by other devices that have established a communication connection with the electronic device, wherein the other devices may be tablet computers, smart wearable devices, mobile phones, computers, and smart home devices, etc., which are not specifically limited here; or, the data acquisition instruction may also be an instruction triggered by a specified operation on the electronic device, wherein the specified operation on the electronic device may be an operation to start a voice data collection function, for example, the specified operation may be a click operation or a sliding operation on a specified control (for example, a control that controls the voice data collection to be turned on / off).
[0038] As another method, the target audio may be voice data sent by a voice acquisition device, wherein the voice acquisition device is a smart device that establishes a wireless communication connection with the electronic device. Specifically, after the voice acquisition device acquires the target audio, it may send the acquired target audio to the electronic device so that the electronic device can perform voice recognition on the target audio.
[0039] Specifically, when the target audio is collected by the voice collection device, the voice data of different users in different scenes can be collected by the voice collection device as the target audio. Optionally, when the voice data of different users in different scenes is collected by the voice collection device, if the voice data of multiple users is needed, in order to distinguish the voice data of different users, the voice data of different users can be collected by different voice collection devices. Exemplarily, in order to make the voice data of the interviewer and the interviewee as the target audio in the interview process, the voice recognition can be performed separately in the future, so as to avoid the accuracy of the voice recognition of the interviewer and the interviewee and the speed of the voice recognition in a complex environment. The first microphone and the second microphone set on the voice collection device can be used to collect the voice data of the interviewer and the interviewee. Specifically, during the interview process, the interviewer and the interviewee are usually not the same person. In order to make the first microphone set on the voice collection device collect the voice data of the interviewer and the second microphone set on the voice collection device collect the voice data of the interviewee, the first microphone and the second microphone set on the voice collection device can be pointed to different directions respectively. Further, the first microphone on the voice collection device can be set to point to the interviewer, and the second microphone on the voice collection device can be set to point to the interviewee. Furthermore, the voice data of different users can be collected by the first microphone and the second microphone of the voice collection device, and then the voice collection device can send the collected voice data of the interviewer and the voice data of the interviewee to the electronic device as the target audio.
[0040] When the voice collection device sends the collected voice data of the interviewer and the voice data of the interviewee to the electronic device, the electronic device can receive the voice data of the interviewer collected by the first microphone of the voice collection device through the first channel of the multi-channel signal receiver, and receive the voice data of the interviewee collected by the second microphone of the voice collection device through the second channel of the multi-channel signal receiver, so that the electronic device can recognize the voice data of the interviewer and the interviewee respectively.
[0041] As another way, the target audio may also be voice data obtained by the electronic device from the cloud server after receiving the voice recognition instruction, which is not specifically limited here.
[0042] It should be noted that the electronic device is a voice recognition terminal. In the embodiment of the present application, the voice recognition terminal can be: a mobile phone, a personal computer, a tablet computer and other terminals, and of course it can also be a server. Specifically, the application does not limit what kind of device the voice recognition terminal is, as long as the voice recognition terminal can perform voice recognition on the target audio received by each channel in the multi-channel signal receiver.
[0043] It is known that speech recognition is achieved through a speech recognition engine (or system) trained online or offline. The speech recognition process can generally be divided into a training phase and a recognition phase. In the training phase, the acoustic model (AM) and the vocabulary (lexicon) are statistically obtained from the training data according to the mathematical model on which the speech recognition engine (or system) is based. In the recognition phase, the speech recognition engine (or system) uses the acoustic model and vocabulary to process the input speech and obtain the speech recognition result, that is, the text content corresponding to the target audio.
[0044] In an embodiment of the present application, the text data corresponding to the target audio can be pre-stored in a preset storage area. When the text data corresponding to the target audio needs to be obtained, the text data corresponding to the target audio can be directly obtained from the preset storage area. Optionally, a correspondence between audio and text data can be established in the preset storage area. When the text data corresponding to the target audio needs to be obtained, the text data corresponding to the target audio can be found in the preset storage area according to the aforementioned correspondence.
[0045] Step S120: Input the target prompt word and the text data into a pre-trained large language model, and obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea of each paragraph content.
[0046] In an embodiment of the present application, the target prompt word is a pre-set prompt word (prompt) for instructing the large language model to divide the input text data into paragraphs according to the main idea, and output the main idea outline of each paragraph content. The target prompt word can, for example: keep the original text content of the input text data unchanged, treat the content of the same main idea together to form a paragraph, separate paragraphs with different main ideas with line breaks, and output the main idea outline of each paragraph. The pre-trained large language model can perform content extraction and paragraph division on the text data according to the target prompt word, wherein content extraction can be understood as summarizing the content of the text data and extracting the main idea outline. For example, when the target audio is a conference audio, the text data is content extracted to extract the conference outline. When the large language model extracts the content of the text data, it can create a detailed main idea outline for each text data by identifying keywords, key sentences, and turning points in the text data.
[0047] After obtaining the text data corresponding to the target audio, the text data corresponding to the target audio and the target prompt word can be input into a pre-trained large language model together, so that the large language model can divide the text data corresponding to the target audio into paragraphs according to the requirements of the target prompt word while ensuring that the content of the original sentence remains unchanged, and output the main idea of each paragraph, that is, output the content division result.
[0048] The content division result includes a plurality of paragraph contents whose original text contents are separated by line breaks and remain unchanged, and a main idea outline corresponding to each of the plurality of paragraph contents; the main idea outline is output in a bold form or in a title form.
[0049] In an embodiment of the present application, when the large language model divides the text data corresponding to the input target audio into paragraphs according to the target prompt word, sentences with the same general meaning in the text data will be divided into one paragraph to obtain multiple paragraph contents, and the content of the multiple paragraphs can be refined to output the main idea of each paragraph content.
[0050] Multiple paragraph contents can be separated by line breaks, and the large language model can output the main idea of each paragraph content in bold, or in the form of a title, or in the form of a bold title. The main idea of each paragraph content can be displayed before the corresponding paragraph content, or after the corresponding paragraph content, or on the side of the corresponding paragraph content, which is not specifically limited here.
[0051] Optionally, when the large language model outputs the content division result, it can output the multiple paragraph contents in the order of their occurrence in the text data, so that the sequence numbers of the multiple paragraph contents and the sequence numbers of the multiple main idea outlines can be determined according to the output sequence of the multiple paragraph contents. The main idea outlines and paragraph contents are in one-to-one correspondence.
[0052] After obtaining the serial numbers of multiple paragraph contents and multiple main idea outlines, the specific paragraph contents or the specific main idea outline can be located by the serial numbers of the paragraph contents or the serial numbers of the main idea outline.
[0053] The present application provides an audio processing method, which first obtains the text data corresponding to the target audio, wherein the text data is the data obtained by splicing the text content according to the index of each character in the text content corresponding to the target audio, and then inputs the target prompt word and the text data into a pre-trained large language model to obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea outline of each paragraph content. Through the above method, users can quickly locate the specific paragraph content in the target audio through the main idea outline, which significantly improves the user's review efficiency.
[0054] See also Figure 4 , an audio processing method provided in an embodiment of the present application is applied to Figure 1 or Figure 2 The electronic device or server shown, the method comprising:
[0055] Step S210: Acquire target audio.
[0056] Step S220: Perform speech recognition on the target audio to determine the text content corresponding to the target audio and the structured information of each character in the text content, wherein the structured information at least includes an index and a start and end time.
[0057] In an embodiment of the present application, the structured information of each character may include at least the index of each character and the start and end time of each character. The index of each character refers to the order in which each character appears in the target audio, which can be represented by 1, 2, 3, ..., n; the start and end time of each character includes the start time of each character and the end time of each character. The start time refers to the time when each character starts to appear in the target audio; the end time refers to the time when each character ends to appear in the target audio. Of course, the structured information of each character may also include the speaker information and status (here divided into None and Modified) of each character, etc., which are not specifically limited here. Exemplarily, the structured information of each character can be recorded in the following form:
[0058] index:1
[0059] text: old
[0060] textEdit: Old
[0061] startTime:0:00
[0062] endTime:0:02
[0063] speaker:spk1
[0064] speakerEdit:spk2
[0065] status:None
[0066] In the above example, index represents the index of the character; text represents the character; startTime represents the start time of the character; endTime represents the end time of the character; speaker represents the speaker to whom the character belongs; status represents the status of the character, None means that the character has not been modified, and Modified means that the character has been modified.
[0067] The speech recognition of the target audio can be performed in real time or offline, which is not specifically limited here. Offline means that the speech recognition is performed after the complete target audio is acquired; real time means that the speech recognition is started when the target audio is collected.
[0068] After performing speech recognition on the target audio, the speech recognition content at the word level can be formatted and recorded, that is, the structured information of each character can be obtained.
[0069] Step S230: Based on the structured information of each character in the text content, the text content is spliced to obtain text data corresponding to the target audio.
[0070] In the embodiment of the present application, after obtaining the structured information of each character, all characters corresponding to the target audio can be spliced according to the structured information of each character to obtain the text data corresponding to the target audio. The splicing process refers to arranging the characters in index order.
[0071] As one method, multiple sentences corresponding to the text content are obtained; based on the index of each character in the text content, an index form corresponding to the text content is created, wherein the index form includes the serial numbers of the multiple sentences and the index of each character included in each sentence; according to the index form, the text content is spliced to obtain text data corresponding to the target audio.
[0072] The sequence numbers of the sentences can be represented by S1, S2, ..., Sn, where Sn represents the nth sentence. For example, the index form can be:
[0073] S1: 1 2 3 4 5
[0074] S2: 6 7 8 ...
[0076] S10: 16 17 18 19
[0077] In the above example, S1 represents the first sentence, and similarly S2 represents the second sentence; the digital number assigned after each sentence number is the index of each character in the structured information of each character obtained in step S220.
[0078] The obtaining of the plurality of sentences corresponding to the text content includes: segmenting the text content based on terminal punctuation marks to obtain the plurality of sentences corresponding to the text content.
[0079] In the embodiment of the present application, the sentence segmentation principle is based on the terminal punctuation mark, that is, from left to right, if it is not a terminal punctuation mark, the index number of the character will continue to be added in the current sentence, otherwise, a new sentence number will be created and the index number of the character in the sentence will be recorded. Among them, the terminal punctuation mark can include ".", "?" and "!", etc., which are not specifically limited here.
[0080] In the embodiment of the present application, the index form can be used as a primary index, that is, the corresponding sentences and characters can be located through the index of sentence numbers and characters.
[0081] Step S240: Input the target prompt word and the text data into a pre-trained large language model, and obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea of each paragraph content.
[0082] In the embodiment of the present application, multiple paragraph contents and the outline of each paragraph content can be recorded in the following form: the sequence number of the outline: the sequence number of the paragraph content, for example, Title 1: Paragraph 1, Title 2: Paragraph 2. In the above example, Title 1 can be understood as the outline of the first paragraph content, and Paragraph 1 can be understood as the first paragraph content; Title 2 can be understood as the outline of the second paragraph content, and Paragraph 2 can be understood as the second paragraph content.
[0083] Optionally, the content of the above record can be used as a secondary index, that is, the corresponding paragraph content can be located according to the serial number of the main idea of the record and the serial number of the paragraph content.
[0084] Step S250: Obtain the character string length and start and end characters corresponding to each paragraph content.
[0085] In the embodiment of the present application, the character string length corresponding to each paragraph content can be understood as the number of characters included in each paragraph; the start and end characters of each paragraph can be understood as the beginning and end characters of each paragraph content.
[0086] After the large language model divides the text data corresponding to the target audio into multiple paragraph contents, the number of characters and the start and end characters included in each paragraph content can be counted respectively, that is, the character string length and the start and end characters of each paragraph content can be obtained.
[0087] Step S260: Based on the length of the character string and the start and end times of the start and end characters, determine the time information of each paragraph content in the target audio.
[0088] In an embodiment of the present application, after obtaining the string length and start and end characters of each paragraph content, the start and end times in the corresponding structured information can be located through the respective indexes of the start and end characters, that is, the start and end times of the starting characters and the start and end times of the ending characters can be determined.
[0089] Then, the start and end time of each paragraph content can be further determined according to the length of the string and the start and end time of the start and end characters, that is, the time information of each paragraph content in the target audio can be determined. Through the above method, the purpose of direction search is achieved.
[0090] The present application provides an audio processing method that can help users quickly locate important moments while maintaining a comprehensive understanding of text data.
[0091] See also Figure 5 , an audio processing method provided in an embodiment of the present application is applied to Figure 1 or Figure 2 The electronic device or server shown, the method comprising:
[0092] Step S310: Acquire text data corresponding to the target audio, wherein the text data is data obtained by concatenating the text content according to the index of each character in the text content corresponding to the target audio.
[0093] Step S320: Input the target prompt word and the text data into a pre-trained large language model, and obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea of each paragraph content.
[0094] Step S330: Obtain the character string length and start and end characters corresponding to each paragraph content.
[0095] Step S340: Based on the length of the character string and the start and end times of the start and end characters, determine the time information of each paragraph content in the target audio.
[0096] Step S350: Obtain the target paragraph content corresponding to the target main idea from the content division result.
[0097] In the embodiment of the present application, the target main idea outline may be the main idea outline of the paragraph content that currently needs to be located, and the target main idea outline may be determined by the sequence number of the main idea outline.
[0098] After obtaining the primary index and secondary index corresponding to the target audio through the above-mentioned embodiment, different levels of content can be located through indexes of different basic levels. For example, a sentence or a character in a sentence in the target audio can be located through the primary index; a general idea or a paragraph content in the target audio can be located through the secondary index.
[0099] Step S360: based on the time information of the target paragraph content, play the audio corresponding to the paragraph content.
[0100] In the embodiment of the present application, after the target paragraph content is acquired, the time information of the target paragraph content can be determined in step S340, so that the audio corresponding to the target paragraph content can be located. The audio collected within the period is played according to the time information of the target paragraph content.
[0101] The audio processing method provided by the present application can not only help users quickly locate specific information in text data, but also provide a direct link to the original audio, so that users can quickly go back and view detailed audio data.
[0102] See also Figure 6 The present application embodiment provides an audio processing device 400, which operates as follows Figure 1 or Figure 2 The electronic device or server shown, the device 400 includes:
[0103] The data acquisition unit 410 is used to acquire text data corresponding to the target audio, where the text data is obtained by concatenating the text content corresponding to the target audio according to the index of each word in the text content.
[0104] As a method, the data acquisition unit 410 is specifically used to acquire target audio; perform speech recognition on the target audio to determine the text content corresponding to the target audio and the structured information of each character in the text content, wherein the structured information includes at least an index and a start and end time; based on the structured information of each character in the text content, the text content is spliced to obtain text data corresponding to the target audio.
[0105] Furthermore, the data acquisition unit 410 is specifically used to acquire multiple sentences corresponding to the text content; based on the index of each character in the text content, create an index form corresponding to the text content, wherein the index form includes the serial numbers of the multiple sentences and the index of each character included in each sentence; according to the index form, the text content is spliced to obtain text data corresponding to the target audio.
[0106] Optionally, the data acquisition unit 410 is further configured to segment the text content based on terminal punctuation marks to obtain a plurality of sentences corresponding to the text content.
[0107] The result output unit 420 is used to input the target prompt word and the text data into a pre-trained large language model to obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea and output the main idea of each paragraph content.
[0108] The content division result includes a plurality of paragraph contents whose original text contents are separated by line breaks and remain unchanged, and a main idea outline corresponding to each of the plurality of paragraph contents; the main idea outline is output in a bold form or in a title form.
[0109] See also Figure 7 , the device 400 further includes:
[0110] The information determination unit 430 is used to obtain the string length and start and end characters corresponding to each paragraph content; based on the string length and the start and end time of the start and end characters, determine the time information of each paragraph content in the target audio.
[0111] The audio playing unit 440 is used to obtain the target paragraph content corresponding to the target main idea from the content division result; and play the audio corresponding to the paragraph content based on the time information of the target paragraph content.
[0112] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.
[0113] The following will be combined Figure 8 An electronic device or server provided in the present application is described.
[0114] See also Figure 8 Based on the above audio processing method and device, the embodiment of the present application also provides another electronic device or server 800 that can execute the above audio processing method. The electronic device or server 800 includes one or more (only one is shown in the figure) processors 802, a memory 804 and a network module 806 that are coupled to each other. Among them, the memory 804 stores a program that can execute the content of the above embodiment, and the processor 802 can execute the program stored in the memory 804.
[0115] Among them, the processor 802 may include one or more processing cores. The processor 802 uses various interfaces and lines to connect the various parts of the entire electronic device or server 800, and executes various functions and processes data of the electronic device 800 by running or executing instructions, programs, code sets or instruction sets stored in the memory 804, and calling data stored in the memory 804. Optionally, the processor 802 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 802 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 802, but may be implemented separately through a communication chip.
[0116] The memory 804 may include a random access memory (RAM) or a read-only memory (ROM). The memory 804 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 804 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device or server 800 during use.
[0117] The network module 806 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, such as communicating with an audio playback device. The network module 806 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM) cards, memories, etc. The network module 806 may communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices via wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. For example, the network module 806 may exchange information with a base station.
[0118] Please refer to Fig. 9 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 900 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.
[0119] The computer-readable storage medium 900 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 900 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 900 has storage space for program code 910 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 910 can be compressed, for example, in an appropriate form.
[0120] The present application provides an audio processing method, device, electronic device and storage medium, which first obtains the text data corresponding to the target audio, wherein the text data is the data obtained by splicing the text content according to the index of each character in the text content corresponding to the target audio, and then inputs the target prompt word and the text data into a pre-trained large language model to obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea outline of each paragraph content. Through the above method, users can quickly locate the specific paragraph content in the target audio through the main idea outline, which significantly improves the user's review efficiency.
[0121] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.
Claims
1. An audio processing method, characterized in that: The method comprises: Acquire text data corresponding to the target audio, wherein the text data is data obtained by concatenating the text content corresponding to the target audio according to the index of each character in the text content; The target prompt word and the text data are input into a pre-trained large language model to obtain a content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea and output the main idea of each paragraph content.
2. The method according to claim 1, characterized in that The step of obtaining text data corresponding to the target audio includes: Get the target audio; Performing speech recognition on the target audio to determine text content corresponding to the target audio and structured information of each character in the text content, wherein the structured information includes at least an index and a start and end time; Based on the structured information of each character in the text content, the text content is spliced to obtain text data corresponding to the target audio.
3. The method according to claim 2, characterized in that The step of performing splicing processing on the text content based on the structured information of each character in the text content to obtain text data corresponding to the target audio includes: Obtaining multiple sentences corresponding to the text content; Based on the index of each character in the text content, create an index table corresponding to the text content, wherein the index table includes the serial numbers of the plurality of sentences and the index of each character included in each sentence; According to the index form, the text content is spliced to obtain text data corresponding to the target audio.
4. The method according to claim 3, characterized in that The obtaining of a plurality of sentences corresponding to the text content includes: The text content is segmented based on terminal punctuation marks to obtain a plurality of sentences corresponding to the text content.
5. The method according to claim 2, characterized in that: After inputting the target prompt word and the text data into a pre-trained large language model and obtaining the content division result corresponding to the target audio output by the large language model, the method further includes: Get the string length and start and end characters corresponding to each paragraph content; Based on the length of the character string and the start and end times of the start and end characters, the time information of each paragraph content in the target audio is determined.
6. The method according to claim 5, characterized in that The method further comprises: From the content division results, obtaining the target paragraph content corresponding to the target main idea outline; Based on the time information of the target paragraph content, the audio corresponding to the paragraph content is played.
7. The method according to claim 1, characterized in that The content division result includes a plurality of paragraph contents whose original text contents are separated by line breaks and remain unchanged, and a main idea outline corresponding to each of the plurality of paragraph contents; the main idea outline is output in a bold form or in a title form.
8. An audio processing device, characterized in that: The device comprises: A data acquisition unit, used to acquire text data corresponding to the target audio, wherein the text data is data obtained by splicing the text content according to the index of each word in the text content corresponding to the target audio; The result output unit is used to input the target prompt word and the text data into a pre-trained large language model, and obtain the content division result corresponding to the target audio output by the large language model, wherein the target prompt word is used to instruct the large language model to divide the text data into paragraphs according to the main idea, and output the main idea of each paragraph content.
9. An electronic device, characterized in that: The method comprises one or more processors and a memory; one or more programs are stored in the memory and are configured so that the one or more processors execute the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 7 is executed.