Audio file processing method and electronic equipment

Abstract

The invention provides an audio file processing method and electronic equipment, and the method comprises the steps: generating a first storage path of an audio file through a first module of the electronic equipment, and transmitting the first storage path to a second module of the electronic equipment; the third module is used for carrying out speaker positioning on an audio file obtained based on the first storage path through the second module, generating an audio clip index and transmitting the audio clip index and the first storage path to the electronic equipment; the fourth module is used for segmenting an audio file acquired based on the first storage path by utilizing each audio clip index through the third module, generating a plurality of audio clips and corresponding second storage paths, and transmitting the second storage paths to the electronic equipment; and performing text conversion on each audio clip acquired based on the second storage path through a fourth module to generate text contents of different speakers and corresponding third storage paths. According to the embodiment of the invention, the risk of audio file processing failure can be effectively reduced.

Description

Technical Field

[0001] The present disclosure relates to but is not limited to the field of audio processing technology, and in particular to an audio file processing method and electronic device. Background Art

[0002] In the field of audio processing, audio files are a very common and important file format, which are widely used in different scenarios. For example, in scenarios such as meeting minutes, audio files are used to assist in recording meeting content.

[0003] In the process of processing an audio file into target data (for example, text), multiple processing modules are usually required. Different modules have upstream and downstream relationships. These processing modules generally use interfaces to directly interact with data. For example, the upstream processing module will transmit the audio-related data generated by its processing of the audio file to the downstream processing module through the interface for corresponding processing.

[0004] However, when the upstream processing module outputs audio-related data to the downstream processing module, instantaneous burst data traffic may be generated, which may cause excessive pressure on the downstream processing module, and may directly lead to the failure to successfully process the audio file into target data. Summary of the invention

[0005] Embodiments of the present disclosure provide an audio file processing method and an electronic device.

[0006] A first aspect of an embodiment of the present disclosure provides an audio file processing method, which is applied to an electronic device, and the method includes:

[0007] Generate a first storage path of the audio file by a first module of the electronic device, and transmit the first storage path to a second module of the electronic device;

[0008] Perform speaker location on the audio file obtained based on the first storage path through the second module, generate an audio segment index, and transmit the audio segment index and the first storage path to a third module of the electronic device;

[0009] The third module uses each of the audio segment indexes to segment the audio file obtained based on the first storage path to generate a plurality of audio segments and corresponding second storage paths, and transmits the second storage path to a fourth module of the electronic device;

[0010] The fourth module performs text conversion on each of the audio segments obtained based on the second storage path to generate text contents of different speakers and corresponding third storage paths.

[0011] In some embodiments, the method further comprises:

[0012] Determine multiple message topics in the message queue; wherein the multiple message topics include:

[0013] a first message topic, used by the first module to transmit the first storage path to the second module;

[0014] a second message topic, used by the second module to transmit the audio segment index and the first storage path to the third module;

[0015] A third message topic, used by the third module to transmit the second storage path to the fourth module;

[0016] A fourth message topic is used by the fourth module to transmit the third storage path to the first module.

[0017] In some embodiments, the method further comprises:

[0018] The audio file, each audio segment in the audio file, and the text content of the different speakers are stored based on a shared storage system.

[0019] In some embodiments, the second module performs speaker localization on the audio file obtained based on the first storage path to generate an audio segment index, including:

[0020] Performing jump point detection on the audio file by the second module to obtain timestamps corresponding to the audio segments to be segmented in the audio file;

[0021] The audio segment index is generated according to the timestamp corresponding to the audio segment to be segmented.

[0022] In some embodiments, the method further comprises:

[0023] The text contents of the different speakers are obtained based on the third storage path through the first module, and the text minutes corresponding to the audio file are obtained according to the text contents of the different speakers.

[0024] In some embodiments, before the step of obtaining the text minutes corresponding to the audio file, the method further includes:

[0025] Sending each of the audio segment indexes, the group identifier of the index group to which each of the audio segment indexes belongs, and the first storage path of the audio file to the fifth module of the electronic device through the second module; wherein different index groups correspond to different speakers;

[0026] After acquiring the audio file based on the first storage path, the fifth module performs voiceprint recognition on audio segments belonging to different speakers in the audio file according to the audio segment indexes and the group identifiers of the index groups to which the audio segment indexes belong, so as to determine the speaker identity information of each audio segment in the audio file;

[0027] The obtaining of the text minutes corresponding to the audio file according to the text contents of the different speakers includes:

[0028] After acquiring the speaker identity information of each audio segment in the audio file, the first module acquires the text minutes corresponding to the audio file according to the speaker identity information of each audio segment and the text contents of the different speakers.

[0029] In some embodiments, the method further comprises:

[0030] The fifth module stores the speaker identity information of each audio segment in the audio file in a relational database, wherein the relational database is used for the first module to query and obtain the speaker identity information of each audio segment in the audio file when acquiring the text content of the different speakers.

[0031] In some embodiments, obtaining the text minutes corresponding to the audio file according to the speaker identity information of each of the audio clips and the text content of the different speakers includes:

[0032] The first module calls the sixth module of the electronic device to generate a text summary corresponding to the audio file according to the speaker identity information of each audio segment and the text content of the different speakers.

[0033] In some embodiments, calling the sixth module of the electronic device to generate a text summary corresponding to the audio file according to the speaker identity information of each of the audio clips and the text content of the different speakers through the first module includes:

[0034] Extract keywords from the text contents of the different speakers through the sixth module to obtain summary contents corresponding to the different speakers;

[0035] A text summary corresponding to the audio file is generated according to the speaker identity information of each audio segment and the summary content corresponding to the different speakers.

[0036] A second aspect of an embodiment of the present disclosure provides an electronic device, the electronic device comprising a first module, a second module, a third module and a fourth module;

[0037] The first module is used to generate a first storage path of the audio file and transmit the first storage path to the second module;

[0038] The second module is used to locate the speaker of the audio file obtained based on the first storage path, generate an audio segment index, and transmit the audio segment index and the first storage path to the third module;

[0039] The third module is used to segment the audio file obtained based on the first storage path by using each of the audio segment indexes to generate multiple audio segments and corresponding second storage paths, and transmit the second storage path to the fourth module;

[0040] The fourth module is used to perform text conversion on each of the audio segments obtained based on the second storage path to generate text contents of different speakers and corresponding third storage paths.

[0041] A third aspect of the present disclosure provides an electronic device, including:

[0042] processor;

[0043] a memory for storing processor-executable instructions;

[0044] Wherein, the processor is configured to: implement any audio file processing method described in the first aspect when running the executable instructions.

[0045] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores an executable program, and when the executable program is executed by a processor, the audio file processing method described in any of the first aspects is implemented.

[0046] The technical solution provided by the embodiment of the present disclosure can process audio files into text contents of different speakers through the first module, the second module, the third module and the fourth module of the electronic device performing different processing on the audio files in sequence, thereby meeting the processing requirements for the audio files.

[0047] The first storage path of the audio file is transmitted to the second module through the first module, so that the second module can obtain the audio file based on the first storage path, reducing the possibility of instantaneous burst data traffic generated by the first module directly transmitting the audio file to the second module, and can alleviate the processing pressure of the second module.

[0048] The audio segment index generated by speaker localization of the audio file and the first storage path of the audio file are transmitted to the third module through the second module, so that the third module can obtain the audio file based on the first storage path, reducing the possibility of instantaneous burst data traffic generated by the second module directly transmitting the audio file to the third module, and can alleviate the processing pressure of the third module.

[0049] The third module uses the audio segment indexes to segment the audio file obtained based on the first storage path to generate multiple audio segments, and transmits the second storage path of the audio segment to the fourth module. In this way, the fourth module can obtain the audio segment based on the second storage path, thereby reducing the possibility of instantaneous burst data traffic generated by the third module directly transmitting the audio segment to the fourth module, and can alleviate the processing pressure of the fourth module.

[0050] Therefore, the technical solution provided by the embodiment of the present disclosure can effectively buffer the upstream and downstream instantaneous burst data traffic generated by multiple processing modules of the electronic device during the audio file processing process, reduce the risk of audio file processing failure due to excessive pressure on the downstream processing module, and improve the reliability of audio file processing. In addition, due to the loose coupling between different processing modules of the electronic device, unnecessary interactions between modules are reduced, and the development of the processing module is simplified to a certain extent.

[0051] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0053] Figure 1 The figure is a flowchart of a method for processing an audio file according to an exemplary embodiment.

[0054] Figure 2 The figure is a flowchart of a method for processing an audio file according to an exemplary embodiment.

[0055] Figure 3 The figure is a flowchart of a method for processing an audio file according to an exemplary embodiment.

[0056] Figure 4 is a flowchart of an audio file processing method according to an exemplary embodiment

[0057] Figure 5 The figure is a flowchart of a method for processing an audio file according to an exemplary embodiment.

[0058] Figure 6 The figure is a flowchart of a method for processing an audio file according to an exemplary embodiment.

[0059] Figure 7a The diagram is a schematic diagram of the architecture of an audio file processing system according to an exemplary embodiment.

[0060] Figure 7b The present invention is a schematic diagram of an interactive process of an audio text processing method according to an exemplary embodiment.

[0061] Figure 7c to Figure 7d The figure is a schematic diagram of a voiceprint recognition process according to an exemplary embodiment.

[0062] Figure 8 The figure is a schematic diagram showing the structure of an audio file processing device according to an exemplary embodiment.

[0063] Fig. 9 The diagram is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the embodiments of the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the embodiments of the present disclosure as detailed in the appended claims.

[0065] The terms used in the disclosed embodiments are only for the purpose of describing specific embodiments and are not intended to limit the disclosed embodiments. The singular forms of "a", "said" and "the" used in the disclosed embodiments and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0066] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the disclosed embodiments, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the disclosed embodiments, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0067] It can be understood that the description of various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0068] like Figure 1 As shown, the embodiment of the present disclosure provides an audio file processing method, which is applied to an electronic device, and the method includes the following steps:

[0069] 101: Generate a first storage path of an audio file through a first module of an electronic device, and transmit the first storage path to a second module of the electronic device;

[0070] 102: Perform speaker location on the audio file obtained based on the first storage path through the second module, generate an audio segment index, and transmit the audio segment index and the first storage path to a third module of the electronic device;

[0071] 103: using the third module to segment the audio file obtained based on the first storage path using the audio segment indexes to generate multiple audio segments and corresponding second storage paths, and transmitting the second storage paths to the fourth module of the electronic device;

[0072] 104: Perform text conversion on each audio segment obtained based on the second storage path through a fourth module to generate text contents of different speakers and corresponding third storage paths.

[0073] In this embodiment, the electronic device may be a device with file processing capabilities, for example, the electronic device may be a server, the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server. For another example, the electronic device may be a terminal, and the terminal may be a mobile terminal such as a smart phone or a tablet computer, or other non-portable computer devices.

[0074] In some examples, the electronic device includes multiple processing modules, and the multiple processing modules include at least a first module, a second module, a third module, and a fourth module. Here, the processing module can be in the form of software and / or hardware, and is used to implement a specific function. The processing module can also be called: a service module or a functional module.

[0075] Different processing modules of the electronic device may be used to perform different processing stages involved in the audio file processing process.

[0076] In some examples, the different processing stages involved in the audio file processing process may include: a file storage stage, a segment index generation stage, an audio segment segmentation stage, a voiceprint recognition stage, and a text conversion stage.

[0077] In some examples, the audio file processing process may also involve a textual summary generation stage.

[0078] For any processing stage among the multiple processing stages involved in the file processing process, if the processing stage is not the first processing stage in the file processing process, then the processing module executing the processing stage is the downstream processing module of the processing module executing the previous processing stage; if the processing stage is not the last processing stage in the file processing process, the processing module executing the processing stage is the upstream processing module of the processing module executing the next processing stage.

[0079] It should be noted that the downstream processing module of a processing module may be the upstream processing module of other processing modules; the upstream processing module of a processing module may be the downstream processing module of other processing modules.

[0080] In some examples, the audio file may be a file to be processed in any audio format. For example, the audio file may be a conference audio file, a dubbing audio file, or a music audio file.

[0081] In some examples, the electronic device is a server, and the audio file to be processed may be uploaded to the electronic device by a client. For example, the audio file may be included in an audio processing request sent by the client to the first module of the electronic device.

[0082] In some examples, the audio file to be processed may be an audio file pre-stored in the electronic device, or the audio file to be processed may be an audio file converted based on a video file, etc., but is not limited thereto.

[0083] In some examples, for any processing module of the electronic device, when the processing module fails to successfully execute the processing stage corresponding to the processing module, the processing module can re-execute the processing stage corresponding to the processing module.

[0084] In this way, it is possible to prevent the audio file from being processed into text content due to a processing module failing to successfully execute a processing stage corresponding to the processing module.

[0085] In this embodiment, the first storage path of the audio file can be used to determine the storage location of the audio file; the second storage path corresponding to the audio segment of the audio file can be used to determine the storage location of the audio segment; and the third storage path corresponding to the text content of different speakers can be used to determine the storage location of the text content of different speakers.

[0086] In some examples, the storage locations indicated by the first storage path of the audio file, the second storage path corresponding to the audio segment of the audio file, and the third storage path corresponding to the text content of different speakers may be located in the same storage system, for example, the storage system is the first storage system; or, the storage locations indicated by the above-mentioned different storage paths may also be located in different storage systems.

[0087] In some examples, in order to reduce the network transmission time during the file processing, the first storage system may be a storage system with a file sharing function, for example, the first storage system may be a ceph FS (ceph file system) or a NFS (Network File System). CephFS is a file system based on a Ceph cluster and compatible with the Portable Operating System Interface (POSIX) standard.

[0088] In this embodiment, the processing results generated by different processing modules of the electronic device during the audio file processing process can all be stored in the first storage system to reduce the amount of data directly transmitted between different processing modules.

[0089] In some examples, the first storage path of the audio file may be directly transmitted to the second module of the electronic device through the first module, or the first storage path of the audio file may be transmitted to the second module of the electronic device through a message queue through the first module.

[0090] In some examples, the audio segment index of the audio file and the first storage path of the audio file can be directly transmitted to the third module through the second module, or the audio segment index and the first storage path of the audio file can be transmitted to the third module through the message queue.

[0091] In some examples, the second storage paths corresponding to multiple audio segments generated by slicing the audio file can be directly transmitted to the fourth module through the third module, or the second storage paths corresponding to each audio segment can be transmitted to the fourth module via a message queue through the third module.

[0092] In some examples, in the above step 101, when the first module stores the received audio file in a shared storage system, it can generate a first storage path for the audio file and task information corresponding to the audio file, and transmit a first message to the second module through a message queue; wherein the first message includes: the first storage path for the audio file and task information.

[0093] Here, the task information corresponding to the audio file may include but is not limited to: task ID, user ID, task time information; the task time information may include task creation time, task end time, etc.

[0094] In some examples, the task information corresponding to the audio file may also include task execution progress and / or task priority.

[0095] Task priority can be used to determine the execution order of tasks. The higher the task priority, the earlier the task is executed; the lower the task priority, the later the task is executed.

[0096] Here, the task priority can be determined based on the task creation time of the audio file and / or the attribute information of the client sending the audio file. For example, the attribute information may include the IP address of the device to which the client belongs or the user account, etc.

[0097] In some examples, before the first module generates the first storage path of the audio file, it can also determine whether the format of the audio file is a target format, and if the format of the audio file is not the target format, convert the format of the audio file to a preset format. For example, the target format can be Moving Picture Experts Group Audio Layer III (MP3).

[0098] In some examples, in the above step 102, after generating the audio segment index, the second module transmits a second message to the third module through the message queue; wherein the second message includes: the audio segment index, the first storage path of the audio file, and task information.

[0099] In some examples, in the above step 103, after generating multiple audio clips and corresponding second storage paths, the third module transmits a third message to the fourth module through the message queue; wherein the third message includes: the second storage path of the audio clip and task information.

[0100] In some examples, in the above step 104, after generating the text contents of different speakers and the corresponding third storage paths, the fourth module stores the fourth message in the message queue; wherein the fourth message includes: the third storage paths corresponding to the text contents of different speakers.

[0101] In some examples, after obtaining the audio segment according to the second storage path of the audio segment, the fourth module can convert the audio segment corresponding to each audio segment index into corresponding text in parallel to obtain the text content of different speakers, and store the text content of different speakers to obtain the corresponding third storage path.

[0102] In some examples, converting the audio segments corresponding to the audio segment indexes into corresponding texts to generate text contents of different speakers may include:

[0103] Automatic Speech Recognition (ASR) technology is used to convert audio clips into corresponding texts, and the converted texts are post-processed to obtain the text contents of different speakers.

[0104] Exemplarily, post-processing of the converted text may include, but is not limited to, at least one of the following: digital conversion (for example, converting date, time, etc. into common expressions), punctuation (for example, modifying and correcting corresponding punctuation in the recognized text), and smooth detection (removing colloquialisms, interjections, etc. in the recognized text).

[0105] Here, textual content from different speakers can be included in the meeting transcript.

[0106] In some examples, the method may further include:

[0107] Different modules in the process of executing audio file processing can respectively output processing status information for the audio file; wherein, the processing status information corresponding to any module can be used to indicate whether the module has completed the execution of the corresponding processing stage of the module in the audio file processing process, and / or whether the corresponding processing stage is successfully executed.

[0108] In this way, the user can more intuitively know the processing status of different modules of the electronic device during the audio file processing process.

[0109] The embodiment of the present disclosure provides an audio file processing method, which is applied to an electronic device. The first module, the second module, the third module and the fourth module of the electronic device sequentially perform different processing on the audio file, so that the audio file can be processed into text content of different speakers, thereby meeting the processing requirements of the audio file.

[0110] The first storage path of the audio file is transmitted to the second module through the first module, so that the second module can obtain the audio file based on the first storage path, reducing the possibility of instantaneous burst data traffic generated by the first module directly transmitting the audio file to the second module, and can alleviate the processing pressure of the second module.

[0111] The audio segment index generated by speaker localization of the audio file and the first storage path of the audio file are transmitted to the third module through the second module, so that the third module can obtain the audio file based on the first storage path, reducing the possibility of instantaneous burst data traffic generated by the second module directly transmitting the audio file to the third module, and can alleviate the processing pressure of the third module.

[0112] The third module uses the audio segment indexes to segment the audio file obtained based on the first storage path to generate multiple audio segments, and transmits the second storage path of the audio segment to the fourth module. In this way, the fourth module can obtain the audio segment based on the second storage path, thereby reducing the possibility of instantaneous burst data traffic generated by the third module directly transmitting the audio segment to the fourth module, and can alleviate the processing pressure of the fourth module.

[0113] Therefore, the technical solution provided by the embodiment of the present disclosure can effectively buffer the upstream and downstream instantaneous burst data traffic generated by multiple processing modules of the electronic device during the audio file processing process, reduce the risk of file processing failure due to excessive pressure on the downstream processing module, and improve the reliability of audio file processing. In addition, due to the loose coupling between different processing modules of the electronic device, unnecessary interactions between modules are reduced, and the development of the processing module is simplified to a certain extent.

[0114] In some embodiments, the method may further include:

[0115] Determine multiple message topics in the message queue; wherein the multiple message topics include:

[0116] a first message topic, used by the first module to transmit the first storage path to the second module;

[0117] a second message topic, used by the second module to transmit the audio segment index and the first storage path to the third module;

[0118] A third message topic, used by the third module to transmit the second storage path to the fourth module;

[0119] A fourth message topic is used by the fourth module to transmit the third storage path to the first module.

[0120] In some examples, the message queue may be a distributed message queue, and the message queue may be a Kafka message queue.

[0121] The message queue includes multiple message topics, and different message topics are used to transmit processing messages output by different processing modules of the electronic device during the audio file processing process.

[0122] Each message topic in the message queue is an independent message stream. In two processing modules in an electronic device that are upstream and downstream of each other, the upstream processing module can publish messages to a specified message topic, and the downstream processing module can subscribe to messages from the specified message topic.

[0123] In some examples, the upstream processing module may specify a valid time period for the message when publishing the message to a specified message topic; if the message published by the upstream processing module is not processed by the downstream processing module within the specified valid time period, the message may be automatically deleted.

[0124] In some examples, multiple message topics in a message queue may be determined based on a message queue configuration parameter set.

[0125] The message queue configuration parameter set includes multiple different message queue configuration parameters, and each message queue configuration parameter includes producer information, consumer information, and message subject.

[0126] Here, the producer sends messages to the message topic, and the consumer subscribes to the message topic to consume messages.

[0127] For example, the first module sends a message to the first message topic as a producer, and the second module subscribes to the first message topic as a consumer to consume the message. For another example, the second module sends a message to the second message topic as a producer, and the third module subscribes to the second message topic as a consumer to consume the message.

[0128] In this embodiment, by determining different message topics in the message queue, the message topics can be used to transmit the processing messages generated by the upstream processing module to the downstream processing module, thereby reducing the coupling between different modules. While effectively buffering the upstream and downstream instantaneous burst data traffic in the audio file processing process, it can realize asynchronous processing of audio files and reduce the possibility of blocking of audio processing tasks.

[0129] In some embodiments, the method may further include:

[0130] The audio files, the individual audio segments in the audio files, and the text contents of different speakers are stored based on a shared storage system.

[0131] The shared storage system may be a Ceph file system (Ceph FS), which uses a Ceph storage cluster to store data.

[0132] The first storage path of the audio file is used to determine the storage location of the audio file in the shared storage system. The second storage path of the audio clip is used to determine the storage location of the audio file in the shared storage system. The third storage path of the text content of different speakers is used to determine the storage location of the text content of different speakers in the shared storage system.

[0133] In this embodiment, audio files, audio clips and text content are stored in a shared storage system, and different modules of the electronic device are used to transmit storage path information of audio files, audio clips or text content, thereby realizing data sharing between different modules and reducing the possibility of instantaneous burst data traffic between upstream and downstream modules.

[0134] In some embodiments, Figure 2 As shown, in the above step 102, the second module performs speaker location on the audio file obtained based on the first storage path to generate an audio segment index, which may include the following steps:

[0135] 201: Performing transition point detection on the audio file through the second module to obtain time stamps corresponding to the audio segments to be segmented in the audio file;

[0136] 202: Generate an audio segment index according to the timestamp corresponding to the audio segment to be segmented.

[0137] In some examples, the second module can subscribe to a message in a first message topic of a message queue to obtain a first storage path of an audio file. For example, after the first module publishes a first message to a first message topic of a message queue, the second module can read the first message from the first message topic. The first message includes the first storage path of the audio file.

[0138] The second module searches the shared storage system for the audio file according to the first storage path of the audio file in the first message.

[0139] The second module can perform jump point detection on the audio file, determine the timestamp corresponding to the audio segment to be segmented according to the jump point detection result, and generate an audio segment index according to the timestamp corresponding to the audio segment to be segmented.

[0140] For example, a jumping point detection model may be used to detect jumping points of an audio file to obtain the position of each jumping point in the audio file, and the timestamps of the audio segments to be segmented may be determined according to the positions of each jumping point in the audio file.

[0141] The timestamp of any audio segment may include: the start time and the end time of the audio segment in the audio file.

[0142] In some examples, the audio segment index to which the audio segment corresponds may include a timestamp of the audio segment.

[0143] In some examples, the second module may perform VAD (Voice Activity Detection) detection on the audio file to obtain an audio segment index corresponding to the audio segment.

[0144] In some examples, before performing transition point detection on the audio file, the second module may also filter out background noise in the audio file to reduce interference of the background noise on the transition point detection of the audio file.

[0145] In some examples, the second module can also transmit the speaker category and audio segment index corresponding to each audio segment to the fifth module of the electronic device, so that the fifth module can use the speaker category and audio segment index corresponding to each audio segment to perform voiceprint recognition on the audio files obtained based on the first storage path to obtain the speaker identity information corresponding to each audio segment.

[0146] In this embodiment, the second module performs jump point detection on the audio file to obtain the timestamps corresponding to the audio segments to be segmented in the audio file, and generates audio segment indexes based on the timestamps corresponding to the audio segments to be segmented. This facilitates the subsequent more accurate segmentation of audio segments corresponding to different speakers from the audio file based on the audio segment indexes.

[0147] In some embodiments, Figure 3 As shown, based on Figure 1 , the method may further comprise the following steps:

[0148] 301: Obtain text contents of different speakers based on the third storage path through the first module, and obtain text minutes corresponding to the audio files according to the text contents of different speakers.

[0149] In some examples, the text minutes may be, for example, meeting minutes, including: meeting summary content and speaker summary content corresponding to each speaker identity information. The meeting summary content may include the topics, themes, purposes, meeting decisions and / or results discussed at the meeting; the speaker summary content corresponding to each speaker identity information may include the speech content, opinions and / or suggestions of the speaker corresponding to each speaker identity information.

[0150] In some examples, after the first module obtains the text minutes, it can respond to the client's audio processing request and send the text minutes corresponding to the audio file to the client or send the storage path of the text minutes to the client. The audio processing request can be used to request the electronic device to process the audio file into a text minutes.

[0151] In this embodiment, by obtaining the text content of different speakers based on the third storage path through the first module, the possibility of instantaneous burst data traffic generated by the fourth module directly transmitting the text content of different speakers to the first module can be reduced, and the processing pressure of the first module can be alleviated; in addition, by obtaining the text minutes corresponding to the audio file based on the text content of different speakers through the first module, the automatic generation of the text minutes can be realized.

[0152] In some embodiments, Figure 4 As shown, based on Figure 3 , the method may further comprise the following steps:

[0153] 401: Sending each audio segment index, the group identifier of the index group to which each audio segment index belongs, and the first storage path of the audio file to a fifth module of the electronic device through the second module; wherein different index groups correspond to different speakers;

[0154] 402: After acquiring the audio file based on the first storage path, the fifth module performs voiceprint recognition on audio segments belonging to different speakers in the audio file according to the audio segment index and the group identifier of the index group to which the audio segment index belongs, and determines the speaker identity information of each audio segment in the audio file;

[0155] In the above step 301, obtaining the text minutes corresponding to the audio file according to the text contents of different speakers may include:

[0156] 403: After acquiring the speaker identity information of each audio segment in the audio file through the first module, the text minutes corresponding to the audio file are acquired according to the speaker identity information of each audio segment and the text contents of different speakers.

[0157] In some examples, in step 401 above, when generating the audio segment index, the second module may divide the multiple audio segment indexes into different index groups corresponding to different speakers, and obtain the group identifier of the index group to which each audio segment index belongs.

[0158] In some examples, dividing the plurality of audio segment indexes into different index groups corresponding to different speakers may include:

[0159] According to the similarity between the audio segments corresponding to the audio segment indexes, the speaker to which each audio segment belongs is determined by clustering. There may be one or more audio segments belonging to the same speaker.

[0160] In some examples, the similarity between the audio segments may include: the similarity of sound features between the audio segments.

[0161] For example, the second module may extract the embeddings (feature representations) corresponding to each audio segment, and determine the similarity of the sound features between the audio segments according to the distance between the feature representations corresponding to the audio segments. The distance may be, for example, Euclidean distance or Manhattan distance.

[0162] Here, the similarity between the audio segments belonging to the same speaker is less than a preset similarity threshold.

[0163] For example, an audio file may include five audio segments to be segmented in chronological order, namely audio segment 1 to audio segment 5, and different audio segments correspond to different audio segment indexes. Among them, audio segment 1 and audio segment 3 may belong to speaker a, audio segment 2 and audio segment 4 may belong to speaker b, and audio segment 5 may belong to speaker c.

[0164] In some examples, the second module may send each audio segment index, the group identifier of the index group to which each audio segment index belongs, and the first storage path of the audio file directly to the fifth module, or may send them to the fifth module through a message queue.

[0165] In some examples, the fifth module may use unsupervised learning (also known as unsupervised learning) to perform voiceprint recognition on the audio segments corresponding to each audio segment index in the audio file to obtain speaker identity information corresponding to the audio segment.

[0166] Unsupervised learning can automatically classify or group input audio data based on the commonalities in the audio data without labeling.

[0167] Here, the speaker identity information is used to uniquely identify the speaker. For example, the speaker identity information may include the speaker ID, name, etc.

[0168] In some examples, in step 402, performing voiceprint recognition on audio segments belonging to different speakers in the audio file according to the audio segment index and the group identifier of the index group to which the audio segment index belongs, and determining the speaker identity information of each audio segment in the audio file may include:

[0169] Extracting feature representations corresponding to multiple audio clips belonging to the same speaker according to the group identifier of the index group to which each audio clip index belongs;

[0170] The feature representation corresponding to each audio clip belonging to the same speaker is matched with each preset voiceprint feature, and each preset voiceprint feature is pre-bound with the corresponding speaker identity information;

[0171] According to the matching results between the feature representations corresponding to the audio clips belonging to the same speaker and the preset voiceprint features, the speaker identity information corresponding to the audio clips belonging to the same speaker is determined.

[0172] In some examples, extracting feature representations corresponding to multiple audio segments belonging to the same speaker according to the group identifier of the index group to which each audio segment index belongs may include:

[0173] According to the group identifier of the index group to which each audio segment index belongs, an audio segment of each speaker among the multiple speakers is determined in the audio file; and the following processing is performed on the audio segment of each speaker:

[0174] Determining whether the number of audio clips of the speaker is greater than a preset number threshold;

[0175] When the number of the audio segments of the speaker is greater than a preset number threshold, selecting a specified number of target audio segments from the multiple audio segments of the speaker, and extracting feature representations corresponding to the specified number of target audio segments; wherein the target audio segments are different from the first audio segment belonging to the speaker in the audio file, and are different from the last audio segment belonging to the speaker in the audio file;

[0176] When the number of the audio segments of the speaker is less than or equal to a preset number threshold, feature representations corresponding to all the audio segments of the speaker are extracted.

[0177] In this way, the amount of calculation for voiceprint recognition of audio clips can be reduced, thereby improving the efficiency of voiceprint recognition.

[0178] In some examples, feature matching may include calculating the similarity of features, and when the similarity between two features is less than or equal to a preset value, the two features match; when the similarity between two features is greater than a preset value, the two features do not match. The similarity between two features may be, for example, the cosine similarity of the features.

[0179] In some examples, determining the speaker identity information corresponding to the audio clips belonging to the same speaker according to the matching results between the feature representations corresponding to the audio clips belonging to the same speaker and the preset voiceprint features may include:

[0180] When the matching result indicates that there is no preset voiceprint feature and the similarity between the feature representation of the audio segment belonging to the same speaker is less than or equal to a preset value, outputting a prompt message, wherein the prompt message is used to indicate the audio segment for which the voiceprint recognition fails;

[0181] When the matching result indicates that the similarity between at least one preset voiceprint feature and the feature representation of the audio segment belonging to the same speaker is less than or equal to a preset value, at least one candidate speaker is determined to be matched; the identity information corresponding to different candidate speakers is pre-bound to different voiceprint features;

[0182] Determine speaker identity information corresponding to each audio clip belonging to the same speaker according to the number of matches corresponding to each candidate speaker;

[0183] For example, the identity information corresponding to the candidate speaker with the greatest number of matches may be determined as the speaker identity information corresponding to the audio segments belonging to the same speaker.

[0184] In this way, the speaker identity information corresponding to each audio segment can be accurately determined.

[0185] In some examples, the fifth module may send speaker identity information of each audio segment to the first module.

[0186] In this embodiment, the speaker identity information of each audio clip is determined by the fifth module, so that after the first module obtains the text content of different speakers, it can obtain the text minutes corresponding to the audio file based on the speaker identity information of each audio clip and the text content of different speakers, thereby making the accuracy and usability of the text minutes higher.

[0187] In some embodiments, the method may further include the following steps:

[0188] The fifth module stores the speaker identity information of each audio segment in the audio file in a relational database, wherein the relational database is used for the first module to query and obtain the speaker identity information of each audio segment in the audio file when acquiring text content of different speakers.

[0189] In some examples, the fifth module may store the speaker information corresponding to each audio segment in association with the task information of the audio file in a preset database. Here, the task information of the audio file may be sent from the first module to the second module, and then from the second module to the fifth module.

[0190] In some examples, the database may be a relational database, for example, the database may be a MySQL database.

[0191] In this embodiment, the fifth module stores the speaker identity information corresponding to each audio clip in a preset database, so that the downstream processing module of the fifth module can obtain the speaker information corresponding to each audio clip from the database for processing in subsequent stages, such as generating text minutes corresponding to the audio clip.

[0192] In some embodiments, Figure 5 As shown, based on Figure 4 In the above step 403, obtaining the text minutes corresponding to the audio file according to the speaker identity information of each audio clip and the text content of different speakers may include:

[0193] 501: The first module calls the sixth module of the electronic device to generate a text summary corresponding to the audio file according to the speaker identity information of each audio segment and the text content of different speakers.

[0194] In this embodiment, the first module can send a request message to the sixth module; the request message includes the speaker identity information of each audio clip and the text content of different speakers, so as to request the sixth module to generate a text minutes corresponding to the audio file based on the speaker identity information of each audio clip and the text content of different speakers.

[0195] In some embodiments, Figure 6 As shown, in the above step 501, the sixth module of the electronic device is called to generate a text summary corresponding to the audio file according to the speaker identity information of each audio segment and the text content of different speakers through the first module, which may include:

[0196] 601: extracting keywords from text contents of different speakers through the sixth module to obtain summary contents corresponding to different speakers;

[0197] 602: Generate a text summary corresponding to the audio file according to the speaker identity information of each audio segment and the summary contents corresponding to different speakers.

[0198] In some examples, the sixth module may use one or more keyword extraction algorithms of TF-IDF (Term Frequency_Inverse Document Frequency), TextRank, and word2vec to extract keywords.

[0199] In some examples, the sixth module can determine, for each speaker identity information, all text content corresponding to the speaker identity information in the text content of each speaker, perform keyword extraction on the determined text content, generate speaker summary content corresponding to the speaker identity information, and generate a text summary containing the summary content based on the speaker summary content of each speaker identity information.

[0200] After the sixth module generates the text minutes corresponding to the audio file, the text minutes corresponding to the audio file is returned to the first module, so that the first module returns the text minutes corresponding to the audio file to the client that sent the audio file.

[0201] It should be noted that those skilled in the art can understand that the method provided in the embodiments of the present disclosure can be executed alone or together with some methods in the embodiments of the present disclosure or some methods in related technologies.

[0202] In order to further explain any embodiments of the present disclosure, some specific embodiments are provided below.

[0203] For meeting minutes, its writing direction is relatively flexible, and it can be upward, downward, and parallel. Meeting minutes are different from meeting records. Meeting records are only an objective documentary material that records everyone's speech, while meeting minutes reflect the main agreed matters of the meeting in a concentrated and comprehensive manner, playing a role of specific guidance and regulation. A high-quality meeting minutes often requires a combination of the two, which can not only clearly grasp the core content of the meeting, but also understand the details corresponding to the time and content of each person's speech. Therefore, for corporate staff who compile meeting minutes, to output a high-quality meeting minutes, not only do they need to have a good ability to summarize and summarize, but they also need to be able to grasp the details of each meeting and compare the audio to understand the core content of the meeting. Therefore, the compilation of meeting minutes is very time-consuming and energy-consuming.

[0204] The embodiment of the present disclosure provides an audio file processing method. The method can be based on an automatic meeting minutes generation system in a cloud conference scenario, used to automatically generate meeting records for audio data collected from a meeting, and generate meeting minutes based on text records.

[0205] The audio files collected from the meeting are parsed offline and converted into audio with a specific sampling rate and frame rate. Combined with artificial intelligence processing technology, automatic audio-to-text conversion is achieved, and accurate speaker correspondence is achieved through voiceprint recognition technology. Then, combined with keyword extraction technology and meeting summary generation technology, accurate meeting records and preliminary meeting minutes are automatically generated, and sentence-by-sentence listening is supported, helping recorders to output high-quality meeting minutes, greatly improving their work efficiency.

[0206] The disclosed embodiments can be applied to the scenario of generating meeting minutes in cloud conferences. Based on the intelligent speech recognition system, through intelligent speech processing technology, the network audio data is connected with the speech recognition system to realize automatic generation of meeting minutes. While greatly improving the efficiency of the recorders in outputting the meeting minutes, it can buffer the instantaneous burst traffic of upstream and downstream based on the message queue and reduce unnecessary interactions between system modules.

[0207] Reference Figure 7a , is a schematic diagram of the architecture of an audio file processing system provided by an embodiment of the present disclosure, and the overall modules of the system may include:

[0208] 1. Data storage.

[0209] To store conference voice data and realize automatic meeting minutes generation in the form of tasks, cloud storage can be used for conference voice data storage, and MySQL can be used to store task related information, including but not limited to task ID, user ID, task creation time, end time, etc. In addition, considering the time-consuming network transmission of audio clips, shared storage CephFS is used to realize data storage.

[0210] 2. Message processing engine Kafka.

[0211] The main reasons for introducing the message processing engine Kafka are as follows: buffering the instantaneous burst traffic of upstream and downstream to make it smoother, especially for the upstream processing modules with strong sending capabilities. Without the protection of the message engine, the "fragile" downstream processing modules may be directly crushed, causing an "avalanche" of the entire link service; the loose coupling of the sender and the receiver also simplifies the application development to a certain extent and reduces unnecessary interactions between system modules.

[0212] 3. Audio data processing.

[0213] In order to ensure format compatibility and subsequent unified processing, audio files in different formats are parsed and content extracted, and then exported to generate audio files in a unified format.

[0214] 4. Speaker segmentation.

[0215] The final output is an accurate voiceprint result. Considering the accuracy of recognition, the audio of the entire meeting is first divided into short audio segments according to the jump points, and the corresponding audio segment Embedding is extracted. Finally, according to the similarity between the audio segments, the corresponding audio is clustered to generate multiple categories to represent multiple speakers.

[0216] 5. Voiceprint recognition.

[0217] 5.1. Unsupervised voiceprint mining.

[0218] Unsupervised learning is also called unsupervised learning. Given unlabeled training data, the input data is automatically classified or grouped according to the commonalities in the data. This paper provides a method for obtaining a voiceprint library based on unsupervised learning using the account information of the participants. This method avoids the process of manually entering voiceprints.

[0219] 5.2. Speaker identification.

[0220] This module is mainly based on the speaker clustering results described in step 4. According to the corresponding audio clips, it calculates the similarity with the sound clips in the voiceprint library described in step 5.1 to determine the final speaker.

[0221] 6. Convert speech to text.

[0222] A speech recognition system based on deep neural networks (for example, using Automatic Speech Recognition (ASR)) can be built to convert the corresponding speech fragments into text, and a speech post-processing system can be built, including smoothness detection, digital conversion, and punctuation processing, so as to ultimately present users with more readable text.

[0223] 7. Meeting minutes.

[0224] Based on the detailed meeting records, the keyword extraction technology is used to display the keywords of the entire meeting text, so that users can directly grasp the key points of the entire meeting. At the same time, according to the meeting records, the meeting minutes are extracted and generated to assist the recorder to complete the output of the meeting minutes.

[0225] like Figure 7b As shown, the interactive process of the audio text processing method provided in the embodiment of the present disclosure can be used to realize the generation of automatic meeting minutes. Based on the target client (for example, the conference client), the user initiates a meeting and turns on the meeting minutes function. After the meeting, the audio automatically calls the service interface s2t-api, and stores the audio file in the cloud storage system in an asynchronous manner, and creates a corresponding task record in the MySQL database. After that, the corresponding audio is transmitted to the speech recognition system, and finally the corresponding meeting record result is generated.

[0226] Kafka can act as a message queue. The finer-grained module in Kafka is called a Topic.

[0227] exist Figure 7b In the example, Kafka may include four topics, namely: s2t-speaker-in (speaker input topic, for example, the first message topic in the aforementioned embodiment), s2t-speaker-out (speaker output topic, for example, the second message topic in the aforementioned embodiment), s2t-asr-in (ASR input topic, for example, the third message topic in the aforementioned embodiment), s2t-asr-out (ASR output topic, for example, the fourth message topic in the aforementioned embodiment). For the overall data flow, please refer to Figure 7b Indicated by the arrow.

[0228] 1. The s2t-api (for example, the first module in the aforementioned embodiment) interface receives the audio file uploaded by the client, the server transcodes the audio and uploads it to the cloud and the shared file system cephfs for storage, then creates the corresponding task data information and writes it to the database. At this point, the task creation is completed, and the task message is written to the s2t-speaker-in topic for subsequent consumers to execute the task.

[0229] 2. s2t-speaker (speaker segmentation module; for example, the second module in the aforementioned embodiment) monitors s2t-speaker-in. After a message is written, it will perform speaker segmentation processing and generate audio segment index information corresponding to the segmented audio segment, and write the index information to the s2t-speaker-out topic for subsequent consumer execution.

[0230] 3. The s2t-split (audio segmentation module, for example, the third module in the aforementioned embodiment) module is used to monitor the s2t-speaker-out message end. After a message is written, it performs file segmentation processing and writes the file fragments to the shared file system cephfs. At the same time, it writes the corresponding file information into the s2t-asr-in topic for subsequent consumer execution.

[0231] 4. The s2t-asr (speech recognition module; for example, the fourth module in the aforementioned embodiment) module monitors the s2t-asr-in message module. After a message is written, it will perform the audio-to-text process according to the file index and the corresponding audio address and write the processed result into the s2t-asr-out message module for returning to the client.

[0232] 1) This system mainly stores conference voice data and realizes automatic conference minutes generation in the form of tasks. Therefore, cloud storage is used to store conference voice data, and MySQL is used to store related task Meta information. At the beginning of the meeting, the client turns on the meeting minutes function. After the meeting, the client actively uploads the conference audio file and calls the system interface. The server uploads the corresponding audio to the cloud storage for persistent backup, creates the corresponding task record and task status in MySQL, and stores the corresponding audio in the CephFS shared storage system for subsequent data reading. Finally, the s2t-api module stores the corresponding task in the s2t-speaker-in message queue in json format.

[0233] 2) The speaker segmentation module reads the corresponding task sequence from s2t-speaker-in and the task audio file from the CephFS sharing system, and performs VAD detection on the audio file (to identify speech segments and non-speech segments from a speech (clean or noisy) signal), obtains the corresponding index of the valid speech segment and the clustered speaker category number, sends the corresponding index to the voiceprint recognition module (s2t-voice, for example, the fifth module in the aforementioned embodiment), and stores the corresponding index in the s2t-speaker-out message queue.

[0234] 3) The execution process of the voiceprint recognition module can refer to Figure 7c to Figure 7d As shown ( Figure 7c The arrow A and Figure 7d The arrow A is the same arrow), the process may include the following steps:

[0235] a. After receiving the audio segment index from s2t-speaker, read the corresponding audio segment according to the index based on the original audio file.

[0236] b. If the number of audio clips in the same category (i.e., audio clips clustered as corresponding to the same speaker) is greater than a preset threshold (e.g., 10), then audio clips with a preset proportion (e.g., three quarters) of all audio clips are selected; otherwise, all audio clips are selected.

[0237] c. Calculate the embedding representation of the corresponding audio clip, query the user voiceprint stored in the MySQL database, and calculate the cosine similarity score with the sound features of the audio clip.

[0238] d. If the score is greater than the set threshold, the corresponding score and the matched candidate speaker are obtained; otherwise, the speaker is set to "None" and returned.

[0239] e. If there are multiple candidate speakers, determine whether the number of matches of the speaker with the highest number of matches is a preset multiple (for example, a multiple of 3 or 4) or more of the number of matches of the second candidate speaker with the highest number of matches, and whether the speaker with the highest number of matches is not "None" based on the number of matches corresponding to each candidate speaker. If so, the final speaker is the candidate with the highest number of matches. If not, the final speaker is "None".

[0240] f. If there is only one candidate speaker and the speaker is "None", the final speaker is "None", otherwise the final speaker is the corresponding candidate speaker.

[0241] 4) The s2t-split service is responsible for slicing the corresponding audio and writing the audio segments into the shared memory, and writing the segmented segments (indexes) and the corresponding speaker clustering information into the s2t-asr-in message topic.

[0242] 5) Automatic speech recognition (ASR) is a technology that converts human speech into text. The s2t-asr module obtains the audio segment index and related speaker clustering information from the message queue, and obtains and processes the actual audio segment from the shared storage, converting the audio into the corresponding text. After that, it goes through the post-processing module: digital conversion (converting date, time, etc. into common expressions), punctuation (modifying and correcting the corresponding punctuation in the recognized text) and smooth detection (removing colloquialisms, interjections, etc. in the recognized text) to obtain the final detailed meeting record.

[0243] 6) Generation of meeting minutes.

[0244] For the entire meeting text, an algorithm summary generation module (for example, the sixth module in the aforementioned embodiment) is used to extract meeting keywords from the results generated by asr using Text-Rank keyword extraction technology. At the same time, the algorithm summary generation module is used to obtain the entire meeting minutes to assist the recorder in completing the meeting minutes output.

[0245] The audio file processing method provided by the embodiment of the present disclosure can realize the transcription of audio and video content, quickly generate searchable, translatable, and locatable text, and efficiently obtain information; in addition, it can realize the intelligent extraction of key information and quickly generate structured notes, so that the key points are clear at a glance, the meeting is more efficient, and the efficiency of recorders in outputting meeting minutes is greatly improved.

[0246] like Figure 8 As shown, an embodiment of the present disclosure provides an electronic device, wherein the electronic device 100 includes a first module 110, a second module 120, a third module 130 and a fourth module 140;

[0247] The first module 110 is used to generate a first storage path of the audio file and transmit the first storage path to the second module;

[0248] The second module 120 is used to locate the speaker of the audio file obtained based on the first storage path, generate an audio segment index, and transmit the audio segment index and the first storage path to the third module;

[0249] The third module 130 is used to segment the audio file obtained based on the first storage path by using each of the audio segment indexes to generate multiple audio segments and corresponding second storage paths, and transmit the second storage path to the fourth module;

[0250] The fourth module 140 is configured to perform text conversion on each of the audio segments obtained based on the second storage path to generate text contents of different speakers and corresponding third storage paths.

[0251] In some embodiments, the electronic device 100 further includes:

[0252] A determination module is used to determine multiple message topics in the message queue; wherein the multiple message topics include:

[0253] a first message topic, used by the first module to transmit the first storage path to the second module;

[0254] a second message topic, used by the second module to transmit the audio segment index and the first storage path to the third module;

[0255] A third message topic, used by the third module to transmit the second storage path to the fourth module;

[0256] A fourth message topic is used by the fourth module to transmit the third storage path to the first module.

[0257] In some embodiments, the electronic device 100 further includes:

[0258] The storage module is used to store the audio file, each audio segment in the audio file and the text content of different speakers based on a shared storage system.

[0259] In some embodiments, the second module 120 is used to:

[0260] Performing jump point detection on the audio file to obtain time stamps corresponding to the audio segments to be segmented in the audio file;

[0261] The audio segment index is generated according to the timestamp corresponding to the audio segment to be segmented.

[0262] In some embodiments, the first module 110 is used to:

[0263] The text contents of the different speakers are obtained based on the third storage path, and the text minutes corresponding to the audio file are obtained according to the text contents of the different speakers.

[0264] In some embodiments, the electronic device 100 further includes a fifth module;

[0265] The second module 120 is further configured to send each of the audio segment indexes, the group identifier of the index group to which each of the audio segment indexes belongs, and the first storage path of the audio file to the fifth module of the electronic device; wherein different index groups correspond to different speakers;

[0266] The fifth module is configured to, after acquiring the audio file based on the first storage path, perform voiceprint recognition on audio segments belonging to different speakers in the audio file according to the audio segment indexes and the group identifiers of the index groups to which the audio segment indexes belong, and determine the speaker identity information of each audio segment in the audio file;

[0267] The first module 110 is further configured to obtain, after acquiring speaker identity information of each audio segment in the audio file, a text summary corresponding to the audio file according to the speaker identity information of each audio segment and the text content of the different speakers.

[0268] In some embodiments, the fifth module is further configured to:

[0269] The speaker identity information of each audio segment in the audio file is stored in a relational database, wherein the relational database is used for the first module to query and obtain the speaker identity information of each audio segment in the audio file when acquiring the text content of the different speakers.

[0270] In some embodiments, the electronic device 100 further includes a sixth module,

[0271] The first module 110 is used to call the sixth module of the electronic device to generate a text summary corresponding to the audio file according to the speaker identity information of each audio segment and the text content of the different speakers.

[0272] In some embodiments, the sixth module is used to:

[0273] Extract keywords from the text contents of the different speakers through the sixth module to obtain summary contents corresponding to the different speakers;

[0274] A text summary corresponding to the audio file is generated according to the speaker identity information of each audio segment and the summary content corresponding to the different speakers.

[0275] Regarding the electronic device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the relevant file processing method, and will not be elaborated here.

[0276] An embodiment of the present disclosure further provides an electronic device, including:

[0277] processor;

[0278] a memory for storing processor-executable instructions;

[0279] The processor is configured to implement the audio file processing method of any embodiment of the present disclosure when running the executable instructions.

[0280] The electronic device can at least be a server and / or a terminal. One or more servers can constitute a service end.

[0281] The memory may include various types of storage media, which are non-transitory computer storage media that can continue to remember information stored thereon after the communication device loses power.

[0282] The processor may be connected to the memory via a bus or the like, and may be used to read an executable program stored in the memory, for example, to implement the audio file processing method provided in any of the aforementioned embodiments.

[0283] like Fig. 9 FIG. 2 is a diagram showing a structure of an electronic device 200 according to an exemplary embodiment. For example, the electronic device 200 may be provided as a server. Fig. 9The electronic device 200 includes a processing component 210, which further includes one or more processors, and a memory resource represented by the processor 220, for storing instructions that can be executed by the processing component 210, such as an application. The application stored in the processor 220 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 210 is configured to execute instructions to perform the audio file processing method provided by any of the above embodiments.

[0284] The electronic device 200 may further include a power supply component 230 configured to perform power management of the electronic device 200, a wired or wireless network interface 240 configured to connect the electronic device 200 to a network, and an input / output (I / O) interface 250. The electronic device 200 may operate based on an operating system stored in the processor 220, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0285] The embodiments of the present disclosure further provide a computer-readable storage medium, which stores an executable program, wherein when the executable program is executed by a processor, the audio file processing method of any embodiment of the present disclosure is implemented. For example, the audio file processing method provided by any of the above embodiments is implemented.

[0286] An embodiment of the present disclosure provides a computer-readable storage medium, wherein the readable storage medium stores an executable program, wherein when the executable program is executed by a processor, the audio file processing method provided by any of the aforementioned technical solutions is implemented.

[0287] The computer-readable storage medium may specifically be: a non-temporary computer-readable storage medium.

[0288] Exemplarily, such as a memory including an executable program, the above instructions can be executed by a processing component of an electronic device to complete the above audio file processing method.

[0289] The non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0290] Those skilled in the art will readily appreciate other implementations of the disclosed embodiments after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosed embodiments, which follow the general principles of the disclosed embodiments and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the disclosed embodiments are indicated by the following claims.

[0291] It should be understood that the embodiments of the present disclosure are not limited to the precise structures described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the embodiments of the present disclosure is limited only by the appended claims.

Claims

1. A method for processing an audio file, characterized in that: Applied to electronic equipment, the method comprises: Generate a first storage path of the audio file by a first module of the electronic device, and transmit the first storage path to a second module of the electronic device; Perform speaker location on the audio file obtained based on the first storage path through the second module, generate an audio segment index, and transmit the audio segment index and the first storage path to a third module of the electronic device; The third module uses each of the audio segment indexes to segment the audio file obtained based on the first storage path to generate a plurality of audio segments and corresponding second storage paths, and transmits the second storage path to a fourth module of the electronic device; The fourth module performs text conversion on each of the audio segments obtained based on the second storage path to generate text contents of different speakers and corresponding third storage paths.

2. The method according to claim 1, characterized in that The method further comprises: Determine multiple message topics in the message queue; wherein the multiple message topics include: a first message topic, used by the first module to transmit the first storage path to the second module; a second message topic, used by the second module to transmit the audio segment index and the first storage path to the third module; A third message topic, used by the third module to transmit the second storage path to the fourth module; A fourth message topic is used by the fourth module to transmit the third storage path to the first module.

3. The method according to claim 1, characterized in that The method further comprises: The audio file, each audio segment in the audio file, and the text content of the different speakers are stored based on a shared storage system.

4. The method according to any one of claims 1 to 3, characterized in that: The second module performs speaker location on the audio file obtained based on the first storage path to generate an audio segment index, including: Performing jump point detection on the audio file by the second module to obtain timestamps corresponding to the audio segments to be segmented in the audio file; The audio segment index is generated according to the timestamp corresponding to the audio segment to be segmented.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The text contents of the different speakers are obtained based on the third storage path through the first module, and the text minutes corresponding to the audio file are obtained according to the text contents of the different speakers.

6. The method according to claim 5, characterized in that Before the step of obtaining the text minutes corresponding to the audio file, the method further includes: Sending each of the audio segment indexes, the group identifier of the index group to which each of the audio segment indexes belongs, and the first storage path of the audio file to the fifth module of the electronic device through the second module; wherein different index groups correspond to different speakers; After acquiring the audio file based on the first storage path, the fifth module performs voiceprint recognition on audio segments belonging to different speakers in the audio file according to the audio segment indexes and the group identifiers of the index groups to which the audio segment indexes belong, so as to determine the speaker identity information of each audio segment in the audio file; The obtaining of the text minutes corresponding to the audio file according to the text contents of the different speakers includes: After acquiring the speaker identity information of each audio segment in the audio file, the first module acquires the text minutes corresponding to the audio file according to the speaker identity information of each audio segment and the text contents of the different speakers.

7. The method according to claim 6, characterized in that The method further comprises: The fifth module stores the speaker identity information of each audio segment in the audio file in a relational database, wherein the relational database is used for the first module to query and obtain the speaker identity information of each audio segment in the audio file when acquiring the text content of the different speakers.

8. The method according to claim 6 or 7, characterized in that: The acquiring of the text minutes corresponding to the audio file according to the speaker identity information of each audio clip and the text contents of the different speakers includes: The first module calls the sixth module of the electronic device to generate a text summary corresponding to the audio file according to the speaker identity information of each audio segment and the text content of the different speakers.

9. The method according to claim 8, characterized in that The first module calls the sixth module of the electronic device to generate a text summary corresponding to the audio file according to the speaker identity information of each audio clip and the text content of the different speakers, including: Extract keywords from the text contents of the different speakers through the sixth module to obtain summary contents corresponding to the different speakers; A text summary corresponding to the audio file is generated according to the speaker identity information of each audio segment and the summary content corresponding to the different speakers.

10. An electronic device, characterized in that: The electronic device comprises a first module, a second module, a third module and a fourth module; The first module is used to generate a first storage path of the audio file and transmit the first storage path to the second module; The second module is used to locate the speaker of the audio file obtained based on the first storage path, generate an audio segment index, and transmit the audio segment index and the first storage path to the third module; The third module is used to segment the audio file obtained based on the first storage path by using each of the audio segment indexes to generate multiple audio segments and corresponding second storage paths, and transmit the second storage path to the fourth module; The fourth module is used to perform text conversion on each of the audio segments obtained based on the second storage path to generate text contents of different speakers and corresponding third storage paths.

Images