Data processing method and apparatus

Abstract

This application provides a data processing method and apparatus. The data processing method includes: receiving a speech conversion request; sending an audio stream to a speech conversion platform in response to the speech conversion request; receiving at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier; and displaying each audio field based on a preset segmentation rule and the field type identifier of each audio field. This application's data processing method enables segmented display of audio fields based on preset segmentation rules, improving the clarity of audio field display and thus enhancing the user's processing efficiency for audio fields.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method. This application also relates to a data processing apparatus, a computing device, and a computer-readable storage medium. Background Technology

[0002] Currently, in order to make efficient use of audio, real-time received audio or saved audio files can be converted into text, which makes it easier to organize and view audio data.

[0003] However, currently, audio is usually converted into text and displayed to the user, which requires the user to look up the required text content in a large amount of text, thus affecting the processing efficiency of audio data. Summary of the Invention

[0004] In view of this, embodiments of this application provide a data processing method to address the technical deficiencies existing in the prior art. Embodiments of this application also provide a data processing apparatus, a computing device, and a computer-readable storage medium.

[0005] According to a first aspect of the embodiments of this application, a data processing method is provided, including:

[0006] Receive a voice conversion request and, in response to the voice conversion request, send an audio stream to the voice conversion platform;

[0007] Receive at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier;

[0008] Each audio field is displayed based on preset segmentation rules and the field type identifier of each audio field.

[0009] Optionally, in response to the speech conversion request, sending an audio stream to the speech conversion platform includes:

[0010] In response to the speech conversion request, the initial audio stream is obtained;

[0011] The initial audio stream is processed according to preset audio processing rules to obtain an audio stream, and the audio stream is sent to the speech conversion platform.

[0012] Optionally, each audio field is displayed based on preset segmentation rules and the field type identifier of each audio field, including:

[0013] Determine the target audio field and obtain the total number of current audio fields;

[0014] The field display position is determined based on the current total number of audio fields and the preset threshold in the preset segmentation rule;

[0015] The target audio field is displayed based on its field type identifier and its display position.

[0016] Optionally, the field display position is determined based on the current total number of audio fields and a preset threshold in the preset segmentation rule, including:

[0017] If the total number of current audio fields is greater than the preset threshold, then the field is determined to be displayed in a new paragraph.

[0018] If the total number of current audio fields is less than or equal to the preset threshold, then the field display position is determined to be the current paragraph.

[0019] Optionally, the field type identifier includes a first type and a second type:

[0020] Accordingly, the target audio field is displayed based on its field type identifier and display position, including:

[0021] If the field type identifier of the target audio field is a first type, then the target audio field is displayed at the field display position;

[0022] If the field type identifier of the target audio field is the second type, then delete the audio fields of the first type in the current paragraph, and display the target audio field at the field display position.

[0023] Optionally, before displaying each audio field based on preset segmentation rules and the field type identifier of each audio field, the following steps are also included:

[0024] Count the total number of audio fields of the second type at the current moment.

[0025] Optionally, if the total number of current audio fields is greater than or equal to the preset threshold, then the field display position is determined to be a new paragraph, including:

[0026] If the total number of current audio fields is greater than the preset threshold and the target audio field contains a preset identifier, then the total number of current audio fields is adjusted, and the field display position is determined to be the current paragraph.

[0027] If the total number of current audio fields is greater than the preset threshold and the target audio field does not contain a preset identifier, then the field display position is determined to be a new paragraph.

[0028] Optionally, the method further includes:

[0029] Receive the audio stream and store the audio stream in the audio database.

[0030] Optionally, before receiving the audio stream, the following is also included:

[0031] Create a target player, wherein the target player is used to play the audio stream in the audio database and display the audio fields returned by the speech conversion platform based on the audio stream.

[0032] Optionally, playing the audio stream from the audio database and displaying the audio fields returned by the voice platform based on the audio stream includes:

[0033] Determine the playback frame value in the audio field;

[0034] Calculate the target playback time point of the audio field based on the playback frame values;

[0035] Monitor the current playback time of the target player, and play the audio stream and audio field corresponding to the current playback time based on the current playback time and the target playback time.

[0036] According to a second aspect of the embodiments of this application, a data processing apparatus is provided, comprising:

[0037] The request receiving module is configured to receive a speech conversion request and, in response to the speech conversion request, send an audio stream to the speech conversion platform.

[0038] An audio field receiving module is configured to receive at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier;

[0039] The display module is configured to display each audio field based on preset segmentation rules and the field type identifier of each audio field.

[0040] According to a third aspect of the embodiments of this application, a computing device is provided, comprising:

[0041] Memory and processor;

[0042] The memory is used to store computer-executable instructions, and the processor executes the computer-executable instructions to implement the steps of the data processing method.

[0043] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the data processing method.

[0044] According to a fifth aspect of the present application, a chip is provided that stores a computer program, which, when executed by the chip, implements the steps of the data processing method.

[0045] The data processing method provided in this application receives a speech conversion request, sends an audio stream to a speech conversion platform in response to the speech conversion request, receives at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier, and displays each audio field based on a preset segmentation rule and the field type identifier of each audio field.

[0046] The system receives voice conversion requests to determine the audio stream to be converted, facilitating subsequent conversion by the voice conversion platform. It also receives audio fields returned by the voice conversion platform to display the corresponding audio fields in the audio stream. Based on preset segmentation rules and field type identifiers, the system displays the audio fields, enabling segmented display of audio fields that conform to the rules, thus improving the clarity of the audio fields and enhancing the user's processing efficiency. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the structure of a data processing method provided in an embodiment of this application;

[0048] Figure 2 This is a flowchart of a data processing method provided in an embodiment of this application;

[0049] Figure 3 This is a flowchart illustrating a data processing method for conference audio provided in one embodiment of this application;

[0050] Figure 4a This is a schematic diagram of a player provided in one embodiment of this application;

[0051] Figure 4b This is a schematic diagram of another player provided in one embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0053] Figure 6 This is a structural block diagram of a computing device provided in one embodiment of this application. Detailed Implementation

[0054] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0055] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a,” “the,” and “the” used in one or more embodiments of this application and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0056] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first.

[0057] First, the terminology used in one or more embodiments of the present invention will be explained.

[0058] Voice conversion platform: refers to the terminal where conversion software capable of bidirectional conversion between voice and text resides.

[0059] Audio stream: refers to audio information that needs to be converted into text information.

[0060] Audio field: refers to the text information obtained by converting an audio stream.

[0061] Field type identifier: refers to the characters that represent the field type, such as 1, 2, or a, b, etc.

[0062] Preset segmentation rules: These are the rules for segmenting audio fields.

[0063] Preset audio processing rules: These are the rules for compressing or converting audio streams.

[0064] ElementUI: A desktop component library based on Vue.js 2.0.

[0065] WebSocket is a protocol for full-duplex communication over a single TCP connection.

[0066] The span tag: The span tag is an inline tag in Hypertext Markup Language (HTML) used to group inline elements in a document.

[0067] offsetTop: is a commonly used property in JavaScript for creating scrolling code. Specifically, it refers to the distance from the current object to the top of its parent element.

[0068] int type: int is a data type. In programming languages, int is an identifier used to define integer type variables.

[0069] float type: Floating-point data type. The FLOAT data type is used to store single-precision floating-point numbers or double-precision floating-point numbers.

[0070] ID (identity document): A relatively unique code within a certain system.

[0071] This application provides a data processing method. This application also relates to a data processing apparatus, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0072] See Figure 1 The schematic diagram of the data processing method illustrates the following steps: A voice conversion platform is established, and a transmission connection is created between the voice conversion platform and the webpage. Upon receiving a voice conversion request generated by a user clicking the voice conversion button, the webpage determines the audio stream to be converted based on the audio stream identifier in the request. The audio stream to be converted is then sent to the voice conversion platform via the pre-established transmission connection. Upon receiving the audio stream, the voice conversion platform performs real-time conversion, obtaining at least one audio field. The converted audio field is then returned to the webpage in real-time, with each returned audio field containing a field type identifier. Upon receiving the target audio field, the webpage counts the total number of audio fields of the current second type and determines the display position of the target audio field based on the total number of fields and preset segmentation rules. Finally, the target audio field is displayed on the webpage according to its field type identifier and display position.

[0073] Figure 2 A flowchart of a data processing method according to an embodiment of this application is shown, which specifically includes the following steps:

[0074] Step S202: Receive a voice conversion request and send an audio stream to the voice conversion platform in response to the voice conversion request.

[0075] Among them, a voice conversion request refers to a conversion request that converts voice information into text information, such as a conversion request that converts voice A into the text corresponding to voice A; a voice conversion platform refers to the terminal where the conversion software that can realize bidirectional conversion between voice and text is located; an audio stream refers to audio information that needs to be converted into text information. In practical applications, an audio stream refers to an audio stream obtained based on an existing audio file or an audio stream generated based on real-time acquired audio data.

[0076] In practical applications, users can trigger voice conversion by clicking the voice conversion button on a webpage or using other triggering methods, generating a voice conversion request. The voice conversion request contains an audio stream identifier, which determines which audio streams will be sent to the voice conversion platform. The audio stream identifier is a field that uniquely identifies the audio stream, such as the audio stream name, audio stream ID, etc. In response to the voice conversion request, the voice conversion request is parsed to obtain the audio stream identifier contained in the voice conversion request. Based on the audio stream identifier, the audio stream to be converted is determined, and the audio stream is sent to the pre-set voice conversion platform used for converting the audio stream.

[0077] It is important to note that in order to efficiently send the determined audio stream to the corresponding speech conversion platform and enable the platform to convert the audio stream, a connection needs to be established between the web page and the speech conversion platform before receiving the speech conversion request. Specifically, the target speech conversion platform needs to be determined, which can be any one or more platforms capable of speech conversion; this application does not limit the speech conversion platform. After determining the target speech conversion platform, the conversion connection interface in the target speech conversion platform needs to be determined. The transmission connection between the web page and the target speech conversion platform is established through the connection interface, thereby facilitating the sending of the audio stream to the target speech conversion platform and improving data transmission efficiency.

[0078] For example, user A clicks the voice conversion button on the webpage to trigger the conversion of the meeting audio and generate a voice conversion request. Before generating the voice conversion request, the voice conversion platform C is determined, and a connection between the webpage frontend and the voice conversion platform C is established based on the webpage frontend's websocket. The voice conversion request is parsed to determine the audio stream name a. Based on the audio stream name, the audio stream a to be converted is determined, and the audio stream a is sent to the voice conversion platform C through the transmission connection.

[0079] By receiving voice conversion requests and determining the audio stream to be converted based on the audio stream identifier in the voice conversion request, and sending the audio stream to the voice conversion platform based on the transmission connection, the efficiency of data determination and transmission is improved, which facilitates the subsequent conversion of the audio stream by the voice conversion platform.

[0080] In practical applications, to further improve audio transmission efficiency and enhance the quality of the audio stream, the method of sending an audio stream to the voice conversion platform in response to the voice conversion request includes S2022 to S2024:

[0081] Step S2022: In response to the speech conversion request, obtain the initial audio stream;

[0082] Step S2024: Process the initial audio stream based on preset audio processing rules to obtain an audio stream, and send the audio stream to the speech conversion platform.

[0083] The initial audio stream refers to the unprocessed audio stream; the preset audio processing rules refer to the rules for processing the audio stream, such as rules for compressing the audio stream, rules for format conversion of the audio stream, etc.

[0084] In practical applications, the size of audio packets and the quality of audio streams vary greatly depending on the recording equipment. Therefore, after obtaining the initial audio stream, it can be adjusted based on audio processing rules to obtain an audio stream with less memory usage and less noise, thereby improving the transmission efficiency and quality of the audio stream. For example, if the preset audio processing rule is compression algorithm S, the initial audio stream can be compressed into an audio stream to be transmitted to the speech conversion platform. Or, if the preset audio processing rule is data format conversion rule H, the initial audio stream of type float can be converted into an audio stream of type int before being sent to the speech conversion platform.

[0085] Specifically, the initial audio stream corresponding to the audio stream identifier is determined based on the speech conversion request; the initial audio stream is obtained and processed according to preset audio processing rules to obtain an audio stream; the processed audio stream is sent to the speech conversion platform.

[0086] For example, the system obtains a voice conversion request generated by user B by triggering the voice conversion button; parses the voice conversion request to obtain the audio stream ID number, and obtains the initial audio stream based on the audio stream ID number; obtains the compression algorithm S, and compresses the audio stream to be processed based on the compression algorithm S to obtain the audio stream; and sends the audio stream to the voice conversion platform E with a pre-established transmission connection.

[0087] By processing the audio stream according to preset processing rules before transmitting it to the voice transmission platform, noise in the audio stream is eliminated, the size of the audio file is reduced, thereby improving the transmission efficiency and quality of the audio stream.

[0088] Step S204: Receive at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier.

[0089] Audio fields refer to the text information obtained by converting audio streams. Any speech conversion method that can obtain audio fields is acceptable, and this application does not impose specific limitations. Field type identifiers refer to characters that indicate the field type, such as 1, 2, or a, b, etc. Field type refers to the type of audio field. During the process of converting audio streams into audio fields, a field type is added to each audio field based on the audio conversion of each audio field. The field type can be a final type or an intermediate type. When the field type of an audio field is an intermediate type, it means that the content of the audio field is not the final text content. When the field type of an audio field is a final type, it means that the content of the audio field is the final text content. For example, in order to display audio fields in real time on the client and reduce user waiting time, intermediate type audio fields can be displayed first. After obtaining the final type audio field, the current intermediate type audio field can be deleted, and the final type audio field can be displayed.

[0090] To enable users to view the text information corresponding to the audio stream in real time, the speech-to-text platform converts the audio stream and returns audio fields in real time. Since the semantics of the returned audio fields are not complete—that is, a semantically complete audio field can only be obtained after determining the audio stream corresponding to the entire sentence—the speech-to-text platform needs to set a field type identifier for the returned audio fields. Based on the field type identifier, it determines whether the audio field is a complete or incomplete semantic field. Subsequently, after obtaining the complete semantic field, the currently displayed incomplete semantic field can be deleted, and the complete semantic field can be displayed, thus improving the accuracy of speech conversion while ensuring a good user experience.

[0091] Specifically, the system receives at least one audio field returned by the voice conversion platform based on the audio stream. The specific number of audio fields depends on the audio stream. For example, the audio fields can be divided based on the time interval corresponding to the audio stream. For instance, based on the time interval corresponding to the audio stream, the audio stream corresponding to each second can be converted into audio fields, thus obtaining multiple audio fields. Each received audio field contains a field type identifier, and each received audio field can be displayed based on the field type identifier.

[0092] For example, the audio field s returned by the voice conversion platform E based on the audio stream A is received. The audio field s carries a field type identifier "1". The field type identifier "1" indicates that the audio field s is an intermediate type audio field, that is, an audio field with incomplete semantics.

[0093] For example, the audio field j returned by the voice conversion platform E based on the audio stream A is received. The audio field j carries a field type identifier "0". The field type identifier "0" indicates that the audio field j is an audio field of the final type, that is, an audio field with complete semantics.

[0094] By receiving audio fields that carry field type identifiers, it is easier to display the audio fields based on the field type identifiers; receiving audio fields also facilitates the real-time display of text information corresponding to the audio stream.

[0095] Step S206: Display each audio field based on the preset segmentation rules and the field type identifier of each audio field.

[0096] In practical applications, when displaying each received audio field, if the audio fields are not segmented and are directly concatenated and displayed, a large amount of text information will be obtained. Users will find it difficult to quickly find the content they need in a large amount of text, which will affect the efficiency of users processing data. Therefore, it is necessary to display the audio fields in segments.

[0097] Among them, the preset segmentation rule refers to the rule for segmenting audio fields. For example, the preset segmentation rule is to accumulate 10 audio fields as audio fields in the same paragraph and display each audio field in one paragraph.

[0098] Specifically, the method for displaying each audio field based on preset segmentation rules and the field type identifier of each audio field includes S2062-S2066:

[0099] Step S2062: Determine the target audio field and obtain the total number of current audio fields;

[0100] Step S2064: Determine the field display position based on the current total number of audio fields and the preset threshold in the preset segmentation rule;

[0101] Step S2066: Display the target audio field based on the field type identifier and the field display position.

[0102] Here, the target audio field refers to the audio field currently received based on the audio stream; each audio field has a corresponding field type identifier, which can be either a first type or a second type; the target audio field can be either a first type audio field or a second type audio field; the first type refers to the field type corresponding to an audio field containing incomplete semantics; the second type refers to the field type corresponding to an audio field containing complete semantics; the total number of current audio fields refers to the total number of second type audio fields received in the current paragraph; the preset threshold refers to the upper limit of the second type audio fields that can be displayed in the current paragraph; the field display position refers to the display position of the target audio field, for example, displaying the target audio field in the current paragraph, or displaying the target audio field in a newly created paragraph.

[0103] In practical applications, after receiving the target audio field sent by the speech conversion platform, the target audio field is parsed to determine the field type identifier of the target audio field; the total number of current audio fields and the preset threshold in the preset segmentation rules are determined, and the field display position of the target audio field is determined by comparing the total number of current audio fields with the preset threshold; then the target audio field is displayed according to the field display position and field type identifier of the target audio field.

[0104] For example, after receiving the target audio field Aj, the total number of audio fields of the second type in the current paragraph is obtained, which is 5; the preset audio field threshold is determined to be 10; the field display position of the target audio field is determined to be the current paragraph by comparing the total number of audio fields of 5 and the preset audio field threshold of 10; the field type identifier of the target audio field is determined; and the target audio field Aj is displayed according to the field type identifier and the field display position.

[0105] Specifically, the method for determining the field display position based on the current total number of audio fields and the preset threshold in the preset segmentation rule includes:

[0106] If the total number of current audio fields is greater than the preset threshold, then the field is determined to be displayed in a new paragraph.

[0107] If the total number of current audio fields is less than or equal to the preset threshold, then the field display position is determined to be the current paragraph.

[0108] That is, after receiving the target audio field, if the total number of current audio fields is greater than a preset threshold, the target audio field needs to be displayed in a new paragraph, that is, the field display position of the target audio field is determined to be a new paragraph; if after receiving the target audio field, if the total number of current audio fields is less than or equal to the preset threshold, the target audio field is displayed in the current paragraph, that is, the field display position of the target audio field is determined to be the current paragraph.

[0109] For example, after receiving the target audio field m1, if the preset threshold is determined to be 8 and the current total number of audio fields is 9, the preset threshold and the current total number of audio fields are compared. If the current total number of fields is greater than the preset threshold, then the field display position of the target audio field m1 is a new paragraph.

[0110] Following the previous example, after receiving the target audio field m2, we determine that the preset threshold is 8 and the current total number of audio fields is 2. We compare the preset threshold with the current total number of audio fields and determine that the current total number of fields is less than the preset threshold. Then, the field display position of the target audio field m2 is the current paragraph.

[0111] By determining the display position of audio fields based on the total number of audio fields and preset thresholds, it is determined when to display audio fields in segments, which facilitates a clearer display of audio fields and improves the efficiency of users in processing data.

[0112] In practical applications, the field type identifier may include a first type and a second type; specifically, the method for displaying the target audio field based on the field type identifier and the field display position includes:

[0113] If the field type identifier of the target audio field is a first type, then the target audio field is displayed at the field display position;

[0114] If the field type identifier of the target audio field is the second type, then delete the audio fields of the first type in the current paragraph, and display the target audio field at the field display position.

[0115] The first type refers to the field type corresponding to an audio field containing incomplete semantics; the second type refers to the field type corresponding to an audio field containing complete semantics.

[0116] Specifically, if the target audio field is determined to be of type 1 based on the field type identifier, the target audio field can be directly displayed in the field display position; if the target audio field is determined to be type 2, the currently displayed type 1 audio field is deleted, and the target audio field is displayed in the field display position; if the current paragraph contains at least two type 1 audio fields, and the currently received target audio field is determined to be type 2, then at least two type 1 audio fields are deleted, and the type 2 target audio field is displayed based on the field display position.

[0117] For example, if the field type identifier of the target audio field n1 is Type 1, then the target audio field n1 will be displayed in the field display position.

[0118] For example, if the field type identifier of the target audio field n2 is the second type, then after deleting the currently displayed first type audio field, the target audio field n2 will be displayed in the field display position.

[0119] For example, if it is determined that the audio fields c1 and c2 currently being displayed in the paragraph are both of type 1; and if an audio field c3 of type 2 is received, after deleting the audio fields c1 and c2 of type 1, the audio field c3 of type 2 is displayed in the field display position.

[0120] It should be noted that if the target audio field is determined to be of the first type, you can also delete the currently displayed first-type audio field and then display the first-type target audio field in the field display position.

[0121] In practical applications, the number of second-type audio fields contained in the current paragraph is counted. That is, before displaying each audio field based on preset segmentation rules and the field type identifier of each audio field, the following steps are also taken:

[0122] Count the total number of audio fields of the second type at the current moment.

[0123] For example, when creating the current paragraph q, the number of second-type audio fields in the current paragraph q is 0; when receiving second-type audio field A, the number is changed from 0 to 1; when receiving second-type audio field B again, the number is changed from 1 to 2; when the number of second-type audio fields in the current paragraph q exceeds the preset threshold of 10, a new paragraph u is created; and the number of second-type audio fields in the new paragraph u is then counted.

[0124] By counting the number of second-type audio fields in a paragraph, the total number of current audio fields can be obtained, which makes it easier to determine the display position of the received target audio field based on the total number of current audio fields and a preset threshold.

[0125] In practical applications, if the total number of current audio fields exceeds a preset threshold, it can be determined whether the target audio field contains a preset identifier, thereby ensuring the integrity of the segmented content.

[0126] Specifically, methods for determining the display position of a target audio field by checking whether a preset identifier exists in the target audio field include:

[0127] If the total number of current audio fields is greater than the preset threshold and the target audio field contains a preset identifier, then the total number of current audio fields is adjusted, and the field display position is determined to be the current paragraph.

[0128] If the total number of current audio fields is greater than the preset threshold and the target audio field does not contain a preset identifier, then the field display position is determined to be a new paragraph.

[0129] Among them, the preset identifier refers to the symbol that can indicate that the audio field content is incomplete, such as commas, pause marks, etc.

[0130] For example, after receiving the target audio field A, if it is determined that the total number of current audio fields (11) is greater than the preset threshold (10), then the target audio field A is parsed. If a preset identifier comma is found in the target audio field A, the total number of current audio fields is modified to be less than the preset threshold. For example, the total number of current audio fields (11) can be reduced by 5, that is, the total number of current audio fields can be modified to 6, and the target audio field A can be displayed in the current paragraph. After receiving the second type of audio field, the number of audio fields is counted with the current total number of audio fields (6) as the initial value.

[0131] For example, continuing with the previous example, if it is determined that the target audio field A does not contain the preset identifier, then the target audio field A can be displayed in a new paragraph.

[0132] Furthermore, to enhance the user experience, this embodiment implements a method for synchronously displaying each audio field and the corresponding audio stream, as detailed below:

[0133] After collecting the user's audio data, the process also includes:

[0134] Receive the audio stream and store the audio stream in the audio database.

[0135] Among them, audio stream refers to audio information that needs to be converted into text information; audio database refers to a database that stores audio streams.

[0136] Specifically, voice recording settings can be configured on the webpage to collect sounds generated in business scenarios and generate audio streams, such as collecting meeting sounds in a meeting scenario and generating an audio stream; determine the audio database to store the audio stream, and upload the real-time generated audio stream to the audio database.

[0137] Storing the audio stream makes it easier to synchronize and display the audio stream and its corresponding audio fields later.

[0138] In practical applications, methods for configuring voice recording settings on a webpage before receiving an audio stream include:

[0139] Create a target player, wherein the target player is used to play audio streams from the audio database or real-time acquired audio streams, as well as audio fields returned by the speech conversion platform based on the aforementioned audio streams.

[0140] The target player refers to a player that can play audio streams and corresponding audio fields on a webpage. Specifically, after obtaining the corresponding audio fields based on the audio stream, the player synchronously displays the audio stream and the corresponding audio fields. For example, if the current player is playing audio at 3:25, then it will simultaneously play the audio fields corresponding to the audio at 3:25.

[0141] For example, by using ElementUI to import a player and embedding a WAV format playback link into the player, the player can play the audio stream and its corresponding audio fields.

[0142] Specifically, the method for playing the audio stream from the audio database and displaying the audio fields returned by the voice platform based on the audio stream includes steps 1-3:

[0143] Step 1: Determine the playback frame value in the audio field;

[0144] Step 2: Calculate the target playback time point of the audio field based on the playback frame values;

[0145] Step 3: Monitor the current playback time of the target player, and play the audio stream and audio field corresponding to the current playback time based on the current playback time and the target playback time.

[0146] Among them, the playback frame value refers to the number of frames of the audio field being played. For example, the playback frame value of audio field A is 60, which means that 60 frames are needed to play audio field A completely. The playback start time of audio field A can be calculated based on the display time of each frame and the playback frame value. The target playback time point refers to the time point when the audio field is played. The current playback time point refers to the time point when the audio stream is played.

[0147] Specifically, the audio field corresponding to the currently playing audio stream can be displayed with a text background of a different color than the background. For example, the audio field corresponding to the currently playing audio stream can be displayed with a blue text background. The specific methods for rendering audio fields include: every preset time interval, such as every 100ms, comparing the current time point monitored by the player with the time point corresponding to the audio field, and rendering the background of the audio field corresponding to the matching time point. For example, after determining the audio field A to be rendered, the span tag of audio field A can be assigned a value to realize the background rendering of audio field A; after obtaining the next matching audio field (i.e., obtaining the current audio field), the background rendering of the previous audio field can be cleared, and the current audio field can be rendered.

[0148] In practical applications, if a user drags the playback progress bar in the player or clicks on an audio field that is not currently playing, in order to achieve the corresponding display of the audio stream and the audio field, it is necessary to bind the progress bar to the audio field above it. For example, when the progress bar jumps to time 'a', the audio field also jumps to time 'a' and starts playing.

[0149] The following is in conjunction with the appendix Figure 3 Taking the application of the data processing method provided in this application to conference audio as an example, the data processing method will be further explained. Among other things, Figure 3 This application provides a flowchart illustrating a data processing method for conference audio according to an embodiment of the present application, which specifically includes the following steps:

[0150] Step S302: Determine the voice conversion platform G and establish a transmission connection between the webpage H and the voice conversion platform G.

[0151] Specifically, access the server of the voice conversion platform G and establish a transmission connection with the server using the websocket of webpage H.

[0152] Step S304: Web page H receives the voice conversion request and obtains the initial audio stream based on the voice conversion request.

[0153] Specifically, the user clicks the voice conversion button on webpage H to generate a voice conversion request; the initial audio stream is obtained based on the audio stream identifier in the voice conversion request.

[0154] Step S306: Process the initial audio stream based on preset audio processing rules to obtain the audio stream to be converted.

[0155] Specifically, the preset audio processing rule is a format conversion rule, which converts the initial audio stream of data type float into an audio stream of data type int to be converted.

[0156] Step S308: Send the audio stream to be converted to the speech conversion platform G.

[0157] Specifically, the audio stream to be converted, which is of type int, is sent to the speech conversion platform G via a pre-established transmission connection.

[0158] Step S310: The voice conversion platform G converts the audio stream to be converted to obtain at least one audio field, and sends each audio field to the webpage H.

[0159] Step S312: Web page H receives at least one audio field returned by voice conversion platform G based on the audio stream.

[0160] Step S314: Obtain the preset segmentation rules and the field type identifier of each audio field.

[0161] Specifically, obtain the preset segmentation rules, which include a preset threshold of 10; parse each audio field to obtain the field type identifier for each audio field.

[0162] Step S316: Count the total number of current audio fields of the second type of audio fields at the current moment.

[0163] Specifically, determine the current paragraph d, and count the total number of second-type audio fields contained in the current paragraph d, and determine that the total number of current audio fields is 5.

[0164] Step S318: Determine the target audio field and obtain the total number of current audio fields.

[0165] Specifically, determine the target audio field B and obtain the total number of current audio fields, which is 5.

[0166] Step S320: Determine the field display position based on the current total number of audio fields and the preset threshold in the preset segmentation rules.

[0167] Specifically, determine the current total number of audio fields (5) and the preset threshold (10); compare the current total number of audio fields with the preset threshold, and determine that the current total number of audio fields (5) is less than the preset threshold (10), then determine that the field display position of the target audio field B is the current paragraph.

[0168] Step S322: Display the target audio field based on the field type identifier and the field display position.

[0169] Specifically, if the field type of the target audio field B is determined to be of type 2 based on the field type identifier of the target audio field B, then the audio fields of type 1 in the current paragraph will be deleted, and the target audio field B will be displayed in the current paragraph.

[0170] The data processing method of this application includes receiving a speech conversion request, sending an audio stream to a speech conversion platform in response to the speech conversion request, receiving at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier, and displaying each audio field based on a preset segmentation rule and the field type identifier of each audio field.

[0171] The system receives voice conversion requests to determine the audio stream to be converted, facilitating subsequent conversion by the voice conversion platform. It also receives audio fields returned by the voice conversion platform to display the corresponding audio fields in the audio stream. Based on preset segmentation rules and field type identifiers, the system displays the audio fields, enabling segmented display of audio fields that conform to the rules, thus improving the clarity of the audio fields and enhancing the user's processing efficiency.

[0172] The following combination Figure 4a and Figure 4b Taking the application of the data processing method provided in this application to conference audio as an example, the demonstration of the audio field in the data processing method is further explained. Among them, Figure 4a This illustration shows a schematic diagram of a player according to an embodiment of this application. Figure 4b A schematic diagram of another player provided in one embodiment of this application is shown.

[0173] In this embodiment, a player is created on the webpage based on ElementUI. After determining the audio stream to be played, the playback link corresponding to the audio stream can be embedded into the player, enabling the player to play the audio stream. The audio stream is recorded on the webpage and stored in the corresponding audio database on the webpage. Each audio field returned by the speech conversion platform corresponds to a playback frame value. Based on the display time of each frame and the playback frame value, the starting playback time of each audio field can be calculated. For example, if the playback frame value corresponding to audio field A is 820 and the display time of each frame is 10ms, the starting playback time of audio field A is calculated based on a preset calculation rule as: 820 * 10ms / 1000 = 8.2 seconds. The player listens to the current time returned every 100ms and compares it with the time of the currently displayed audio field A. After a match is determined, the matched audio field A is displayed with an underline. After matching the next audio field, the underline style of audio field A is cleared, and a style value is assigned to the matched audio field. For example, the span tag of the audio field is copied to an underline style. Figure 4a As shown, the audio field at the top of the webpage is linked to the player's progress bar below. When the progress bar is at point A, the audio field playing is synchronized with the audio stream; for example... Figure 4b As shown, when the user drags the progress bar to point B, the corresponding audio stream and audio field at point B are played synchronously. If the user drags the player's progress bar, the scrollbar of the current DOM element (span tag) is checked. If the progress bar's offsetTop is negative, the scrollbar is redirected. If the user clicks on an audio field in the current webpage, the scrollbar is redirected, thus achieving two-way binding between the progress bar and the audio field.

[0174] The data processing method of this application, by synchronously playing the audio stream and the corresponding audio fields, makes it easier for users to quickly locate the text information corresponding to the audio, thereby improving the efficiency of users in obtaining information.

[0175] Corresponding to the above method embodiments, this application also provides data processing apparatus embodiments. Figure 5 A schematic diagram of the structure of a data processing apparatus according to an embodiment of this application is shown. Figure 5 As shown, the device includes:

[0176] Request receiving module 502 is configured to receive a voice conversion request and send an audio stream to the voice conversion platform in response to the voice conversion request;

[0177] The audio field receiving module 504 is configured to receive at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier;

[0178] The display module 506 is configured to display each audio field based on preset segmentation rules and the field type identifier of each audio field.

[0179] Optionally, the request receiving module 502 is further configured to:

[0180] In response to the speech conversion request, the initial audio stream is obtained;

[0181] The initial audio stream is processed based on preset audio processing rules to obtain an audio stream, and the audio stream is sent to the speech conversion platform.

[0182] Optionally, the display module 506 is further configured to:

[0183] Determine the target audio field and obtain the total number of current audio fields;

[0184] The field display position is determined based on the current total number of audio fields and the preset threshold in the preset segmentation rule;

[0185] The target audio field is displayed based on its field type identifier and its display position.

[0186] Optionally, the display module 506 is further configured to:

[0187] If the total number of current audio fields is greater than the preset threshold, then the field is determined to be displayed in a new paragraph.

[0188] If the total number of current audio fields is less than or equal to the preset threshold, then the field display position is determined to be the current paragraph.

[0189] Optionally, the field type identifier includes a first type and a second type; the display module 506 is further configured to:

[0190] If the field type identifier of the target audio field is a first type, then the target audio field is displayed at the field display position;

[0191] If the field type identifier of the target audio field is the second type, then delete the audio fields of the first type in the current paragraph, and display the target audio field at the field display position.

[0192] Optionally, the device further includes a statistics module configured to:

[0193] Count the total number of audio fields of the second type at the current moment.

[0194] Optionally, the display module 506 is further configured to:

[0195] If the total number of current audio fields is greater than the preset threshold and the target audio field contains a preset identifier, then the total number of current audio fields is adjusted, and the field display position is determined to be the current paragraph.

[0196] If the total number of current audio fields is greater than the preset threshold and the target audio field does not contain a preset identifier, then the field display position is determined to be a new paragraph.

[0197] Optionally, the device further includes a receiving submodule configured to:

[0198] Receive the audio stream and store the audio stream in the audio database.

[0199] Optionally, the apparatus further includes a creation submodule configured to:

[0200] Create a target player, wherein the target player is used to play the audio stream in the audio database and display the audio fields returned by the speech conversion platform based on the audio stream.

[0201] Optionally, the creation submodule is further configured as follows:

[0202] Determine the playback frame value in the audio field;

[0203] Calculate the target playback time point of the audio field based on the playback frame values;

[0204] Monitor the current playback time of the target player, and play the audio stream and audio field corresponding to the current playback time based on the current playback time and the target playback time.

[0205] The data processing apparatus provided in this application receives a speech conversion request, and in response to the speech conversion request sends an audio stream to a speech conversion platform; receives at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier; and displays each audio field based on a preset segmentation rule and the field type identifier of each audio field.

[0206] The system receives voice conversion requests to determine the audio stream to be converted, facilitating subsequent conversion by the voice conversion platform. It also receives audio fields returned by the voice conversion platform to display the corresponding audio fields in the audio stream. Based on preset segmentation rules and field type identifiers, the system displays the audio fields, enabling segmented display of audio fields that conform to the rules, thus improving the clarity of the audio fields and enhancing the user's processing efficiency.

[0207] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the aforementioned data processing method belong to the same concept. Details not described in detail in the technical solution of the data processing apparatus can be found in the description of the technical solution of the aforementioned data processing method. Furthermore, the components in the apparatus embodiment should be understood as functional modules necessary to implement each step of the program flow or each step of the method; these functional modules are not actual functional divisions or separations. An apparatus claim defined by such a set of functional modules should be understood as a functional module architecture that primarily implements the solution through the computer program described in the specification, and not as a physical device that primarily implements the solution through hardware.

[0208] Figure 6 A structural block diagram of a computing device 600 according to an embodiment of this application is shown. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0209] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0210] In one embodiment of this application, the aforementioned components of the computing device 600 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0211] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 600 can also be a mobile or stationary server.

[0212] The processor 620 is used to execute computer-executable instructions for the data processing method.

[0213] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0214] One embodiment of this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, are used for a data processing method.

[0215] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method described above.

[0216] One embodiment of this application also provides a chip that stores a computer program, which, when executed by the chip, implements the steps of the data processing method.

[0217] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0218] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0219] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0220] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0221] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, characterized in that, include: Receive a voice conversion request and, in response to the voice conversion request, send an audio stream to the voice conversion platform; The system receives at least one audio field returned by the speech conversion platform based on the audio stream, wherein each audio field carries a field type identifier, which is a character that identifies the field type. The field type includes intermediate type and final type. The audio field of intermediate type is used to be displayed to the user first, and the audio field of final type is used to be displayed after the current audio field of intermediate type is deleted. Each audio field is displayed based on preset segmentation rules and the field type identifier of each audio field.

2. The data processing method as described in claim 1, characterized in that, In response to the speech conversion request, an audio stream is sent to the speech conversion platform, including: In response to the speech conversion request, the initial audio stream is obtained; The initial audio stream is processed based on preset audio processing rules to obtain an audio stream, and the audio stream is sent to the speech conversion platform.

3. The data processing method as described in claim 1, characterized in that, Each audio field is displayed based on preset segmentation rules and the field type identifier of each audio field, including: Determine the target audio field and obtain the total number of current audio fields; The field display position is determined based on the current total number of audio fields and the preset threshold in the preset segmentation rule; The target audio field is displayed based on its field type identifier and its display position.

4. The data processing method as described in claim 3, characterized in that, The field display position is determined based on the current total number of audio fields and the preset threshold in the preset segmentation rules, including: If the total number of current audio fields is greater than the preset threshold, then the field is determined to be displayed in a new paragraph. If the total number of current audio fields is less than or equal to the preset threshold, then the field display position is determined to be the current paragraph.

5. The data processing method as described in claim 3, characterized in that, The field type identifier includes a first type and a second type: Accordingly, the target audio field is displayed based on its field type identifier and display position, including: If the field type identifier of the target audio field is a first type, then the target audio field is displayed at the field display position; If the field type identifier of the target audio field is the second type, then delete the audio fields of the first type in the current paragraph, and display the target audio field at the field display position.

6. The data processing method as described in claim 3, characterized in that, Before displaying each audio field based on preset segmentation rules and the field type identifier of each audio field, the following also includes: Count the total number of audio fields of the second type at the current moment.

7. The data processing method as described in claim 4, characterized in that, If the total number of current audio fields is greater than or equal to the preset threshold, then the field display position is determined to be a new paragraph, including: If the total number of current audio fields is greater than the preset threshold and the target audio field contains a preset identifier, then the total number of current audio fields is adjusted, and the field display position is determined to be the current paragraph. If the total number of current audio fields is greater than the preset threshold and the target audio field does not contain a preset identifier, then the field display position is determined to be a new paragraph.

8. The data processing method as described in claim 1, characterized in that, The method further includes: Receive the audio stream and store the audio stream in the audio database.

9. The data processing method as described in claim 8, characterized in that, Before receiving the audio stream, the following is also included: Create a target player, wherein the target player is used to play the audio stream in the audio database and display the audio fields returned by the speech conversion platform based on the audio stream.

10. The data processing method as described in claim 9, characterized in that, Playing audio streams from the audio database and displaying audio fields returned by the speech conversion platform based on the audio streams, including: Determine the playback frame value in the audio field; Calculate the target playback time point of the audio field based on the playback frame values; Monitor the current playback time of the target player, and play the audio stream and audio field corresponding to the current playback time based on the current playback time and the target playback time.

11. A data processing apparatus, characterized in that, include: The request receiving module is configured to receive a speech conversion request and, in response to the speech conversion request, send an audio stream to the speech conversion platform. An audio field receiving module is configured to receive at least one audio field returned by the speech conversion platform based on the audio stream. Each audio field carries a field type identifier, which is a character that identifies the field type. The field type includes an intermediate type and a final type. The audio field of the intermediate type is used to be displayed to the user first, and the audio field of the final type is used to be displayed after the current intermediate type audio field is deleted. The display module is configured to display each audio field based on preset segmentation rules and the field type identifier of each audio field.

12. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the data processing method according to any one of claims 1 to 10.

13. A computer-readable storage medium storing computer instructions, characterized in that, When executed by the processor, this instruction implements the steps of the data processing method according to any one of claims 1 to 10.

Images