Audio data processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to an audio data processing method, device, electronic device, storage medium and computer program product, including: in response to a speed change processing operation on audio data, a target sound wave spectrogram after speed change is displayed in an audio processing interface; the target sound wave spectrogram is obtained by stretching the initial sound wave spectrogram according to the target speed change multiple corresponding to the speed change processing operation; in the process of playing the audio data after speed change, the electronic device responds to a manual card point marking operation on the audio data, and displays a manual card point mark on the target sound wave spectrogram. The method is adopted to improve the convenience and flexibility of manual card point marking.

Description

Technical Field

[0001] The present disclosure relates to the field of audio processing technology, and in particular to an audio data processing method, device, electronic device, storage medium and computer program product. Background Art

[0002] With the development of audio processing technology, audio card point technology has emerged. Users can mark card points on the sound wave spectrogram corresponding to the audio data, so that when special effects are subsequently processed on the audio clips, the position of the card point mark can be used to locate the position where special effects are added.

[0003] In the current method of marking audio data points, a fixed-length sound wave spectrogram corresponding to each audio segment is displayed in the audio processing interface. In the sound wave spectrogram, the electronic device automatically marks the audio data based on pre-identified audio regular points (also called audio action points).

[0004] However, in the current audio data card point marking method, the card points are only marked according to the audio regular points, the card point marking rules are single, and the card point flexibility is low. Summary of the invention

[0005] The present disclosure provides an audio data processing method, device, electronic device, storage medium and computer program product to at least solve the problem of single card point marking form and low card point flexibility in the related art. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an audio data processing method, comprising:

[0007] In response to a speed change processing operation on audio data, a target sound wave spectrogram after speed change is displayed in an audio processing interface; the target sound wave spectrogram is obtained by stretching an initial sound wave spectrogram according to a target speed change multiple corresponding to the speed change processing operation;

[0008] During the playback of the speed-changed audio data, in response to a manual card point marking operation on the audio data, a manual card point mark is displayed on the target sound wave spectrogram.

[0009] In one embodiment, in response to the speed change processing operation on the audio data, displaying the speed-changed target sound wave spectrogram in the audio processing interface includes:

[0010] In response to a speed change processing operation on the audio data, determining a speed change sound wave spectrogram length corresponding to the audio data after the speed change according to a target speed change multiple corresponding to the speed change processing operation and an initial sound wave spectrogram length corresponding to the audio data;

[0011] According to the length of the speed-varied sound wave spectrogram, the initial sound wave spectrogram corresponding to the original audio data in the audio processing interface is stretched;

[0012] In the audio processing interface, a target sound wave spectrogram after speed change is displayed after being stretched to a length equal to that of the speed change sound wave spectrogram.

[0013] In one embodiment, the method further comprises:

[0014] In a case where an original manual card point mark is displayed on the initial sound wave spectrogram, the original manual card point mark is displayed on the target sound wave spectrogram.

[0015] In one embodiment, before displaying the sound wave spectrogram after speed change in the audio processing interface, the method further includes:

[0016] Determine the speed-changed audio duration corresponding to the speed-changed audio data according to the audio duration corresponding to the original audio data and the target speed-changed multiple of the original audio data;

[0017] The audio timeline is displayed on the audio processing interface according to the time scale corresponding to the variable speed audio duration and the preset time scale interval.

[0018] In one embodiment, a size line is displayed on the target sound wave spectrogram in the audio processing interface; and in response to a manual card point marking operation on the audio data, a manual card point mark is displayed on the speed-changed sound wave spectrogram, including:

[0019] In response to a manual card point marking operation on the audio data, a manual card point mark is displayed at a position corresponding to the size line on the target sound wave spectrogram.

[0020] In one embodiment, the method further comprises:

[0021] In response to the automatic card point marking operation on the audio data, the automatic card point marking of the target audio frame is displayed on the speed-changed sound wave spectrogram of the audio processing interface.

[0022] In one of the embodiments, after the automatic card point marking operation is performed on the audio data, the method further includes:

[0023] In response to a cancel operation of the automatic card point mark, the automatic card point mark is canceled from being displayed on the speed-changed sound wave spectrogram of the audio processing interface.

[0024] In one embodiment, the method further comprises:

[0025] In response to a play operation on the speed-changed audio data, play the speed-changed audio data;

[0026] In response to the playback progress adjustment operation of the speed-changed audio data, the playback start position is re-determined on the speed-changed audio timeline, and the speed-changed audio data is played from the playback start position; the audio playback timeline is an audio playback axis corresponding to the speed-changed audio duration.

[0027] In one embodiment, in response to the speed change processing operation on the audio data, after displaying the speed-changed target sound wave spectrogram in the audio processing interface, the method further includes:

[0028] In response to the speed-changing recovery operation on the audio data, the restored sound wave spectrogram is displayed in the audio processing interface; the manual point card mark is displayed on the restored sound wave spectrogram.

[0029] According to a second aspect of an embodiment of the present disclosure, there is provided an audio data processing apparatus, including:

[0030] The spectrogram display unit is configured to perform a speed change processing operation on the audio data and display a target sound wave spectrogram after speed change in the audio processing interface; the target sound wave spectrogram is obtained by stretching the initial sound wave spectrogram according to the target speed change multiple corresponding to the speed change processing operation;

[0031] The manual card point unit is configured to display a manual card point mark on the target sound wave spectrogram in response to a manual card point marking operation on the audio data during the playback of the audio data after the speed change.

[0032] In one embodiment, the spectrum display unit comprises:

[0033] A determination subunit is configured to perform a speed change processing operation on the audio data, and determine the speed change sound wave spectrogram length corresponding to the audio data after the speed change according to the target speed change multiple corresponding to the speed change processing operation and the initial sound wave spectrogram length corresponding to the audio data;

[0034] The stretching subunit is configured to perform stretching processing on the initial sound wave spectrogram corresponding to the original audio data in the audio processing interface according to the length of the speed-varied sound wave spectrogram;

[0035] The display subunit is configured to be executed in the audio processing interface to display the speed-changed target sound wave spectrogram that is stretched to be equal to the length of the speed-changed sound wave spectrogram.

[0036] In one embodiment, the audio data processing device further includes:

[0037] The mark display unit is configured to display the original manual card point mark on the target sound wave spectrogram when the original manual card point mark is displayed on the initial sound wave spectrogram.

[0038] In one embodiment, the audio data processing device further includes:

[0039] A processing unit is configured to determine the speed-changed audio duration corresponding to the speed-changed audio data according to the audio duration corresponding to the original audio data and the target speed-changed multiple of the original audio data;

[0040] The display unit is configured to display the audio timeline on the audio processing interface according to the time scale corresponding to the length of the variable-speed audio and the preset time scale interval.

[0041] In one embodiment, a size line is displayed on the target sound wave spectrogram in the audio processing interface;

[0042] The manual card point unit is configured to execute a manual card point marking operation in response to the audio data, and display a manual card point mark at a position corresponding to the size line on the target sound wave spectrogram.

[0043] In one embodiment, the audio data processing device further includes:

[0044] The automatic card point unit is configured to perform an automatic card point marking operation in response to the audio data, and display the automatic card point marking of the target audio frame on the speed-changed sound wave spectrogram of the audio processing interface.

[0045] In one embodiment, the audio data processing device further includes:

[0046] The card point canceling unit is configured to execute a canceling operation in response to the automatic card point mark, and cancel the display of the automatic card point mark on the speed-changed sound wave spectrogram of the audio processing interface.

[0047] In one embodiment, the audio data processing device further includes:

[0048] a playing unit configured to play the audio data after the speed change in response to a playing operation on the audio data after the speed change;

[0049] The progress adjustment unit is configured to execute a playback progress adjustment operation in response to the speed-changed audio data, redetermine the playback start position on the speed-changed audio timeline, and play the speed-changed audio data from the playback start position; the audio playback timeline is an audio playback axis corresponding to the speed-changed audio duration.

[0050] In one embodiment, the audio data processing device further includes:

[0051] The speed recovery unit is configured to execute a speed recovery operation in response to the audio data, and display the restored sound wave spectrogram in the audio processing interface; the manual point card mark is displayed on the restored sound wave spectrogram.

[0052] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0053] processor;

[0054] a memory for storing instructions executable by the processor;

[0055] Wherein, the processor is configured to execute the instructions to implement the above-mentioned audio data processing method.

[0056] According to a fourth aspect of an embodiment of the present disclosure, a computer storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the audio data processing method as described above.

[0057] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided. When the instructions are executed by a processor of an electronic device, the electronic device is enabled to perform the audio data processing method as described above.

[0058] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: the electronic device displays the target sound wave spectrogram after speed change in the audio processing interface in response to the speed change processing operation on the audio data; the target sound wave spectrogram is obtained based on the initial sound wave spectrogram and the speed change multiple; in the process of playing the audio data after speed change, the electronic device responds to the manual card point marking operation on the audio data and displays the manual card point mark on the target sound wave spectrogram. With this method, the electronic device obtains the target sound wave spectrogram by performing speed change stretching processing on the sound wave spectrogram corresponding to the audio data in advance, and manually marks the card points on the target sound wave spectrogram, thereby increasing the length of the sound wave spectrogram that can be used for audio card points, and in the process of playing the audio data after speed change, the operable space of the manual card point marking operation and the flexibility of the audio card points are improved.

[0059] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0061] Figure 1 The present invention is a flowchart of an audio data processing method according to an exemplary embodiment.

[0062] Figure 2 The figure is a flow chart of a method for displaying a sound wave spectrogram after speed change according to an exemplary embodiment.

[0063] Figure 3 It is a schematic diagram of an audio processing interface in a 0.5x speed change trigger state according to an exemplary embodiment.

[0064] Figure 4 The figure is a flowchart of a method for displaying a variable-speed audio timeline according to an exemplary embodiment.

[0065] Figure 5 The present invention is a schematic diagram showing a size line included in an audio processing interface according to an exemplary embodiment.

[0066] Figure 6 It is a schematic diagram of an audio processing interface in an automatic card point triggering state according to an exemplary embodiment.

[0067] Figure 7 The present invention is a flowchart of a method for canceling automatic card point marking according to an exemplary embodiment.

[0068] Figure 8 The present invention is a schematic diagram of an audio processing interface for canceling automatic card point marking according to an exemplary embodiment.

[0069] Fig. 9 It is a block diagram of an audio data processing device according to an exemplary embodiment.

[0070] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0071] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0072] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0073] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0074] The method provided by the present disclosure can be applied to electronic devices, which may be mobile terminals, computers, tablet devices, digital broadcast terminals, etc., and the embodiments of the present disclosure are not limited thereto.

[0075] There are many different scenarios for audio speed change and pitch change, such as speed change without changing pitch, pitch change without changing speed, and speed change and pitch change. The method provided in the present disclosure can be applied to all scenarios with audio speed change.

[0076] Figure 1 is a flowchart of an audio data processing method according to an exemplary embodiment. Figure 1 As shown, the audio data processing method is used in an electronic device and includes the following steps.

[0077] In step S110, in response to the speed change processing operation on the audio data, the target sound wave spectrogram after the speed change is displayed in the audio processing interface.

[0078] The target sound wave spectrogram is obtained by stretching the initial sound wave spectrogram according to the target speed change multiple corresponding to the speed change processing operation.

[0079] In implementation, an application for executing the technical solution of the present disclosure may be installed in the electronic device. When a certain section of audio data needs to be processed, the user selects and obtains the pre-cached audio data of the waiting point through the application. Then, the electronic device displays the sound wave spectrogram of the audio data of the waiting point in the preset display area in the audio processing interface of the application, and displays various audio operation options for processing the audio data. When the user needs to perform speed change processing on the audio data, the user can perform a speed change processing operation (for example, triggering an audio speed change processing option) to select a target speed change multiple of the audio data. In response to the speed change processing operation, the electronic device performs speed change processing on the audio data at the target speed change multiple. At the same time, the electronic device also stretches the length of the sound wave spectrogram corresponding to the audio data accordingly, and displays the target sound wave spectrogram stretched according to the target speed change multiple in the audio processing interface.

[0080] In step S120, during the playback of the audio data after the speed change, in response to the manual card point marking operation on the audio data, a manual card point mark is displayed on the target sound wave spectrogram.

[0081] In implementation, during the playback of the audio data after speed change, the user pre-listens to the audio segment after speed change to determine the position where the card point needs to be located, and in the audio processing interface, the user triggers the manual card point marking operation to trigger the manual card point marking option. In response to the manual card point marking operation on the audio data after speed change, the electronic device displays the manual card point mark at a preset position on the target audio spectrogram of the audio data after speed change.

[0082] In the above-mentioned audio data processing method, the electronic device displays the target sound wave spectrogram after speed change in the audio processing interface in response to the speed change processing operation on the audio data. The target sound wave spectrogram is obtained by stretching the initial sound wave spectrogram and the speed change multiple. Then, in the process of playing the audio data after speed change, the electronic device responds to the manual card point marking operation on the audio data and displays the manual card point mark on the target sound wave spectrogram. With this method, the electronic device obtains the target sound wave spectrogram by performing speed change stretching processing on the sound wave spectrogram corresponding to the audio data in advance, and performs card point marking on the target sound wave spectrogram, thereby increasing the length of the sound wave spectrogram that can be used for audio card points, and improving the convenience of the manual card point marking operation and the flexibility of the audio card points in the pre-listening process after the audio data is speed changed.

[0083] In an exemplary embodiment, if Figure 2 As shown, in step S110, in response to the speed change processing operation on the audio data, the process of displaying the speed-changed target sound wave spectrogram in the audio processing interface can be specifically implemented by the following steps:

[0084] In step S111, in response to the speed change processing operation on the audio data, the speed change sound wave spectrogram length corresponding to the speed change audio data is determined according to the target speed change multiple corresponding to the speed change processing operation and the initial sound wave spectrogram length corresponding to the audio data.

[0085] In implementation, multiple audio speed-changing options are displayed on the audio processing interface of the electronic device, such as Figure 3 As shown, for example, 1 (speed), 0.75 (speed), 0.5 (speed), custom, etc. Among them, for the custom options in the audio speed change options, the user can independently define the multiples that need to be changed, thereby improving the flexibility of the speed change multiples of the audio data. When the user needs to perform audio data speed change processing, the user can trigger the target audio speed change option according to the required speed change multiple, and generate a speed change processing operation instruction for the audio data carrying the target speed change multiple. The electronic device responds to the speed change processing operation instruction for the audio data, calculates the ratio of the initial sound wave spectrogram length to the target speed change multiple according to the target speed change multiple carried by the speed change processing operation instruction, and the initial sound wave spectrogram length corresponding to the audio data, and determines the speed change sound wave spectrogram length corresponding to the audio data after the speed change. As shown Figure 3 As shown, the user needs to perform a 0.5 audio data speed change. After triggering the 0.5 audio speed change option, the 0.5 option in the audio processing interface is displayed as selected (or called triggered). The electronic device responds to the speed change processing operation on the audio data, and determines the speed-changed sound wave spectrogram length corresponding to the audio data after the speed change. The calculation formula for the speed-changed sound wave spectrogram length is: initial sound wave spectrogram length / target speed change multiple=speed-changed sound wave spectrogram length. For example, if the sound wave length in the sound wave spectrogram corresponding to the original audio data is 15 cm, then the target sound wave spectrogram length corresponding to the audio data after the 0.5 speed change is: 15 / 0.5=30 cm.

[0086] Step S112, stretching the initial sound wave spectrogram corresponding to the original audio data in the audio processing interface according to the length of the speed-varied sound wave spectrogram.

[0087] In implementation, the electronic device stretches the sound wave spectrogram corresponding to the original audio data based on the determined length of the speed-shifted sound wave spectrogram, and displays the target sound wave spectrogram obtained after the stretching process in a preset display area in the audio processing interface with the length of the speed-shifted sound wave spectrogram. Figure 3 As shown, the target sound wave spectrogram corresponding to the audio data after a speed change of 0.5 is longer than the initial sound wave spectrogram. Therefore, the distance between the two peaks of the original sound wave in the sound wave spectrogram is doubled, providing more space for card point operation.

[0088] In implementation, the electronic device uses a logarithmic filter to stretch the energy amplitude of the initial sound wave spectrogram corresponding to the original audio data according to the length of the variable speed sound wave spectrogram (i.e., stretch it by the corresponding target multiple). For example, if the initial sound wave spectrogram corresponding to the original audio data is 15 centimeters (cm), and the target sound wave spectrogram length determined after the 0.58 variable speed processing is 30 centimeters, the electronic device uses a logarithmic filter to stretch each energy amplitude in the sound wave spectrogram by one time, so that the stretched sound wave spectrogram is equal to 30 centimeters.

[0089] Step S113, in the audio processing interface, displaying the speed-changed target sound wave spectrogram that is stretched to be equal to the length of the speed-changed sound wave spectrogram.

[0090] In implementation, in the audio processing interface, the electronic device displays the stretched target sound wave spectrogram (also called sound wave trajectory or audio track) in a preset display area, and the length of the sound waves in the stretched sound wave spectrogram is equal to the determined length of the target sound wave spectrogram after the speed change.

[0091] In this embodiment, by slowing down the playback speed of the audio data, the sound wave spectrogram corresponding to the audio data is also stretched accordingly, so that the sound wave spectrogram and the audio data are more closely matched during the playback process. At the same time, more space is provided for the user to manually mark the card points.

[0092] Optionally, the sound wave spectrogram is displayed in a scrolling manner in the audio processing interface. That is, as the audio data is played, the sound wave spectrogram is scrolled in a preset display area of ​​the audio processing interface along with the progress of the audio data being played. Part of the sound wave spectrogram in the display area can be stacked into progress blocks for display to correspond to the progress of the audio data being played.

[0093] In an exemplary embodiment, the audio data processing method further comprises the following steps:

[0094] When the original manual card point mark is displayed on the initial sound wave spectrogram, the original manual card point mark is displayed on the target sound wave spectrogram.

[0095] In implementation, when the original manual card point mark is displayed on the initial sound wave spectrogram, each original manual card point mark corresponds to a target audio frame in the initial sound wave spectrogram. Therefore, when the initial sound wave spectrogram is stretched and displayed as a target sound wave spectrogram, the display position of the target audio frame included in the target sound wave spectrogram changes, so that the display position of the original manual card point mark corresponding to the target audio frame also changes accordingly. Furthermore, the electronic device can determine the display position of the original manual card point mark according to the display position of the target audio frame in the target sound wave spectrogram, so that the electronic device can display the original manual card point mark on the target sound wave spectrogram.

[0096] In an exemplary embodiment, if Figure 4 As shown, before displaying the target sound wave spectrogram after speed change in the audio processing interface in step S110, the audio data processing method further includes the following steps:

[0097] Step S401, determining the speed-changed audio duration corresponding to the speed-changed audio data according to the audio duration corresponding to the original audio data and the target speed-changed multiple of the original audio data.

[0098] In implementation, the electronic device determines the speed-changed audio duration corresponding to the audio data after speed change according to the audio duration corresponding to the original audio data and the target speed-changed multiple that the user needs to perform on the original audio data. For example, when the audio duration corresponding to the original audio data is 2 minutes, and the user needs to perform a 0.58-fold speed change on the original audio data, the electronic device obtains a speed-changed audio duration corresponding to the audio data after speed change based on the audio duration speed-changed formula: original audio duration / target speed-changed multiple=speed-changed audio duration, and the speed-changed audio duration corresponding to the audio data after speed change is 4 minutes.

[0099] Step S402: Display the audio timeline on the audio processing interface according to the time scale corresponding to the variable speed audio duration and the preset time scale interval.

[0100] In implementation, in the audio processing interface, the playback progress of the audio data can be displayed in the form of a scrolling sound wave spectrogram, and the audio duration and the corresponding audio moment can be displayed through the time scale of the audio time axis. Then, when the corresponding audio duration changes after the audio data is speeded up, the electronic device displays the audio time axis corresponding to the speed-changed audio duration in the audio processing interface according to the time scale corresponding to the speed-changed audio duration (for example, in seconds S as a unit) and the preset time scale interval (with 0.1S as the time scale interval). For example, when the audio duration corresponding to the original audio data is 2 minutes, after the speed-changed processing, the speed-changed audio duration is determined to be 4 minutes. The electronic device uses seconds as the time scale and 0.1 seconds as the time scale interval to form an audio time axis with a total duration of 4 minutes, which is displayed in the preset display area of ​​the audio processing interface. Among them, the sound wave spectrogram corresponding to the audio data and the audio time axis corresponding to the audio data are scrolled and played at the same speed and in the same direction.

[0101] In an exemplary embodiment, if Figure 5 As shown, a size line is displayed on the target sound wave spectrogram in the audio processing interface. The size line is vertical, perpendicular to the scrolling direction of the sound wave spectrogram, and the position displayed in the audio processing interface is fixed. The size line is used to provide the user with a fixed point position when marking a point. The specific processing process of step S120 may include the following steps:

[0102] Step S121, in response to a manual card point marking operation on the audio data, a manual card point mark is displayed at a position corresponding to the size line on the target sound wave spectrogram.

[0103] In implementation, the position of the size line on the target sound wave spectrogram remains fixed, and the target sound wave spectrogram scrolls forward as the audio data is played. The user pre-listens to the played audio data and manually marks the audio data points, such as Figure 5 As shown, at a certain moment, the user triggers the manual card point marking option (also called the "add point" option) in the audio processing interface, and the electronic device receives the manual card point marking operation instruction. The electronic device responds to the manual card point marking operation on the audio data and displays the manual card point mark at the current position corresponding to the size line on the target sound wave spectrogram.

[0104] In an exemplary embodiment, if Figure 6 As shown, in addition to providing a manual card point marking operation for audio data, the present disclosure can also automatically card the audio data. Specifically, the audio data processing method also includes the following processing steps:

[0105] Step S130, in response to the automatic card point marking operation on the audio data, the automatic card point mark of the target audio frame is displayed on the speed-changed sound wave spectrogram of the audio processing interface.

[0106] In implementation, when a user triggers the automatic card point marking option (also called the "smart card point" option) in the audio processing interface, the electronic device receives the automatic card point marking operation instruction, and the electronic device responds to the automatic card point marking operation on the audio data and displays the automatic card point marking of the target audio frame on the speed-shifted sound wave spectrogram of the audio processing interface. The position of the automatic card point marking is displayed in a preset area of ​​the audio processing interface.

[0107] In an exemplary embodiment, if Figure 7 As shown, the audio data processing method also includes the following processing steps:

[0108] Step S140, in response to the cancellation operation of the automatic card point mark, cancel the display of the automatic card point mark on the sound wave spectrogram after the speed change in the audio processing interface.

[0109] During implementation, the electronic device pre-determines the current state of the automatic stick point mark. If the current automatic stick point mark option is in a triggered state, the user triggers the automatic stick point mark option in the audio processing interface again, and the electronic device receives the automatic stick point mark cancellation operation. The electronic device responds to the automatic stick point mark cancellation operation and cancels the display of the automatic stick point mark on the target sound wave spectrogram of the audio processing interface.

[0110] Optionally, if the current automatic card point marking option is not triggered, the electronic device executes step S130.

[0111] Step S150, in response to the playback progress adjustment operation of the speed-changed audio data, the playback start position is re-determined on the speed-changed audio time axis, and the speed-changed audio data is played from the playback start position.

[0112] Among them, the audio playback time axis is the audio playback axis corresponding to the length of the accelerated audio.

[0113] During implementation, if the user needs to adjust the playback progress of the audio data during the process of pre-listening to the audio data, the user can send a playback progress adjustment instruction to the background by triggering operations such as dragging the scrolling direction of the sound wave spectrogram. The electronic device will re-determine the playback start position on the audio timeline after the speed change, and play the audio data after the speed change from the re-determined playback start position.

[0114] In an exemplary embodiment, after manually marking the audio data, the user may choose to restore the audio speed change so that the manual mark position is displayed at the length of the initial sound wave spectrogram. Specifically, the audio data processing method further includes the following steps:

[0115] Step S160, in response to the speed-changing recovery operation on the audio data, displaying the restored sound wave spectrogram in the audio processing interface.

[0116] In implementation, the audio processing interface provides multiple audio playback multiple options, such as Figure 8 As shown, the options include "1", "0.75", "0.5" and "Custom". If the audio data needs to be restored to the original playback speed, the user can trigger the "1" option in the audio playback multiple option to send an audio playback speed change recovery operation instruction to the background. In response to the speed change recovery operation instruction for the audio data, the electronic device displays the restored sound wave spectrogram in the audio processing interface, wherein the restored sound wave spectrogram is displayed with a manual card mark.

[0117] It should be understood that although Figure 1 , Figure 2 , Figure 4 and Figure 7 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 , Figure 2 , Figure 4 and Figure 7At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0118] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0119] Fig. 9 is a block diagram of an audio data processing device according to an exemplary embodiment. Fig. 9 The device 900 includes a spectrum display unit 902 and a manual card point unit 904.

[0120] The spectrogram display unit 902 is configured to perform a speed change processing operation on the audio data and display a target sound wave spectrogram after speed change in the audio processing interface. The target sound wave spectrogram is obtained by stretching the initial sound wave spectrogram according to the target speed change multiple corresponding to the speed change processing operation.

[0121] The manual check point unit 904 is configured to display a manual check point mark on the target sound wave spectrogram in response to a manual check point mark operation on the audio data during the playback of the audio data after the speed change.

[0122] In an exemplary embodiment, the spectrogram display unit 902 includes:

[0123] A determination subunit is configured to perform a speed change processing operation on the audio data, and determine the speed change sound wave spectrogram length corresponding to the audio data after the speed change according to the target speed change multiple corresponding to the speed change processing operation and the initial sound wave spectrogram length corresponding to the audio data;

[0124] The stretching subunit is configured to perform stretching processing on the initial sound wave spectrogram corresponding to the original audio data in the audio processing interface according to the length of the variable speed sound wave spectrogram;

[0125] The display subunit is configured to be executed in the audio processing interface to display the speed-changed target sound wave spectrogram that is stretched to be equal to the length of the speed-changed sound wave spectrogram.

[0126] In an exemplary embodiment, the audio data processing apparatus 900 further includes:

[0127] A processing unit is configured to determine the speed-changed audio duration corresponding to the speed-changed audio data according to the audio duration corresponding to the original audio data and the target speed-changed multiple of the original audio data;

[0128] The display unit is configured to display the audio timeline on the audio processing interface according to the time scale corresponding to the length of the variable-speed audio and the preset time scale interval.

[0129] In an exemplary embodiment, a size line is displayed on the target sound wave spectrogram in the audio processing interface;

[0130] The manual card point unit 904 is configured to execute a manual card point marking operation in response to the audio data, and display a manual card point mark at a position corresponding to the size line on the target sound wave spectrogram.

[0131] In an exemplary embodiment, the audio data processing apparatus 900 further includes:

[0132] The automatic card point unit is configured to perform an automatic card point marking operation in response to the audio data, and display the automatic card point mark of the target audio frame on the speed-changed sound wave spectrogram of the audio processing interface.

[0133] In an exemplary embodiment, the audio data processing apparatus 900 further includes:

[0134] The card point canceling unit is configured to execute a canceling operation in response to the automatic card point mark, and cancel the display of the automatic card point mark on the sound wave spectrogram after the speed change of the audio processing interface.

[0135] In an exemplary embodiment, the audio data processing apparatus 900 further includes:

[0136] a playing unit configured to play the audio data after the speed change in response to a playing operation on the audio data after the speed change;

[0137] The progress adjustment unit is configured to execute a playback progress adjustment operation in response to the speed-changed audio data, redetermine the playback start position on the speed-changed audio timeline, and play the speed-changed audio data from the playback start position; the audio playback timeline is an audio playback axis corresponding to the speed-changed audio duration.

[0138] In an exemplary embodiment, the audio data processing apparatus 900 further includes:

[0139] The variable speed recovery unit is configured to execute a variable speed recovery operation in response to the audio data, and display the restored sound wave spectrogram in the audio processing interface; the restored sound wave spectrogram is displayed with manual point card marks.

[0140] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0141] Fig.10 1 is a block diagram of an electronic device 1000 for audio data processing according to an exemplary embodiment. For example, the electronic device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0142] Reference Fig.10 , the electronic device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0143] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0144] The memory 1004 is configured to store various types of data to support operations on the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, an optical disk, or a graphene memory.

[0145] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 1000.

[0146] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0147] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the electronic device 1000 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting an audio signal.

[0148] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0149] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the electronic device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000, and the sensor assembly 1014 can also detect changes in the position of the electronic device 1000 or components of the electronic device 1000, the presence or absence of contact between the user and the electronic device 1000, the orientation or acceleration / deceleration of the device 1000, and the temperature change of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0150] The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0151] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0152] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, and the above instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0153] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions. The instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method.

[0154] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0155] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0156] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio data processing method, characterized in that: include: In response to the speed change processing operation on the audio data, displaying the target sound wave spectrogram after the speed change in the audio processing interface; The target sound wave spectrogram is obtained by stretching the initial sound wave spectrogram according to the target speed change multiple corresponding to the speed change processing operation; During the playback of the speed-changed audio data, in response to a manual card point marking operation on the audio data, a manual card point mark is displayed on the target sound wave spectrogram.

2. The audio data processing method according to claim 1, characterized in that: In response to the speed change processing operation on the audio data, displaying the speed-changed target sound wave spectrogram in the audio processing interface includes: In response to a speed change processing operation on the audio data, determining a speed change sound wave spectrogram length corresponding to the audio data after the speed change according to a target speed change multiple corresponding to the speed change processing operation and an initial sound wave spectrogram length corresponding to the audio data; According to the length of the speed-varied sound wave spectrogram, the initial sound wave spectrogram corresponding to the original audio data in the audio processing interface is stretched; In the audio processing interface, a target sound wave spectrogram after speed change is displayed after being stretched to a length equal to that of the speed change sound wave spectrogram.

3. The audio data processing method according to claim 2, characterized in that: After displaying the stretched target sound wave spectrogram having a length equal to that of the speed-changed sound wave spectrogram, the method further comprises: In a case where an original manual card point mark is displayed on the initial sound wave spectrogram, the original manual card point mark is displayed on the target sound wave spectrogram.

4. The audio data processing method according to any one of claims 1 to 3, characterized in that: Before displaying the target sound wave spectrogram after speed change in the audio processing interface, the method further includes: Determine the speed-changed audio duration corresponding to the speed-changed audio data according to the audio duration corresponding to the original audio data and the target speed-changed multiple of the original audio data; The audio timeline is displayed on the audio processing interface according to the time scale corresponding to the variable speed audio duration and the preset time scale interval.

5. The audio data processing method according to claim 1, characterized in that: The target sound wave spectrogram in the audio processing interface is provided with a size line; and in response to a manual card point marking operation on the audio data, a manual card point mark is displayed on the speed-changed sound wave spectrogram, including: In response to a manual card point marking operation on the audio data, a manual card point mark is displayed at a position corresponding to the size line on the target sound wave spectrogram.

6. The audio data processing method according to claim 1, characterized in that: The method further comprises: In response to the automatic card point marking operation on the audio data, the automatic card point marking of the target audio frame is displayed on the speed-changed sound wave spectrogram of the audio processing interface.

7. The audio data processing method according to claim 6, characterized in that: After the automatic card point marking operation is performed in response to the audio data, the method further includes: In response to a cancel operation of the automatic card point mark, the automatic card point mark is canceled from being displayed on the speed-changed sound wave spectrogram of the audio processing interface.

8. The audio data processing method according to any one of claims 1 to 2, characterized in that: The method further comprises: In response to a play operation on the speed-changed audio data, play the speed-changed audio data; In response to the playback progress adjustment operation of the speed-changed audio data, the playback start position is re-determined on the speed-changed audio timeline, and the speed-changed audio data is played from the playback start position; the audio timeline is an audio playback axis corresponding to the speed-changed audio duration.

9. The audio data processing method according to claim 1, characterized in that: In response to the speed change processing operation on the audio data, displaying the target sound wave spectrogram after the speed change in the audio processing interface, the method further includes: In response to the speed-changing recovery operation on the audio data, a restored sound wave spectrogram is displayed in the audio processing interface; the manual card point mark is displayed on the restored sound wave spectrogram.

10. An audio data processing device, characterized in that: include: A spectrogram display unit, configured to perform a speed-changing processing operation on the audio data and display a speed-changed target sound wave spectrogram in the audio processing interface; The target sound wave spectrogram is obtained by stretching the initial sound wave spectrogram according to the target speed change multiple corresponding to the speed change processing operation; The manual card point unit is configured to display a manual card point mark on the target sound wave spectrogram in response to a manual card point marking operation on the audio data during the playback of the audio data after the speed change.

11. The audio data processing device according to claim 10, characterized in that: The spectrogram display unit comprises: A determination subunit is configured to perform a speed change processing operation on the audio data, and determine the speed change sound wave spectrogram length corresponding to the audio data after the speed change according to the target speed change multiple corresponding to the speed change processing operation and the initial sound wave spectrogram length corresponding to the audio data; A stretching subunit is configured to perform stretching processing on an initial sound wave spectrogram corresponding to the original audio data in the audio processing interface according to the length of the speed-varied sound wave spectrogram; The display subunit is configured to be executed in the audio processing interface to display the speed-changed target sound wave spectrogram that is stretched to be equal to the length of the speed-changed sound wave spectrogram.

12. The audio data processing device according to claim 11, characterized in that: The audio data processing device further comprises: The mark display unit is configured to display the original manual card point mark on the target sound wave spectrogram when the original manual card point mark is displayed on the initial sound wave spectrogram.

13. The audio data processing device according to any one of claims 10 to 12, characterized in that: The audio data processing device further comprises: A processing unit is configured to determine the speed-changed audio duration corresponding to the speed-changed audio data according to the audio duration corresponding to the original audio data and the target speed-changed multiple of the original audio data; The display unit is configured to display the audio timeline on the audio processing interface according to the time scale corresponding to the length of the variable-speed audio and the preset time scale interval.

14. The audio data processing device according to claim 10, characterized in that: A size line is displayed on the target sound wave spectrogram in the audio processing interface; The manual card point unit is configured to execute a manual card point marking operation in response to the audio data, and display a manual card point mark at a position corresponding to the size line on the target sound wave spectrogram.

15. The audio data processing device according to claim 10, characterized in that: The audio data processing device further comprises: The automatic card point unit is configured to perform an automatic card point marking operation in response to the audio data, and display an automatic card point mark for the target audio frame on the speed-changed sound wave spectrogram of the audio processing interface.

16. The audio data processing device according to claim 15, characterized in that: The audio data processing device further comprises: The card point canceling unit is configured to execute a canceling operation in response to the automatic card point mark, and cancel the display of the automatic card point mark on the speed-changed sound wave spectrogram of the audio processing interface.

17. The audio data processing device according to any one of claims 10 to 11, characterized in that: The audio data processing device further comprises: a playing unit configured to play the audio data after the speed change in response to a playing operation on the audio data after the speed change; The progress adjustment unit is configured to execute a playback progress adjustment operation in response to the speed-changed audio data, redetermine the playback start position on the speed-changed audio timeline, and play the speed-changed audio data from the playback start position; the audio timeline is an audio playback axis corresponding to the speed-changed audio duration.

18. The audio data processing device according to claim 10, characterized in that: The audio data processing device further comprises: The variable speed recovery unit is configured to execute a variable speed recovery operation in response to the audio data, and display a restored sound wave spectrogram in the audio processing interface; the manual mark is displayed on the restored sound wave spectrogram.

19. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the audio data processing method according to any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the audio data processing method according to any one of claims 1 to 9.

21. A computer program product, comprising instructions, characterized in that: When the instruction is executed by a processor of an electronic device, the electronic device is enabled to execute the audio data processing method as described in any one of claims 1 to 9.

Images