Audio labeling method, device and electronic equipment

Abstract

The application provides an audio labeling method and device and electronic equipment; the method comprises the following steps: receiving at least one label corresponding to a first segment in audio; displaying the at least one label corresponding to the first segment in the process that a target user plays the first segment; in response to receiving a first operation of a target label in the at least one label by the target user, obtaining a timestamp associated with the first operation; the timestamp indicates an audio frame currently played by the audio when the first operation is executed; adding a mark at a position corresponding to the timestamp on a progress bar of the audio. Through the application, the process of audio labeling can be simplified, and the participation rate of users can be improved.

Description

Technical Field

[0001] This application relates to audio interaction technology, and more particularly to an audio annotation method, apparatus, and electronic device. Background Technology

[0002] Currently, the content feedback mechanism of audio platforms mainly relies on likes or comments. When users resonate with a segment while listening to long audio (such as podcasts), they need to manually pause playback, jump to the comment section, enter the time and text comment, and then submit it. This process usually takes more than 10 seconds, which seriously disrupts the listening experience in accompanying scenarios such as commuting or exercising, resulting in a user participation rate of less than 5%. Summary of the Invention

[0003] This application provides an audio annotation method, apparatus, and electronic device that can simplify the audio annotation process and improve user participation.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides an audio annotation method executed by a client. The method includes: receiving at least one tag corresponding to a first segment in audio; displaying at least one tag corresponding to the first segment while a target user is playing the first segment; in response to receiving a first operation from the target user on a target tag among the at least one tags, obtaining a timestamp associated with the first operation; the timestamp indicates the audio frame currently being played when the first operation is executed; and adding a mark at the position corresponding to the timestamp on the audio's progress bar.

[0006] In some possible implementations, the method further includes: sending a timestamp to the server for the server to verify the timestamp; and receiving an indication from the server that the timestamp verification was successful.

[0007] In some possible implementations, the method further includes: generating structured data based on a timestamp, the user identifier of the target user, the audio identifier of the audio, and the target tag; and sending the structured data to the server via asynchronous upload.

[0008] In some possible implementations, structured data is sent to the server via asynchronous upload, including: initiating an asynchronous upload request carrying structured data to the server; receiving response information returned by the server; and the response information indicating that the server has successfully accepted the asynchronous upload request.

[0009] In some possible implementations, the method further includes at least one of the following: displaying a target label in response to receiving a click on the label by a target user; and withdrawing the label in response to receiving a second action by the target user on the label.

[0010] This application provides an audio annotation method executed by a server. The method includes: generating at least one tag for a first segment of audio; sending at least one tag to a client; receiving structured data sent by the client via asynchronous upload; the structured data includes a timestamp of a target tag clicked by a target user, an audio identifier of the audio, a user identifier of the target user, and the target tag clicked by the target user; and storing the structured data.

[0011] In some possible implementations, generating at least one tag for a first segment of audio includes: acquiring the audio; extracting text information from the first segment of audio; extracting keywords from the audio based on the text information; and generating at least one tag based on the keywords and the target user's historical tagging habits.

[0012] In some possible implementations, at least one tag is generated based on keywords and the target user's historical tagging habits, including: matching multiple initial tags related to keywords in a preset tag library according to keywords; calculating the weight of each initial tag according to historical tagging habits; generating at least one tag according to the weight of each initial tag; and the weight of at least one tag being greater than a preset threshold.

[0013] In some possible implementations, the method further includes: receiving structured data sent by the client via asynchronous upload; the structured data includes a timestamp of the target user clicking the target tag, an audio identifier of the audio, a user identifier of the target user, and the target tag clicked by the target user; and storing the structured data.

[0014] In some possible implementations, after sending at least one tag to the client, the method further includes: receiving a timestamp from the client indicating that the target user performed a first operation on the target tag; and verifying the timestamp.

[0015] This application provides an audio annotation device applied to a client. The device includes: a first data transceiver module for receiving at least one tag corresponding to a first segment in audio; a display module for displaying at least one tag corresponding to the first segment while a target user is playing the first segment; a first data processing module for obtaining a timestamp associated with the first operation in response to receiving a first operation from the target user on a target tag among the at least one tags; the timestamp indicates the audio frame currently being played when the first operation is executed; and a dotting module for adding a mark at the position corresponding to the timestamp on the audio's progress bar.

[0016] In some possible implementations, the first data transceiver module is also used to send a timestamp to the server for the server to verify the timestamp; and to receive an indication from the server that the timestamp verification was successful.

[0017] In some possible implementations, the first data processing module is further configured to generate structured data based on the timestamp, the user identifier of the target user, the audio identifier of the audio, and the target tag; and the first data sending and receiving module is configured to send the structured data to the server via asynchronous upload.

[0018] In some possible implementations, the first data transceiver module is used to initiate an asynchronous upload request to the server; receive response information returned by the server; the response information indicates that the server has successfully accepted the asynchronous upload request.

[0019] In some possible implementations, the first data processing module is further configured to: display a target label in response to receiving a click on the label by a target user; and withdraw the label in response to receiving a second operation by the target user on the label.

[0020] This application provides an audio annotation device applied to a server. The device includes: a second data processing module for generating at least one tag for a first segment of audio; a second data transceiver module for sending at least one tag to a client; the second data transceiver module is also used to receive structured data sent by the client asynchronously; the structured data includes a timestamp of a target tag clicked by a target user, an audio identifier of the audio, a user identifier of the target user, and the target tag clicked by the target user; and a data storage module for storing the structured data.

[0021] In some possible implementations, the second data processing module is used to acquire audio; extract text information from a first segment of the audio; extract keywords from the audio based on the text information; and generate at least one tag based on the keywords and the target user's historical tagging habits.

[0022] In some possible implementations, the second data processing module is used to match multiple initial tags related to the keywords in a preset tag library; calculate the weight of each initial tag according to historical tagging habits; generate at least one tag according to the weight of each initial tag; and the weight of at least one tag is greater than a preset threshold.

[0023] In some possible implementations, the second data transceiver module is further configured to receive a timestamp sent by the client of the target user performing a first operation on the target tag; the second data processing module is further configured to verify the timestamp.

[0024] This application provides an electronic device, including:

[0025] processor;

[0026] Memory used to store computer programs or instructions;

[0027] The processor executes computer programs or instructions to implement the steps in any of the methods provided in the embodiments of this application.

[0028] This application provides a computer-readable storage medium storing a computer program or instructions thereon, which, when executed by a processor, implements the steps of any method provided in this application.

[0029] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of any method provided in this application.

[0030] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0031] The client receives multiple tags generated by the server for the first segment of audio. While the target user is playing the first segment, the client displays these tags. The target user selects a target tag and performs a first action on it to complete their comment on the first segment of audio. By simplifying the commenting process, a one-step annotation is achieved in "accompanied" scenarios. The target user does not need to pause playback, switch interfaces, or manually input anything; they only need to perform a first action to provide instant feedback on the currently playing audio segment. This lowers the interaction threshold and significantly increases user engagement.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0034] Figure 1 This is a schematic diagram of the architecture of the audio annotation system provided in the embodiments of this application.

[0035] Figure 2 This is a schematic diagram of the terminal structure provided in the embodiments of this application.

[0036] Figure 3 This is a flowchart illustrating the audio annotation method provided in the embodiments of this application. Figure 1 .

[0037] Figure 4 This is a flowchart illustrating the audio annotation method provided in the embodiments of this application. Figure 2 .

[0038] Figure 5This is a schematic diagram of the client interaction interface provided in the embodiments of this application. Figure 1 .

[0039] Figure 6 This is a schematic diagram of the client interaction interface provided in the embodiments of this application. Figure 2 .

[0040] Figure 7 This is a flowchart illustrating the audio annotation method provided in the embodiments of this application. Figure 3 .

[0041] Figure 8 This is a structural block diagram of a client according to an exemplary embodiment.

[0042] Figure 9 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0045] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0046] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0047] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0048] Current audio playback platforms primarily rely on "liking the entire episode" or "leaving comments" for content feedback. When a user listens to a long audio program (such as a 30-minute podcast) and resonates with a particular segment or wants to provide feedback, the client typically performs the following steps: 1. Pause playback: The client receives the user's manual click of the pause button. 2. Navigate to the comment section: The client receives the user's action of exiting the playback interface or swiping down to the comment section. 3. Manual input: The client receives the user's manual input of the time point (e.g., "10:05") and the corresponding text comment (e.g., "This is a great point"). 4. Submit: Clicking the send button completes the feedback. 5. Data storage: The client stores this feedback as an unstructured text comment, only for client display.

[0049] The above feedback methods have the following problems: 1. Coarse annotation granularity, unable to capture instantaneous feedback: Current feedback methods only support whole-episode feedback or minute-level comment positioning. However, user emotions are "instantaneous" (such as a memorable quote), making "second-level" positioning impossible, resulting in the inability to obtain core value fragments of the content. 2. Annotation timing is out of sync with the playback flow, resulting in high operational costs: Annotation requires interrupting the listening process (pause → jump → input), and the operation time is usually greater than 10 seconds. In "accompanying scenarios" such as commuting and exercising, this high-cost operation leads to extremely low user participation rates (less than 5%).

[0050] This application provides an audio annotation method, apparatus, and electronic device, which can simplify the audio annotation process and improve user participation. The following describes exemplary applications of the electronic device provided in this application. The electronic device provided in this application can be implemented as various types of user terminals such as laptops, tablets, desktop computers, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices). Exemplary applications when the electronic device is implemented as a terminal will be described below.

[0051] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the audio annotation system provided in the embodiments of this application. The terminal (terminal 200-1 and terminal 200-2 are shown as examples) connects to the server 400 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0052] Terminal 200 can be used to display an audio playback interface and receive user operations on audio, and display the audio playback application on graphical interface 210 (graphical interfaces 210-1 and 210-2 are shown as examples); server 400 can be used to analyze audio and generate audio-related annotations.

[0053] In some embodiments, server 400 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. Terminal 200 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited thereto. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0054] See Figure 2 , Figure 2 This is a schematic diagram of the terminal structure provided in the embodiments of this application. Figure 2 The terminal 200 shown includes at least one processor 210, a memory 250, at least one network interface 220, and a user interface 230. The various components in the terminal 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 240.

[0055] It should be noted that, in the embodiments of this application, "terminal" may also refer to "client".

[0056] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0057] User interface 230 includes one or more output devices 231 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0058] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices physically located away from the processor 210.

[0059] The memory 250 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 250 described in this application embodiment is intended to include any suitable type of memory.

[0060] In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0061] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0062] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including: Bluetooth, Wi-Fi, and Universal Serial Bus (USB), etc.

[0063] Presentation module 253 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 231 associated with user interface 230 (e.g., a display screen, a speaker, etc.).

[0064] The input processing module 254 is used to detect and translate one or more user inputs or interactions from one or more input devices 232.

[0065] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2An audio annotation device 255 stored in memory 250 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a first data transceiver module 2551, a display module 2552, a first data processing module 2553, and a dot-marking module 2554. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0066] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0067] In some embodiments, the terminal or server can implement the methods provided in the embodiments of this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a local application (APP), that is, a program that needs to be installed in the operating system to run, such as a live streaming APP or an instant messaging APP; it can also be a mini-program, that is, a program that only needs to be downloaded into a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plugin.

[0068] The audio annotation method provided in this application will be described below with reference to exemplary applications and implementations of the terminals provided in the embodiments of this application.

[0069] See Figure 3 , Figure 3 This is a flowchart illustrating the audio annotation method provided in the embodiments of this application. Figure 1 This method is executed by the client (i.e., the terminal) and will combine Figure 3 The steps shown are explained.

[0070] In step 301, at least one tag corresponding to the first segment of the audio is received.

[0071] In some embodiments, before the target user listens to the audio through the client, the client receives at least one tag corresponding to a first segment of the audio. The audio can be divided into multiple audio segments, each corresponding to at least one tag.

[0072] In step 302, while the target user is playing the first segment, at least one tag corresponding to the first segment is displayed.

[0073] In some embodiments, the client plays audio, and when the audio reaches the first segment, a label corresponding to the first segment is displayed on the client's display interface.

[0074] Understandably, during audio playback, the client displays a corresponding tag for each audio segment played. As audio segments switch, the displayed tags also change accordingly.

[0075] In step 303, in response to receiving a first operation from the target user for a target tag among at least one tags, a timestamp associated with the first operation is obtained. The timestamp indicates the audio frame currently playing when the first operation is executed.

[0076] In some embodiments, during the playback of a first segment of audio, the target user performs a first operation on one of at least one of the tags (i.e., the target tag). After performing the first operation, the client obtains the timestamp at the time the first operation was performed. The timestamp refers to the currently playing audio frame when the first operation was performed.

[0077] In one example, the first action may include, but is not limited to, any of the following: clicking, swiping, swiping, dragging, etc.

[0078] In some embodiments, after receiving a first operation performed by the target user on the target tag, the client immediately calls the player kernel interface to capture a timestamp.

[0079] In one example, when the audio plays for 3 minutes, 42 seconds, and 500 milliseconds, the target user clicks the target label, and the client calls the player kernel interface to obtain the timestamp "3:42.5".

[0080] In one example, when the client's operating system is Android, the player kernel interface is MediaPlayer.onProgressChanged; when the client's operating system is iOS, the player kernel interface is AVPlayer.addPeriodicTimeObserver.

[0081] In step 304, a marker is added to the progress bar of the audio at the position corresponding to the timestamp.

[0082] In some embodiments, after obtaining the timestamp, a dot operation (i.e., adding a mark) is performed on the progress bar at the position corresponding to the timestamp, and a dot mark appears on the progress bar after dotting.

[0083] In one example, after the target user clicks on the target label, the client controls the target label on the display screen to blink and controls the vibration settings. For example, the target label blinks twice (with a period of 0.5 seconds) + vibrates slightly for 0.3 seconds.

[0084] In one example, after the target user clicks the target label, the client generates a dot at the corresponding position on the progress bar. The dot can be colored, the diameter of the dot can be set to 6px, and the color can match the color of the target label.

[0085] In this embodiment, by receiving annotations generated for the first segment of audio from the server, and while the first segment is playing, the target user performs a first operation on the target tag to complete the comment operation for the first segment of audio. This simplifies the comment operation from the traditional 5 to 7 steps (pause, jump, input, etc.) to a single click during playback (without pausing), simplifying the audio annotation process and significantly improving user engagement.

[0086] In some possible implementations, after step 303, the client may also perform the following actions: send a timestamp to the server for the server to verify the timestamp; and receive an indication from the server that the timestamp verification was successful.

[0087] In some embodiments, to ensure accurate timestamp positioning, the client can send the timestamp to the server after obtaining it. The server will verify the timestamp against the original timeline of the audio. After the server successfully verifies the timestamp, the client will receive a response message from the server confirming the successful timestamp verification.

[0088] In this embodiment of the application, the timestamp is verified with the server through the kernel port interface, which can improve the accuracy of timestamp positioning. For example, timestamp capture with an accuracy of 0.10 seconds is achieved, and the core segment recognition accuracy is ≥90%.

[0089] In some possible implementations, after receiving the server's indication of successful timestamp verification, the client can also generate structured data based on the timestamp, the target user's user identifier, the audio identifier, and the target tag; and send the structured data to the server via asynchronous upload.

[0090] In some embodiments, after the client confirms that the timestamp location is accurate, the client further obtains the target user's user identifier and the audio identifier of the audio. Then, the timestamp, user identifier, audio identifier, and target tag are encapsulated into structured data, and the structured data is sent to the server asynchronously.

[0091] In one example, the audio identifier is A001, the timestamp is "3:42.5", the user identifier is U00123, and the target tag is "brainstorming". The client associates the timestamp, user identifier, audio identifier, and target tag to generate four-tuple structured data. The execution process is as follows:

[0092] {

[0093] "audioId": "A001",

[0094] "timestamp": "3:42.5",

[0095] "tag": "mind-blowing",

[0096] "userId": "U00123",

[0097] "deviceType": "Android",

[0098] "createTime": "2024-06-10T15:20:30"

[0099] }

[0100] In some embodiments, after the client generates structured data, the structured data is stored in the client's local database to prevent data loss due to network interruption.

[0101] In some embodiments, if the client fails to send structured data to the server, the structured data is stored in the client's local database for a preset time period (e.g., 7 days). After the network is restored (by listening to navigator.onLine), the asynchronous upload is automatically triggered, and the client's display shows "Synchronizing offline annotations". After successful synchronization, a message "n annotations have been synchronized" is displayed.

[0102] In some possible implementations, the above-described asynchronous upload method for sending structured data to the server may include: initiating an asynchronous upload request carrying the structured data to the server; and receiving a response from the server. The response indicates that the server successfully accepted the asynchronous upload request.

[0103] In some embodiments, after generating structured data, the client initiates an asynchronous upload request carrying the structured data to the server. Upon receiving the asynchronous upload request, the server parses it and, upon successful parsing, sends a response message to the client. The client then sends the structured data back to the server based on the received response message.

[0104] In one example, the client initiates an asynchronous HTTP POST request via Axios (URL: / api / annotation / save), with the user token carried in the request header to avoid blocking the interaction; after receiving the request, the server first verifies the validity of the token and then executes the deduplication logic.

[0105] In one example, the client initiates an asynchronous HTTP POST request via Axios, and the execution process is as follows;

[0106] const uploadAnnotation = async (data) => {

[0107] try {

[0108] const response = await axios.post(

[0109] '<https: / / your-server.com / api / annotation / save> ',

[0110] data,

[0111] {

[0112] timeout: 5000,

[0113] retry: 3, / / Retry 3 times, with a 3-second interval.

[0114] retryDelay: 3000,

[0115] headers: { 'Authorization': `Bearer ${localStorage.getItem('token')}`}

[0116] } );

[0118] if (response.data.code === 0) {

[0119] localStorage.removeItem(`anno_${data.audioId}_${data.timestamp}`);

[0120] / / UI Update: Loading animation → Green checkmark (displayed for 2 seconds)

[0121] updateTagStatus(data.timestamp, 'success');

[0122] }

[0123] } catch (err) {

[0124] / / UI prompts: "Retrying..." appears next to the label, and "Network fluctuation, temporarily saved" pops up at the bottom.

[0125] updateTagStatus(data.timestamp, 'retry');

[0126] console.error('Upload failed, already saved:', err);

[0127] }

[0128] };

[0129] / / Execute upload after local feedback (without blocking interaction)

[0130] triggerTagFeedback('Brainstorming');

[0131] uploadAnnotation(annotationData);

[0132] In some embodiments, the process of a client sending structured data to a server via asynchronous upload can be configured with a retry mechanism to retries if the asynchronous upload fails. For example, a 3-retry mechanism (with a 3-second interval) can be set, meaning that if the asynchronous upload fails, it will retry after 3 seconds, and can be retried 3 times.

[0133] In one example, the asynchronous upload method described above can reduce upload latency. For instance, the latency for a single upload can be ≤300 milliseconds, and the server-side verification response time can be ≤100 milliseconds.

[0134] In this embodiment of the application, structured data is sent via asynchronous upload, which can transmit data without interrupting the audio playback process, thus improving the user experience.

[0135] In some possible implementations, the above method may further include at least one of the following: displaying a target label in response to receiving a click on the label by a target user; and withdrawing the label in response to receiving a second action by the target user on the label.

[0136] In some embodiments, after the target user clicks on the target tag on the client's display interface, the client adds a mark (i.e., a dot mark left during the dotting process) at the corresponding position on the audio progress bar. If the target user wants to review the previous tags, they can click on the mark on the progress bar. The client receives the target user's click operation on the mark and displays the target tag corresponding to the mark.

[0137] In some embodiments, if the target user wishes to withdraw the target label, a second operation can be performed on the marker on the progress bar, and the client responds to the target user's second operation by withdrawing the marker on the progress bar.

[0138] In one example, the second action could be a long press. For instance, if the target user long presses a marker on the progress bar within 10 seconds, triggering a recall request, the client confirms the marker is invalid and removes the marker from the progress bar.

[0139] In one example, if you press and hold the progress bar for 10 seconds, the client displays an "×" delete button. After clicking, the client initiates a / api / annotation / delete request, the server marks the data as "invalid", and the interface displays a message "retracted".

[0140] In some possible implementations, the above method may further include: after the target user performs a first operation on the target tag, if the client receives a message within a predetermined time indicating that the target user has performed a third operation on the target tag, then the first operation performed by the target user is determined to be a misclick, and the locally stored timestamp obtained after performing the first operation is deleted. The third operation may include, but is not limited to, any of the following: clicking, swiping, tapping, dragging, etc.

[0141] In one example, after the target user clicks the target label, the target label flashes twice. If the target user then clicks the target label again within 2 seconds, it is identified as a misclick. The client then grays out the target label and deletes the timestamp temporarily stored from the first click.

[0142] In one example, if the target user repeatedly clicks the "Brainstorming" label within 2 seconds, the client's display will show "Label canceled", the label will turn gray, the progress bar markers will disappear, and the temporary data stored in localStorage (i.e., the timestamp corresponding to the "Brainstorming" label) will be deleted.

[0143] This application provides an audio annotation method, see [link to relevant documentation] Figure 4 , Figure 4This is a flowchart illustrating the audio annotation method provided in the embodiments of this application. Figure 2 This method is executed by the server and will combine Figure 4 The steps shown are explained.

[0144] In step 401, at least one tag is generated for the first segment of the audio.

[0145] In some embodiments, before the target user plays the audio, the server analyzes the audio and generates at least one corresponding tag for each audio segment.

[0146] In some possible implementations, step 401 above may include: acquiring audio; extracting text information from a first segment of the audio; extracting keywords from the audio based on the text information; and generating at least one tag based on the keywords and the target user's historical tagging habits.

[0147] In some embodiments, the server acquires audio and extracts text information from each audio segment using automatic speech recognition (ASR) technology. Based on the text information, keywords are extracted from the audio using the TF-IDF (term frequency-inverse document frequency) algorithm. Then, at least one tag is generated based on the keywords and the target user's historical tagging habits.

[0148] In one example, the server extracted the keyword "quantum computing" from the audio and generated the tag "mind-blowing".

[0149] In some possible implementations, generating the at least one tag based on the keywords and the target user's historical tagging habits includes: matching multiple initial tags related to the keywords in a preset tag library; calculating the weight of each initial tag based on historical tagging habits; and generating at least one tag based on the weight of each initial tag. Wherein, the weight of at least one tag is greater than a preset threshold.

[0150] In some embodiments, after the server extracts keywords from the audio, it matches multiple initial tags corresponding to the keywords in a preset tag library. Then, based on the target user's tagging habits, it calculates the weight of each initial tag. For example, if user U00123 frequently clicks on target tags including "brainstorming," then the weight of "brainstorming" in the initial tags is increased by 0.2. After calculating the weight of each initial tag, the initial tags are sorted, and the top-ranked initial tags or at least one initial tag are selected as the final tags displayed on the client.

[0151] In some embodiments, having at least one tag with a weight greater than a preset threshold means that an initial tag with a weight greater than the preset threshold is displayed as a tag on the client.

[0152] In step 402, at least one tag is sent to the client.

[0153] In some embodiments, after the server determines the tag, it sends the tag and the corresponding audio segment to the client. Thus, while the target user is playing an audio segment, the client displays the tag corresponding to the audio segment on the display interface.

[0154] In one example, the server extracts text information from a real-time playback segment (0-10 seconds) using ASR, and extracts the keywords "quantum computing" and "AI breakthrough" using the TF-IDF algorithm. The server matches the keywords to a preset tag library ("quantum computing" can match "mind-blowing" and "knowledge point"), and combines the user U00123's historical annotations (if "mind-blowing" is frequently clicked, the weight of the tag is increased by 0.2). The server generates a tag list: "mind-blowing", "knowledge point", "surprised", "deeply agree", and "conflicting viewpoints". The server sends the tag list to the client, and the client's display interface loads the tags.

[0155] Figure 5 This is a schematic diagram of the client interaction interface provided in the embodiments of this application. Figure 1 See Figure 5 As shown, when a user plays audio in the background, a mini player is displayed on the interactive interface. On the right side of the mini player's control area, a semi-transparent label drawer (60px wide, labels are emoji icons, 80% transparency) is displayed, located within thumb reach (lower right side of the screen). Users can click on the emoji labels to quickly comment on a specific audio segment. It should be noted that the emoji labels can be replaced with text, such as "Mind-blowing," "Knowledge Point," "Surprised," "I totally agree," etc.

[0156] Figure 6 This is a schematic diagram of the client interaction interface provided in the embodiments of this application. Figure 2 See Figure 6 As shown, when the target user plays audio in full screen, an immersive playback page is displayed on the interactive interface. In the immersive playback page, a tab bar (50px high, semi-transparent background) is loaded below the progress bar. The tab bar contains multiple tabs, each 80×36px in size and 20px apart.

[0157] In some possible implementations, during the interaction between the server and the client, the server sends a tag to the client (i.e., step 402). In response to receiving the first operation of the target user on the target tag among the at least one tags, the client obtains the timestamp associated with the first operation (i.e., step 302). Then, the client sends the timestamp to the server, and the server will perform the following steps: receive the timestamp sent by the client on the target user performing the first operation on the target tag; verify the timestamp.

[0158] In some embodiments, the client obtains the timestamp of the target user performing the first operation on the target tag by calling the player kernel interface, and then sends the timestamp to the server. The server verifies the timestamp against the original audio timeline to ensure accurate timestamp positioning.

[0159] In some embodiments, the server-side verification process may include: 1. Checking whether the timestamp is within the range of the original timeline. 2. Achieving time alignment between the client and server through time synchronization (such as Network Time Protocol (NTP) and the Cristian algorithm), verifying the timestamp, so as to accurately associate the timestamp with the server's original timeline.

[0160] In one example, when audio A001 plays to "3 minutes 42.5 seconds", the target user clicks the "Brainstorming" label. UI feedback: the label is pressed (opause 80% → 60%) → flashes twice (purple → white → purple, period 0.5 seconds), a purple dot (6px in diameter) is generated at the "3 minutes 42.5 seconds" mark on the progress bar, and a small spinning loading animation is displayed next to the label. The client calls the `player.getCurrentTimestamp()` interface to obtain a 0.1-second precision timestamp "3:42.5", which is then compared with the original timeline of audio A001 on the server (the server returns frame number 10325 corresponding to "3:42.5", and the timestamp is valid if the time difference between the local and server times is 0.1 seconds ≤ 0.2 seconds).

[0161] In step 403, the server receives structured data sent by the client via asynchronous upload. This structured data includes the timestamp of the target user clicking on at least one target tag, the audio identifier of the audio, the user identifier of the target user, and the target tag clicked by the target user.

[0162] In some embodiments, the server receives structured data sent by the client asynchronously via upload. Then, based on the "audio ID + timestamp + user ID" in the structured data, it queries the server's storage layer database and discards duplicate data within a preset time (e.g., 10 seconds), thus achieving deduplication. In this way, a deduplication accuracy of 100% and a data duplication rate of ≤0.5% can be achieved.

[0163] In this embodiment of the application, the combination of local temporary storage (i.e., client-side temporary data storage), resume transmission after network interruption, and server-side deduplication mechanism can improve the upload success rate (≥99.5%) in mobile network environment and effectively solve the problem of data loss.

[0164] In one example, after receiving structured data via the interface / api / annotation / save, the server verifies the validity of the user's token to exclude illegal requests; and executes deduplication SQL.

[0165] SELECT id FROM annotation

[0166] WHERE audioId='A001' AND timestamp='3:42.5' AND userId='U00123'

[0167] AND create_time >= DATE_SUB(NOW(), INTERVAL 10 SECOND)

[0168] No duplicate records → Execute INSERT to insert into the database and synchronize the Redis cache (SET anno:A001:3:42.5:U00123"Brainstorming" EX 604800);

[0169] The response to the front end (time ≤ 100ms) returns {"code":0,"msg":"marking successful"}.

[0170] In step 404, the structured data is stored.

[0171] Understandably, after receiving structured data and deduplicating it, the server stores the deduplicated structured data in the database.

[0172] In some embodiments, the server includes a processing layer and a storage layer. The processing layer performs deduplication, and the storage layer stores structured data. Upon receiving the structured data, the server stores the complete structured data in the MySQL database of the storage layer. Simultaneously, the Redis instance in the storage layer synchronizes daily hot data within a preset time period (e.g., 2 PM to 4 PM) for rapid retrieval. Hot data refers to the most popular tags of the day (i.e., target tags with high click counts).

[0173] In some possible implementations, after storing structured data and trending data in the storage layer, the structured data and trending data can be used to train models (such as sentiment prosody synthesis models and content recommendation models). The structured data is adapted to the model's input requirements and can be directly used for training without changing the format. This improves model optimization efficiency by ≥25%, significantly enhancing the technical value of the data.

[0174] In one example, taking the emotional prosody synthesis model as a TTS (text-to-speech) model, the server reads audio segments corresponding to specific tags (such as "brainstorming") from Redis as training samples to optimize the speech rate and intonation parameters of the TTS model.

[0175] In one example, the server reads trending tags from Redis daily. Taking "brainstorming" as an example, it extracts the corresponding audio segment (e.g., 3:42.5-3:45.5, 3 seconds long) and uses this audio segment as a training sample. The model parameters are adjusted (the speech rate is increased from 150 words / minute to 180 words / minute, and the peak pitch is increased by 15%) to generate synthesized speech adapted to the "brainstorming" scenario for subsequent science and technology audio creation.

[0176] In one example, the server generates fine-grained interest features based on records of target users clicking on target tags, which are then used for precise fragment-level distribution in the recommendation system.

[0177] In one example, the server adds the interest tag "Tech podcast - Mind-Blowing" (weight 0.8) based on user U00123's historical annotation records. The recommendation system filters audio with "Mind-Blowing" tag ≥ 30% (such as "Tech Curiosities"). The server adds the tag "You may be interested in the 2:10 and 5:30 segments" to the recommendation card (based on similar annotations from other users), thereby improving the segment-level click-through rate.

[0178] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario. Figure 7 This is a flowchart illustrating the audio annotation method provided in the embodiments of this application. Figure 3 See Figure 7 The audio annotation system comprises a client and a server. The server includes a processing layer and a storage layer. In the audio annotation system, the client is responsible for displaying tags and responding to user interactions, the processing layer is responsible for verifying timestamps, deduplicating data, and asynchronous transmission, and the storage layer is responsible for persistently storing data and providing training samples to the algorithm model.

[0179] See one example. Figure 7 As shown, audio annotation methods include:

[0180] In step 701, the server-side processing layer analyzes the audio and extracts keywords from it.

[0181] In step 702, the processing layer generates multiple tags based on keywords and the target user's historical tagging habits.

[0182] In step 703, the processing layer sends the tag to the client.

[0183] In step 704, the client receives the first operation performed by the target user on the target tag.

[0184] In step 705, the client sends the timestamp corresponding to the first operation to the processing layer.

[0185] In step 706, the processing layer verifies the timestamp.

[0186] In step 707, the verification result is returned to the client.

[0187] In step 708, if the verification is successful, the client generates structured data based on the target user's user identifier, the audio identifier of the audio, the timestamp, and the target tag.

[0188] In step 709, the client sends structured data to the processing layer via asynchronous upload.

[0189] In step 710, the processing layer queries the MySQL database in the storage layer, discards duplicate data within 10 seconds, and stores the deduplicated structured data in the storage layer.

[0190] In step 711, the storage layer stores structured data in MySQL and hot data of the day in Redis.

[0191] This completes the above audio annotation method.

[0192] In this embodiment of the application, the server-side storage layer is divided into two parts (i.e., MySQL and Redis), which store structured data and hot data respectively. In this way, during the model training process, structured data can be used for long-term analysis, while hot data of the day can be used for quick model calls to reduce the pressure on MySQL.

[0193] In this embodiment, the client receives multiple tags generated by the server for the first segment of audio. While the target user is playing the first segment, the client displays these tags. The target user selects a target tag and performs a first operation on it to complete the commentary on the first segment of audio. By simplifying the commentary process, one-step annotation is achieved in "accompanied" scenarios. The target user does not need to pause playback or switch interfaces, nor does they need to manually input anything; they only need to perform a first operation to provide instant feedback on the currently playing audio segment. This lowers the interaction threshold and significantly increases user engagement.

[0194] The following description continues to illustrate the exemplary structure of the audio annotation device 255 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the audio annotation device 255 in the memory 250 may include: a first data transceiver module 2551, used to receive at least one tag corresponding to a first segment in the audio; a display module 2552, used to display at least one tag corresponding to the first segment during the playback of the first segment by the target user; a first data processing module 2553, used to obtain a timestamp associated with the first operation in response to receiving a first operation from the target user on a target tag among the at least one tags; the timestamp indicates the audio frame currently being played when the first operation is executed; and a dot-marking module 2554, used to add a mark on the progress bar of the audio at the position corresponding to the timestamp.

[0195] In some possible implementations, the first data transceiver module 2551 is also used to send a timestamp to the server for the server to verify the timestamp; and to receive an indication from the server that the timestamp verification was successful.

[0196] In some possible implementations, the first data processing module 2553 is further configured to generate structured data based on the timestamp, the user identifier of the target user, the audio identifier of the audio, and the target tag; the first data transceiver module 2551 is configured to send the structured data to the server via asynchronous upload.

[0197] In some possible implementations, the first data transceiver module 2551 is used to initiate an asynchronous upload request carrying structured data to the server; receive response information returned by the server; the response information indicates that the server has successfully accepted the asynchronous upload request.

[0198] In some possible implementations, the first data processing module 2553 is further configured to: display a target label in response to receiving a click on the label by a target user; and withdraw the label in response to receiving a second operation by the target user on the label.

[0199] This application also provides an audio annotation device applied to a server. The device includes: a second data processing module for generating at least one tag for a first segment of audio; a second data transceiver module for sending at least one tag to a client; the second data transceiver module is also used to receive structured data sent by the client asynchronously; the structured data includes a timestamp of a target tag clicked by a target user, an audio identifier of the audio, a user identifier of the target user, and the target tag clicked by the target user; and a data storage module for storing the structured data.

[0200] In some possible implementations, the second data processing module is used to acquire audio; extract text information from a first segment of the audio; extract keywords from the audio based on the text information; and generate at least one tag based on the keywords and the target user's historical tagging habits.

[0201] In some possible implementations, the second data processing module is used to match multiple initial tags related to the keywords in a preset tag library; calculate the weight of each initial tag according to historical tagging habits; generate at least one tag according to the weight of each initial tag; and the weight of at least one tag is greater than a preset threshold.

[0202] In some possible implementations, the second data transceiver module is further configured to receive a timestamp sent by the client of the target user performing a first operation on the target tag; the second data processing module is further configured to verify the timestamp.

[0203] This application provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or executable instructions from the computer-readable storage medium and executes the computer program or executable instructions, causing the electronic device to perform the method described in this application.

[0204] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions. When the computer program or executable instructions are executed by a processor, the processor will execute the audio annotation method provided in this application. For example, ... Figure 7 The audio annotation method is shown.

[0205] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0206] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0207] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0208] As an example, computer-executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0209] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

[0210] Figure 8 This is a structural block diagram of a client according to an exemplary embodiment. For example, the client 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0211] Reference Figure 8 The client 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output (I / O) interface 812, sensor component 814, and communication component 816.

[0212] Processing component 802 typically controls the overall operation of client 800, such as operations associated with at least one of display, telephone call, data communication, camera operation, and recording operation. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0213] Memory 804 is configured to store various types of data to support operation on client 800. Examples of such data include at least one of the following: instructions for any application or method operating on client 800, contact data, phonebook data, messages, pictures, and videos. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0214] Power supply component 806 provides power to various components of client 800. Power supply component 806 may include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to client 800.

[0215] Multimedia component 808 includes a screen that provides an output interface between client 800 and user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When client 800 is in an operating mode, such as a shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0216] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when client 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0217] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, and buttons. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0218] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of client 800. For example, sensor assembly 814 may detect the on / off state of client 800, the relative positioning of components such as the display and keypad of client 800, changes in position of client 800 or one of its components, the presence or absence of user contact with client 800, the orientation or acceleration / deceleration of client 800, and temperature changes of client 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include, but is not limited to, at least one of the following: an accelerometer, a gyroscope, a magnetometer, a pressure sensor, and a temperature sensor.

[0219] Communication component 816 is configured to facilitate wired or wireless communication between client 800 and other devices. Client 800 can access wireless networks based on communication standards, such as Wi-Fi, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0220] In an exemplary embodiment, the client 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0221] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 804 including executable instructions or a computer program, which can be executed by a processor 820 of a client 800 to perform the above-described method. For example, a non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0222] Figure 9 This is a block diagram illustrating a server according to an exemplary embodiment. For example, server 900 can be provided as a server. (See also...) Figure 9 The server 900 includes a processing component 922, which further includes one or more processors, and memory resources represented by memory 932 for storing instructions, such as applications, that can be executed by the processing component 922. The applications stored in memory 932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 922 is configured to execute instructions to perform the aforementioned audio annotation method. For example, the method includes: generating at least one tag for a first segment of audio; and sending at least one tag to a client.

[0223] The server 900 may also include a power supply component 926 configured to perform power management for the server 900, a wired or wireless network interface 950 configured to connect the server 900 to a network, and an input / output (I / O) interface 958. The server 900 can operate an operating system stored in memory 932, such as Windows Server™, Mac OSX™, Unix™, Linux™, FreeBSD™, or similar.

Claims

1. An audio annotation method, characterized in that, The method is executed by the client, and the method includes: Receive at least one tag corresponding to the first segment of the audio; During the playback of the first segment by the target user, at least one tag corresponding to the first segment is displayed; In response to receiving a first operation from the target user for a target tag among the at least one tags, a timestamp associated with the first operation is obtained; the timestamp indicates the audio frame currently being played when the first operation is executed. Add a marker to the progress bar of the audio at the position corresponding to the timestamp.

2. The method according to claim 1, characterized in that, The method further includes: Structured data is generated based on the timestamp, the user identifier of the target user, the audio identifier of the audio, and the target tag; The structured data is sent to the server via asynchronous upload.

3. The method according to claim 2, characterized in that, The step of sending the structured data to the server via asynchronous upload includes: Initiate an asynchronous upload request carrying the structured data to the server; Receive the response information returned by the server; the response information indicates that the server has successfully accepted the asynchronous upload request.

4. An audio annotation method, characterized in that, The method is executed by the server, and the method includes: Generate at least one tag for the first segment of the audio; Send at least one of the tags to the client; The system receives structured data sent by the client via asynchronous upload; the structured data includes the timestamp of the target user clicking on the target tag in the at least one tag, the audio identifier of the audio, the user identifier of the target user, and the target tag clicked by the target user; Store the structured data.

5. The method according to claim 4, characterized in that, The process of generating at least one tag for the first segment of the audio includes: Obtain the audio; Extract the text information of the first segment from the audio; Based on the text information, extract keywords from the audio; The at least one tag is generated based on the keywords and the target user's historical tagging habits.

6. The method according to claim 5, characterized in that, The step of generating at least one tag based on the keywords and the target user's historical tagging habits includes: Based on the keyword, match multiple initial tags related to the keyword in a preset tag library; Calculate the weight of each initial label based on the historical labeling habits; At least one label is generated based on the weight of each initial label; the weight of the at least one label is greater than a preset threshold.

7. The method according to claim 4, characterized in that, After sending the at least one tag to the client, the method further includes: Receive the timestamp sent by the client showing the target user performing a first operation on the target tag; Verify the timestamp.

8. An audio annotation device, characterized in that, Applied to a client, the device includes: The first data transceiver module is used to receive at least one tag corresponding to the first segment of the audio; The display module is used to display at least one tag corresponding to the first segment during the playback of the first segment; A first data processing module is configured to, in response to receiving a first operation from a target user on a target tag among the at least one tags, obtain a timestamp associated with the first operation; the timestamp indicates the audio frame currently being played when the first operation is executed; The dot module is used to add markers to the progress bar of the audio at the positions corresponding to the timestamps.

9. An audio annotation device, characterized in that, Applied to the server side, the device includes: The second data processing module is used to generate at least one tag for the first segment of the audio. The second data transceiver module is used to send the at least one tag to the client; The second data transceiver module is further configured to receive structured data sent by the client via asynchronous upload; the structured data includes the timestamp of the target user clicking on the target tag in the at least one tag, the audio identifier of the audio, the user identifier of the target user, and the target tag clicked by the target user; A data storage module is used to store the structured data.

10. An electronic device, characterized in that, include: processor; Memory used to store computer programs or instructions; The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 3 or claims 4 to 7.

Images