An audio management method and an audio device

Abstract

The application discloses an audio management method applied to an audio device with a touchable display screen, the audio device performing information interaction with a remote device through a terminal device, and the method comprises the following steps: displaying a user interface based on the touchable display screen; if a first user operation on the user interface is detected, acquiring audio data in response to the first user operation; wherein the audio data comprises local audio stream data obtained by the audio device through audio collection on an environment where the audio device is located, and audio stream data sent by the terminal device. The application further discloses an audio device.

Description

Technical Field

[0001] This application relates to the field of computer application technology, and in particular to an audio management method and an audio device. Background Technology

[0002] With the continuous development of smart devices and multimedia technologies, smart devices are increasingly being used in online communication scenarios such as meetings to collect and record communication content and generate multimedia files such as minutes. This allows for review of the communication content after the meeting and effectively prevents the possibility of communication leaks due to recording the content using personal devices. As a result, these smart devices are being used more and more in daily and office life.

[0003] However, how to effectively and quickly manage minutes during the application of the aforementioned smart devices has become a pressing technical problem that needs to be solved.

[0004] Application content

[0005] To address the aforementioned technical problems, this application aims to provide an audio management method and audio device that solves the current issue that smart devices cannot quickly and effectively manage the minutes process. It proposes an audio device capable of efficiently and quickly managing minutes generation. By operating the touchscreen of the smart device, the communication minutes process can be effectively managed and controlled, improving the interaction efficiency between the user and the audio device and ensuring a superior user experience.

[0006] The technical solution of this application is implemented as follows:

[0007] This application provides an audio management method, applied to an audio device with a touch-screen display, wherein the audio device interacts with a remote device via a terminal device, and the method includes:

[0008] The user interface is displayed based on the touch-sensitive display screen;

[0009] If a first user operation is detected in response to the user interface, audio data is acquired in response to the first user operation; wherein, the audio data includes local audio stream data obtained by the audio device from audio acquisition of its environment, and audio stream data sent by the terminal device.

[0010] The method in the above scheme further includes:

[0011] If a second user action is detected on the user interface, in response to the second user action, the display state of the user interface is switched from the first state to the second state;

[0012] Determine the audio segment data acquired within the preset duration of switching to the second state;

[0013] The audio segment data is tagged with tagging information; wherein the tagging information is used to differentiate and display the target information corresponding to the audio segment data.

[0014] In the above scheme, the information content of the target information is determined by the current summary mode of the audio device.

[0015] The method in the above scheme further includes:

[0016] The audio data is subjected to audio-to-text transcription to obtain the transcribed text content;

[0017] The transcribed text content is output in real time to obtain transcribed subtitles.

[0018] The method in the above scheme further includes:

[0019] The transcribed text content is sent to the terminal device, so that the terminal device uploads the transcribed text content to other participating devices and displays the transcribed text content as subtitles on the display screen of the participating devices; wherein, the participating devices and the terminal device are located in the same conference, and the participating devices belong to the remote devices.

[0020] The method in the above scheme further includes:

[0021] If a third user action is detected on the user interface, the acquisition of the audio data is stopped in response to the third user action;

[0022] Generate meeting minutes.

[0023] The method in the above scheme further includes:

[0024] Output meeting minutes, or the meeting minutes and the recorded audio of the meeting; wherein the recorded audio of the meeting is recorded based on the minutes mode of the audio device.

[0025] The method in the above scheme further includes:

[0026] If a fourth user action is detected on the user interface, the minutes configuration page is displayed in response to the fourth user action;

[0027] If a configuration update operation based on the minutes configuration page is detected, the updated minutes configuration is obtained based on the configuration update operation;

[0028] Based on the updated minutes configuration, the acquired real-time audio data is managed.

[0029] The method in the above scheme further includes:

[0030] If a fifth user action is detected on the user interface, in response to the fifth user action, the display state of the user interface is switched from the third state to the fourth state;

[0031] Receive the audio stream data sent by the terminal device;

[0032] The audio stream data is managed based on the minutes configuration.

[0033] The method in the above scheme further includes:

[0034] If a sixth user action is detected for the user interface, in response to the sixth user action, the display state of the user interface is switched from the fourth state to the third state;

[0035] Audio is captured from the current environment to obtain the local audio stream data;

[0036] Receive the audio stream data sent by the terminal device;

[0037] Based on the current minutes configuration, the local audio stream data and the audio stream data are managed.

[0038] The method in the above scheme further includes:

[0039] If a seventh user action is detected on the user interface, in response to the seventh user action, the display state of the user interface is switched from the fifth state to the sixth state;

[0040] Pause the acquisition of the audio data.

[0041] The method in the above scheme further includes:

[0042] If an eighth user action is detected for the user interface, in response to the eighth user action, the display state of the user interface is switched from the sixth state to the fifth state;

[0043] Audio data is acquired starting from the moment the eighth user action is detected.

[0044] The audio data is managed based on the current minutes configuration.

[0045] In the above scheme, the step of switching the display state of the user interface from the sixth state to the fifth state in response to the eighth user operation if an eighth user operation is detected includes:

[0046] If an eighth user action is detected on the user interface, in response to the eighth user action, a first prompt message is displayed to prompt the user to confirm whether to cancel the paused operation;

[0047] If a confirmation operation to cancel the pause operation based on the first prompt information is detected, in response to the confirmation operation, the display state of the user interface is switched from the sixth state to the fifth state.

[0048] In the above scheme, the audio device includes a minutes configuration, which includes at least one of the following configurations: minutes mode configuration, preset duration configuration, and information storage path configuration; the information storage method includes at least local storage and / or cloud storage.

[0049] In the above scheme, the minutes mode includes a first mode and a second mode. The first mode is used to instruct the recording of minutes while simultaneously transcribing them to obtain the recorded audio and the minutes. The second mode is used to instruct the transcribing of minutes only to obtain the minutes.

[0050] This application provides an audio management device, which is applied to an audio device with a touch-screen display. The audio device interacts with a remote device via a terminal device. The device includes: a display unit and an acquisition unit; wherein:

[0051] The display unit is used to display a user interface based on the touch-sensitive display screen;

[0052] The acquisition unit is configured to, if a first user operation is detected on the user interface, acquire audio data based on the current minutes configuration in response to the first user operation; wherein the audio data includes local audio stream data obtained by the audio device from audio acquisition of its environment, and audio stream data sent by the terminal device.

[0053] This application provides an audio device that interacts with a remote device via a terminal device. The device includes at least: a touch-sensitive display screen, a memory, a processor, and a communication bus; wherein:

[0054] The touch-sensitive display screen is at least used to display a user interface;

[0055] The memory is used to store executable instructions;

[0056] The communication bus is used to realize the communication connection between the touch screen, the processor and the memory;

[0057] The processor is configured to execute the audio management program stored in the memory to implement the steps of the audio management method as described in any of the preceding claims.

[0058] This application provides a storage medium storing an audio management program, which, when executed, implements the steps of the audio management method as described in any of the preceding claims.

[0059] This application provides an audio management method and an audio device. The audio device, equipped with a touch-sensitive display screen, interacts with participating devices via a terminal device. It displays a user interface on the touch-sensitive screen. If a first user operation is detected on the user interface, the device responds by acquiring audio data based on the current minutes configuration. This solves the problem that current smart devices cannot quickly and effectively manage the minutes process. It proposes an audio device that can efficiently and quickly manage minutes generation. By operating the touch-sensitive display screen of the smart device, effective management and control of the communication minutes process is achieved, improving the interaction efficiency between the user and the audio device and ensuring a better user experience. Attached Figure Description

[0060] Figure 1 A flowchart illustrating an audio management method provided in an embodiment of this application;

[0061] Figure 2 This application provides a schematic diagram of the user display interface in an embodiment. Figure 1 ;

[0062] Figure 3 This application provides a schematic diagram of the user display interface in an embodiment. Figure 2 ;

[0063] Figure 4 This application provides a schematic diagram of the user display interface in an embodiment. Figure 3 ;

[0064] Figure 5 This application provides a schematic diagram of the user display interface in an embodiment. Figure 4 ;

[0065] Figure 6 This application provides a schematic diagram of the user display interface in an embodiment. Figure 5 ;

[0066] Figure 7 This application provides a schematic diagram of the user display interface in an embodiment. Figure 6 ;

[0067] Figure 8 This application provides a schematic diagram of the user display interface in an embodiment. Figure 7 ;

[0068] Figure 9 This application provides a schematic diagram of the user display interface in an embodiment. Figure 8 ;

[0069] Figure 10 This application provides a schematic diagram of the user display interface in an embodiment. Figure 9 ;

[0070] Figure 11 This application provides a schematic diagram of the user display interface in an embodiment. Figure 10 ;

[0071] Figure 12 This application provides a schematic diagram of the user display interface in an embodiment. Figure 10 one;

[0072] Figure 13 This application provides a schematic diagram of the user display interface in an embodiment. Figure 10 two;

[0073] Figure 14 This is a schematic diagram of the structure of an audio management device provided in an embodiment of this application;

[0074] Figure 15 This is a schematic diagram of the structure of an audio device provided in an embodiment of this application. Detailed Implementation

[0075] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0076] The embodiments of this application provide an audio management method, referring to... Figure 1 As shown, the method is applied to an audio device with a touch-screen display, where the audio device interacts with a remote device via a terminal device. The method includes the following steps:

[0077] Step 101: Display the user interface based on the touch-sensitive display screen.

[0078] In this embodiment, the audio device is a device with at least audio acquisition and audio playback functions, and it has a touch-operable display screen. The terminal device is a device that can install a client corresponding to the audio device, such as a mobile phone, desktop computer, tablet computer, or laptop computer, and it has communication data network functions, and can communicate and connect with the participating devices through the communication data network.

[0079] For example, such as Figure 2The diagram illustrates a system architecture for communication between an audio device with a touchscreen display, a terminal device, and a remote device. The remote device includes at least participating devices, which are other terminal devices located within the same meeting as the terminal device. The remote device may also include a cloud storage device for storing meeting content. During application, the audio device acquires the audio data to be played from the terminal device and sends the locally acquired audio data back to the terminal device. The terminal device decides whether to send the data to the participating devices. In other words, the audio device does not directly communicate with the participating devices, thus reducing the risk of information leakage at the audio device. In some application scenarios, when the audio device interacts with the cloud device through the terminal device, the interaction content is processed locally on the audio device, eliminating the need to upload information to the cloud device, effectively reducing the risk of information such as a user's voiceprint characteristics being leaked in the cloud. The audio device and the terminal device can establish a communication connection via a wired connection or a near-field communication (NFC) method. In some embodiments, during a meeting, the audio device is typically configured not to communicate with other devices besides the terminal device, such as data communication networks or wireless communication networks. This effectively reduces the possibility of the audio device being controlled by malicious actors who could obtain confidential meeting information from it.

[0080] The user interface is an operating interface used to quickly and efficiently manage and control audio devices. Users control the audio devices to manage the acquired audio data by touching the user interface displayed on the touch screen, which improves the interaction efficiency between users and audio devices and ensures the user experience.

[0081] For example, such as Figure 3 This is a schematic diagram of the structure of an audio device provided in an embodiment of this application. The audio device includes: an audio processing module a1, a connector a2, and a support base a3 with a touch-screen display.

[0082] The touchscreen display is mounted on the support base a3 of the audio device. The audio processing module a1 may include a microphone for audio acquisition and a speaker for audio playback, and is mounted on a cylindrical main body component. It should be noted that the microphone and speaker in the audio processing module can also be two independent modules, located at different positions on the same main body component, depending on the specific circumstances.

[0083] The main body of the cylindrical structure housing the audio processing module a1 is connected to the supporting base a3 via a connector a2. This connector a2 can be a retractable component, internally housing a circuit board for connecting the touchscreen display, microphone, and speaker, thus enabling communication between the touchscreen display and the speakers and microphone. The distance between the main body of the cylindrical structure and the supporting base a3 can be adjusted by changing the length of the retractable component, thereby adjusting the overall height of the audio device. In some applications, the bottom of the main body of the cylindrical structure can overlap with the touchscreen display, allowing for quick power-off shutdown of the audio device. It should be noted that... Figure 3 The structure of the audio device shown is only one implementation method and is not specifically limited here. In some implementation scenarios, the touch display screen can also be placed on the outer surface of the audio processing module, which can be determined according to the actual situation.

[0084] Step 102: If a first user action is detected on the user interface, in response to the first user action, acquire audio data.

[0085] The audio data includes local audio stream data obtained by the audio device from its environment, as well as audio stream data sent by the terminal device.

[0086] In this embodiment, the minutes configuration is used to indicate certain configuration parameters required for the current operation of the audio device. The first user operation is implemented by touching the user interface displayed on the touch-screen display. That is, the audio device detects the user operation trajectory on the touch-screen display to obtain the first user operation. The first user operation is the user touching the user interface to instruct the audio device to start the minutes mode, that is, to acquire audio data and manage the audio data accordingly. Thus, after the audio device detects the first user operation, it responds to the first user operation, acquires audio data, and manages the acquired audio data according to the minutes configuration. For example, when the minutes configuration is configured not to collect local audio stream data, the acquired audio data only includes the audio stream data sent by the terminal device. Then, when the minutes configuration is configured to only transcribe and not record, the acquired audio stream data sent by the terminal device is transcribed to obtain the meeting minutes.

[0087] It should be noted that when responding to the first user operation, the user interface state can be updated. For example, if the first user operation is that the user touches the virtual button to open the minutes mode, the state of the virtual button can be switched.

[0088] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0089] If a second user action is detected on the user interface, in response to the second user action, the display state of the user interface is switched from the first state to the second state;

[0090] Confirm the audio segment data acquired within the preset duration of switching to the second state;

[0091] The audio segment data is labeled using tagging information; the tagging information is used to differentiate and display the target information corresponding to the audio segment data.

[0092] In this embodiment, the second user operation is an operation to instruct the audio device to mark the audio data collected at the current moment. The marking information can be explicit distinguishing information, such as text highlighting techniques or identifying time segments of the audio. The preset duration can be an empirical duration set based on extensive experiments or practical experience; in actual application scenarios, it can be set according to specific needs. The target information is correlated with the audio segment data.

[0093] With the minutes mode already enabled, when the audio device detects a second user operation, it responds by switching the user interface display state from a first state where no countdown is displayed to a second state where a countdown display of the processing duration is shown. Then, it determines the audio segment data within a preset duration starting from the second state and marks the audio segment data with pre-set marking information. In this way, the marking information can remind the user to pay attention to this part of the data, indicating that the user wants to focus on a section of the audio segment data, making it easier for the user to find it quickly later.

[0094] Based on the foregoing embodiments, in other embodiments of this application, the information content of the target information is determined by the current summary mode of the audio device.

[0095] In this application embodiment, the minutes mode is usually configured in the minutes configuration to indicate the working mode of the audio device during operation, that is, related to whether or not local audio stream data is recorded.

[0096] In some application scenarios, when the target information is an audio recording file, such as when the target information is displayed on the meeting device, the corresponding marking information can be the time position information marked in the audio recording file, or some prompts, such as red arrows, or prompt dialog boxes to remind users to pay special attention to the content at the time position marked in the audio recording file.

[0097] In some application scenarios, when the target information is meeting log information, that is, text information obtained by real-time conversion of meeting content, marking information such as text highlighting can be used to mark the text content at the marked position. For example, the text at the marked position can be highlighted in yellow to remind the user to pay attention to this part of the content.

[0098] In some application scenarios, when the target information is meeting minutes—that is, a document summarizing the meeting content—if the marked content is recorded in the meeting minutes, this part of the content recorded in the meeting minutes can be marked with identification information, such as highlighting. If the meeting minutes provide a general summary of the marked content, it can be marked at the corresponding location using identification information, such as attaching the corresponding audio content or meeting log information, so that users can quickly access the original content at the marked location and reduce the possibility of missing important information.

[0099] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0100] Audio data is processed into audio-to-text transcription to obtain the transcribed text content;

[0101] The transcribed text content is output in real time to obtain transcribed subtitles.

[0102] In this embodiment, audio-to-text transcription of audio data can be achieved using speech recognition technology, such as a pre-trained speech recognition model. The audio device performs audio-to-text transcription on the audio data locally to obtain transcribed text content, achieving real-time speech transcription. The transcribed text content is then output to obtain transcribed subtitles corresponding to the acquired audio data.

[0103] The audio device outputs transcribed text to obtain transcribed subtitles. This can be achieved by outputting the transcribed text to a touch-sensitive display screen on the audio device, thus displaying the transcribed subtitles in real time; or by outputting the transcribed text to a terminal device, displaying the transcribed text on the terminal device's screen in real time. Furthermore, the terminal device can project the transcribed text in real time via a connected projection device; or by the audio device outputting transcribed text to a terminal device, which then outputs the transcribed text to participating devices, which process the received transcribed text to obtain subtitles. The specific method depends on the actual application scenario and is not limited here. This allows users to obtain the desired real-time speech-to-subtitle conversion through touch operation of the audio device, improving the user experience.

[0104] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0105] The transcribed text content is sent to the terminal device, so that the terminal device can upload the transcribed text content to other participating devices and display the transcribed text content as subtitles on the display screens of the participating devices; wherein, the participating devices and the terminal device are located in the same conference, and the participating devices are the remote devices.

[0106] In this embodiment, the audio device synchronously sends the real-time transcribed text content to the terminal device, which then synchronously transmits the transcribed text content to other participating devices. This allows other participating devices to display the transcribed text content obtained from the audio device's speech recognition in real time, ensuring the consistency of subtitles between participating devices and the terminal device, reducing the consumption of computing resources on participating devices, and further improving the user experience when participating devices cannot transcribe subtitles.

[0107] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0108] If a third-user action is detected on the user interface, audio data acquisition is stopped in response to the third-user action;

[0109] Generate meeting minutes.

[0110] In this embodiment of the application, the third user operation is an operation used to indicate the end of the minutes mode.

[0111] Given that the minutes mode is currently enabled, if the audio device detects a third-user operation on the user interface, in response to the third-user operation, the audio device stops acquiring audio data and generates meeting minutes based on the entire audio data from the start of minutes mode to the receipt of the third-user operation. In some application scenarios, meeting minutes can be generated based on real-time transcribed text content. It should be noted that meeting minutes can be generated from the entire audio data, transcribed text content, or a combination of both. The specific generation method is determined by the meeting minutes generation method. For example, if the meeting minutes are generated by a minutes generation AI model, the specific method is determined by that model.

[0112] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0113] Output meeting minutes, or meeting minutes and recorded audio; the recorded audio is determined by the minutes mode of the audio device.

[0114] In this embodiment, after generating meeting minutes, the audio device can output the minutes. Specifically, the minutes can be displayed on the audio device's touchscreen, which can be achieved through prompts based on user actions. The audio device can also output the generated minutes to a terminal device for display, or the terminal device may display the minutes only upon user input. Furthermore, after receiving the meeting minutes from the audio device, the terminal device can send them to participating devices for display, storage, updating, or other management purposes, depending on the specific circumstances.

[0115] In some implementations, if the audio device's minutes mode indicates that audio recording is required, the audio device, while acquiring and transcribing the audio data, also records the acquired audio data to obtain the meeting recording audio. During this process, when the audio device detects a third user's operation, it generates the meeting minutes and also obtains the meeting recording audio. In this case, whether the audio device outputs the meeting recording audio when outputting the meeting minutes can be determined by the transmission configuration between the audio device and the terminal device. If the audio device outputs both the meeting minutes and the meeting recording audio to the terminal device, and the terminal device then sends the meeting minutes to the server or other participating devices, whether to send the meeting recording audio is determined by the terminal device's transmission configuration; no specific limitation is made here.

[0116] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0117] If a fourth user action is detected on the user interface, the minutes configuration page is displayed in response to the fourth user action.

[0118] If a configuration update operation based on the minutes configuration page is detected, the updated minutes configuration is obtained based on the configuration update operation;

[0119] Based on the updated minutes configuration, manage the acquired real-time audio data.

[0120] In this embodiment, the fourth user operation is a minutes configuration update operation. This process can be performed before or after any of the aforementioned steps, depending on the order of the user's operations, and is not specifically limited here.

[0121] When the audio device detects a fourth user's operation, it responds to the fourth user's operation by switching the display interface to show the minutes configuration interface for minutes configuration. Then, it detects the user's operation on the minutes configuration interface, obtains the configuration update operation, responds to the configuration update operation, determines the configuration parameters updated by the user, and obtains the updated minutes configuration. Then, based on the updated minutes configuration, it manages the audio data acquired after detecting the fourth user's operation, including the generation and output of transcribed text content, the generation and output of meeting minutes, and other operations.

[0122] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0123] If a fifth user action is detected on the user interface, in response to the fifth user action, the display state of the user interface is switched from the third state to the fourth state.

[0124] Receive audio stream data sent by the terminal device;

[0125] Manage audio stream data based on minutes configuration.

[0126] In this embodiment, the fifth user operation is a mute setting operation. When the audio device detects the fifth user operation, the audio device switches the display state of the user interface from the non-mute state (third state) to the mute state (fourth state). At this time, the audio device no longer collects local audio stream data in its environment, but only manages the audio stream data sent by the terminal device based on the minutes configuration.

[0127] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0128] If a sixth user action is detected on the user interface, in response to the sixth user action, the display state of the user interface is switched from the fourth state to the third state.

[0129] The system captures audio from the current environment to obtain local audio stream data.

[0130] Receive audio stream data sent by the terminal device;

[0131] Based on the current minutes configuration, manage local audio stream data and audio stream data.

[0132] In this embodiment of the application, when the current state of the audio device is the fourth state, if the audio device detects a sixth user operation, i.e., an operation to unmute, the audio device responds to the sixth user operation by first switching the display state of the user interface, then resuming the audio acquisition of the local environment to obtain local audio stream data, and receiving audio stream data sent by the terminal device. Then, based on the current minutes configuration, the acquired local audio stream data and audio stream data are managed.

[0133] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0134] If a seventh user action is detected on the user interface, in response to the seventh user action, the display state of the user interface is switched from the fifth state to the sixth state.

[0135] Pause audio data acquisition.

[0136] In this embodiment, the seventh user operation is a pause operation used to indicate the pause of the minutes. When the audio device detects the seventh user operation, it responds by switching the display state of the user interface from the fifth state, i.e., the normal minutes processing state, to the paused state. At the same time, the audio device pauses the acquisition of local audio stream data and audio stream data sent by the terminal device.

[0137] Based on the foregoing embodiments, in other embodiments of this application, the method further includes:

[0138] If an eighth user action is detected on the user interface, in response to the eighth user action, the display state of the user interface is switched from the sixth state to the fifth state.

[0139] Audio data acquisition begins from the moment the eighth user action is detected;

[0140] Based on the current minutes configuration, manage the audio data.

[0141] In this embodiment, the eighth user operation is an operation to continue the minutes processing. When the audio device detects the eighth user operation, the display state of the user interface is switched from the sixth state to the fifth state, and the acquisition of audio data resumes from the moment the eighth user operation is detected. The acquired audio data is determined by the minutes mode configured in the minutes configuration, and may be audio stream data sent by the terminal device, or may include audio stream data sent by the terminal device and local audio stream data obtained by the audio device from audio acquisition of the local environment. Finally, the recovered audio data is managed according to the current minutes configuration.

[0142] Based on the foregoing embodiments, in other embodiments of this application, if an eighth user operation on the user interface is detected, the step of switching the display state of the user interface from the sixth state to the fifth state in response to the eighth user operation can be implemented by the following steps:

[0143] If an eighth user action is detected on the user interface, in response to the eighth user action, a first prompt message is displayed to ask the user to confirm whether to cancel the paused operation;

[0144] If a confirmation operation to cancel the pause operation based on the first prompt information is detected, respond to the confirmation operation and switch the display state of the user interface from the sixth state to the fifth state.

[0145] In this embodiment, when an eighth user operation is detected, to reduce the risk of erroneous operation, a first prompt message is generated in response to the eighth user operation to determine whether it is an erroneous operation. Then, after detecting a user's confirmation operation to cancel the pause operation based on the first prompt message, the confirmation operation is responded to, and the display state of the user interface is switched from the sixth state to the fifth state, thereby resuming audio data acquisition and managing the acquired audio data based on the current minutes configuration. After detecting a user's denial operation to not cancel the pause operation based on the first prompt message, the display state of the user interface is maintained, and the audio data acquisition operation continues to be paused until the eighth user operation or the third user operation is detected again. It should be noted that if the eighth user operation has been detected during the process from the detection of the first user operation to the detection of the third user operation, after the pause operation is performed, when the meeting minutes are generated after the third user operation is detected, the meeting minutes will not include the content from the start of the eighth user operation until the end of the pause operation.

[0146] Based on the foregoing embodiments, in other embodiments of this application, the audio device includes a minutes configuration, which includes at least one of the following configurations: minutes mode configuration, preset duration configuration, and information storage path configuration; the information storage method includes at least local storage and / or cloud storage.

[0147] In this embodiment, the configuration of the information storage path includes storage path information for meeting minutes and recorded audio. Typically, a default path can be configured, which the user can modify as needed. The specific path can be determined based on actual circumstances and is not specifically limited here. The audio device can be configured separately to obtain a minutes configuration, which guides various processes performed by the audio device during the meeting, such as acquiring and storing audio data based on the minutes configuration.

[0148] Based on the foregoing embodiments, in other embodiments of this application, the minutes mode includes a first mode and a second mode. The first mode is used to instruct that minutes be transcribed while recording, so as to obtain the meeting recording audio and the meeting minutes; the second mode is used to instruct that minutes be transcribed only, so as to obtain the meeting minutes.

[0149] Based on the foregoing embodiments, this application provides an audio device with a touch-screen display. Optionally, the audio device decouples various relationships during its implementation. For example, it includes decoupling of transcription configuration, enabling three minutes modes: recording and transcription, recording only, and transcription only. It can also include decoupling of storage methods, allowing for local storage and cloud storage, as well as local storage only and cloud storage only. Furthermore, it can include decoupling of file viewing and management methods, meaning that after generating a minutes file (corresponding to the aforementioned meeting minutes), a shortcut button to automatically open and view the minutes file is set to allow viewing without exiting the current user interface. Users would otherwise need to perform a series of search operations based on their experience to find and view the minutes file.

[0150] In some application scenarios, audio devices have already been activated via a personal computer (PC) client and registered with their identity document (ID). Audio devices support at least two operating modes: recording and transcription mode, and transcription-only mode; their storage modes include at least local storage mode, cloud storage mode, and a combination of local and cloud storage modes.

[0151] For example, such as Figure 4 The image shows a user interface for quickly starting the recording function on the touchscreen display of an audio device. Virtual button A in the user interface is the shortcut key to activate the minutes function. At this time, the user... Figure 4 When virtual button A in the user interface shown is touched, the audio device detects the first user action. At this time, the audio device can switch the user interface to, as shown in the image. Figure 5The user interface shown indicates that the process has entered the note-taking stage. Correspondingly, the audio device's note-taking configuration can be the default configuration, such as the default note-taking mode being recording and transcription mode, with the local storage path configuration set by default for recording and transcription mode; or the default note-taking mode being transcription-only mode, with the local storage path configuration set by default for transcription-only mode. It should be noted that users can modify the storage path configuration information during subsequent operations.

[0152] If a user performs a factory reset on an audio device, the audio device will follow the original default configuration, meaning the system will default to using local storage to store the relevant information.

[0153] In cloud storage mode, audio devices can send transcripts that need to be stored in the cloud to a PC client for cloud backup. In one implementation, the PC client uploads the transcripts to the cloud in real-time and stores them in the storage area corresponding to the cloud ID only after connecting to a Wi-Fi network and binding a cloud ID. If the PC client's cloud backup operation is interrupted, the cloud will mark the backed-up transcripts and automatically upload them again once the communication network between the cloud and the PC client is restored and the transmission channel between them is idle. During this process, the PC client can notify the user with prompts such as "Network error, please check network," "File has been saved offline," or "Network connection restored, files will automatically resume synchronization to the cloud." These prompts can be implemented using toast notifications. It should be noted that the local transcript transcription process on the audio device is not interrupted during the PC client's offline saving process.

[0154] Both the PC client and the audio device support real-time viewing of the translated content obtained from Automatic Speech Recognition (ASR). During viewing, the translated text output by the audio device can be displayed as subtitles. Furthermore, during the ASR process, the audio device can automatically identify the speaker's identity; for example, based on voiceprint characteristics, it can identify the current speaker as Zhang San, and display the subtitle as "Zhang San: The weather is really nice today." During ASR recognition and transcription, the audio device can default to recognizing the user's language. If the user speaks multiple languages, it can transcribe multiple languages. For example, if the detected speech includes Chinese, English, and French, the audio device can automatically switch between different languages ​​such as Chinese, English, and French.

[0155] The microphone state and recording state of the audio device are decoupled. Specifically, this can be achieved as follows: After enabling the minutes function, the audio device detects a touch operation on the mute button, corresponding to the fifth user operation mentioned above, indicating that the user wishes the audio device to not currently capture local audio information. However, this does not affect the audio device's speech recognition of the audio stream information sent by the terminal device, and the minutes processing process for generating minutes files. At this time, the audio device responds to the touch operation on the mute button by switching the mute button's current non-mute state to a state indicating that the audio device is currently in a mute state. For example, if the user... Figure 5 The mute button D in the user interface shown is touched (where, Figure 5 The state of the mute button D indicates that the audio device is currently in a non-mute state and is acquiring audio stream data from the current environment. Afterwards, the audio device control user interface will... Figure 5 Switch to Figure 6 The user interface shown is about to be Figure 5 The user interface shown has the mute button D's state switched to the following: Figure 6 The state of the mute button D shown in the user interface indicates to the user that the audio device is currently muted, and the audio device is no longer collecting audio stream data from the current environment.

[0156] When the audio device is in mute mode, if it detects a user's action on the button to start meeting recording / transcription, for example, if the user... Figure 6 When the mute button D in the user interface is touched, corresponding to the sixth user operation mentioned above, the audio device can display a pop-up prompting the user to turn on the microphone, i.e., prompting the user to confirm whether to turn on the microphone for local audio acquisition. If the user's instruction to confirm turning on the microphone is detected, the microphone is turned on, local audio stream data is acquired, and the mute is released. At this time, for example, the user interface starts from... Figure 6 Switch back Figure 5 The user interface shown, if it detects a user instruction to confirm the current microphone status, keeps the microphone muted, thus maintaining mute. For example, the user interface maintains... Figure 6 The user interface shown remains unchanged.

[0157] After the audio device enables the minutes function, the user can pause recording at any time and resume recording by clicking resume. For example, the audio device... Figure 4 Clicking the virtual button A in the user interface shown corresponds to the first user action mentioned above. Figure 4 The user interface shown can be switched to, for example... Figure 5 The virtual interface shown allows users to click... Figure 5The virtual pause button B in the user interface shown corresponds to the audio device detecting the aforementioned seventh user operation. At this time, the audio device responds to the user's touch operation on the virtual button B, pausing recording, and the audio device controls the display interface to... Figure 5 Switch to Figure 7 The displayed interface shows that after the user clicks virtual button B to pause, virtual button B in the user interface toggles its state, switching to... Figure 7 The status of virtual button B indicates to the user that the audio device is currently in a paused recording state, so that the user is clearly aware that the audio device is currently in a paused recording state. The recording time unit can be: hours, minutes, seconds, etc.

[0158] After the audio device activates the minutes function, users can mark recordings at any time, with no limit on the number of times. The user interface can differentiate between the recording in progress and recording finished states, allowing users to determine whether the marking is complete, for example, through color changes. For instance, one marking process could be: the user clicks... Figure 8 The user interface shown includes a marker button E. When the audio device detects the second user action mentioned earlier, it automatically records the data for 3 seconds starting from the moment the marker is clicked. On the user interface, a countdown timer pops up from the moment the marker is clicked, and the marking operation is completed when the countdown reaches 0. In some application scenarios, when the countdown reaches 0, the audio device can also generate... Figure 9 The user interface shown is for users to make secondary confirmations. If user confirmation is detected... Figure 9 After a confirmation touch operation on the marking button E on the user interface, the audio device completes the marking operation. In the underlying implementation, the content marked by the audio device is displayed on the PC client side based on the time the marking was initiated. When displayed on the PC client, the marked content requires easily identifiable identifiers for feature processing to facilitate rapid recognition. The marked content can be subtitles obtained from speech recognition, recording information, or subtitles only. The number of marks made so far can be displayed on the audio device's touch-sensitive screen. In some application scenarios, such as when the audio device is recording or paused, if the user clicks the marking button, the audio device completes the marking process, and the user is not allowed to cancel the completed marking.

[0159] It should be noted that for audio devices, recording duration, save location, recording method, and real-time ASR can all be displayed in the user interface.

[0160] For microphone / speaker control, users can touch the volume up and down buttons displayed on the user interface. For example, shortcuts to the volume up and down buttons displayed on the user interface could be... Figure 10 As shown, you can adjust the speaker volume and microphone volume as needed.

[0161] The audio device can be set to the maximum allowed recording time, for example, 12 hours. If the recording time exceeds the maximum allowed time, the audio device can directly stop recording by displaying a prompt message that the recording has exceeded 12 hours and automatically end the recording function.

[0162] The on / off status of the microphone on the audio display device can be controlled by the indicator lights on the display interface of the audio device. For example, when the microphone is on, the green light is turned on and keeps the green light in a breathing state; when the microphone is off, the red light is turned on and keeps the red light in a breathing state.

[0163] In some embodiments, when the minutes function is enabled, the touchscreen display cannot be turned off. That is, the user interface is always displayed on the touchscreen display so that the user can quickly and clearly know that the audio device is currently recording by observing the status of the touchscreen display.

[0164] After enabling the minutes feature, users can click the "Recording in Progress" button and then click the "Back" virtual button to exit the minutes feature. At this point, the file will not be saved, and a pop-up window will prompt the user to exit the minutes feature without saving the file.

[0165] After enabling the minutes feature, you can click the virtual "End Minutes" button to exit the minutes feature, as shown below. Figure 5 The virtual button C in the user interface shown is triggered when the audio device detects the aforementioned third user interaction; in this case, the audio device can pop up a pop-up window as follows: Figure 11 The displayed interface prompts the user to choose between ending directly or saving and ending. If the user selects the virtual button to end directly, the note-taking function ends without saving the file. If the user selects the virtual button to save and end, the note-taking function ends after saving the file. Furthermore, after saving the file, a pop-up window may appear... Figure 12 The displayed interface allows users to click... Figure 12 The "Open" button in the displayed interface will take you directly to the recorded file interface for playback. Figure 12 The "Open" button's functionality can be implemented using Snackbar technology. In some application scenarios, after the user touches the virtual button C, the user interface can switch to... Figure 13 The displayed interface requires the user to... Figure 13The audio device only determines that the detected operation is the aforementioned third user operation after the virtual button C shown is pressed and held for a certain period of time, such as 3 seconds (s), in order to prevent user misoperation.

[0166] In some application scenarios, the above file saving formats are exemplified as follows: Minutes files can be saved as txt or srt format. In txt or srt format, the conversation text in the minutes file can include timestamps. Recorded audio can be in wav, mp3, or other formats. Files can be named according to the year, month, day, and recording time. New minutes files can have the following markers: a new minutes file combined with a storage method marker, recording year, month, day, and recording duration.

[0167] In this way, users can directly control the audio device to start meeting recording to obtain high-quality transcripts, or transcripts and recordings, and view the recorded files on the audio device. This effectively improves the interaction efficiency between the user and the audio device and ensures the user experience.

[0168] The audio management method provided in this application involves an audio device that interacts with participating devices via a terminal device. This device displays a user interface on a touch-sensitive screen. If a first user operation is detected on the user interface, the method responds by acquiring audio data based on the current minutes configuration. This solves the problem that current smart devices cannot quickly and effectively manage the minutes process. It proposes an audio device capable of efficiently and quickly managing minutes generation. By operating the touch-sensitive screen of the smart device, effective management and control of the communication minutes process are achieved, improving the interaction efficiency between the user and the audio device and ensuring a better user experience.

[0169] Based on the foregoing embodiments, embodiments of this application provide an audio management device. The method is applied to an audio device with a touch-screen display, wherein the audio device interacts with a remote device via a terminal device. (Refer to...) Figure 14 As shown, the audio management device 2 includes at least: a display unit 21 and an acquisition unit 22; wherein:

[0170] Display unit 21 is used to display a user interface based on a touch-sensitive display screen;

[0171] The acquisition unit 22 is configured to acquire audio data based on the current minutes configuration in response to a first user operation on the user interface if a first user operation is detected; wherein the audio data includes local audio stream data obtained by the audio device from audio acquisition of its environment, and audio stream data sent by the terminal device.

[0172] The audio management device also includes functions for achieving, for example Figure 1 The units and modules implementing the audio management method provided in the corresponding embodiments will not be described in detail here.

[0173] Based on the foregoing embodiments, embodiments of this application provide an audio device that interacts with a remote device via a terminal device. This device can be applied to... Figure 1 In the corresponding embodiment of the audio management method, refer to Figure 8 As shown, the audio device 3 may include: a touch-sensitive display screen 31, a memory 32, a processor 33, and a communication bus 34; wherein:

[0174] A touch-sensitive display screen 31 is used at least to display a user interface;

[0175] Memory 32 is used to store executable instructions;

[0176] The communication bus 34 is used to realize the communication connection between the touch screen 31, the processor 33 and the memory 32;

[0177] Processor 33 is used to execute audio management methods stored in memory, for implementing, etc. Figure 1 The implementation process of the audio management method provided in the corresponding embodiments will not be elaborated here.

[0178] Based on the foregoing embodiments, embodiments of this application provide a computer-readable storage medium, simply referred to as a storage medium, which stores one or more programs that can be executed by one or more processors to implement the reference. Figure 1 The implementation process of the audio management method provided in the corresponding embodiments will not be elaborated here.

[0179] Based on the foregoing embodiments, this application also provides a computer program product, including a computer program that can be executed by the processor 33 of the audio device 3 to complete the implementation steps provided by any of the foregoing methods.

[0180] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0181] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0182] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0183] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0184] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. An audio management method, characterized in that, The method is applied to an audio device with a touch-screen display, wherein the audio device interacts with a remote device via a terminal device, and the method includes: The user interface is displayed based on the touch-sensitive display screen; If a first user operation is detected in response to the user interface, audio data is acquired in response to the first user operation; wherein, the audio data includes local audio stream data obtained by the audio device from audio acquisition of its environment, and audio stream data sent by the terminal device.

2. The method according to claim 1, characterized in that, The method further includes: If a second user action is detected on the user interface, in response to the second user action, the display state of the user interface is switched from the first state to the second state; Determine the audio segment data acquired within a preset time period after switching to the second state; The audio segment data is tagged with tagging information; wherein the tagging information is used to differentiate and display the target information corresponding to the audio segment data.

3. The method according to claim 2, characterized in that, The information content of the target information is determined by the current minutes mode of the audio device.

4. The method according to claim 1, characterized in that, The method further includes: The audio data is subjected to audio-to-text transcription to obtain the transcribed text content; The transcribed text content is output in real time to obtain transcribed subtitles.

5. The method according to claim 4, characterized in that, The method further includes: The transcribed text content is sent to the terminal device, so that the terminal device uploads the transcribed text content to other participating devices and displays the transcribed text content as subtitles on the display screen of the participating devices; wherein, the participating devices and the terminal device are located in the same conference, and the participating devices belong to the remote devices.

6. The method according to claim 1, characterized in that, The method further includes: If a third user action is detected on the user interface, the acquisition of the audio data is stopped in response to the third user action; Generate meeting minutes.

7. The method according to claim 6, characterized in that, The method further includes: Output meeting minutes, or the meeting minutes and the recorded audio of the meeting; wherein the recorded audio of the meeting is recorded based on the minutes mode of the audio device.

8. The method according to any one of claims 1-7, characterized in that, The audio device includes a minutes configuration, which includes at least one of the following configurations: minutes mode configuration, preset duration configuration, and information storage path configuration; the information storage method includes at least local storage and / or cloud storage.

9. The method according to claim 8, characterized in that, The minutes mode includes a first mode and a second mode. The first mode is used to instruct the recording to be transcribed simultaneously, resulting in the meeting recording audio and the meeting minutes. The second mode is used to instruct the transcription to be performed only, resulting in the meeting minutes.

10. An audio device, characterized in that, The audio device interacts with a remote device via a terminal device, and the device includes at least: a touch-sensitive display screen, a memory, a processor, and a communication bus; wherein: The touch-sensitive display screen is at least used to display a user interface; The memory is used to store executable instructions; The communication bus is used to realize the communication connection between the touch screen, the processor and the memory; The processor is configured to execute the audio management program stored in the memory, and implement the steps of the audio management method as described in any one of claims 1 to 9.

Images