Audio data transmission method and electronic device

Abstract

The application provides an audio data transmission method and an electronic device. An audio processor periodically sends a data reading request to an audio data buffer. If the data reading request is a non-first data reading request, the audio data buffer needs to perform data emptying processing before the audio processor reads audio data from the audio data buffer. The audio processor reads the audio data from the audio data buffer only after the data emptying processing of the audio data buffer is completed. In this way, it is ensured that the audio data read by the audio processor from the audio data buffer in each cycle is newly cached audio data, rather than audio data remaining in the audio data buffer in the last cycle, so that noise in the audio data transmission process is avoided, and thus the karaoke effect is improved, and the user experience is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to an audio data transmission method and an electronic device. BACKGROUND

[0002] With the improvement of people's living standards, people's pursuit of leisure and entertainment has gradually improved. Among them, K song gradually becomes one of the important projects of people's leisure and entertainment. In addition to going out for K song entertainment, more and more people choose to sing K songs at home using home electronic devices. The audio playing process in the K song process is usually: using an audio acquisition device to acquire the sound signal input by the user, sending the processed sound signal to a playing device, and playing the processed sound signal by the playing device.

[0003] However, the current audio data transmission process is easy to introduce residual noise audio data. If these residual noise audio data are played together with the user's sound signal, it will cause the played sound to have noise, resulting in poor K song effect and poor user experience. SUMMARY

[0004] The present application provides an audio data transmission method and an electronic device, which are used to solve the problem that the current audio data acquisition process is easy to introduce residual noise audio data. If such residual noise audio data are played together with the user's sound signal, it will cause the played sound to have noise, resulting in poor K song effect and poor user experience.

[0005] In a first aspect, the embodiment provides an electronic device, comprising:

[0006] a sound collector configured to acquire audio data input by a user;

[0007] an audio data buffer configured to continuously buffer the audio data from the sound collector;

[0008] an audio processor configured to perform:

[0009] periodically sending a data reading request to the audio data buffer;

[0010] if the data reading request is a non-first data reading request, receiving the audio data of the current period fed back by the audio data buffer according to the data reading request after the audio data buffer completes data emptying processing, wherein the audio data buffer performs data emptying processing to clear the audio data remaining in the audio data buffer when the audio processor reads the audio data from the audio data buffer in the last period;

[0011] sending the audio data of the current period to an audio player to make the audio player play sound according to the audio data.

[0012] In a second aspect, the embodiments provide an audio data transmission method. The method is applied to an audio processor of an electronic device. The electronic device further comprises a sound collector configured to collect audio data input by a user, and an audio data buffer configured to continuously buffer the audio data from the sound collector. The method comprises:

[0013] periodically sending a data read request to the audio data buffer;

[0014] if the data read request is a non-first data read request, receiving the audio data of a current period fed back by the audio data buffer according to the data read request after the audio data buffer completes data clearing processing, wherein the audio data buffer performs the data clearing processing to clear the audio data remaining in the audio data buffer when the audio data is read from the audio data buffer by the audio processor in a previous period;

[0015] sending the audio data of the current period to an audio player to enable the audio player to play sound according to the audio data.

[0016] The audio data transmission method and the electronic device provided by the embodiments of the present application, the electronic device comprises a sound collector configured to collect audio data input by a user, an audio data buffer configured to continuously buffer the audio data from the sound collector, and an audio processor. The audio processor periodically sends a data read request to the audio data buffer. If the data read request is a non-first data read request, the audio data buffer needs to perform data clearing processing before the audio data is read from the audio data buffer by the audio processor. The audio processor reads the audio data from the audio data buffer only after the data clearing processing of the audio data buffer is completed. The audio data buffer performs the data clearing processing to clear the audio data remaining in the audio data buffer when the audio data is read from the audio data buffer by the audio processor in a previous period. In this way, it is ensured that the audio data read from the audio data buffer by the audio processor in each period is newly buffered audio data, rather than the audio data remaining in the audio data buffer in a previous period, thereby avoiding noise in the audio data transmission process, and further improving the karaoke effect and the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles behind the application.

[0018] Figure 1 Fig. 1 shows an operating scenario between a display device and a control device according to some embodiments;

[0019] Figure 2 A hardware configuration block diagram of the control device 100 is shown according to some embodiments;

[0020] Figure 3 A hardware configuration block diagram of the display device 200 is shown according to some embodiments;

[0021] Figure 4 A software configuration diagram in the display device 200 is shown according to some embodiments;

[0022] Figure 5 A schematic diagram of an audio data transmission system architecture is shown according to some embodiments;

[0023] Figure 6 A schematic diagram of yet another audio data transmission system architecture is shown according to some embodiments;

[0024] Figure 7 A signaling diagram of an audio data transmission process is shown according to some embodiments;

[0025] Figure 8 A signaling diagram of yet another audio data transmission process is shown according to some embodiments;

[0026] Figure 9 A schematic diagram of an audio data buffer emptying principle is shown according to some embodiments;

[0027] Figure 10 A schematic diagram of an audio data packetized transmission principle is shown according to some embodiments;

[0028] Figure 11 A schematic diagram of a user interface provided by the display device 200 is shown according to some embodiments;

[0029] Figure 12 A schematic diagram of yet another user interface provided by the display device 200 is shown according to some embodiments;

[0030] Figure 13 A schematic diagram of yet another user interface provided by the display device 200 is shown according to some embodiments;

[0031] Figure 14 A schematic diagram of yet another user interface provided by the display device 200 is shown according to some embodiments;

[0032] Figure 15 A schematic diagram of yet another user interface provided by the display device 200 is shown according to some embodiments;

[0033] Figure 16Fig. 6 shows yet another user interface diagram provided by the display device 200 according to some embodiments;

[0034] Figure 17 Fig. 6 shows yet another user interface diagram provided by the display device 200 according to some embodiments;

[0035] Figure 18 Fig. 7 shows a flowchart of an audio data transmission method according to some embodiments. DETAILED DESCRIPTION

[0036] In order to make the purposes, technical solutions, and advantages of some embodiments of the present application clearer, the following will describe some embodiments of the present application and corresponding drawings in detail. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application.

[0037] It should be noted that the brief description of the terms in some embodiments of the present application is only for the convenience of understanding the following described embodiments, and is not intended to limit the implementation of some embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0038] The terms "first", "second", "third", and the like in the description and claims of some embodiments of the present application and the above-mentioned drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit the specific order or sequence, unless otherwise specified. It should be understood that the terms used in this way can be interchanged under appropriate circumstances.

[0039] The terms "include" and "have" and any variations thereof are intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not have to be limited to all components clearly listed, but can include other components not clearly listed or inherent to these products or devices.

[0040] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or / and software code capable of performing functions associated with the element.

[0041] The term "remote control" in the present application refers to a component of a display device (such as the display device disclosed in the present application), which can generally control the display device wirelessly within a short distance range. It is generally connected with the display device by infrared and / or radio frequency (RF) signals and / or Bluetooth, and can also include WiFi, wireless USB, Bluetooth, motion sensor, and other functional modules. For example: a handheld touch remote control replaces most of the physical built-in hard keys in a general remote control device with a touch screen user interface.

[0042] The electronic device in the present application can be a display device, or other electronic device with voice assistant function. The following description takes the display device as an example. Figure 1 The schematic diagram of the operation scenario between the display device and the terminal device is provided for some embodiments of the present application. As shown in Figure 1 The user can operate the display device 200 through the mobile terminal 300 and the terminal device 100.

[0043] In some embodiments, the terminal device 100 can be a remote controller. The communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-distance communication methods, etc. The display device 200 is controlled by wireless or other wired ways. The user can input user instructions through the keys, voice input, control panel input, etc. on the remote controller to control the display device 200.

[0044] In some embodiments, the mobile terminal 300 can install software applications on the display device 200, and realize connection communication through network communication protocol, so as to realize one-to-one control operation and data communication. The mobile terminal 300 can also transmit the audio and video content displayed on the mobile terminal 300 to the electronic device 200, so as to realize synchronous display function.

[0045] As shown in Figure 1 It is also shown in that the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 can be allowed to communicate and connect through local area network (LAN), wireless local area network (WLAN) and other networks.

[0046] In addition to providing the function of receiving broadcast television, the display device 200 can also provide the function of smart network television supported by computer, including but not limited to network television, smart television, Internet protocol television (IPTV), etc.

[0047] Figure 2 The schematic diagram of the hardware configuration of the display device 200 is provided for some embodiments of the present application. Figure 1

[0048] In some embodiments, the display device 200 includes at least one of a tuning demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0049] In some embodiments, the detector 230 is used to collect signals of external environment or interaction with the outside.

[0050] ​In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving the image display, for receiving the image signal from the controller output, and for displaying the video content, the image content, and the menu operation interface component, and the user operation UI interface, etc.

[0051] In some embodiments, the communicator 220 is a component for communicating with the external device or the server 400 according to various communication protocol types.

[0052] In some embodiments, the controller 250 controls the operation of the display device and the response to the user's operation by storing various software control programs in the memory. The controller 250 controls the overall operation of the display device 200.

[0053] In some embodiments, the user can input the user command through the graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI).

[0054] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0055] Figure 3 The hardware configuration block diagram of the terminal device provided in some embodiments of the present application is shown in FIG. 2. Figure 1 As shown in FIG. 2, the terminal device 100 includes a controller 111, a communication interface 130, a user input / output interface, a memory, and a power supply. Figure 3

[0056] The terminal device 100 is configured to control the display device 200 and can receive the user's input operation instruction and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, thereby playing a role of an intermediary between the user and the display device 200.

[0057] In some embodiments, the terminal device 100 can be a smart device. For example, the terminal device 100 can install various applications for controlling the display device 200 according to the user's demand.

[0058] In some embodiments, as shown in FIG. 3, the mobile terminal 300 or other smart display device can play a similar function to the terminal device 100 after installing the application for controlling the display device 200. Figure 1

[0059] The controller 111 includes a processor 112 and a RAM 113 and a ROM 114, a communication interface 130, and a communication bus. The controller 111 is used to control the operation and operation of the terminal device 100, and the communication and cooperation between the internal components, and the external and internal data processing functions.

[0060] ​​The communication interface 130, under the control of the controller 111, realizes communication of control signals and data signals with the display device 200. The communication interface 130 can include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, and other near field communication modules.

[0061] The user input / output interface 140, wherein the input interface includes at least one of a microphone 141, a touchpad 142, a sensor 143, a key 144, and other input interfaces.

[0062] In some embodiments, the terminal device 100 includes at least one of the communication interface 130 and the input / output interface 140. The terminal device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC, etc. module, which can encode user input instructions through a WiFi protocol, or a Bluetooth protocol, or an NFC protocol and send them to the display device 200.

[0063] The memory 190 is used to store various running programs, data and applications for driving and controlling the terminal device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0064] The power supply 180 is used to provide operating power support for the elements of the terminal device 100 under the control of the controller.

[0065] Figure 4 The software configuration schematic diagram provided in some embodiments of the present application Figure 1 The software configuration schematic diagram provided in some embodiments of the present application

[0066] In some embodiments, at least one application program is running in the application program layer, which can be a window (Window) program, a system setting program, a clock program, a camera application, etc. provided by the operating system, or an application program developed by a third-party developer.

[0067] The framework layer provides an application programming interface (Aplication Pogramming Interface, API) and a programming framework for the application programs in the application program layer. The application program framework layer includes some pre-defined functions. The application program framework layer is equivalent to a processing center, which decides the actions of the application programs in the application layer.

[0068] AsFigure 4 As shown, in some embodiments of the present application, the application framework layer includes managers, content providers, view systems, and the like.

[0069] In some embodiments, the activity manager is used to manage the life cycle of each application and the general navigation back function.

[0070] In some embodiments, the window manager is used to manage all window programs.

[0071] In some embodiments, the system runtime library layer provides support for the upper layer, i.e., the framework layer. When the framework layer is accessed, the Android operating system runs the C / C++ library contained in the system runtime library layer to realize the functions of the framework layer.

[0072] In some embodiments, the kernel layer is the layer between hardware and software. As shown, the kernel layer at least includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, touch sensor, pressure sensor, etc.), and the like. Figure 4

[0073] In some embodiments, the kernel layer also includes a power supply driving module for power management.

[0074] In some embodiments, Figure 4 The software programs and / or modules corresponding to the software architecture in Figure 2 or Figure 3 The first memory or the second memory shown.

[0075] In some embodiments, the audio playing process in the karaoke process is: using the audio acquisition device to acquire the sound signal input by the user, sending the processed sound signal to the playing device, and playing the processed sound signal by the playing device. For example, Figure 5 As shown in the audio data transmission system architecture diagram, the microphone collects audio data of the user and continuously writes the audio data to the cache buffer (data buffer). At the same time, the audio processing process continuously reads data from the cache buffer. Therefore, the process of the audio processing process reading audio data from the cache buffer and the process of writing audio data to the cache buffer are performed simultaneously.

[0076] ​There is a certain probability of data residual problem when read and write operations are performed in the same cache buffer simultaneously. For example, when the audio data is read from the cache buffer currently, the audio data cached in the last period may be left in the cache buffer. Although the time for reading the audio data is in milliseconds, for K-song, which is an audio sensitive service, if the residual audio data is played together with the voice signal of the user, the sound played will have noise, resulting in poor K-song effect and poor user experience.

[0077] To solve the problems in the above embodiments, the embodiments of the present application provide an audio data transmission method. The audio data transmission method provided by the embodiments of the present application can be applied to Figure 6 the system as shown in Figure 6 The system can include a sound collector, an audio processor, an audio data cache, a K-song application, and an audio player. The sound collector can be a microphone, for example. The audio data cache can be an audio cache buffer, for example. The audio player can be a loudspeaker, for example.

[0078] The sound collector is used to collect audio data input by the user. For example, the user inputs his / her singing voice signal, i.e. audio data, through the microphone during K-song. The audio processor is used to read the audio data from the audio data cache and send the read audio data to the K-song application. The audio data cache is used to cache the audio data input by the user. The K-song application is used to process the audio data input by the user and the accompaniment audio data, for example, to mix the audio data. The audio player is used to play the mixed audio data and the accompaniment audio data. It should be noted that Figure 6 The system as shown in Figure 1 The display device 200 as shown in

[0079] The signaling diagram as shown in Figure 7 The method in the signaling diagram as shown in Figure 7 The system as shown in Figure 6 In the signaling diagram, the sound collector collects audio data input by the user. The audio data cache continuously caches the audio data from the sound collector. Here, continuous means that the audio data cache continuously caches the audio data from the sound collector without interruption. That is, the sound collector continuously writes the audio data to the audio data cache.

[0080] When a karaoke application is initialized, a default audio capture frequency is set. The audio processor reads audio data from the audio data buffer according to the set audio capture frequency. That is, the audio processor periodically reads audio data from the audio data buffer. Specifically, the audio processor can periodically send data read requests to the audio data buffer, and the audio data buffer responds with audio data based on these data read requests.

[0081] Before sending audio data back to the audio processor, the audio data buffer needs to determine whether the current data read request is the first data read request, that is, whether the audio processor is reading data from the audio data buffer for the first time. If the current data read request is the first data read request, that is, the audio processor is reading audio data from the audio data buffer for the first time, then there is no possibility that there is any residual audio data in the audio data buffer. Therefore, the audio processor can directly read the audio data from the audio data buffer without needing to clear the audio data buffer before reading the audio data.

[0082] If the current data read request is not the first data read request, meaning the audio processor is not reading audio data from the audio data buffer for the first time, there may be residual audio data in the audio data buffer. Therefore, it is necessary to first clear the audio data buffer to remove any residual audio data from the previous cycle's audio data read. This ensures that each cycle's audio processor reads newly cached audio data from the audio data buffer, avoiding the reading of residual audio data from the previous cycle in the current cycle.

[0083] The audio processor periodically reads audio data from the audio data buffer and sends the audio data for the current period to the karaoke application. The karaoke application mixes the audio data obtained from the audio processor with the accompaniment audio and outputs it to the audio player, so that the audio player can play sound based on the mixed audio. Since the audio data read by the audio processor from the audio data buffer is free of noise, the audio player will also play sound without noise. Therefore, the audio data transmission method of this application can improve the karaoke effect and enhance the user experience.

[0084] For example, when a user continuously inputs audio data through a microphone, the microphone continuously buffers the audio data into an audio data buffer as it captures the data. This application can utilize ALsa (Advanced Linux Sound Architecture) to provide audio and MIDI (Musical Instrument Digital Interface) support on the Linux operating system. ALSA is a low-level library for audio processing in Android, and karaoke applications use the ALsa library to set the sampling rate for each microphone sound capture.

[0085] The audio processor reads audio data from the audio data buffer every 10ms, meaning it sends a data read request to the buffer every 10ms. If this is the first time the audio processor is sending a read request, the buffer directly sends the audio data back without needing to clear it. If this is not the first time, the buffer is cleared before sending the data back. It's important to note that because the stored data is in 10ms segments, the amount is small. When clearing the buffer, the reading process for the current acquisition cycle is already complete, and the writing process doesn't pause (the time is negligible). Clearing and data transfer are synchronized; therefore, clearing the buffer does not cause any audio delay.

[0086] In some embodiments, the specific process of determining whether the audio processor is sending a data read request to the audio data buffer for the first time may be as follows: A flag is set in the audio data buffer to indicate whether a data read request sent by the audio processor has been received. When the audio data buffer receives the first data read request from the audio processor, the flag is set to 0, indicating that the audio data buffer has not received a data read request from the audio processor. After the audio data buffer receives the first data read request from the audio processor, the flag is updated from 0 to 1, indicating that the audio data buffer has received a data read request from the audio processor.

[0087] After the audio processor sends a data read request to the audio data buffer, if this flag is 0, it indicates that the audio processor is sending a data read request to the audio data buffer for the first time, and the audio data buffer does not need to be cleared. If the flag is 1, it indicates that the audio processor is not sending a data read request to the audio data buffer for the first time, and the audio data buffer needs to be cleared before the audio data is fed back to the audio processor. It should be noted that after a system restart or initialization, this flag returns to 0 from 1. In this way, every time the user sends a data read request to the audio data buffer for the first time, the audio data buffer does not need to be cleared.

[0088] In some embodiments, asynchronous audio transmission can lead to chaotic audio data transmission. For example, the acquired audio data may arrive at the application out of sequence due to asynchronous processing. Therefore, the audio data transmission process in this application is a serial process. That is, the audio transmission process in this application is as follows: after the audio processor reads one cycle of audio data from the audio data buffer, the audio processor sends one cycle of audio data to the karaoke application.

[0089] The conditions for the audio data buffer to clear data include: the audio processor has read one cycle's worth of audio data from the audio data buffer, where one cycle's worth of audio data is the amount of data calculated based on a preset sampling frequency, and the preset sampling frequency is the frequency at which the audio processor sends the data read request to the audio data buffer. Simultaneously, the conditions for sending the audio data of the current cycle to the audio player include: the amount of audio data read from the audio data buffer in the current cycle reaches one cycle's worth of audio data.

[0090] For example, the sampling frequency can be set to 100MHz, resulting in a sampling period of 10ms. This means the audio processor reads audio data from the audio data buffer every 10ms. The frequency setting affects the overall latency of the service. Sampling rate: refers to the number of times the recording device samples the sound signal per second. Sample size: the size of the data storage space for each sample. The formula for calculating the amount of data in one cycle can be: Data volume (bytes / second) = Sampling rate (Hz) * Sample size (bits) * Number of channels / 8.

[0091] After setting the sampling frequency, the amount of data M that the audio processor can read from the audio data buffer in one cycle is calculated using the above formula. If the audio processor has read M amounts of audio data from the audio data buffer in the current cycle, it means that the audio processor has read one cycle of audio data from the audio data buffer, and at this time, the audio data buffer can be triggered to clear the data. Conversely, if the audio processor has not read M amounts of audio data from the audio data buffer in the current cycle, it means that the audio processor has not read one cycle of audio data from the audio data buffer, and at this time, the audio data buffer is not triggered to clear the data. Instead, the audio processor continues to read audio data from the audio data buffer until the amount of audio data read reaches M, and then the current cycle ends.

[0092] Similarly, if the audio processor has read M units of audio data from the audio data buffer in the current cycle, it means that the audio processor has read one cycle of audio data from the audio data buffer. At this time, the audio data of the current cycle can be sent to the audio player. Conversely, if the audio processor has not read M units of audio data from the audio data buffer in the current cycle, it means that the audio processor has not read one cycle of audio data from the audio data buffer. At this time, the audio data is not sent to the audio player, but waits until the audio processor has read M units of audio data from the audio data buffer in the current cycle.

[0093] In some embodiments, if the data read request is not the first data read request, the audio processor is further configured to perform the following before receiving the audio data of the current period from the audio data buffer in accordance with the data request: clearing the audio data in the memory of the audio processor.

[0094] like Figure 8The signaling diagram shown illustrates that, in cases where the data read request is not the first read request, the audio processor clears the audio data transmitted in the previous cycle from its memory before sending the data read request to the audio data buffer. Only then does the audio processor send the data read request to the audio data buffer. The audio data buffer then feeds back the audio data to the audio processor based on the data read request. After feeding back the audio data, the audio data buffer clears its own data while continuing to receive data read requests from the audio processor in the next cycle. This process of clearing the data transmitted by the audio processor before reading it and clearing the audio data in the audio data buffer after reading it ensures that the audio data read from the audio data buffer and sent to the audio player in each cycle is free of contaminant data, further guaranteeing the purity of the data during karaoke audio acquisition.

[0095] In some embodiments, the electronic device further includes a timer that starts counting when the audio data buffer begins buffering audio from the sound collector, thus timestamping the buffered audio data. Specifically, the audio data is in the form of audio frames, and each audio frame is timestamped based on the timing. The process of clearing data from the audio data buffer involves clearing the audio data remaining in the buffer from the previous cycle when the audio processor read the audio data from the buffer, based on the timestamps marked on the audio frames. Thus, if the speed at which the audio data buffer buffers audio data is faster than the speed at which the audio processor reads audio data from the buffer, clearing the audio data from the buffer can be done by only clearing the audio data remaining in the buffer from the previous cycle when the audio processor read the audio data from the buffer, based on the timestamps.

[0096] For example, such as Figure 9 In the schematic diagram illustrating the audio data buffer clearing principle, the audio data buffer caches audio data faster than the audio processor reads audio data from the audio data buffer. The audio data buffer caches 11 audio frames per cycle, while the audio processor reads 10 audio frames per cycle. Within a data transmission cycle, after the audio processor reads audio data from the audio data buffer, it may retain not only the audio frames needed for the current cycle but also those needed for the next cycle.

[0097] At this point, based on the audio frame timestamps, only the audio frames needed for the current cycle should be cleared. The audio data buffer caches 11 audio frames per cycle, with timestamps from 1ms to 11ms. However, the audio processor only needs to read audio frames with timestamps from 1ms to 10ms from the audio data buffer in the current cycle. After the audio processor finishes reading data from the audio data buffer, audio frames with timestamps from 1ms to 10ms may remain. When clearing the audio data buffer, only audio frames with timestamps from 1ms to 10ms are cleared, not those with timestamps from 11ms.

[0098] When transmitting audio in the next cycle, the audio frame with a timestamp of 11ms is still stored in the audio data buffer, and the audio processor can read the audio frame with the timestamp of 11ms from the audio data buffer. This avoids both the residual audio data in the audio data buffer and the loss of audio frames caused by excessive clearing.

[0099] In some embodiments, the audio data buffer continuously buffers the audio data collected by the sound collector. After buffering the audio data from the sound collector, it can also package and label the audio data for one cycle, one label per cycle. The audio data buffer then feeds back the packaged audio data for the current cycle to the audio processor according to the labels. Alternatively, each audio frame in each package can be marked with a corresponding cycle marker.

[0100] For example, such as Figure 10 In the schematic diagram illustrating the audio data packet transmission principle, when the audio data buffer continuously buffers audio data from the sound acquisition unit, it buffers audio frames 1-10 of the first cycle. After packaging these audio frames 1-10, it marks them with the first cycle's marker 1. Audio frame 11, however, is not marked with the first cycle's marker 1. If the audio data buffer buffers audio frame 11 for one cycle, and the audio processor reads audio frame 10 from the buffer for one cycle, then audio frame 11 remains in the buffer. When the audio data buffer performs a clearing process, it only clears the audio frames of the current cycle, i.e., only clears the audio frames marked with 1. Therefore, audio frame 11 will not be cleared during this clearing process of the audio data buffer.

[0101] In some embodiments, such as Figure 11The diagram illustrates another audio data transmission system framework. The electronic device can be equipped with multiple sound collectors, and correspondingly, multiple audio data buffers are configured to buffer the audio data from each sound collector. The audio processor periodically sends data read requests to each of the audio data buffers. Each audio data buffer then returns the audio data collected by its respective sound collector to the audio processor. All audio data buffers undergo the data clearing process described in the previous embodiment. This design effectively avoids the impact of background noise on the karaoke experience, even in scenarios where multiple users simultaneously input audio data.

[0102] In some embodiments, if the audio data buffer clearing process described in the above embodiments is not performed, the presence of residual data can be determined during subsequent mixing based on the timestamps of the audio frames. Specifically, if the timestamps of the audio frames are duplicated, one of the audio frames with the duplicate timestamps is cleared. This way, even without the audio data buffer clearing process described in the above embodiments, the impact of residual audio data on sound quality can be avoided.

[0103] For example, in the first data transmission cycle, the audio processor reads audio frames 1-10 from the audio data buffer. These audio frames have timestamps of 1-10ms respectively. After reading the audio data from the audio data buffer, the audio processor retains audio frames from the first cycle, with a timestamp of 5ms. If the audio data buffer is not cleared before the second data transmission cycle, the audio processor will read these residual audio frames from the buffer during the second data transmission cycle. Finally, the audio data sent by the audio processor to the audio player will contain duplicate audio frames with a timestamp of 5ms. The audio processor can directly clear one of these duplicate timestamp audio frames. This ensures that the final audio data sent to the audio player will not contain residual audio data, preventing noise during playback.

[0104] In some embodiments, the audio processor can re-detect whether there is noise in the audio data before sending it to the audio player. If noise is detected, a prompt message can be displayed on the screen to indicate that there is noise in the currently input audio data, and that continued playback may result in noise and a poor karaoke effect. This detection of noise in the audio data can be performed according to the method described in the above embodiments.

[0105] For example, electronic devices can be Figure 1The display device 200 shown has an audio processor that, after receiving audio data from the audio data buffer and before sending the audio data to the audio player, checks whether there is noise data in the audio data buffer for the current cycle. If there is noise data in the audio data buffer for the current cycle, then... Figure 11 The user interface shown displays the message "The captured sound has noise; would you like to sing again?" Both "OK" and "Cancel" buttons can also be displayed on this message.

[0106] If a user input command to select the "OK" button control is received, the user interface will be redirected to... Figure 12 The user interface shown allows users to... Figure 12 The song will be played again in the user interface shown. If a user input is received to select the "Cancel" button control, the user interface will jump to... Figure 13 The user interface shown allows users to... Figure 13 The user interface shown allows the user to continue singing the song as before. This allows the user to choose whether to restart or continue singing if background noise is detected.

[0107] In some embodiments, since clearing the audio data buffer may consume system computing resources, users can choose not to clear it, indicating that even with background noise, the user can accept it. Users can also choose to clear the buffer, indicating that they cannot accept background noise and have high requirements for singing quality. In this way, by configuring the system to determine whether to clear the audio data buffer during audio data acquisition, the needs of different users can be met. For example, ... Figure 14 The user interface shown allows users to... Figure 14 The user interface shown allows you to configure whether to clear the audio data buffer during audio data acquisition, specifically via a slider button. If the user selects to clear the audio data buffer during audio data acquisition, then... Figure 15 The user interface shown will display a special "Clear Noise" icon in the singing interface. If the user chooses not to clear the audio data buffer during audio data acquisition, the "Clear Noise" icon will not be displayed in the singing interface.

[0108] In some embodiments, a noise threshold can be set. If the noise data in the audio data buffer does not exceed the noise threshold within a period, the audio data buffer is not cleared. If the noise data in the audio data buffer exceeds the noise threshold within a period, the audio data buffer is cleared. This noise threshold can be set based on empirical values. For example, if the noise data in the audio data exceeds a certain value and has a significant impact on the karaoke effect, that value is set as the noise threshold. If the noise data in the audio data buffer does not exceed the noise threshold within a period, the noise data will not have a significant impact on the karaoke effect, so there is no need to clear the audio data buffer. Conversely, if the noise data in the audio data buffer exceeds the noise threshold within a period, the noise data will have a significant impact on the karaoke effect, so the audio data buffer needs to be cleared.

[0109] In some embodiments, the audio processor can re-detect the audio data for noise before sending it to the audio player. If noise is detected, a warning message can be displayed on the screen to indicate that the currently input audio data contains noise, and that continued playback may result in noise and a poor karaoke effect. Figure 11 The user interface shown displays the message "The captured sound has noise; would you like to sing again?". Both "OK" and "Cancel" buttons can be displayed simultaneously on this message. If the user selects the "OK" button, the user interface will then redirect to... Figure 16 The user interface shown displays further prompts: "Sing the whole song again, or sing this segment again?". Both "Whole Song" and "Segment" buttons can be displayed simultaneously on this prompt. If a user input is received to select the "Whole Song" button, the user interface will jump back to... Figure 12 The user interface shown. If a control command to select the "fragment" button control is received from the user, the user interface will jump to... Figure 17 The user interface shown. Figure 17 In the user interface shown, the song also jumps back to the part that needs to be sung again, and the user can... Figure 17 The user interface shown re-records the segment with noise.

[0110] In some embodiments, if the audio data buffer is not cleared, the audio processor can determine whether there is noise in the audio data based on the number of repeated characters after reading the audio data from the audio data buffer, and clear the number of repeated characters, thereby achieving the purpose of clearing noise.

[0111] For example, the number of occurrences of each character in each cycle of a song can be counted in advance. For example, the number of occurrences of "I" is 2 times, the number of occurrences of "of" is 10 times, the number of occurrences of "ancestral" is 3 times, and the number of occurrences of "country" is 3 times. If within a cycle, the number of occurrences of "I" is 3 times, the number of occurrences of "of" is 11 times, the number of occurrences of "ancestral" is 4 times, and the number of occurrences of "country" is 4 times, it may indicate that the audio segment of "my country" has repeated, which is noise data. Therefore, the audio data corresponding to "my country" can be cleared, thereby achieving the effect of clearing noise.

[0112] This application provides an audio data transmission method. Figure 18 It is a flowchart of an audio data transmission method shown according to an exemplary embodiment. This audio data transmission method is applicable to Figure 6 the system shown. As Figure 18 shown, this audio data transmission method may include the following steps:

[0113] In step S101, a data reading request is periodically sent to the audio data buffer. The audio data buffer continuously caches audio data from the sound collector. The sound collector collects audio data input by the user.

[0114] In step S102, if the data reading request is a non-first data reading request, after the audio data buffer completes the data clearing process, the audio data of the current cycle feedback by the audio data buffer according to the data reading request is received, where the audio data buffer performs the data clearing process to clear the audio data remaining in the audio data buffer when the audio processor reads the audio data from the audio data buffer in the previous cycle.

[0115] In step S103, the audio data of the current cycle is sent to the audio player so that the audio player plays sound according to the audio data.

[0116] In some embodiments, if the data reading request is a non-first data reading request, before receiving the audio data of the current cycle feedback by the audio data buffer according to the data request, the method further includes: clearing the audio data in the memory of the audio processor.

[0117] In some embodiments, the method further includes:

[0118] If the data read request is the first data read request, then the audio data of the current period fed back by the audio data buffer according to the data read request is received directly. Before receiving the audio data of the current period fed back by the audio data buffer according to the data read request, the audio data buffer does not perform data clearing.

[0119] Those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Accordingly, aspects of this application can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. All of the above hardware or software may be referred to as a “data block,” “controller,” “engine,” “unit,” “component,” or “system.” Furthermore, aspects of this application may manifest as a computer product located on one or more computer-readable media, the product including computer-readable program code.

[0120] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this application are not intended to limit the order of the processes and methods of this application. Although some currently considered useful embodiments of the invention have been discussed in the foregoing disclosure through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the substance and scope of the embodiments of this application. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely through software solutions, such as installing the described system on existing servers or mobile devices.

[0121] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this disclosure method does not imply that the subject matter of the application requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

Claims

1. An electronic device, characterized in that, include: A sound acquisition device is used to collect audio data input by the user. An audio data buffer is used to continuously buffer the audio data from the sound collector; An audio processor is used to perform: The audio data buffer is periodically sent with a data read request, so that when the data read request is not the first data read request, the audio data buffer can obtain the noise data in the audio data buffer in the current period, and when the noise data exceeds the noise threshold, a data clearing process is performed. The noise data is the audio data remaining in the audio data buffer after the audio processor read and cleared the audio data in the previous period, and the data clearing process is used to clear the noise data. The audio data of the current period is received from the audio data buffer after the data cleaning process is performed, based on the data read request. The audio data for the current period is sent to the audio player so that the audio player plays sound based on the audio data.

2. The electronic device according to claim 1, characterized in that, Before receiving the audio data of the current period from the audio data buffer after performing data cleanup processing according to the data read request, the audio processor is further configured to: clear the audio data in the memory of the audio processor.

3. The electronic device according to claim 1, characterized in that, The audio processor is also used to perform: When the data read request is the first data read request, the audio data of the current period is received from the audio data buffer according to the data read request, wherein the audio data buffer does not perform data clearing before receiving the audio data of the current period from the audio data buffer according to the data read request.

4. The electronic device according to claim 1, characterized in that, The conditions for the audio data buffer to perform data clearing include that the audio processor has read one cycle of audio data from the audio data buffer, wherein one cycle of audio data is the amount of data calculated according to a preset acquisition frequency, and the preset acquisition frequency is the frequency at which the audio processor sends the data read request to the audio data buffer.

5. The electronic device according to claim 4, characterized in that, The conditions for sending the audio data of the current period to the audio player include that the amount of audio data read from the audio data buffer in the current period reaches the amount of audio data for one period.

6. The electronic device according to claim 5, characterized in that, The audio processor is also used to perform: If the amount of audio data read from the audio data buffer in the current period does not reach the amount of audio data for one period, then the audio data for the current period will not be sent to the audio player.

7. The electronic device according to claim 1, characterized in that, The electronic device further includes a timer, which is used to start timing when the audio data buffer begins to buffer the audio data, wherein the buffered audio data is an audio frame, and the audio frame is timestamped according to the timing mark; The specific process of clearing data in the audio data buffer is as follows: Based on the timestamp of the audio frame, the audio data remaining in the audio data buffer when the audio processor read and cleared the audio data in the previous cycle is cleared.

8. An audio data transmission method, characterized in that, The method is applied to an audio processor of an electronic device, the electronic device further comprising a sound acquisition unit for acquiring user-input audio data, and an audio data buffer for continuously buffering the audio data from the sound acquisition unit. The method includes: The audio data buffer is periodically sent with a data read request, so that when the data read request is not the first data read request, the audio data buffer can obtain the noise data in the audio data buffer in the current period, and when the noise data exceeds the noise threshold, a data clearing process is performed. The noise data is the audio data remaining in the audio data buffer after the audio processor read and cleared the audio data in the previous period, and the data clearing process is used to clear the noise data. The audio data of the current period is received from the audio data buffer after the data cleaning process is performed, based on the data read request. The audio data for the current period is sent to the audio player so that the audio player plays sound based on the audio data.

9. The audio data transmission method according to claim 8, characterized in that, Before receiving the audio data of the current period fed back by the audio data buffer after performing data cleanup processing according to the data read request, the method further includes: clearing the audio data in the memory of the audio processor.

10. The audio data transmission method according to claim 8, characterized in that, The method further includes: When the data read request is the first data read request, the audio data of the current period is received from the audio data buffer according to the data read request, wherein the audio data buffer does not perform data clearing before receiving the audio data of the current period from the audio data buffer according to the data read request.

Images