Audio processing method, communication device, computer equipment and readable storage medium

Abstract

The invention provides an audio processing method, a communication device, computer equipment and a readable storage medium, the method is applied to Internet of Things equipment, the Internet of Things equipment is connected with a communication module, the method comprises the following steps: sending a first instruction to the communication module, the first instruction comprising a to-be-converted first text; a first response from the communication module is received, the first response comprises first audio, and the first audio is audio converted based on the first text; performing audio encoding and decoding processing on the first audio to obtain a second audio; and playing the second audio in response to the second audio meeting the playing condition. According to the invention, the cost for realizing the voice playing function of the Internet of Things equipment can be reduced.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an audio processing method, a communication device, a computer equipment, and a readable storage medium. Background Art

[0002] Currently, when IoT devices expand their voice playback functions, they usually connect a codec (coder-decoder) device to the communication module of the IoT host device (referred to as the host). The codec device is responsible for encoding and decoding the audio signal and converting it into a playable digital signal to achieve voice output. In particular, in application scenarios where text needs to be converted into voice, the host often relies on a text-to-speech (TTS) library to first convert text information into audio data, and then encode and decode the audio data through the codec device and play it.

[0003] However, this technical implementation has significant defects: on the one hand, purchasing the TTS library will increase the software configuration cost of the host; on the other hand, the external codec device will increase the hardware cost. Therefore, how to effectively reduce the overall cost of expanding the voice playback function of IoT devices has become a key issue that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide an audio processing method, a communication device, a computer device, and a readable storage medium, which can reduce the cost of expanding the voice playback function of an Internet of Things device.

[0005] In a first aspect, an embodiment of the present application provides an audio processing method, which is applied to an Internet of Things device, and the Internet of Things device is connected to a communication module; the method includes: sending a first instruction to the communication module, the first instruction includes a first text to be converted; receiving a first response from the communication module, the first response includes a first audio, and the first audio is an audio converted based on the first text; performing audio codec processing on the first audio to obtain a second audio; in response to the second audio meeting a playback condition, playing the second audio.

[0006] It can be seen that the IoT device can realize the text-to-speech function through the communication module to obtain the first audio. Through the audio codec processing process, the second audio can be obtained, and the second audio is played when the second audio meets the playback condition. In this way, the IoT device does not need to have the text-to-speech capability, which can reduce the cost of expanding the voice playback function of the IoT device.

[0007] In a possible implementation, in response to the second audio satisfying the play condition, playing the second audio may include: in response to the priority of the second audio being higher than the priority of the audio to be played in the IoT device, playing the second audio.

[0008] In this way, IoT devices can manage the playback order of multiple audios according to priority, thereby achieving audio conflict management, ensuring smooth audio playback and optimizing user experience.

[0009] In a possible implementation, the method may further include: in response to a priority of the second audio being lower than a priority of the audio to be played, discarding the second audio.

[0010] In a possible implementation, in response to the second audio satisfying the play condition, playing the second audio includes: in response to obtaining mixed audio integrated into the second audio, playing the mixed audio.

[0011] In this way, a richer and more three-dimensional sound effect can be created, enhancing the audio listening experience.

[0012] In a possible implementation, the method may further include: determining a first port and a second port through a port enumeration process with a communication module, wherein the first port is used to send a first instruction, and the second port is used to receive a first response.

[0013] In a possible implementation manner, the method may further include: generating a first text based on the application notification information and / or the received input information.

[0014] In a second aspect, an embodiment of the present application provides another audio processing method, which is applied to a communication module, which is connected to an Internet of Things device, and the communication module includes a text-to-speech library; the method includes: receiving a first instruction from the Internet of Things device, the first instruction including a first text to be converted; converting the first text into a first audio through the text-to-speech library; and sending a first response to the Internet of Things device, the first response including the first audio.

[0015] The communication module can convert the first text into the first audio through the text-to-speech library, and return the first audio to the IoT device. In this way, the communication module does not need to have the ability to play audio, and can send the audio to the IoT device, which will perform further processing, such as audio encoding and decoding, and can reduce the cost of configuring the communication module.

[0016] In a possible implementation, the method may further include: determining a first port and a second port through a port enumeration process with the IoT device, wherein the first port is used to receive a first instruction, and the second port is used to send a first response.

[0017] In a third aspect, an embodiment of the present application provides a communication device, which is connected to a communication module, and further includes a transceiver unit and a processing unit. The transceiver unit is used to send a first instruction to the communication module, the first instruction includes a first text to be converted; receive a first response from the communication module, the first response includes a first audio, and the first audio is an audio converted based on the first text. The processing unit is used to perform audio codec processing on the first audio to obtain a second audio; in response to the second audio meeting a playback condition, play the second audio.

[0018] In a possible implementation, the processing unit is further configured to play the second audio in response to a priority of the second audio being higher than a priority of the audio to be played in the IoT device.

[0019] In a possible implementation, the processing unit is further configured to discard the second audio in response to a priority level of the second audio being lower than a priority level of the audio to be played.

[0020] In a possible implementation, the processing unit is further configured to, in response to acquiring the mixed audio integrated with the second audio, play the mixed audio.

[0021] In a possible implementation, the processing unit is further used to determine the first port and the second port through a port enumeration process with the communication module, the first port is used to send the first instruction, and the second port is used to receive the first response.

[0022] In a possible implementation manner, the processing unit is further configured to generate the first text based on the application notification information and / or the received input information.

[0023] In a fourth aspect, an embodiment of the present application provides another communication device, which is connected to an Internet of Things device, and includes a text-to-speech library. The communication device also includes a transceiver unit and a processing unit. The transceiver unit is used to receive a first instruction from the Internet of Things device, and the first instruction includes a first text to be converted. The processing unit is used to convert the first text into a first audio through the text-to-speech library. The transceiver unit is also used to send a first response to the Internet of Things device, and the first response includes the first audio.

[0024] In a possible implementation, the processing unit is further used to determine a first port and a second port through a port enumeration process with the IoT device, where the first port is used to receive a first instruction and the second port is used to send a first response.

[0025] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing programs; a processor for executing the programs stored in the memory, and when the programs are executed by the processor, the processor executes a method as in the first aspect and any possible implementation of the first aspect, or executes a method as in the second aspect and any possible implementation of the second aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a computer storage medium, in which a computer program is stored. The computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the method in the first aspect and any possible implementation of the first aspect, or executes the method in the second aspect and any possible implementation of the second aspect.

[0027] In the seventh aspect, an embodiment of the present application provides a computer program product, which includes: instructions or computer programs; when the instructions or the computer programs are executed, the method in the first aspect and any possible implementation method of the first aspect is implemented, or the method in the second aspect and any possible implementation method of the second aspect is implemented.

[0028] In an eighth aspect, an embodiment of the present application provides a chip, the chip includes a processor, the processor is used to execute instructions, when the processor executes the instructions, the chip executes the method in the first aspect and any possible implementation of the first aspect, or executes the method in the second aspect and any possible implementation of the second aspect. Optionally, the chip also includes an input / output interface, the input / output interface is used to receive or send signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0030] Figure 2 A flowchart of an audio processing method provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of another system architecture provided for an embodiment of the present application;

[0032] Figure 4 A flowchart of another audio processing method provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the structure of a communication device provided in an embodiment of the present application;

[0034] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0036] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0037] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0038] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0039] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0040] First, some terms in this application are explained to facilitate understanding by those skilled in the art.

[0041] 1. IoT devices

[0042] IoT devices refer to devices that can communicate and interact through the Internet. IoT devices can include sensor devices (such as temperature sensors, humidity sensors, etc.), actuator devices (such as smart switches, smart door locks, etc.), communication devices (such as wireless modules, IoT gateways, etc.) and embedded system devices (such as smart TVs, smart refrigerators, etc.).

[0043] The IoT host device, referred to as the host, is the core device in the IoT system. The host usually has strong computing and storage capabilities and is responsible for managing and controlling other devices in the IoT network. The host can be a smart router, smart home control center, industrial IoT gateway, etc. The host can manage the entire IoT system through communication with other IoT devices.

[0044] The IoT device may have a built-in communication module, or provide an interface such as a universal serial bus (USB) interface to connect to the communication module. The IoT device can access the network through the communication module and realize wireless data transmission.

[0045] 2. Communication module

[0046] A communication module is a communication device that provides a standard interface and supports various communication protocols. It is widely used in smart homes, smart security, logistics and transportation, remote sensing control and other fields.

[0047] The communication module involved in the embodiments of the present application is introduced from the following aspects.

[0048] (1) Communication module type

[0049] From the technical type of communication modules, communication modules can be divided into cellular communication modules and non-cellular communication modules. Among them, cellular communication modules are modules based on cellular mobile communication technology, such as communication modules based on the 4th generation mobile communication technology (4G), 5th generation mobile communication (5G), 6th generation mobile communication (6G) systems and narrow band Internet of Things (NB-IoT).

[0050] The standard interfaces provided by the communication module include standard serial port, Ethernet interface and universal serial bus (USB) interface, etc. Through the standard interface, the communication module can be connected to a variety of devices such as the host, sensor and actuator to achieve data transmission and control command sending.

[0051] (3) Virtual port of communication module

[0052] The communication module can be inserted into the host through its USB interface, and can be identified by the host as a USB device, and perform port enumeration with the host to enumerate one or more virtual ports. Each type of virtual port can perform different functions.

[0053] Among them, the port enumeration process between the host and the communication module includes: the communication module reports the descriptor information of the communication module to the host, such as the device type, configuration, one or more virtual ports supported, etc.; accordingly, the host obtains the above information, identifies the communication module, and determines the basic situation of the communication module (such as the above descriptor information) and communication requirements (such as which virtual ports need to be used) in order to communicate with the communication module correctly.

[0054] When the communication module enumerates a virtual port, such as port A, it means that the communication module supports port A and reports to the host that it supports port A, and the host has completed the resource configuration of port A, so that data can be exchanged between the communication module and the host through port A. The host's resource configuration of port A may include loading the driver corresponding to port A, configuring the port parameters of port A (for example, baud rate, data bit, stop bit, check bit), etc.

[0055] For example, the virtual ports commonly enumerated by the communication module may include an attention (AT) port. The AT port, also known as the AT command port, is used to receive and respond to AT commands, which are a standard command set for controlling modems and other devices. The host can configure the parameters of the communication module, query its status, or perform specific communication tasks by sending AT commands to the communication module.

[0056] Currently, when IoT devices expand their voice playback functions, they usually connect a codec (coder-decoder) device to the communication module of the IoT host device (referred to as the host). The codec device is responsible for encoding and decoding the audio signal and converting it into a playable digital signal to achieve voice output. In particular, in application scenarios where text needs to be converted into voice, the host often relies on a text-to-speech (TTS) library to first convert text information into audio data, and then encode and decode the audio data through the codec device and play it.

[0057] However, this technical implementation has significant defects: on the one hand, purchasing the TTS library will increase the software configuration cost of the host; on the other hand, the external codec device will increase the hardware cost.

[0058] In view of this, the embodiments of the present application provide an audio processing method, a communication device, a computer device and a readable storage medium. The audio processing method can be applied to IoT devices and communication modules, and can effectively reduce the cost of expanding the voice playback function of IoT devices.

[0059] The following is an introduction to a system architecture applicable to an embodiment of the present application.

[0060] See also Figure 1 , Figure 1 A schematic diagram of a system architecture provided in an embodiment of the present application. Figure 1 As shown, the system architecture includes an IoT device 110 and a communication module 120. The communication module 120 can be inserted into the IoT device 110. Data can be exchanged between the IoT device 110 and the communication module 120. The IoT device 110 includes a codec that can encode and decode audio data. The communication module 120 includes a text to speech (TTS) library that can convert text data into audio data. The IoT device 110 accesses the network through the communication module 120, thereby establishing a network connection with the network device. Among them, the network device can be a gateway, an authentication server, or a base station, such as a 4G gateway, a 5G gateway, Figure 1 Not shown.

[0061] Among them, the Internet of Things device 110 provides a standard interface and can be equipped with a variety of module devices, such as a communication module, a storage module, a camera module, etc. In the embodiment of the present application, the Internet of Things device 110 equipped with a communication module 120 is taken as an example for description.

[0062] The communication module 120 supports a variety of standard interfaces and communication protocols and can be mounted on various types of IoT devices. For example, the communication module 120 can be inserted into the IoT device 110 through its USB interface, so that the communication module 120 can be connected to the IoT device 110. In an embodiment of the present application, after the communication module 120 is connected to the IoT device 110, a port enumeration process can be performed between the two to enumerate multiple virtual ports, such as a first port and a second port. The first port can be an AT port, and the IoT device 110 can send AT commands to the communication module 120 through the AT port. Accordingly, the communication module 120 can receive AT commands from the IoT device 110 through the AT port and respond. The communication module 120 can send audio data to the IoT device 110 through the second port; accordingly. The IoT device 110 can receive audio data from the communication module 120 through the second port.

[0063] The following describes the audio processing method provided by the embodiments of the present application.

[0064] See also Figure 2 , Figure 2 A flowchart of an audio processing method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the audio processing method may include but is not limited to the following steps:

[0065] S201, the Internet of Things device sends a first instruction to the communication module, where the first instruction includes a first text to be converted. Correspondingly, the communication module receives the first instruction from the Internet of Things device.

[0066] The first text refers to text data that needs to be played by voice. In the IoT scenario, the first text may include various texts that need to be played by voice in various scenarios such as smart home systems, industrial IoT systems, home security systems, medical IoT systems, smart car systems, smart city systems, smart wearable devices, smart weather stations, and educational IoT devices. For example, the first text may include notification and reminder information (such as smart door lock status information, sweeping robot working status information), security alarm information (such as fire alarm information, earthquake alarm information), health detection information (such as exercise status information, heart rate reminder information, sedentary reminder information), navigation and location service information (such as route guidance information, traffic condition broadcast information, public transportation information), social and communication information (such as incoming call reminders, friend messages), etc.

[0067] In a possible implementation, the IoT device may generate the first text based on the application notification information and / or the received input information. For example, the application notification information may be information used to remind the working status of the smart home, such as information that the sweeping robot has completed cleaning, and its text content may be "sweeping completed". The input information received by the IoT device may be information input by the user. For example, the user may input the text content "Please ask customer No. 108 to go to window No. 1 to pick up the meal", and the IoT device may generate the first text based on the received text content.

[0068] Converting these text data into audio data and playing it back can enable users to obtain information and control devices more conveniently, and enhance the interactivity and user experience of the system.

[0069] In the embodiments of the present application, an IoT device refers to an IoT host device, which may be referred to as a host. The IoT device may provide a USB socket, and the communication module may be inserted into the IoT device through the USB interface. In this way, the communication module may be connected to the IoT device. For example, the communication module is inserted into the USB socket of the IoT device through its USB interface. When the IoT device detects that the USB socket is connected to a USB interface, it may identify that the device corresponding to the USB interface is a USB device. After the communication module is inserted into the IoT device based on the USB interface, a USB connection may be established with the IoT device, so that the IoT device may manage and control the communication module through the USB connection.

[0070] In one possible implementation, the communication module can also access the IoT device through other physical interfaces. For example, a standard serial port, an Ethernet interface, etc. According to the design and functional requirements of the communication module, the interface types supported by the IoT device, and factors such as communication speed and transmission distance, a suitable interface can be selected to ensure that the communication module can be correctly and stably inserted into the IoT device and realize data transmission. The embodiment of the present application is described by taking the example of a communication module accessing an IoT device through its USB interface.

[0071] The first instruction may be an AT instruction. Through this AT instruction, the IoT device may send the first text to be converted to the communication module, and instruct the communication module to perform text-to-speech processing on the first text.

[0072] In a possible implementation, the IoT device may send the first instruction to the communication module via an AT port, wherein the AT port may be determined through a port enumeration process between the IoT device and the communication module.

[0073] In a possible implementation, the first port and the second port may be determined through a port enumeration process. The IoT device may use the first port to send a first instruction; accordingly, the communication module may use the first port to receive the first instruction. The communication module may use the second port to send a first response; accordingly, the IoT device may use the second port to receive the first response. For an introduction to the first response, please refer to the description in S203 below.

[0074] Optionally, the first port may be an AT port, and the IoT device may send an AT command to the communication module through the AT port; accordingly, the communication module may receive the AT command from the IoT device through the AT port and respond. The communication module may send a first response to the IoT device through the second port; accordingly, the IoT device may receive the first response from the communication module through the second port.

[0075] S202, the communication module converts the first text into the first audio through the text-to-speech library.

[0076] After receiving the first instruction, the communication module converts the first text into the first audio according to the instruction of the first instruction.

[0077] The communication module is configured with a text-to-speech library, such as a text-to-speech (TTS) library. The TTS library is a software package that integrates text analysis and speech synthesis technology. The TTS library can use algorithms and speech models to convert input text into corresponding audio waveforms, thereby converting the first text into the first audio.

[0078] In one possible implementation, the communication module preprocesses the input text data through the TTS library, such as word segmentation, intonation annotation and other preparatory work. The language model and speech synthesis model in the TTS library are used to convert the text into corresponding audio feature parameters. These parameters can describe the frequency, pitch, volume and other characteristics of the sound. The original audio waveform can be generated based on these parameters. After obtaining the original audio waveform, the generated audio data can be optimized through processing such as removing noise and enhancing sound quality, making it more natural and smooth when playing.

[0079] Optionally, the first audio may be a digital audio data stream consisting of a plurality of digital audio signals, for example, the first audio may be a PCM data stream consisting of a plurality of pulse code modulation (PCM) signals. The analog signal (i.e., the original audio waveform described above) may be sampled, quantized, and encoded to convert it into a series of discrete digital values. These digital values ​​are organized according to specific formats and rules to obtain a PCM signal. In a PCM signal, the amplitude of each sampling point is quantized into a finite digital value and represented by a binary code. These binary codes are arranged in sequence to obtain a digital audio data stream that can be used for transmission, storage, and processing, i.e., a PCM data stream. In this way, when the first audio is a PCM data stream, it is conducive to storing high-quality audio information and to obtaining a lossless audio file in the subsequent processing of the first audio.

[0080] S203, the communication module sends a first response to the IoT device, where the first response includes a first audio.

[0081] The first response refers to the response information to the first instruction. After completing the conversion of the first text and obtaining the first audio, the communication module can carry the first audio in the first response, thereby returning the first audio to the IoT device.

[0082] In a possible implementation, the IoT device may regard the communication module as a virtual sound card device through the second port determined by the port enumeration process. The communication module sends the first response to the IoT device, which is equivalent to the communication module sending the first response including the first audio to the IoT device through the second port.

[0083] In this way, IoT devices can realize the text-to-speech function through the communication module without the need to purchase TTS libraries for IoT devices, thus saving costs.

[0084] S204: The IoT device performs audio encoding and decoding processing on the first audio to obtain a second audio.

[0085] After receiving the first audio, the IoT device may perform audio encoding and decoding processing on the first audio through a codec therein, such as a codec device, to generate a second audio.

[0086] In one possible implementation, the codec device may perform one or more processes on the first audio, including but not limited to signal conversion, audio path control, and audio signal processing. Signal conversion may refer to the codec device converting the first audio from a digital audio signal (such as a PCM signal) into an analog signal, so that the speaker of the IoT device can output the analog signal to achieve audio playback. Audio path control may refer to the codec device being able to control the audio path of the IoT device to ensure that the audio signal can flow along the correct route when playing music, listening to the radio, or answering the phone. Audio signal processing may refer to the codec device being able to perform processing on the audio signal, such as volume control, power amplification, and equalization control, to meet different sound quality requirements.

[0087] S205: In response to the second audio satisfying the playing condition, the IoT device plays the second audio.

[0088] Among them, the IoT device can play the second audio through speakers, stereos and other playback devices.

[0089] In one possible implementation, the second audio meets the playback condition, which can be expressed as: the priority of the second audio is higher than the priority of the audio to be played in the IoT device. The priority of the second audio is the same as the priority of the first audio. Optionally, the audio to be played in the IoT device may include one or more audios such as streaming audio (such as music to be played in a music application), alarm audio (such as an alarm that detects that the smart door lock is not locked normally), notification and reminder audio (such as navigation route reminder audio in the car navigation).

[0090] Optionally, the priority of the audio may be determined based on its source, which is not limited in this application. For example, the second audio may be the audio corresponding to the incoming call reminder; the audio to be played may be the music in the music application in the IoT device. In this way, the priority of the second audio may be higher than the priority of the audio to be played. For another example, the second audio may be the weather forecast audio generated by the weather application in the IoT device; the audio to be played may be the alarm audio generated by the application security center in the IoT device regarding the abnormal locking of the smart door lock. In this way, the priority of the second audio is lower than the audio to be played.

[0091] Optionally, if the priority of the second audio is lower than the priority of the audio to be played in the IoT device, the second audio may be discarded. Optionally, the second audio may be stored in a corresponding storage area, such as a memory, and if an instruction for instructing to play the second audio is received, the second audio may be retrieved from the storage area and played.

[0092] Optionally, the IoT device may include one or more audios to be played. The second audio meets the playback condition and can be expressed as follows: the priority of the second audio is higher than the priority of at least n audios in the audios to be played, where n is a positive integer and can be set according to factors such as the total number of audios to be played. For example, assume that the audios to be played included in the IoT device are audio A, audio B, and audio C, and n is set to 2. If the priority of each audio is in the order of audio A, second audio, audio B, and audio C from large to small, then in this case, the priority of the second audio is higher than the priority of 2 audios in the total number of 3 audios to be played, so it can be said that the second audio meets the playback condition and each audio can be played in order from large to small according to the priority of each audio. Correspondingly, if the priority of each audio is in the order of audio A, audio B, second audio, and audio C from large to small, it means that the second audio does not meet the playback condition.

[0093] This allows for audio conflict management, ensuring that only one audio is played at a time, thus avoiding possible audio interference.

[0094] In a possible implementation, the second audio meets the playback condition, which can be represented as: obtaining a mixed audio that is integrated into the second audio. Optionally, when the IoT device receives the second audio, it can check whether its audio playback setting information indicates that the second audio is to be mixed. If it is indicated that the second audio is to be mixed, the IoT device can mix the second audio and the audio to be played in the IoT device through the codec device to generate mixed audio. In this way, the IoT device can play the mixed audio.

[0095] Optionally, mixing the second audio and the audio to be played may include but is not limited to the following steps (1)-(6):

[0096] (1) Format processing: Processing the formats of the second audio and the audio to be played to make the formats of each audio compatible (such as consistent sampling rate and bit depth).

[0097] (2) Time alignment: Adjust the start time of the second audio and the audio to be played to ensure that the time axis of each audio is aligned during mixing to avoid sound quality problems caused by phase shift.

[0098] (3) Amplitude adjustment: Adjust the volume level of the second audio and the audio to be played so that each audio can be played clearly after mixing without affecting each other. This process can be completed using a volume slider or an automation curve.

[0099] (4) Frequency balance: Use an equalizer to adjust the frequency of the second audio and the audio to be played, enhance or weaken specific frequency bands to achieve a balanced and harmonious overall sound effect.

[0100] (5) Mixing synthesis: The adjusted audio signals are combined in a preset manner (such as juxtaposition, superposition, etc.) to generate mixed audio.

[0101] (6) Post-processing: Adding effects such as echo and reverberation to the mixed audio can enhance the stereoscopic and spatial sense of the mixed audio.

[0102] In this way, through mixing processing, not only the sound quality and spatial sense of audio playback can be improved, but also the flexibility of audio playback of IoT devices can be improved.

[0103] It can be seen that by implementing the embodiments of the present application, the IoT device does not need to purchase an additional TTS library, but can use the TTS library in the communication module to which it is connected to convert the first text into the first audio; the communication module does not need an external codec device, but can encode and decode the first audio and play it through the audio hardware device in the IoT device (such as a codec device and a speaker, etc.). In this way, not only can costs be saved, but the audio playback of the IoT device and the communication module can also be managed in a unified manner, such as audio conflict management and mixed playback, which is conducive to avoiding audio interference caused by playing multiple audios at the same time, and improving the flexibility and diversity of audio processing.

[0104] See also Figure 3 , Figure 3 A schematic diagram of another system architecture provided in an embodiment of the present application. Figure 3 As shown, the system architecture includes a host 301, which is an IoT device, and the host 301 includes a communication module 302 and a codec. The communication module 302 includes a text to speech (TTS) library, through which text data can be converted into audio data. The codec can be used to encode and decode audio data. Among them, the communication module 302 is a communication module built into the host 301. For example, the communication module 302 is integrated in the hardware of the host 301, and the communication module 302 can communicate with the host 301 through an internal bus or interface. When the communication module 302 is initialized, a configuration process needs to be performed between the host 301 and the communication module 302. In this configuration process, hardware initialization, configuration parameter setting, communication protocol negotiation, etc. can be performed to ensure that data transmission and communication can be performed normally between the host 301 and the communication module 302.

[0105] Optionally, in the above configuration process, a first port and a second port may be virtualized. The first port may be an AT port, and the host 301 may send AT commands to the communication module 302 through the AT port. Accordingly, the communication module 302 may receive AT commands from the host 301 through the AT port and respond. The communication module 302 may send audio data to the host 301 through the second port; accordingly, the host 301 may receive audio data from the communication module 302 through the second port.

[0106] See also Figure 4 , Figure 4 A flowchart of another audio processing method provided in an embodiment of the present application is applicable to Figure 3 The system architecture is shown in Figure 1. Figure 4 As shown, the audio processing method may include but is not limited to the following steps:

[0107] S401, the host sends a first instruction to the communication module, the first instruction including a first text to be converted. Correspondingly, the communication module receives the first instruction from the host.

[0108] The host can send a first instruction to the communication module through the AT port. The first instruction can be an AT instruction. Figure 2 The description in S201 shown.

[0109] S402, the communication module converts the first text into the first audio through the text-to-speech library.

[0110] The communication module has a built-in text-to-speech library, a text-to-speech (TTS) library. The TTS library can convert the first text into the first audio. Figure 2 The description in S202 shown.

[0111] S403, the communication module sends a first response to the host, where the first response includes a first audio.

[0112] The communication module can be virtualized as a sound card device and can send a first response to the host through the second port.

[0113] Specifically, see Figure 2 The description in S203 shown.

[0114] S404: The host performs audio encoding and decoding processing on the first audio to obtain a second audio.

[0115] The host is equipped with a built-in codec device, and the codec device can be used to implement audio encoding and decoding processing of the first audio.

[0116] Specifically, see Figure 2 The description in S204 shown.

[0117] S405: In response to the second audio satisfying the playing condition, the host plays the second audio.

[0118] Specifically, see Figure 2 The description in S205 shown is not repeated here.

[0119] Based on this, the host does not need to purchase an additional TTS library, nor does it need to connect an external codec device to the communication module to realize the voice playback function, which can greatly save costs.

[0120] Next, a communication device provided in an embodiment of the present application is introduced.

[0121] See also Figure 5 , Figure 5 A schematic diagram of the structure of a communication device provided in an embodiment of the present application. The communication device may include a transceiver unit 510 and a processing unit 520. The transceiver unit 510 may be a device with a signal input (receiving) or output (sending), and is used to transmit signals with other devices or other devices in the device.

[0122] The processing unit 520 may be a device with a processing function, and may include one or more processors. The processor may be a general-purpose processor or a dedicated processor, etc. The processor may be a baseband processor or a central processing unit. The baseband processor may be used to process the communication protocol and the communication data, and the central processing unit may be used to control the device (such as a host node, a relay node or a chip, etc.), execute the software program, and process the data of the software program.

[0123] The communication device may be an IoT device or a communication module, or may be a device for realizing the functions of an IoT device or a communication module, such as a chip.

[0124] When the communication device is an IoT device, the communication device includes:

[0125] The transceiver unit 510 is used to send a first instruction to the communication module, the first instruction includes a first text to be converted; receive a first response from the communication module, the first response includes a first audio, and the first audio is an audio converted based on the first text.

[0126] The processing unit 520 is configured to perform audio codec processing on the first audio to obtain a second audio; and in response to the second audio satisfying a playing condition, play the second audio.

[0127] In a possible implementation, the processing unit 520 is further configured to play the second audio in response to a priority of the second audio being higher than a priority of the audio to be played in the IoT device.

[0128] In a possible implementation, the processing unit 520 is further configured to discard the second audio in response to a priority level of the second audio being lower than a priority level of the audio to be played.

[0129] In a possible implementation, the processing unit 520 is further configured to, in response to acquiring the mixed audio integrated with the second audio, play the mixed audio.

[0130] In a possible implementation, the processing unit 520 is further used to determine a first port and a second port through a port enumeration process with the communication module, the first port is used to send a first instruction, and the second port is used to receive a first response.

[0131] In a possible implementation, the processing unit 520 is further configured to generate a first text based on the application notification information and / or the received input information.

[0132] Or when the communication device is a communication module, the communication device includes:

[0133] The transceiver unit 510 is used to receive a first instruction from an IoT device, where the first instruction includes a first text to be converted.

[0134] The processing unit 520 is configured to convert the first text into a first audio by using a text-to-speech library.

[0135] The transceiver unit 510 is further configured to send a first response to the IoT device, where the first response includes a first audio signal.

[0136] In a possible implementation, the processing unit 520 is further configured to determine a first port and a second port through a port enumeration process with the IoT device, wherein the first port is configured to receive a first instruction and the second port is configured to send a first response.

[0137] For the introduction of the technical effects brought about by the communication device and any possible implementation manner thereof, reference may be made to the introduction of the technical effects of the aforementioned method embodiment, which will not be repeated here.

[0138] According to the embodiment of the present application, Figure 5 The various units in the communication device shown in the figure can be separately or completely combined into one or more other units to form, or one (or more) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In practical applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit.

[0139] The present application also provides a computer device. Figure 6 , is a structural diagram of a computer device provided in an embodiment of the present application.

[0140] like Figure 6 As shown, the computer device may include: one or more processors 610, one or more memories 630, one or more communication interfaces 620 and a bus 640, wherein the processors 610, memories 630 and communication interfaces 620 are connected via the bus 640. The computer device may be the communication device in the foregoing description.

[0141] Among them, the memory 630 is used to store programs; the processor 610 is used to execute the above-mentioned program stored in the above-mentioned memory. When the above-mentioned program is executed, the above-mentioned processor 610 executes a method in any possible implementation manner of the above-mentioned audio processing method.

[0142] It should be understood that in the embodiment of the present application, the above-mentioned memory 630 includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CDROM), as well as external memory other than computer memory and processor cache. A part of the above-mentioned memory 630 may also include a non-volatile random access memory. For example, the memory 630 may also store information about the device type.

[0143] The processor 610 may be one or more central processing units (CPUs). When the processor 610 is a CPU, the CPU may be a single-core CPU or a multi-core CPU. The processor 610 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0144] The steps performed in the above embodiments can be based on the above Figure 6 The structure of the computer device shown in the figure is implemented, and the processor 610 can execute the implementation described in any optional embodiment of the audio processing method provided in the embodiment of the present application, and can also execute the implementation of the communication device described in the embodiment of the present application. Specifically, the processor 610 can implement Figure 5 The communication interface 620 can implement the functions of the processing unit 520 in the communication device shown in FIG. Figure 5 The memory 630 can provide a cache when the processor 610 executes the implementation of the communication device described in the embodiment of the present application, and can also store the computer program required for the processor 610 to execute the implementation of the communication device described in the embodiment of the present application.

[0145] The embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor can implement the above Figure 2 or Figure 4 The method shown in .

[0146] The present application also provides a computer program product, which includes: instructions or computer programs; when the instructions or computer programs are executed, the above-mentioned Figure 2 or Figure 4 The method shown in .

[0147] The embodiment of the present application further provides a chip, the chip includes a processor, the processor is used to execute instructions, when the processor executes the instructions, the chip can achieve the above Figure 2 or Figure 4 Optionally, the chip further comprises a communication interface, and the communication interface is used to receive a signal or send a signal.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by hardware related to the computer program, and the above-mentioned computer program can be stored in a computer storage medium. When the above-mentioned computer program is executed, the processes in the above-mentioned method embodiments can be implemented. The aforementioned computer storage medium includes: a read-only memory ROM or a random access memory RAM, a magnetic disk or an optical disk, and other media that can store computer program codes.

Claims

1. An audio processing method, characterized in that: Applied to an Internet of Things device, the Internet of Things device is connected to a communication module, and the method includes: Sending a first instruction to the communication module, wherein the first instruction includes a first text to be converted; receiving a first response from the communication module, wherein the first response includes a first audio, and the first audio is an audio converted based on the first text; Performing audio encoding and decoding processing on the first audio to obtain a second audio; In response to the second audio satisfying a playing condition, the second audio is played.

2. The method according to claim 1, characterized in that In response to the second audio satisfying a playing condition, playing the second audio includes: In response to the priority of the second audio being higher than the priority of the audio to be played in the IoT device, the second audio is played.

3. The method according to claim 1, characterized in that In response to the second audio satisfying a playing condition, playing the second audio includes: In response to acquiring the mixed audio integrated with the second audio, the mixed audio is played.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: A first port and a second port are determined through a port enumeration process with the communication module, wherein the first port is used to send the first instruction and the second port is used to receive the first response.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The first text is generated based on application notification information and / or received input information.

6. An audio processing method, characterized in that: The method is applied to a communication module, the communication module is connected to an Internet of Things device, the communication module includes a text-to-speech library, and the method includes: Receiving a first instruction from the IoT device, wherein the first instruction includes a first text to be converted; Converting the first text into a first audio by using the text-to-speech library; A first response is sent to the IoT device, where the first response includes the first audio.

7. A communication device, characterized in that: The communication device is connected to a communication module, and the communication device further comprises: a transceiver unit, configured to send a first instruction to the communication module, wherein the first instruction includes a first text to be converted; and receive a first response from the communication module, wherein the first response includes a first audio, and the first audio is an audio converted based on the first text; The processing unit is configured to perform audio codec processing on the first audio to obtain a second audio; and in response to the second audio satisfying a playback condition, play the second audio.

8. A communication device, characterized in that: The communication device is connected to an Internet of Things device, the communication module includes a text-to-speech library, and the communication device further includes: A transceiver unit, configured to receive a first instruction from the IoT device, wherein the first instruction includes a first text to be converted; A processing unit, configured to convert the first text into a first audio by using the text-to-speech library; The transceiver unit is used to send a first response to the Internet of Things device, where the first response includes the first audio.

9. A computer device, characterized in that: include: Memory, processor; wherein: The memory is used to store a computer program, wherein the computer program includes program instructions; The processor is used to call the program instructions so that the communication device executes the method according to any one of claims 1 to 5, or executes the method according to claim 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5, or implements the method according to claim 6.

Images