Audio streaming playback methods, devices, and vehicles

Abstract

This application discloses an audio stream playback method, apparatus, and vehicle, belonging to the field of vehicle technology. The technical solution provided by this application's embodiments receives an online audio stream downloaded from the cloud, which is obtained by converting target text into speech in the cloud. The online audio stream and a local audio stream are played synchronously at different volumes, with the local audio stream's playback volume being lower than that of the online audio stream. In the event of a connection interruption with the cloud, the online audio stream can no longer be received; in this case, simply increasing the playback volume of the local audio stream ensures normal playback of the audio stream, resulting in high continuity of audio stream playback and thus improving the user experience.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to an audio streaming method, apparatus, and vehicle. Background Technology

[0002] Vehicles typically use TTS (Text-to-Speech) technology for human-machine interaction. Because online synthesized audio streams have a pleasant and expressive quality, they are prioritized for playback when vehicles need to respond to customers via voice. However, under poor network conditions, network interaction times can be long, potentially causing vehicles to miss receiving the audio stream from the cloud for extended periods.

[0003] In related technologies, when network conditions are poor, the vehicle will perform an offline process to synthesize the audio stream, and then continue playing the audio stream after the synthesis is complete.

[0004] However, this will cause the current audio stream playback to be interrupted, that is, the human-computer interaction will suddenly stop, resulting in a poor user experience. Summary of the Invention

[0005] This application provides an audio streaming playback method, apparatus, and vehicle that can ensure the continuity of audio streaming playback under poor network conditions, thereby improving user experience. The technical solution is as follows:

[0006] On the one hand, an audio stream playback method is provided, the method comprising:

[0007] Receive an online audio stream sent from the cloud, wherein the online audio stream is obtained by the cloud performing speech conversion on the target text;

[0008] Play the online audio stream and the local audio stream. The local audio stream is obtained by the vehicle terminal after converting the target text into speech. The playback volume of the local audio stream is lower than that of the online audio stream. The local audio stream and the online audio stream maintain the same playback progress during playback.

[0009] In the event of an interruption of the connection to the cloud, increase the playback volume to continue playing the local audio stream.

[0010] In one possible implementation, playing the online audio stream and the local audio stream includes:

[0011] The online audio stream is played on a first audio channel, and the local audio stream is played on a second audio channel, with the first and second audio channels operating independently of each other.

[0012] In one possible implementation, playing the online audio stream on the first audio channel and the local audio stream on the second audio channel includes:

[0013] The local audio stream is played on the first audio channel at a first volume, where the first volume is zero.

[0014] The online audio stream is played at a second volume on the second audio channel, where the second volume is greater than the first volume.

[0015] In one possible implementation, increasing the playback volume to continue playing the local audio stream in the event of an interruption of the connection to the cloud includes any of the following:

[0016] In the event of an interruption of the connection with the cloud, stop playing the online audio stream; adjust the playback volume of the local audio stream to a second volume, which is the playback volume when the online audio stream is played; and continue playing the local audio stream at the second volume.

[0017] In the event of an interruption of the connection with the cloud, the playback volume of the online audio stream is adjusted from the second volume to the first volume, where the first volume is the playback volume of the local audio stream; the playback volume of the local audio stream is adjusted to the second volume; and the local audio stream continues to be played at the second volume.

[0018] In one possible implementation, before increasing the playback volume to continue playing the local audio stream, the method further includes:

[0019] The playback progress of the local audio stream is adjusted based on a target delay, which is the time interval between detecting the interruption of the connection with the cloud and the offline audio engine receiving the information about the interruption of the connection with the cloud. The offline audio engine is used to generate the local audio stream.

[0020] The step of increasing the playback volume to continue playing the local audio stream includes:

[0021] Increase the playback volume to continue playing the local audio stream after adjusting the playback progress.

[0022] In one possible implementation, after increasing the playback volume to continue playing the local audio stream in the event of an interruption of the connection to the cloud, the method further includes:

[0023] Once the connection to the cloud is restored, the online audio stream sent from the cloud will be received again;

[0024] Based on the playback progress of the local audio stream, determine the target playback progress of the re-received online audio stream;

[0025] Reduce the playback volume of the local audio stream, and play the re-received online audio stream at a second volume and the target playback progress, where the second volume is the playback volume of the online audio stream before the connection with the cloud was interrupted.

[0026] In one possible implementation, prior to playing the online audio stream and the local audio stream, the method further includes:

[0027] The target text is input into an offline audio engine, which processes the target text to obtain its spectrogram; the spectrogram of the target text is then subjected to time-frequency transformation to obtain the local audio stream.

[0028] In one possible implementation, before increasing the playback volume to continue playing the local audio stream in the event of an interruption of the connection to the cloud, the method further includes:

[0029] If no online audio stream is received from the cloud within the target duration, the connection with the cloud is determined to be interrupted.

[0030] On one hand, an audio streaming playback device is provided, the device comprising:

[0031] An online audio stream receiving module is used to receive online audio streams sent from the cloud, wherein the online audio streams are obtained by the cloud after converting the target text into speech;

[0032] The playback module is used to play the online audio stream and the local audio stream. The local audio stream is obtained by the vehicle terminal after converting the target text into speech. The playback volume of the local audio stream is lower than that of the online audio stream. The local audio stream and the online audio stream maintain the same playback progress during playback.

[0033] The playback module is also used to increase the playback volume and continue playing the local audio stream in the event of an interruption in the connection with the cloud.

[0034] In one possible implementation, the playback module is configured to play the online audio stream on a first audio channel and the local audio stream on a second audio channel, wherein the first audio channel and the second audio channel are independent of each other.

[0035] In one possible implementation, the playback module is configured to play the local audio stream at a first volume on the first audio channel, where the first volume is zero; and to play the online audio stream at a second volume on the second audio channel, where the second volume is greater than the first volume.

[0036] In one possible implementation, the playback module is further configured to perform any of the following:

[0037] In the event of an interruption of the connection with the cloud, stop playing the online audio stream; adjust the playback volume of the local audio stream to a second volume, which is the playback volume when the online audio stream is played; and continue playing the local audio stream at the second volume.

[0038] In the event of an interruption of the connection with the cloud, the playback volume of the online audio stream is adjusted from the second volume to the first volume, where the first volume is the playback volume of the local audio stream; the playback volume of the local audio stream is adjusted to the second volume; and the local audio stream continues to be played at the second volume.

[0039] In one possible implementation, the device further includes:

[0040] A playback progress adjustment module is used to adjust the playback progress of the local audio stream based on a target delay, wherein the target delay is the time interval between detecting the interruption of the connection with the cloud and the offline audio engine receiving the information of the interruption of the connection with the cloud, and the offline audio engine is used to generate the local audio stream;

[0041] The playback module is also used to increase the playback volume and continue playing the local audio stream after adjusting the playback progress.

[0042] In one possible implementation, the online audio stream receiving module is further configured to re-receive the online audio stream sent from the cloud when the connection with the cloud is restored;

[0043] The playback module is further configured to determine the target playback progress of the re-received online audio stream based on the playback progress of the local audio stream; reduce the playback volume of the local audio stream, and play the re-received online audio stream at a second volume and the target playback progress, wherein the second volume is the playback volume of the online audio stream before the connection with the cloud was interrupted.

[0044] In one possible implementation, the device further includes:

[0045] The local audio stream acquisition module is used to input the target text into an offline audio engine, process the target text through the offline audio engine to obtain the spectrogram of the target text, and perform time-frequency transformation on the spectrogram of the target text to obtain the local audio stream.

[0046] In one possible implementation, the device further includes:

[0047] The connection determination module is used to determine that the connection with the cloud is interrupted if no online audio stream is received from the cloud within a target duration.

[0048] On one hand, a vehicle is provided, the vehicle including an in-vehicle terminal, the in-vehicle terminal including one or more processors and one or more memories, the one or more memories storing at least one computer program, the computer program being loaded and executed by the one or more processors to implement the audio stream playback method.

[0049] On one hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer program, which is loaded and executed by a processor to implement the audio stream playback method.

[0050] On one hand, a computer program product or computer program is provided, which includes program code stored in a computer-readable storage medium. The processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the above-described audio stream playback method.

[0051] The technical solution provided in this application embodiment receives an online audio stream downloaded from the cloud. This online audio stream is obtained by converting target text into speech in the cloud. The online audio stream and a local audio stream are played synchronously at different volumes, with the local audio stream's playback volume being lower than that of the online audio stream. In the event of a connection interruption with the cloud, the online audio stream can no longer be received. In this case, simply increasing the playback volume of the local audio stream ensures normal playback of the audio stream, resulting in high continuity of audio stream playback and thus improving the user experience. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the implementation environment of an audio stream playback method provided in an embodiment of this application;

[0054] Figure 2 This is a flowchart of an audio stream playback method provided in an embodiment of this application;

[0055] Figure 3 This is a flowchart of another audio stream playback method provided in the embodiments of this application;

[0056] Figure 4 This is a logic block diagram of an audio stream playback method provided in an embodiment of this application;

[0057] Figure 5 This is a schematic diagram of the structure of an audio streaming playback device provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of the structure of a vehicle-mounted terminal provided in an embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0060] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0061] Artificial intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0062] Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by demonstration.

[0063] TTS: TTS is an abbreviation for Text To Speech. It is an outstanding achievement that combines linguistics and psychology. With the support of a built-in chip, it uses neural network design to intelligently convert text into a natural speech stream.

[0064] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0065] Figure 1 This is a schematic diagram illustrating the implementation environment of an audio stream playback method provided in this application embodiment. See also... Figure 1The implementation environment may include an in-vehicle terminal 110 and a cloud 120.

[0066] The vehicle-mounted terminal 110 is connected to the cloud 120 via a wireless network. Also known as the vehicle's infotainment system, the vehicle-mounted terminal 110 has data transmission and processing capabilities. An application supporting human-machine interaction runs on the vehicle-mounted terminal 110, allowing users to interact with the system.

[0067] Cloud 120 is an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Cloud 120 can provide necessary computing power support for vehicle terminal 110, such as providing background services for applications running on vehicle terminal 110.

[0068] Those skilled in the art will understand that the number of the aforementioned vehicle-mounted terminals can be more or less. For example, there may be only one vehicle-mounted terminal, or there may be dozens or hundreds of vehicle-mounted terminals, or even more. In this case, the aforementioned implementation environment may also include other vehicle-mounted terminals.

[0069] After introducing the implementation environment provided by the embodiments of this application, the application scenarios of the embodiments of this application will be introduced in conjunction with the above implementation environment. In the following description, the vehicle terminal is the vehicle terminal 110 in the above implementation environment, and the server is the server 120 in the above implementation environment.

[0070] The technical solution provided in this application can be applied to any vehicle with human-computer interaction functionality. After adopting the technical solution provided in this application, the vehicle's in-vehicle terminal receives an online audio stream from the cloud. This online audio stream is obtained by the cloud performing speech-to-speech conversion on the target text, which is the text the vehicle replies to the user during human-computer interaction. Because the cloud has strong computing power, the quality of the online audio stream obtained after speech-to-speech conversion is relatively good. The in-vehicle terminal simultaneously plays the online audio stream and the local audio stream. The playback volume of the online audio stream is greater than that of the local audio stream, meaning that the playback is primarily based on the higher-quality online audio stream, providing the user with a better audio playback experience. If the connection between the in-vehicle terminal and the cloud is interrupted, the in-vehicle terminal cannot continue receiving the online audio stream from the cloud. In this case, the volume of the local audio stream is directly increased to achieve continuous playback of the audio stream, ensuring the continuity of human-computer interaction.

[0071] After introducing the implementation environment and application scenarios of the embodiments of this application, the technical solutions provided by the embodiments of this application will be described below. (See also...) Figure 2 Taking a vehicle-mounted terminal as the executing entity as an example, the method includes the following steps.

[0072] 201. The vehicle terminal receives an online audio stream sent from the cloud, which is obtained by the cloud after converting the target text into speech.

[0073] In this embodiment, the cloud provides remote services to the in-vehicle terminal. Specifically, the cloud provides a text-to-speech service to the in-vehicle terminal. The target text is the text the in-vehicle terminal replies to the user during human-computer interaction. Speech conversion refers to converting text into an audio stream. Correspondingly, the online audio stream is obtained by the cloud after performing speech conversion on the target text. Playing this online audio stream allows the in-vehicle terminal to reply to the user via audio. Since the cloud often has strong computing power, the audio stream obtained by the cloud after performing speech conversion on the target text often has good quality.

[0074] 202. The vehicle terminal plays the online audio stream and the local audio stream. The local audio stream is obtained by the vehicle terminal after converting the target text into speech. The playback volume of the local audio stream is lower than that of the online audio stream. The local audio stream and the online audio stream maintain the same playback progress.

[0075] The local audio stream is obtained by the in-vehicle terminal after converting the target text into speech. The online audio stream and the local audio stream correspond to the same content. Playing the online audio stream and the local audio stream simultaneously means playing both simultaneously. During playback, the volume of the local audio stream is lower than that of the online audio stream; that is, playback is centered on the online audio stream to ensure playback quality. In some embodiments, during simultaneous playback of the online and local audio streams, the volume of the local audio stream is 0, meaning the user can only hear the playback effect of the online audio stream.

[0076] 203. In the event of an interruption of the connection with the cloud, the vehicle terminal increases the playback volume to continue playing the local audio stream.

[0077] In this case, the connection to the cloud is interrupted, meaning the vehicle terminal cannot obtain data from the cloud, and therefore cannot continue to obtain online audio streams from the cloud. At this time, the vehicle terminal increases the volume of the local audio stream, so that the audio stream can be played to ensure the continuity of audio stream playback when the online audio stream cannot be played.

[0078] The technical solution provided in this application embodiment receives an online audio stream downloaded from the cloud. This online audio stream is obtained by converting target text into speech in the cloud. The online audio stream and a local audio stream are played synchronously at different volumes, with the local audio stream's playback volume being lower than that of the online audio stream. In the event of a connection interruption with the cloud, the online audio stream can no longer be received. In this case, simply increasing the playback volume of the local audio stream ensures normal playback of the audio stream, resulting in high continuity of audio stream playback and thus improving the user experience.

[0079] Steps 201-203 above are a brief introduction to the technical solutions provided in the embodiments of this application. The technical solutions provided in the embodiments of this application will be explained more clearly below with some examples. See [link to relevant documentation]. Figure 3 The method includes the following steps.

[0080] 301. In response to the interactive command, the vehicle terminal determines the target text corresponding to the interactive command.

[0081] The interactive command is used for human-machine interaction with the in-vehicle terminal. This interactive command can be a voice interaction command or an action interaction command triggered by performing an interactive action, etc., and this application embodiment does not limit this. Interactive actions include actions performed on the in-vehicle screen and actions performed on physical buttons on the vehicle. In some embodiments, the interactive command is used to instruct the in-vehicle terminal to perform one or more functions, such as instructing the in-vehicle terminal to play music, or instructing the in-vehicle terminal to turn on the air conditioning, or instructing the in-vehicle terminal to turn on the headlights, etc.

[0082] The target text corresponding to the interactive command is used to respond to the interactive command, and human-computer interaction can be realized based on the target text. When the interactive command is used to instruct the vehicle terminal to perform one or more functions, the target text is used to prompt the start of the execution of the one or more functions, or to indicate the execution status of the one or more functions, or to indicate that the one or more functions have been completed.

[0083] In one possible implementation, in response to an interactive command, the vehicle terminal determines the function corresponding to the interactive command. The vehicle terminal then determines the target text corresponding to that function.

[0084] In some embodiments, the target text corresponding to this function is set by a technician according to the actual situation, and this application embodiment does not limit this.

[0085] In this implementation, the vehicle terminal can determine the function corresponding to the interaction command, then determine the target text based on the function, and associate the determined target text with the function.

[0086] To illustrate the above implementation methods more clearly, two examples are provided below.

[0087] Example 1: In response to a voice interaction command, the in-vehicle terminal performs semantic recognition on the command to obtain its semantic information. The in-vehicle terminal then determines the function corresponding to this semantic information. Finally, the in-vehicle terminal determines the target text corresponding to that function.

[0088] In one possible implementation, in response to a voice interaction command, the in-vehicle terminal inputs the voice interaction command into a semantic recognition model. The semantic recognition model performs semantic recognition on the voice interaction command and outputs the semantic information of the voice interaction command. This semantic recognition model is trained and has the ability to extract semantic information. The in-vehicle terminal then performs matching based on this semantic information to obtain the function corresponding to that semantic information. Finally, the in-vehicle terminal obtains the target text corresponding to that function.

[0089] The semantic recognition model in the above embodiments can adopt any structure of semantic recognition model, and this application embodiment does not limit it.

[0090] Example 2: In response to an action interaction command, the vehicle terminal determines the function corresponding to the action interaction command. The vehicle terminal determines the target text corresponding to the function.

[0091] It should be noted that the vehicle-mounted terminal can obtain the target text through any of the above methods, and this application embodiment does not limit this.

[0092] 302. The vehicle terminal sends the target text to the cloud.

[0093] In this embodiment, the cloud provides remote services to the vehicle-mounted terminal. The cloud provides a text-to-speech service to the vehicle-mounted terminal. The vehicle-mounted terminal sends the target text to the cloud so that the cloud can convert the target text into an online audio stream. Because the cloud often has strong computing power, the online audio stream obtained by the cloud from the speech conversion of the target text often has good quality.

[0094] In one possible implementation, the vehicle terminal sends a text conversion request to the cloud, the text conversion request carrying the target text, the text conversion request being used to request that the target text be converted into an audio stream.

[0095] 303. The target text is received in the cloud.

[0096] In one possible implementation, the cloud receives the text conversion request and obtains the target text from the text conversion request.

[0097] 304. The cloud performs speech conversion on the target text to obtain an online audio stream of the target text.

[0098] In one possible implementation, the target text is input into an online audio engine in the cloud, processed by the online audio engine, and output as an online audio stream of the target text.

[0099] The online audio engine is used to convert text into an audio stream; it is also a TTS model.

[0100] In this implementation, the cloud can convert the target text into an online audio stream using an online audio engine, resulting in high efficiency and accuracy in speech conversion.

[0101] For example, the cloud inputs the target text into the online audio engine, which extracts linguistic features from the target text. The cloud then processes these linguistic features using the online audio engine to obtain the acoustic features of the target text. Finally, the cloud maps these acoustic features using the online audio engine to output the spectrum of the target text. The cloud then performs a time-frequency transformation on the spectrum of the target text to obtain the online audio stream of the target text. In some embodiments, the linguistic features of the target text are phoneme features.

[0102] It should be noted that, due to the strong computing power of the cloud, the online audio engine deployed in the cloud can have more model layers or higher complexity, thus achieving better speech conversion results.

[0103] In one possible implementation, the cloud inputs the target text and the target audio stream style into an online audio engine, which then processes the target text based on the target audio stream style and outputs an online audio stream of the target text.

[0104] The target audio stream style refers to the audio stream style output by the online audio engine; different audio stream styles have different listening experiences. This target audio stream style can be specified by the user or determined by the cloud; this embodiment does not limit this.

[0105] In this implementation, the cloud can convert the target text into an online audio stream in the style of the target audio stream through an online audio engine, thereby providing a richer selection of styles for the converted online audio stream.

[0106] For example, the cloud inputs the target text and the target audio stream style into an online audio engine. The online audio engine extracts linguistic features from the target text, obtaining its linguistic features. The cloud then processes these linguistic features based on the target audio stream style using the online audio engine, obtaining the acoustic features of the target text. Finally, the cloud maps these acoustic features using the online audio engine, outputting the spectrum of the target text. The cloud then performs a time-frequency transform on the spectrum of the target text, obtaining the online audio stream of the target text. The online audio engine is obtained through generative adversarial training.

[0107] 305. The cloud sends the online audio stream to the vehicle terminal.

[0108] 306. The vehicle terminal receives online audio streams sent from the cloud.

[0109] 307. The vehicle terminal performs speech conversion on the target text to obtain the local audio stream of the target text.

[0110] The local audio stream is obtained by the vehicle terminal after converting the target text into speech, and the online audio stream and the local audio stream correspond to the same content.

[0111] In one possible implementation, the in-vehicle terminal inputs the target text into an offline audio engine, which processes the target text to obtain its spectrogram. The in-vehicle terminal then performs a time-frequency transformation on the spectrogram to obtain the local audio stream.

[0112] The offline audio engine is used to convert text into an audio stream. The complexity of the offline audio engine is lower than that of the online audio engine. The offline audio engine is a TTS model.

[0113] In this implementation, the vehicle terminal can convert the target text into a local audio stream through an offline audio engine, and the conversion efficiency of the local audio stream is relatively high.

[0114] For example, the in-vehicle terminal inputs the target text into an offline audio engine, which extracts linguistic features from the text. The in-vehicle terminal then processes these linguistic features to obtain the acoustic features of the target text. Finally, the in-vehicle terminal maps these acoustic features to output the spectrum of the target text. The in-vehicle terminal then performs a time-frequency transform on the spectrum to obtain the local audio stream of the target text.

[0115] 308. The vehicle terminal plays the online audio stream and the local audio stream. The playback volume of the local audio stream is lower than that of the online audio stream. The local audio stream and the online audio stream maintain the same playback progress during playback.

[0116] In this context, "playing the online audio stream and the local audio stream simultaneously" means that the online audio stream and the local audio stream are played at the same time. During playback, the volume of the local audio stream is lower than that of the online audio stream, meaning that playback is centered around the online audio stream to ensure playback quality. In some embodiments, during simultaneous playback of the online and local audio streams, the volume of the local audio stream is 0, meaning the user can only hear the online audio stream. In some embodiments, the local and online audio streams maintain the same playback progress, which is related to the words in the target text. Maintaining the same playback progress means that at any given moment, both the online and local audio streams play the same word in the target text.

[0117] In one possible implementation, the vehicle terminal plays the online audio stream on a first audio channel and the vehicle terminal plays the local audio stream on a second audio channel, the first audio channel and the second audio channel being independent of each other.

[0118] The independence of the first and second audio channels means that they do not affect each other when playing audio streams. The first and second audio channels can play different audio streams simultaneously or at different volumes. For example, phone calls and multimedia playback on a mobile phone occur on two different audio channels. Changing the multimedia playback volume will not affect the phone call playback volume, and vice versa.

[0119] In this implementation, the vehicle terminal can play online audio streams and local audio streams on the first audio channel and the second audio channel, respectively. Since the first audio channel and the second audio channel are independent of each other, the vehicle terminal can also independently control the playback of the online audio stream and the local audio stream.

[0120] For example, the in-vehicle terminal plays the local audio stream at a first volume on a first audio channel, where the first volume is zero. The in-vehicle terminal plays the online audio stream at a second volume on a second audio channel, where the second volume is greater than the first volume. In some embodiments, the second volume is set by the user, and therefore is also referred to as the set volume or normal volume. During the playback of the local audio stream and the online audio stream, if the playback volume of the local audio stream is zero and the playback volume of the online audio stream is the second volume, then the user can only hear the online audio stream. Since the quality of the online audio stream is often better, the user's listening experience is better.

[0121] For example, the in-vehicle terminal uses a local audio stream player to load a local audio stream and an online audio stream player to load an online audio stream. The local audio stream player plays the audio stream on the first audio channel, and the online audio stream player plays the audio stream on the second audio channel. The in-vehicle terminal adjusts the playback volume of the local audio stream player to zero and the playback volume of the online audio stream player to the second volume. When playing through the local audio stream player and the online audio stream player, the in-vehicle terminal keeps the playback progress of the local audio stream and the online audio stream consistent. Although the local audio stream has no sound, the playback progress always remains the same as the online audio stream, facilitating subsequent switching.

[0122] 309. In the event of an interruption of the connection to the cloud, the vehicle terminal increases the playback volume to continue playing the local audio stream.

[0123] The interruption of the connection with the cloud means that the vehicle terminal cannot obtain data from the cloud, and therefore cannot continue to obtain online audio streams from the cloud.

[0124] In one possible implementation, if the connection to the cloud is interrupted, the in-vehicle terminal stops playing the online audio stream. The in-vehicle terminal adjusts the playback volume of the local audio stream to a second volume, which is the playback volume when the online audio stream was played. The in-vehicle terminal then continues playing the local audio stream at this second volume.

[0125] In this implementation, if the connection between the in-vehicle terminal and the cloud is interrupted, the in-vehicle terminal can stop playing the online audio stream, avoiding playback errors caused by a sudden restoration of the connection. Simultaneously, the in-vehicle terminal adjusts the playback volume of the local audio stream to a second volume, which is the same as the playback volume of the online audio stream. Since the local audio stream and the online audio stream play synchronously, directly adjusting the playback volume of the local audio stream to be the same as that of the online audio stream enables seamless switching between audio streams.

[0126] For example, if the connection to the cloud is interrupted, the in-vehicle terminal stops playing the online audio stream on the first audio channel. The in-vehicle terminal adjusts the playback volume of the second audio channel to the second volume and plays the local audio stream on the second audio channel at the second volume. Alternatively, if the connection to the cloud is interrupted, the in-vehicle terminal stops playing the online audio stream on the first audio channel via an online audio stream player. The in-vehicle terminal adjusts the playback volume of the second audio channel to the second volume via a local audio stream player and plays the local audio stream on the second audio channel at the second volume.

[0127] In some embodiments, if the in-vehicle terminal stops playing the online audio stream, the in-vehicle terminal will no longer receive the online audio stream from the cloud, but will continue to play the local audio stream at the second volume.

[0128] In one possible implementation, in the event of an interruption of the connection to the cloud, the in-vehicle terminal adjusts the playback volume of the online audio stream from the second volume to a first volume, which is the playback volume of the local audio stream. The in-vehicle terminal then adjusts the playback volume of the local audio stream back to the second volume. The in-vehicle terminal continues to play the local audio stream at the second volume.

[0129] In this implementation, if the connection between the in-vehicle terminal and the cloud is interrupted, the in-vehicle terminal can lower the playback volume of the online audio stream and increase the playback volume of the local audio stream, thus switching from focusing on online audio stream playback to focusing on local audio stream playback, ensuring smooth audio playback. Furthermore, maintaining the online audio playback allows for continued playback of the online audio stream once the cloud connection is restored.

[0130] For example, in the event of a connection interruption with the cloud, the in-vehicle terminal adjusts the playback volume of the first audio channel to a first volume, which in some embodiments is zero. The in-vehicle terminal then adjusts the playback volume of the second audio channel to the second volume, playing the local audio stream at that second volume on the second audio channel. For instance, in the event of a connection interruption with the cloud, the in-vehicle terminal uses an online audio streaming player to adjust the playback volume of the first audio channel to zero. The in-vehicle terminal then uses a local audio streaming player to adjust the playback volume of the second audio channel to the second volume, playing the local audio stream at that second volume on the second audio channel.

[0131] In some embodiments, based on the above implementation, upon restoration of the cloud connection, the vehicle terminal re-receives the online audio stream sent from the cloud. The vehicle terminal determines the target playback progress of the re-received online audio stream based on the playback progress of the local audio stream. The vehicle terminal reduces the playback volume of the local audio stream and plays the re-received online audio stream at a second volume and the target playback progress, where the second volume is the playback volume of the online audio stream before the cloud connection was interrupted.

[0132] The target playback progress is the current playback progress of the local audio stream. Playing the newly received online audio stream at the target playback progress ensures that the playback progress of the newly received online audio stream is synchronized with that of the local audio stream, thus ensuring the continuity of audio playback.

[0133] In some embodiments, step 309 above further includes the following implementation.

[0134] In one possible implementation, in the event of a connection interruption with the cloud, the in-vehicle terminal adjusts the playback progress of the local audio stream based on a target delay, which is the time interval between detecting the connection interruption and the offline audio engine receiving information about the connection interruption. The offline audio engine is used to generate the local audio stream. The in-vehicle terminal then increases the playback volume and continues playing the local audio stream with the adjusted playback progress.

[0135] In this implementation, in the event of a connection interruption with the cloud, the vehicle terminal can adjust the playback progress of the local audio stream based on the target latency, so that when the playback volume of the local audio stream is increased, the playback progress of the local audio stream is the same as the progress when the online audio stream stops playing, further improving the continuity of audio playback and achieving seamless switching between online and local audio streams.

[0136] For example, a local audio streaming player has a data cache that stores previously played local audio streams. The in-vehicle terminal detects the target delay between the interruption of the connection to the cloud and the offline audio engine receiving information about the interruption. Based on this target delay, the in-vehicle terminal adjusts the playback progress of the local audio stream, essentially rolling back the playback progress of the local audio stream [rolling back a certain amount of PCM (Pulse Code Modulation) data]. The in-vehicle terminal then increases the playback volume and continues playing the local audio stream with the adjusted playback progress.

[0137] To provide a clearer explanation of the above implementation methods, the method for determining the interruption of the connection between the vehicle terminal and the cloud is described below.

[0138] In one possible implementation, if the online audio stream sent from the cloud is not received within a target duration, the vehicle terminal determines that the connection with the cloud has been interrupted.

[0139] The following is combined with Figure 4 The technical solutions provided in the embodiments of this application will be described.

[0140] See Figure 4 The in-vehicle terminal determines the target text, such as "Song XXX will be played for you soon." The in-vehicle terminal uploads this target text to the cloud, where it is input into an online audio engine. The online audio engine processes the target text and outputs an online audio stream. The in-vehicle terminal then inputs the target text into an offline audio engine, which processes the target text and outputs a local audio stream. The in-vehicle terminal plays the local audio stream using a local audio stream player, with the playback volume at zero. Simultaneously, the in-vehicle terminal plays the online audio stream using an online audio stream player, with the playback volume at normal volume. The playback progress of the local and online audio streams remains synchronized. If the connection between the in-vehicle terminal and the cloud is lost, the in-vehicle terminal adjusts the playback volume of the local audio stream player to normal and continues playing the local audio stream at normal volume. At the same time, the in-vehicle terminal stops playing the online audio stream using the online audio stream player. Correspondingly, if the connection between the in-vehicle terminal and the cloud remains intact, the in-vehicle terminal maintains the local audio stream's playback volume at zero and the online audio stream's playback volume at normal volume until playback ends.

[0141] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0142] The technical solution provided in this application embodiment receives an online audio stream downloaded from the cloud. This online audio stream is obtained by converting target text into speech in the cloud. The online audio stream and a local audio stream are played synchronously at different volumes, with the local audio stream's playback volume being lower than that of the online audio stream. In the event of a connection interruption with the cloud, the online audio stream can no longer be received. In this case, simply increasing the playback volume of the local audio stream ensures normal playback of the audio stream, resulting in high continuity of audio stream playback and thus improving the user experience.

[0143] The technical solution provided in this application embodiment utilizes both online and offline audio engines, with one set to mute and the other to normal volume. When the online connection is interrupted, the offline audio engine is switched to normal volume and playback continues, achieving seamless playback of the current audio content and improving user experience.

[0144] Figure 5This is a schematic diagram of the structure of an audio streaming playback device provided in an embodiment of this application. See also... Figure 5 The device includes an online audio stream receiving module 501 and a playback module 502.

[0145] The online audio stream receiving module 501 is used to receive online audio streams sent from the cloud, which are obtained by the cloud after converting the target text into speech.

[0146] The playback module 502 is used to play the online audio stream and the local audio stream. The local audio stream is obtained by the vehicle terminal after converting the target text into speech. The playback volume of the local audio stream is lower than that of the online audio stream. The local audio stream and the online audio stream maintain the same playback progress during playback.

[0147] The playback module 502 is also used to increase the playback volume to continue playing the local audio stream in the event of an interruption of the connection with the cloud.

[0148] In one possible implementation, the playback module 502 is configured to play the online audio stream on a first audio channel and the local audio stream on a second audio channel, wherein the first audio channel and the second audio channel are independent of each other.

[0149] In one possible implementation, the playback module 502 is configured to play the local audio stream at a first volume (zero) on a first audio channel, and to play the online audio stream at a second volume (greater than the first volume) on a second audio channel.

[0150] In one possible implementation, the playback module 502 is further configured to perform any of the following:

[0151] If the connection to the cloud is interrupted, stop playing the online audio stream. Adjust the playback volume of the local audio stream to the second volume, which is the same volume the online audio stream was playing at. Resume playing the local audio stream at the second volume.

[0152] In the event of a connection interruption to the cloud, the playback volume of the online audio stream is adjusted from the second volume to the first volume, which is the playback volume of the local audio stream. The playback volume of the local audio stream is then adjusted back to the second volume. The local audio stream continues to play at the second volume.

[0153] In one possible implementation, the device further includes:

[0154] The playback progress adjustment module is used to adjust the playback progress of the local audio stream based on a target delay, which is the time interval between detecting the interruption of the connection with the cloud and the offline audio engine receiving the information about the interruption of the connection with the cloud. The offline audio engine is used to generate the local audio stream.

[0155] The playback module 502 is also used to increase the playback volume and continue playing the local audio stream after adjusting the playback progress.

[0156] In one possible implementation, the online audio stream receiving module 501 is also used to re-receive the online audio stream sent by the cloud when the connection with the cloud is restored.

[0157] The playback module 502 is further configured to determine the target playback progress of the re-received online audio stream based on the playback progress of the local audio stream. The playback volume of the local audio stream is reduced, and the re-received online audio stream is played at a second volume and the target playback progress, where the second volume is the playback volume of the online audio stream before the connection with the cloud was interrupted.

[0158] In one possible implementation, the device further includes:

[0159] The local audio stream acquisition module is used to input the target text into the offline audio engine, which processes the target text to obtain its spectrogram. A time-frequency transformation is then performed on the spectrogram to obtain the local audio stream.

[0160] In one possible implementation, the device further includes:

[0161] The connection determination module is used to determine that the connection with the cloud is interrupted if no online audio stream is received from the cloud within a target duration.

[0162] It should be noted that the audio streaming device provided in the above embodiments is only illustrated by the division of the above functional modules when playing audio streams. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the audio streaming device and the audio streaming method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0163] The technical solution provided in this application embodiment receives an online audio stream downloaded from the cloud. This online audio stream is obtained by converting target text into speech in the cloud. The online audio stream and a local audio stream are played synchronously at different volumes, with the local audio stream's playback volume being lower than that of the online audio stream. In the event of a connection interruption with the cloud, the online audio stream can no longer be received. In this case, simply increasing the playback volume of the local audio stream ensures normal playback of the audio stream, resulting in high continuity of audio stream playback and thus improving the user experience.

[0164] This application provides a vehicle including an on-board terminal for performing the above-described method. The structure of the on-board terminal is described below:

[0165] Figure 6 This is a schematic diagram of the structure of an in-vehicle terminal provided in an embodiment of this application. Typically, the in-vehicle terminal 600 includes one or more processors 601 and one or more memories 602.

[0166] Processor 601 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0167] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 are used to store a computer program that is executed by the processor 601 to implement the method for playing audio streams in an in-vehicle terminal provided in the method embodiments of this application.

[0168] In some embodiments, the vehicle terminal 600 may also optionally include a peripheral device interface 603 and peripheral devices. The processor 601, memory 602, and peripheral device interface 603 can be connected via a bus or signal lines. Each peripheral device can be connected to the peripheral device interface 603 via a bus, signal lines, or a circuit board. Specifically, the peripheral devices include at least one of the following: a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio streaming circuit 607, and a power supply 608.

[0169] Peripheral interface 603 can be used to connect I / O (Input / Output) related peripheral devices to processor 601 and memory 602. In some embodiments, processor 601, memory 602 and peripheral interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 601, memory 602 and peripheral interface 603 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0170] The radio frequency (RF) circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 604 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 604 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc.

[0171] Display screen 605 is used to display a user interface (UI). This UI may include graphics, text, icons, video, and any combination thereof. When display screen 605 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 601 for processing. In this case, display screen 605 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard.

[0172] The camera assembly 606 is used to acquire images or videos. Optionally, the camera assembly 606 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the vehicle terminal, and the rear-facing camera is located on the back of the vehicle terminal.

[0173] The audio stream circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 601 for processing, or input to the radio frequency circuit 604 to realize voice communication.

[0174] The power supply 608 is used to power the various components in the vehicle terminal 600. The power supply 608 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery.

[0175] In some embodiments, the vehicle terminal 600 further includes one or more sensors 609. The one or more sensors 609 include, but are not limited to, an acceleration sensor 610, a gyroscope sensor 611, a pressure sensor 612, an optical sensor 613, and a proximity sensor 614.

[0176] The accelerometer 610 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the vehicle terminal 600.

[0177] The gyroscope sensor 611 can detect the body orientation and rotation angle of the vehicle terminal 600. The gyroscope sensor 611 can work in conjunction with the accelerometer sensor 610 to collect the user's 3D movements on the vehicle terminal 600.

[0178] The pressure sensor 612 can be installed on the side bezel of the vehicle terminal 600 and / or on the lower layer of the display screen 605. When the pressure sensor 612 is installed on the side bezel of the vehicle terminal 600, it can detect the user's grip signal on the vehicle terminal 600, and the processor 601 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 612. When the pressure sensor 612 is installed on the lower layer of the display screen 605, the processor 601 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 605.

[0179] An optical sensor 613 is used to collect ambient light intensity. In one embodiment, a processor 601 can control the display brightness of a display screen 605 based on the ambient light intensity collected by the optical sensor 613.

[0180] The proximity sensor 614 is used to collect the distance between the user and the front of the vehicle terminal 600.

[0181] Those skilled in the art will understand that Figure 6The structure shown does not constitute a limitation on the vehicle terminal 600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0182] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a computer program that can be executed by a processor to perform the audio stream playback method in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0183] In an exemplary embodiment, a computer program product or computer program is also provided, which includes program code stored in a computer-readable storage medium. The processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the above-described audio stream playback method.

[0184] In some embodiments, the computer program involved in the present application embodiments may be deployed and executed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network may constitute a blockchain system.

[0185] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0186] The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An audio stream playing method, characterized in that, The method is executed by a vehicle terminal, and comprises the following steps: receiving an online audio stream issued by a cloud, the online audio stream being obtained by converting a target text into speech by the cloud; playing the online audio stream and a local audio stream, the local audio stream being obtained by converting the target text into speech by the vehicle terminal, a playing volume of the local audio stream being less than that of the online audio stream, and the local audio stream and the online audio stream maintaining the same playing progress when being played; in a case where a connection with the cloud is interrupted, adjusting a playing progress of the local audio stream based on a target delay, so that the playing progress of the local audio stream is the same as that of the online audio stream when the online audio stream stops playing, the target delay being a time interval between detecting that the connection with the cloud is interrupted and receiving information that the connection with the cloud is interrupted by an offline audio engine used for generating the local audio stream; increasing the playing volume to continue playing the local audio stream; wherein the step of increasing the playing volume to continue playing the local audio stream comprises: increasing the playing volume to continue playing the local audio stream after the playing progress is adjusted.

2. The method of claim 1, wherein, The step of playing the online audio stream and the local audio stream comprises: playing the online audio stream in a first audio channel and playing the local audio stream in a second audio channel, the first audio channel and the second audio channel being independent of each other.

3. The method of claim 2, wherein, The step of playing the online audio stream in the first audio channel and playing the local audio stream in the second audio channel comprises: playing the local audio stream in the first audio channel at a first volume, the first volume being zero; playing the online audio stream in the second audio channel at a second volume, the second volume being greater than the first volume.

4. The method of claim 1, wherein, The step of increasing the playing volume to continue playing the local audio stream in the case where the connection with the cloud is interrupted comprises any one of the following: in the case where the connection with the cloud is interrupted, stopping playing the online audio stream; adjusting the playing volume of the local audio stream to a second volume, the second volume being the playing volume of the online audio stream when being played; continuing to play the local audio stream at the second volume; in the case where the connection with the cloud is interrupted, adjusting the playing volume of the online audio stream from the second volume to a first volume, the first volume being the playing volume of the local audio stream; adjusting the playing volume of the local audio stream to the second volume; and continuing to play the local audio stream at the second volume.

5. The method of claim 1, wherein, After the step of increasing the playing volume to continue playing the local audio stream in the case where the connection with the cloud is interrupted, the method further comprises the following steps: in the case where the connection with the cloud is restored, re-receiving the online audio stream issued by the cloud; determining a target playing progress of the re-received online audio stream based on the playing progress of the local audio stream; decreasing the playing volume of the local audio stream to play the re-received online audio stream at a second volume and the target playing progress, the second volume being the playing volume of the online audio stream before the connection with the cloud is interrupted.

6. The method according to any one of claims 1 to 5, characterized in that, Before the step of playing the online audio stream and the local audio stream, the method further comprises the following steps: input the target text into an offline audio engine, process the target text by the offline audio engine to obtain a spectrum graph of the target text, and perform time-frequency conversion on the spectrum graph of the target text to obtain the local audio stream.

7. The method according to any one of claims 1 to 5, characterized in that, Before the playing volume is increased to continue playing the local audio stream in the case where the connection with the cloud is interrupted, the method further includes: In a case where the online audio stream issued by the cloud is not received within a target time length, it is determined that the connection with the cloud is interrupted.

8. An audio stream playing apparatus, characterized by comprising: The apparatus includes: an online audio stream receiving module configured to receive an online audio stream issued by a cloud, the online audio stream being obtained by performing voice conversion on a target text by the cloud; a playing module configured to play the online audio stream and a local audio stream, the local audio stream being obtained by performing voice conversion on the target text by a vehicle-mounted terminal, a playing volume of the local audio stream being less than that of the online audio stream, and the local audio stream and the online audio stream maintaining the same playing progress when being played; the playing module is further configured to, in a case where the connection with the cloud is interrupted, increase the playing volume to continue playing the local audio stream; a playing progress adjusting module configured to adjust a playing progress of the local audio stream based on a target delay to make the playing progress of the local audio stream the same as a progress of the online audio stream when the online audio stream stops being played, the target delay being a time interval between a time when it is detected that the connection with the cloud is interrupted and a time when information about the connection with the cloud being interrupted is received by an offline audio engine configured to generate the local audio stream; the playing module is further configured to increase the playing volume to continue playing the local audio stream after the playing progress is adjusted.

9. A vehicle characterized by comprising: The vehicle includes a vehicle-mounted terminal, the vehicle-mounted terminal including one or more processors and one or more memories, at least one computer program being stored in the one or more memories, the computer program being loaded and executed by the one or more processors to implement the audio stream playing method according to any one of claims 1 to 7.

Images