Resource scheduling method for vehicle-mounted speech synthesis, electronic device, medium and product

Abstract

The application provides a resource scheduling method for vehicle-mounted voice synthesis, an electronic device, a medium and a product, relates to the technical field of vehicle voice synthesis, and comprises the following steps: obtaining a starting sequence event of vehicle-mounted voice; preheating a voice synthesis channel in response to the starting sequence event; and generating a vehicle-mounted voice interaction request based on the starting sequence event; the voice synthesis channel is used for processing the vehicle-mounted voice interaction request; performing scene-aware processing on the vehicle-mounted voice interaction request to obtain a target voice broadcast scene to which the vehicle-mounted voice interaction request belongs; and based on the target voice broadcast scene, allocating inference resources for processing the vehicle-mounted voice interaction request by the voice synthesis channel. The application can solve the problem of high response delay of traditional vehicle-mounted voice interaction for key safety scenarios, thereby improving the reliability of vehicle-mounted voice interaction in safety-related scenarios such as navigation prompts, and meeting the differentiated voice broadcast requirements in the intelligent cabin under multi-scene concurrency.

Description

Technical Field

[0001] This application relates to the field of vehicle speech synthesis (Text-to-Speech, TTS) technology, and in particular to a resource scheduling method, electronic device, medium and product for vehicle speech synthesis. Background Technology

[0002] Currently, in-vehicle voice interaction needs to simultaneously handle the broadcasting needs of multiple scenarios, including navigation prompts, vehicle control operation feedback, entertainment content broadcasting, and intelligent dialogue. Due to the fundamental differences in the functional attributes and safety importance of different scenarios, their sensitivities to voice broadcasting latency vary significantly. For example, critical safety scenarios such as navigation turn prompts and emergency vehicle control alarms have extremely high requirements for response latency; excessive latency can directly impact driving safety. Conversely, non-critical scenarios such as entertainment chat and information broadcasting have a relatively higher tolerance for latency. However, in related technologies, in-vehicle TTS voice synthesis services employ a unified first-in, first-out (FIFO) resource scheduling strategy for voice interaction requests in different scenarios. This means that TTS resources are scheduled for processing according to the order in which requests arrive. This can lead to high-priority critical scenario broadcasts being blocked by low-priority non-critical scenarios, resulting in navigation turn prompts being broadcast later than the user's casual conversation with the vehicle's infotainment system, directly causing excessively high response latency in critical safety scenarios. Summary of the Invention

[0003] The main objective of this application is to propose a resource scheduling method, electronic device, medium, and product for in-vehicle voice synthesis, aiming to solve the problem of excessive response latency in traditional in-vehicle voice interaction for critical safety scenarios, thereby improving the reliability of in-vehicle voice interaction in safety-related scenarios such as navigation prompts, and meeting the differentiated voice broadcasting needs of intelligent cockpits under multiple concurrent scenarios.

[0004] To achieve the above objectives, a first aspect of this application proposes a resource scheduling method for in-vehicle speech synthesis, the method comprising: Obtain the startup sequence events of the in-vehicle voice system; In response to the launch sequence event, the speech synthesis channel is warmed up, and an in-vehicle voice interaction request is generated based on the launch sequence event; the speech synthesis channel is used to process the in-vehicle voice interaction request. The in-vehicle voice interaction request is processed for scene perception to obtain the target voice broadcast scene to which the in-vehicle voice interaction request belongs. Based on the target voice broadcast scenario, inference resources are allocated to the voice synthesis channel to process the in-vehicle voice interaction request.

[0005] In some embodiments, the startup sequence event includes a voice start button trigger event, a voice activity detection event, and a voice end prediction event, and the preheating of the voice synthesis channel in response to the startup sequence event includes: In response to the voice start button trigger event, a first-level warm-up operation is performed on the voice synthesis channel; the first-level warm-up operation is used to establish the communication connection of the voice synthesis channel; In response to the speech activity detection event, a second-level warm-up operation is performed on the speech synthesis channel; the second-level warm-up operation is used to construct the inference context of the speech synthesis channel. In response to the speech end prediction event, a third-level warm-up operation is performed on the speech synthesis channel; the third-level warm-up operation is used to construct the audio output context of the speech synthesis channel.

[0006] In some embodiments, the method further includes at least one of the following: In response to the successful execution of the target preheating operation, the preset preheating state machine is controlled to switch the preheating state of the speech synthesis channel; the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation, and the preheating state includes at least the first-level preheating ready state, the second-level preheating ready state, and the third-level preheating ready state. In response to the fact that the duration of the target preheating operation exceeds a preset duration threshold, the preheating resources allocated for performing the target preheating operation are reclaimed; the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation. The voice synthesis channel is used to process continuous in-vehicle voice interaction requests initiated by the same user; the voice synthesis channel is a voice synthesis channel after performing a target preheating operation, and the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation.

[0007] In some embodiments, the startup sequence event includes a first startup sequence event triggered by a first user and a second startup sequence event triggered by a second user; the speech synthesis channel includes a first speech synthesis channel preheated based on a first preheating resource in response to the first startup sequence event, and a second speech synthesis channel preheated based on a second preheating resource in response to the second startup sequence event; the in-vehicle voice interaction request includes a first in-vehicle voice interaction request generated based on the first startup sequence event, and a second in-vehicle voice interaction request generated based on the second startup sequence event; the first speech synthesis channel is used to process the first in-vehicle voice interaction request, and the second speech synthesis channel is used to process the second in-vehicle voice interaction request.

[0008] In some embodiments, the step of performing scene-aware processing on the in-vehicle voice interaction request to obtain the target voice broadcasting scene to which the in-vehicle voice interaction request belongs includes: Obtain the source from which the in-vehicle voice interaction request is generated; the source includes the vehicle driving control system, navigation system, vehicle cockpit control system, and in-vehicle entertainment system; When the source of the generation is the vehicle driving control system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be an emergency vehicle control broadcast scenario; When the source of the request is the navigation system, the target voice broadcasting scenario to which the in-vehicle voice interaction request belongs is determined to be a navigation broadcasting scenario; In the case where the generation source is the vehicle cockpit control system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be the vehicle control feedback scenario; When the source of the generation is the in-vehicle entertainment system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be an entertainment dialogue scenario.

[0009] In some embodiments, allocating inference resources for the speech synthesis channel to process the in-vehicle voice interaction request based on the target voice broadcast scenario includes: When the target voice broadcast scenario is the emergency vehicle control broadcast scenario, inference resources are allocated from the first reserved inference resources for the voice synthesis channel to process the vehicle voice interaction request according to the first response priority corresponding to the emergency vehicle control broadcast scenario; When the target voice broadcast scenario is the navigation broadcast scenario, inference resources are allocated from the first reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the second response priority corresponding to the navigation broadcast scenario; the second response priority is lower than the first response priority; When the target voice broadcast scenario is the vehicle control feedback scenario, inference resources are allocated from the second reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the third response priority corresponding to the vehicle control feedback scenario; the third response priority is lower than the second response priority, and the second reserved inference resources are greater than the first reserved inference resources; When the target voice broadcast scenario is the entertainment dialogue scenario, inference resources are allocated from the second reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the fourth response priority corresponding to the entertainment dialogue scenario; the fourth response priority is lower than the third response priority.

[0010] In some embodiments, the method further includes: Delay monitoring is performed on multiple preset key nodes to obtain delay information for each of the multiple key nodes; the multiple key nodes include a stage for receiving the vehicle voice interaction request, a stage for allocating inference resources to the voice synthesis channel, a stage for performing voice synthesis processing on the vehicle voice interaction request, and a stage for playing the synthesized audio. The warm-up strategy and / or resource allocation strategy are adjusted based on the delay information; the warm-up strategy is used to warm up the speech synthesis channel, and the resource allocation strategy is used to allocate inference resources to the speech synthesis channel.

[0011] To achieve the above objectives, a second aspect of this application proposes a resource scheduling system for in-vehicle speech synthesis, the system comprising: The acquisition module is used to acquire the startup sequence events of the in-vehicle voice system; A preheating module is used to preheat the speech synthesis channel in response to the start sequence event, and to generate an in-vehicle voice interaction request based on the start sequence event; the speech synthesis channel is used to process the in-vehicle voice interaction request; The scene perception module is used to perform scene perception processing on the vehicle voice interaction request to obtain the target voice broadcast scene to which the vehicle voice interaction request belongs. The resource scheduling module is used to allocate inference resources for the speech synthesis channel to process the in-vehicle voice interaction request based on the target voice broadcast scenario.

[0012] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the resource scheduling method for in-vehicle speech synthesis described in the first aspect.

[0013] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the resource scheduling method for in-vehicle speech synthesis described in the first aspect.

[0014] To achieve the above objectives, a fifth aspect of the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements the resource scheduling method for in-vehicle speech synthesis provided in the first aspect above.

[0015] The resource scheduling method, system, electronic device, computer-readable storage medium, and computer program product for in-vehicle speech synthesis proposed in this application embodiment acquire in-vehicle speech start sequence events; warm up the speech synthesis channel in response to the start sequence events; and generate in-vehicle speech interaction requests based on the start sequence events. The speech synthesis channel is used to process the in-vehicle speech interaction requests; scene perception processing is performed on the in-vehicle speech interaction requests to obtain the target speech broadcast scene to which the in-vehicle speech interaction requests belong; and inference resources are allocated to the speech synthesis channel for processing the in-vehicle speech interaction requests based on the target speech broadcast scene.

[0016] Compared to the traditional approach of using a FIFO unified resource scheduling strategy to process voice interaction requests, this embodiment of the application, upon obtaining the in-vehicle voice start sequence event, responds to the start sequence event to preheat the voice synthesis channel. This voice synthesis channel is used to process the in-vehicle voice interaction request generated based on the start sequence event. Furthermore, by performing scene-aware processing on the in-vehicle voice interaction request, inference resources are allocated to the voice synthesis channel based on the target voice broadcast scene to which the in-vehicle voice interaction request belongs, when processing the in-vehicle voice interaction request based on the preheated voice synthesis channel. Thus, in the TTS resource scheduling scenario, the user behavior that triggers the start sequence event of the in-vehicle voice is used to warm up the voice synthesis channel, and inference resource scheduling is performed through the voice broadcast scenario to which the in-vehicle voice interaction request belongs. This can ensure the rapid response of in-vehicle voice in critical safety scenarios, improve resource utilization and reduce response latency in critical scenarios. In other words, the present application embodiment can solve the problem of excessive response latency of traditional in-vehicle voice interaction in critical safety scenarios, thereby improving the reliability of in-vehicle voice interaction in safety-related scenarios such as navigation prompts, and meeting the differentiated voice broadcast requirements of multiple concurrent scenarios in smart cockpits. Attached Figure Description

[0017] Figure 1 A flowchart illustrating the steps of the resource scheduling method for vehicle-mounted speech synthesis provided in this application in some embodiments; Figure 2 for Figure 1 A detailed flowchart of step S102; Figure 3 A flowchart illustrating the steps of the resource scheduling method for vehicle speech synthesis provided in this application in other embodiments; Figure 4 The resource scheduling method for vehicle-mounted speech synthesis provided in the embodiments of this application includes a schematic diagram of the preheating state machine transition process in some embodiments; Figure 5 for Figure 1A detailed flowchart of step S103; Figure 6 for Figure 1 A detailed flowchart of step S104; Figure 7 A schematic diagram of a scene-aware inference resource hierarchical scheduling algorithm for the vehicle speech synthesis resource scheduling method provided in the embodiments of this application in some embodiments; Figure 8 A flowchart illustrating the steps of the resource scheduling method for vehicle speech synthesis provided in this application in some other embodiments; Figure 9 The resource scheduling method for vehicle speech synthesis provided in the embodiments of this application involves end-to-end latency monitoring and adaptive adjustment pipelines in some embodiments; Figure 10 The system architecture diagram of the resource scheduling method for vehicle speech synthesis provided in the embodiments of this application is shown in some embodiments; Figure 11 This is a schematic diagram of the structure of the resource scheduling system for in-vehicle speech synthesis provided in an embodiment of this application; Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] It should be noted that although functional modules are divided in the device / system schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device / system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0021] First, a brief explanation of the overall concept of the resource scheduling method for in-vehicle speech synthesis provided in the embodiments of this application will be given.

[0022] Currently, the intelligence level of in-vehicle intelligent cockpits continues to improve, and voice interaction has become the core method of human-vehicle interaction. As a key output link in voice interaction, TTS (Text-to-Speech) voice synthesis service needs to simultaneously handle the broadcasting needs of multiple concurrent scenarios, such as navigation prompts, vehicle control operation feedback, entertainment content broadcasting, and intelligent dialogue. The functional attributes and safety importance of different scenarios differ fundamentally, resulting in vastly different sensitivities to voice broadcasting latency: critical safety scenarios such as navigation turn prompts and emergency vehicle control alarms have extremely high requirements for response latency, as excessive latency may directly affect driving safety; while non-critical scenarios such as entertainment chat and information broadcasting have a relatively higher tolerance for latency. Therefore, having scenario-differentiated TTS scheduling capabilities has become a core requirement for ensuring both the in-vehicle voice interaction experience and driving safety.

[0023] In related technologies, the scheduling and processing scheme of in-vehicle TTS voice synthesis services has significant technical defects: On the one hand, the system only initializes the TTS synthesis channel when a voice broadcast request arrives, and the channel initialization process itself has a fixed delay, further exacerbating the overall response latency; on the other hand, the system adopts a unified FIFO scheduling strategy, without prioritizing broadcast requests from different scenarios, and processes them entirely according to the order in which they arrive. This scheduling method is highly susceptible to high-priority critical scenario broadcasts being blocked by low-priority non-critical scenarios, resulting in unreasonable phenomena such as navigation turn prompts being broadcast later than the content of casual conversations between the user and the vehicle's infotainment system, directly causing excessively high response latency in critical safety scenarios.

[0024] To address the aforementioned issues, this application proposes a resource scheduling method, system, electronic device, computer-readable storage medium, and computer program product for in-vehicle voice synthesis. The aim is to resolve the problem of excessively high response latency in critical safety scenarios using traditional in-vehicle voice interaction, thereby improving the reliability of in-vehicle voice interaction in safety-related scenarios such as navigation prompts, and meeting the differentiated voice broadcasting needs of intelligent cockpits under multiple concurrent scenarios.

[0025] In this embodiment, the following steps are taken: A startup sequence event for the in-vehicle voice system is acquired; the voice synthesis channel is preheated in response to the startup sequence event; and an in-vehicle voice interaction request is generated based on the startup sequence event. The voice synthesis channel processes the in-vehicle voice interaction request. Scene perception processing is performed on the in-vehicle voice interaction request to obtain the target voice broadcasting scene to which the in-vehicle voice interaction request belongs. Based on the target voice broadcasting scene, inference resources are allocated to the voice synthesis channel for processing the in-vehicle voice interaction request.

[0026] Compared to the traditional approach of using a FIFO unified resource scheduling strategy to process voice interaction requests, this embodiment of the application, upon obtaining the in-vehicle voice start sequence event, responds to the start sequence event to preheat the voice synthesis channel. This voice synthesis channel is used to process the in-vehicle voice interaction request generated based on the start sequence event. Furthermore, by performing scene-aware processing on the in-vehicle voice interaction request, inference resources are allocated to the voice synthesis channel based on the target voice broadcast scene to which the in-vehicle voice interaction request belongs, when processing the in-vehicle voice interaction request based on the preheated voice synthesis channel. Thus, in the TTS resource scheduling scenario, the user behavior that triggers the start sequence event of the in-vehicle voice is used to warm up the voice synthesis channel, and inference resource scheduling is performed through the voice broadcast scenario to which the in-vehicle voice interaction request belongs. This can ensure the rapid response of in-vehicle voice in critical safety scenarios, improve resource utilization and reduce response latency in critical scenarios. In other words, the present application embodiment can solve the problem of excessive response latency of traditional in-vehicle voice interaction in critical safety scenarios, thereby improving the reliability of in-vehicle voice interaction in safety-related scenarios such as navigation prompts, and meeting the differentiated voice broadcast requirements of multiple concurrent scenarios in smart cockpits.

[0027] In some embodiments, an intelligent preheating status management mechanism may also be included. By designing a complete preheating status lifecycle management, the utilization rate of preheating resources can be further improved and the resource waste rate can be reduced. Furthermore, by performing rapid recovery and adopting a degradation mechanism for abnormal recovery when preheating fails, the abnormal recovery time can be reduced.

[0028] In some embodiments, end-to-end latency monitoring and adaptive strategy adjustment mechanisms can be used to establish end-to-end latency monitoring from request to playback, and the scheduling strategy can be dynamically adjusted according to real-time latency, thereby achieving end-to-end latency observability, reducing timeout rate and automatically optimizing scheduling strategy.

[0029] Next, the resource scheduling method, system, electronic device, computer-readable storage medium, and computer program product for vehicle speech synthesis provided in this application will be specifically described through the following embodiments, and the various detailed embodiments of the resource scheduling method for vehicle speech synthesis provided in this application will be described in detail first.

[0030] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0031] It should be noted that the resource scheduling method for in-vehicle speech synthesis provided in this application relates to the field of vehicle speech synthesis technology. The resource scheduling method for in-vehicle speech synthesis provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a vehicle, an in-vehicle terminal, or an electronic device such as a smartphone, tablet, laptop, or desktop computer associated with the vehicle and capable of communicating and interacting with it via a network. The server can be a backend server terminal device of the terminal, which can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The software can be an application implementing the resource scheduling method for in-vehicle speech synthesis, a computer program, and a storage medium carrying the computer program. It should be understood that, based on different design needs of practical applications, the terminal, server, and software of the in-vehicle speech synthesis resource scheduling method provided in this application embodiment may also be other forms not listed here, and the in-vehicle speech synthesis resource scheduling method provided in this application embodiment does not specifically limit these.

[0032] Furthermore, this application can also be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: vehicles, in-vehicle terminals, personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, personal computers (PCs), minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via communication networks. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0033] For ease of understanding and explanation, the following text will use the vehicle-mounted terminal application of the resource scheduling method for vehicle-mounted speech synthesis provided in the embodiments of this application as examples to describe the various specific embodiments of this application in detail. In some descriptions, the vehicle-mounted terminal may be simply referred to as an intelligent agent. The implementation of the resource scheduling method for vehicle-mounted speech synthesis provided in the embodiments of this application for any of the above-described subjects can refer to the implementation process of the resource scheduling method for vehicle-mounted speech synthesis described below.

[0034] Please refer to Figure 1 , Figure 1 The flowchart illustrates the steps of the resource scheduling method for in-vehicle speech synthesis provided in this application embodiment in some embodiments. It should be understood that, although... Figure 1 The flowcharts illustrating subsequent steps show the execution order of some method steps. However, based on different design needs in practical applications, the resource scheduling method for in-vehicle speech synthesis provided in this application embodiment can, of course, employ a different execution order of method steps than shown in the figures. That is, Figure 1 The order of the method steps shown does not constitute a limitation on the execution logic order of the resource scheduling method for in-vehicle speech synthesis provided in the embodiments of this application. Any other method based on... Figure 1 Reasonable changes to the order of steps shown should be included within the protection scope of the resource scheduling method for vehicle speech synthesis provided in the embodiments of this application.

[0035] like Figure 1 As shown, in some embodiments, the resource scheduling method for vehicle-mounted voice synthesis provided in this application may include steps S101 to S104 as shown below.

[0036] Step S101: Obtain the startup sequence event of the in-vehicle voice system.

[0037] It should be noted that the in-vehicle voice interaction sequence refers to key signal events that the in-vehicle terminal can capture in advance during the in-vehicle voice interaction process, before the output of the Automatic Speech Recognition (ASR) result and the in-vehicle voice interaction request (TTS synthesis request) is initiated. These events indicate that an in-vehicle voice interaction request is about to occur. Examples include the user pressing the voice button on the steering wheel or the voice button on the touchscreen, the voice activity detection (VAD) detecting that the user has started speaking, and the ASR module predicting the upcoming output result. The in-vehicle voice interaction sequence provides advance notice for TTS resource scheduling and is the core foundation for realizing the TTS warm-up mechanism and low-latency voice interaction.

[0038] The in-vehicle terminal continuously monitors the startup sequence events triggered by the user in response to in-vehicle voice commands. For example, the in-vehicle terminal can confirm the startup sequence events triggered by the user in response to in-vehicle voice commands by detecting events such as voice button press, VAD detection activation, or ASR completion.

[0039] In some embodiments, the in-vehicle terminal can obtain the user's trigger sequence events for in-vehicle voice commands through a TTS resource scheduling system. For example, the TTS resource scheduling system may include a warm-up management layer, which includes a warm-up trigger that can be used to monitor voice button presses, VAD detection, and ASR completion events.

[0040] Step S102: In response to the start sequence event, warm up the speech synthesis channel and generate an in-vehicle voice interaction request based on the start sequence event; the speech synthesis channel is used to process the in-vehicle voice interaction request.

[0041] It should be noted that the speech synthesis channel, also known as the TTS channel, is a logical execution unit at the in-vehicle speech synthesis software level. It represents an independent TTS synthesis task execution environment. Each TTS channel can independently process in-vehicle voice interaction requests (i.e., TTS synthesis requests), including a complete process such as text preprocessing, acoustic model inference, vocoder inference, and audio post-processing.

[0042] After the vehicle terminal receives any start sequence event triggered by the user, it can respond to the start sequence event to preheat the speech synthesis channel in advance. This can avoid the fixed delay inherent in the channel initialization process from exacerbating the overall response delay, thereby further reducing the TTS response delay.

[0043] In some embodiments, the vehicle terminal can leverage the predictability of in-vehicle voice interaction to design a multi-level prediction strategy to preheat the voice synthesis channel based on different start sequence events, thereby overlapping the preheating time with the user operation time and improving the effective preheating rate of the voice synthesis channel.

[0044] It should be noted that the predictability of in-vehicle voice interaction means that there is a fixed time window (about 300-500ms) between when the user presses the voice button and when they start speaking, which can be used to warm up the voice synthesis channel.

[0045] After the vehicle terminal warms up the voice synthesis channel in response to the startup sequence event, it further generates the vehicle voice interaction request to be processed based on the startup sequence event.

[0046] In some embodiments, the in-vehicle terminal can generate a currently pending in-vehicle voice interaction request when it detects a voice end prediction event in the startup sequence event. For example, suppose the voice end prediction event is an ASR recognition result event generated when the in-vehicle terminal completes the final decoding through the ASR engine and outputs the text recognition result "open the car window". In this way, the in-vehicle terminal can generate a complete in-vehicle voice interaction request based on the ASR recognition result event. The in-vehicle voice interaction request can at least include fields such as request ID, sender domain ID (e.g., smart cockpit), and sender module ID (e.g., voice assistant).

[0047] Step S103: Perform scene perception processing on the in-vehicle voice interaction request to obtain the target voice broadcast scene to which the in-vehicle voice interaction request belongs.

[0048] It should be noted that scene awareness processing can be a preliminary decision-making method for TTS resource scheduling. That is, after receiving a vehicle voice interaction request and before allocating TTS synthesis and inference resources, the vehicle terminal determines the voice broadcast scene type to which the vehicle voice interaction request belongs, thereby providing a decision basis for the subsequent allocation of TTS synthesis and inference resources.

[0049] After the vehicle terminal warms up the voice synthesis channel for the vehicle voice interaction request, it can further automatically identify which scenario type the vehicle voice interaction request belongs to, such as emergency vehicle control broadcast scenario, navigation broadcast scenario, vehicle control feedback scenario, or entertainment dialogue scenario, based on the source of the vehicle voice interaction request.

[0050] In some embodiments, the in-vehicle terminal can also identify the target voice broadcasting scenario to which the in-vehicle voice interaction request belongs by parsing the field information contained in the in-vehicle voice interaction request. For example, after parsing and determining the request ID, sender domain ID (such as smart cockpit), sender module ID (such as voice assistant), etc. of the in-vehicle voice interaction request, the in-vehicle terminal can determine the target voice broadcasting scenario to which the in-vehicle voice interaction request belongs as a vehicle control feedback scenario based on the content of these fields such as "smart cockpit" and "voice assistant".

[0051] Step S104: Based on the target voice broadcast scenario, allocate inference resources for the voice synthesis channel to process the in-vehicle voice interaction request.

[0052] It should be noted that inference resources refer to the collective hardware and software resources used for TTS model inference, including computing resources, memory resources, bandwidth resources, and concurrency resources. These inference resources can at least include inference board resources, that is, dedicated hardware acceleration card resources used for model inference computation in TTS speech synthesis. For example, inference resources could be a neural network processing unit (NPU) or an artificial intelligence (AI) chip used for TTS model inference in an in-vehicle speech synthesis scenario.

[0053] After determining the target voice broadcast scenario to which the in-vehicle voice interaction request belongs, the in-vehicle terminal further allocates inference resources that are appropriate to the urgency level of the target voice broadcast scenario to process the in-vehicle voice interaction request through the preheated voice synthesis channel.

[0054] In some embodiments, the in-vehicle terminal can dynamically allocate inference resource priorities to the speech synthesis channel for processing in-vehicle voice interaction requests based on the urgency of the target voice broadcast scenario, and then allocate inference resources to the speech synthesis channel according to this priority. The higher the urgency of the target voice broadcast scenario, the higher the priority of the inference resources allocated by the in-vehicle terminal to the speech synthesis channel for processing the in-vehicle voice interaction request. For example, emergency vehicle control broadcast scenarios have the highest urgency, navigation broadcast scenarios have a lower urgency than emergency vehicle control broadcast scenarios, vehicle control feedback scenarios have a lower urgency than navigation broadcast scenarios, and entertainment dialogue scenarios have the lowest urgency. Therefore, when an in-vehicle voice interaction request belongs to an emergency vehicle control broadcast scenario, the in-vehicle terminal allocates the highest priority inference resources to the speech synthesis channel processing that in-vehicle voice interaction request.

[0055] In some embodiments, the in-vehicle terminal can set up a four-level priority queue from high to low based on emergency vehicle control broadcast scenarios, navigation broadcast scenarios, vehicle control feedback scenarios, and entertainment dialogue scenarios, and the high-priority queue can preempt inference resources from the low-priority queue. Thus, after determining the target voice broadcast scenario to which the in-vehicle voice interaction request belongs, the in-vehicle terminal can add the request to the priority queue corresponding to that target voice broadcast scenario. When the voice synthesis channel processes the in-vehicle voice interaction requests in that priority queue, it allocates inference resources to that voice synthesis channel according to that priority queue. Furthermore, if there are still unprocessed / being processed in-process in-vehicle voice interaction requests in the low-priority queue when the current in-vehicle voice interaction request is added to the high-priority queue, the in-vehicle terminal can reallocate the inference resources (allocated / planned / being allocated) of the low-priority queue to the voice synthesis channel processing the in-vehicle voice interaction requests in the high-priority queue.

[0056] In some embodiments, in scenarios where the latency requirements for in-vehicle speech synthesis are not strict, the in-vehicle terminal only sets up a simple priority queue (which does not support preemption and warm-up) based on emergency vehicle control broadcast scenarios, navigation broadcast scenarios, vehicle control feedback scenarios, and entertainment dialogue scenarios.

[0057] In some embodiments, the in-vehicle terminal can pre-allocate different inference resources for emergency vehicle control broadcast scenarios, navigation broadcast scenarios, vehicle control feedback scenarios, and entertainment dialogue scenarios. For example, the in-vehicle terminal can pre-allocate 40% of the inference resources for emergency vehicle control broadcast scenarios and navigation broadcast scenarios, and reserve the remaining 60% of the shared inference resources for vehicle control feedback scenarios and entertainment dialogue scenarios. Thus, after determining the target voice broadcast scenario to which the in-vehicle voice interaction request belongs, the in-vehicle terminal can allocate the inference resources pre-allocated for that target voice broadcast scenario to the speech synthesis channel to process the in-vehicle voice interaction request.

[0058] In some embodiments, for in-vehicle voice synthesis environments with sufficient resources and fixed usage patterns, the in-vehicle terminal can pre-allocate a fixed proportion of resources for emergency vehicle control broadcast scenarios, navigation broadcast scenarios, vehicle control feedback scenarios, and entertainment dialogue scenarios, and these resources do not need to be dynamically adjusted.

[0059] In some embodiments, the in-vehicle terminal can, according to a preemption mechanism, allocate the inference resources currently used for processing low-priority tasks to the speech synthesis channel processing the high-priority request if a high-priority request is received while processing a low-priority task. This allows the speech synthesis channel to directly acquire resources and quickly process the request. Here, the high-priority request refers to an in-vehicle voice interaction request belonging to an emergency vehicle control broadcast scenario, a navigation broadcast scenario, or a vehicle control feedback scenario. When the high-priority request belongs to an emergency vehicle control broadcast scenario, the low-priority task can be a task processing in-vehicle voice interaction requests belonging to a navigation broadcast scenario, a vehicle control feedback scenario, or an entertainment dialogue scenario. When the high-priority request belongs to a navigation broadcast scenario, the low-priority task can be a task processing in-vehicle voice interaction requests belonging to a vehicle control feedback scenario or an entertainment dialogue scenario. When the high-priority request belongs to a vehicle control feedback scenario, the low-priority task can be a task processing in-vehicle voice interaction requests belonging to an entertainment dialogue scenario.

[0060] In some embodiments, the vehicle terminal may adopt a dynamic degradation strategy. If it is determined that the vehicle voice interaction request belongs to an entertainment dialogue scenario and the current inference resources are insufficient (such as high-priority tasks that are preempting resources to process vehicle voice interaction requests belonging to emergency vehicle control broadcast scenarios / navigation broadcast scenarios / vehicle control feedback scenarios), the vehicle voice interaction request may be downgraded to the local TTS of the vehicle terminal for processing.

[0061] In this embodiment, the in-vehicle terminal continuously monitors user-triggered activation sequence events for in-vehicle voice commands. Then, in response to these activation sequence events, it preheats the speech synthesis channel, thereby avoiding the fixed latency inherent in the channel initialization process from exacerbating the overall response delay and further reducing TTS response latency. Based on these activation sequence events, it generates a currently pending in-vehicle voice interaction request. Subsequently, the in-vehicle terminal automatically identifies the target voice broadcast scenario to which the in-vehicle voice interaction request belongs. Based on the urgency of the target voice broadcast scenario, it allocates inference resources appropriate to the preheated speech synthesis channel to process the in-vehicle voice interaction request.

[0062] Compared to the traditional approach of using a FIFO unified resource scheduling strategy to process voice interaction requests, this embodiment of the application, upon obtaining the in-vehicle voice start sequence event, responds to the start sequence event to preheat the voice synthesis channel. This voice synthesis channel is used to process the in-vehicle voice interaction request generated based on the start sequence event. Furthermore, by performing scene-aware processing on the in-vehicle voice interaction request, inference resources are allocated to the voice synthesis channel based on the target voice broadcast scene to which the in-vehicle voice interaction request belongs, when processing the in-vehicle voice interaction request based on the preheated voice synthesis channel. Thus, in the TTS resource scheduling scenario, the user behavior that triggers the start sequence event of the in-vehicle voice is used to warm up the voice synthesis channel, and inference resource scheduling is performed through the voice broadcast scenario to which the in-vehicle voice interaction request belongs. This can ensure the rapid response of in-vehicle voice in critical safety scenarios, improve resource utilization and reduce response latency in critical scenarios. In other words, the present application embodiment can solve the problem of excessive response latency of traditional in-vehicle voice interaction in critical safety scenarios, thereby improving the reliability of in-vehicle voice interaction in safety-related scenarios such as navigation prompts, and meeting the differentiated voice broadcast requirements of multiple concurrent scenarios in smart cockpits.

[0063] Furthermore, by designing a priority queue in the vehicle terminal and enabling high-priority requests to preempt low-priority resources, 40% of inference resources are reserved for critical safety scenarios such as emergency vehicle control broadcasting and navigation broadcasting to ensure that requests in critical scenarios do not queue. Based on the preemption mechanism, high-priority requests can interrupt low-priority tasks and immediately acquire resources for processing, which can further reduce the response time of vehicle voice interaction requests in critical safety scenarios (e.g., <300ms).

[0064] In some embodiments, the start sequence events include a voice start button trigger event, a voice activity detection event, and a voice end prediction event.

[0065] It should be noted that the voice activation button trigger event can be the system event corresponding to the user pressing the steering wheel voice button or the touch screen voice button, the voice activity detection event can be the system event corresponding to the VAD detecting that the user has started speaking, and the voice end prediction event can be the system event corresponding to the ASR module predicting the output result based on the voice end detection.

[0066] Please refer to Figure 2 , Figure 2 for Figure 1 A detailed flowchart of step S102.

[0067] like Figure 2As shown, in some embodiments, the step of "warming up the speech synthesis channel in response to the start sequence event" in step S102 above may include steps S201 to S203 as shown below.

[0068] Step S201: In response to the voice start button trigger event, perform a first-level warm-up operation on the voice synthesis channel; the first-level warm-up operation is used to establish a communication connection for the voice synthesis channel.

[0069] It should be noted that the first-level warm-up operation can be the warm-up operation used to build the communication connection of the speech synthesis channel in the multi-level warm-up strategy pre-set by the in-vehicle terminal. This multi-level warm-up strategy utilizes the predictability of in-vehicle voice interaction to perform different warm-up operations on the speech synthesis channel at different times (the time points when different start sequence events are detected), thereby overlapping the warm-up time for the speech synthesis channel with the time when the user operation triggers the start event, thus improving the effective warm-up rate and reducing TTS response latency. For example, the multi-level warm-up strategy can be a three-level warm-up strategy (L0 / L1 / L2) designed by the terminal device based on the multi-level warm-up mechanism of the in-vehicle scenario, utilizing the temporal characteristics of user operation: pressing the voice button at time T0, starting to speak at time T1, ASR recognition at time T2, and ITS broadcast at time T3. The design of the warm-up timing of each level in this three-level warm-up strategy (L0 / L1 / L2) can be shown in the table below:

[0070] Based on this, the first-stage preheating operation can be used to preheat L0.

[0071] When the vehicle terminal warms up the voice synthesis channel in response to a user-triggered start sequence event, it first performs a first-level warm-up operation on the voice synthesis channel in response to the user-triggered voice start button event, thereby establishing a communication connection for the voice synthesis channel.

[0072] In some embodiments, when the vehicle terminal performs a first-level warm-up operation to establish a communication connection for the voice synthesis channel, the warm-up content of the first-level warm-up operation may include: establishing a Transmission Control Protocol (TCP) connection, a TLS handshake, and upgrading a WebSocket network communication protocol.

[0073] In some embodiments, the vehicle terminal can be designed with a first-level warm-up operation time of ≤50ms. Here, 50ms can be the warm-up start time, and the actual connection establishment can be completed during the user's speech (e.g., 300-500ms).

[0074] For example, the vehicle terminal may perform a first-level warm-up operation on the voice synthesis channel in response to a voice start button trigger event: when the user presses the steering wheel voice button or the touch screen voice button, L0 warm-up is triggered to perform warm-up content such as TCP connection establishment, TLS handshake, and WebSocket upgrade.

[0075] Step S202: In response to the speech activity detection event, perform a second-level warm-up operation on the speech synthesis channel; the second-level warm-up operation is used to construct the inference context of the speech synthesis channel.

[0076] It should be noted that the second-level warm-up operation can be the warm-up operation used to construct the inference context of the speech synthesis channel in the multi-level warm-up strategy mentioned above. For example, when the multi-level warm-up strategy is a three-level warm-up strategy of L0 / L1 / L2, the second-level warm-up operation can be the L1 warm-up.

[0077] After performing the first-level warm-up operation to establish a communication connection for the speech synthesis channel, the vehicle terminal further performs a second-level warm-up operation on the speech synthesis channel in response to a voice activity detection event, thereby constructing the inference context of the speech synthesis channel.

[0078] In some embodiments, when the vehicle terminal performs a second-level warm-up operation to construct the inference context of the speech synthesis channel, the warm-up content of the second-level warm-up operation may include: loading the TTS acoustic model, initializing the vocoder, pre-allocating inference resources, etc.

[0079] In some embodiments, the vehicle terminal may be designed to have a second-level preheating operation time of 100ms.

[0080] For example, the vehicle terminal may perform a second-level warm-up operation on the speech synthesis channel in response to a voice activity detection event: when the VAD detects that the user has started speaking, it triggers L1 warm-up to perform preheating content such as TTS acoustic model loading, vocoder initialization, and inference resource pre-allocation within 100ms.

[0081] Step S203: In response to the speech end prediction event, perform a third-level warm-up operation on the speech synthesis channel; the third-level warm-up operation is used to construct the audio output context of the speech synthesis channel.

[0082] It should be noted that the third-level warm-up operation can be the warm-up operation used to construct the audio output context of the speech synthesis channel in the multi-level warm-up strategy described above. For example, when the multi-level warm-up strategy is a three-level warm-up strategy of L0 / L1 / L2, the third-level warm-up operation can be the L2 warm-up.

[0083] After performing the second-level warm-up operation to construct the inference context of the speech synthesis channel, the vehicle terminal further performs the third-level warm-up operation on the speech synthesis channel in response to the speech end prediction event when it detects the speech end prediction event, thereby constructing the audio output context of the speech synthesis channel.

[0084] In some embodiments, when the vehicle terminal performs a third-level warm-up operation to construct the audio output context of the speech synthesis channel, the warm-up content of the third-level warm-up operation may include: audio decoder initialization, AudioTrack player creation, audio focus acquisition, etc.

[0085] In some embodiments, the vehicle terminal may be designed to have a third-level preheating operation time of 30ms.

[0086] For example, the vehicle terminal may perform a third-level warm-up operation on the speech synthesis channel in response to the speech end prediction event: when the ASR module predicts that the output result is about to be released, it may trigger L2 warm-up to perform warm-up content such as audio decoder initialization, AudioTrack player creation, and audio focus acquisition within 30ms.

[0087] In this embodiment, by taking advantage of the predictability of in-vehicle voice interaction, a multi-level warm-up strategy is designed to overlap the warm-up time with the user operation time, thereby warming up the voice synthesis channel in advance. In this way, the fixed delay inherent in the channel initialization process itself can be avoided from exacerbating the overall response delay, so that the TTS response delay can be further reduced (for example, the TTS response delay can be reduced by another 100-200ms).

[0088] Please refer to Figure 3 , Figure 3 The flowcharts of the resource scheduling method for vehicle-mounted speech synthesis provided in this application are shown in some other embodiments.

[0089] like Figure 3 As shown, in some embodiments, the resource scheduling method for vehicle-mounted voice synthesis provided in this application may further include at least one of the following steps S301 to S303.

[0090] Step S301: In response to the successful execution of the target warm-up operation, control the preset warm-up state machine to switch the warm-up state of the speech synthesis channel; the target warm-up operation includes any one of the first-level warm-up operation, the second-level warm-up operation, and the third-level warm-up operation, and the warm-up state includes at least the first-level warm-up ready state, the second-level warm-up ready state, and the third-level warm-up ready state.

[0091] It should be noted that the warm-up state machine can be designed based on the principle of finite state machine (FSM) to uniformly manage the entire life cycle warm-up state transition of the TTS speech synthesis channel from complete idle to formal synthesis. The core function of this warm-up state machine is to decompose the serial process of completing the initialization of the speech synthesis channel at one time after receiving the synthesis request into a multi-stage, event-driven, hierarchical and progressive warm-up process, thereby achieving an optimal balance between TTS response speed and system resource consumption.

[0092] In some embodiments, the preheating state machine can be designed as a complete state transition from NONE to L0_READY to L1_READY to L2_READY to ACTIVE. Here, L0_READY is the first-level preheating ready state, L1_READY is the second-level preheating ready state, and L2_READY is the third-level preheating ready state.

[0093] After executing each level of preheating operation (i.e., the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation) of the speech synthesis channel based on a multi-level preheating strategy, the vehicle terminal controls the preheating state machine to switch the preheating state of the speech synthesis channel. That is, after successfully executing the first-level preheating operation, the vehicle terminal immediately controls the preheating state machine to switch the preheating state of the speech synthesis channel to the first-level preheating ready state; after successfully executing the second-level preheating operation, it immediately controls the preheating state machine to switch the preheating state of the speech synthesis channel to the second-level preheating ready state; and after successfully executing the third-level preheating operation, it immediately controls the preheating state machine to switch the preheating state of the speech synthesis channel to the third-level preheating ready state.

[0094] For example, the vehicle terminal can be configured as follows: Figure 4 The preheating state machine transition process shown describes the following state transitions during the sequential execution of L0, L1, and L2 preheating of the speech synthesis channel based on the L0 / L1 / L2 three-level preheating strategy: NONE→L0_PREHEATING→L0_READY→L1_PREHEATING→L1_READY→L2_PREHEATING→L2_READY→ACTIVE.

[0095] Step S302: In response to the fact that the duration of the target preheating operation exceeds a preset duration threshold, the preheating resources allocated for the execution of the target preheating operation are reclaimed; the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation.

[0096] When the vehicle terminal performs each level of preheating operation (i.e., the first level preheating operation, the second level preheating operation, and the third level preheating operation) on the speech synthesis channel based on the multi-level preheating strategy, if the duration of one or more of the target preheating operations exceeds the duration threshold corresponding to the target preheating operation, the vehicle terminal will reclaim the preheating resources pre-allocated for performing the target preheating operation.

[0097] In some embodiments, the vehicle terminal may also reclaim the preheating resources allocated to the preheating speech synthesis channel if the second preheating operation is not triggered within a first duration after the execution of the first preheating operation, if the third preheating operation is not triggered within a second duration after the execution of the second preheating operation, if no vehicle voice interaction request is received within a third duration after the execution of the third preheating operation, and also reclaim the preheating resources allocated to the preheating speech synthesis channel after the preheated speech synthesis channel has completed its task of processing the vehicle voice interaction request.

[0098] For example, the vehicle terminal can reclaim the preheating resources by converting L0_READY to NONE if there is no L1 trigger within 10 seconds after L0 preheating, or by converting L1_READY to L0_READY if there is no L2 trigger within 5 seconds after L1 preheating, or by converting L1_READY to L1_READY if there is no request generated within 2 seconds after L2 preheating, and by converting ACTIVE to NONE after the preheating speech synthesis channel completes its task of processing vehicle voice interaction requests.

[0099] In some embodiments, the in-vehicle terminal can also, in any state of warming up the speech synthesis channel, immediately release all resources allocated to the speech synthesis channel in response to the user canceling the in-vehicle speech synthesis task (such as issuing a "cancel" voice command). For example, after detecting a voice key press / VAD detecting speech / ASR recognition completion to warm up the speech channel, if a "cancel" voice command is received from the user, the current L0_READY / L1_READY / L2_READY / ACTIVE states after warming up the speech synthesis channel are directly changed to NONE state, thereby immediately reclaiming the warm-up resources.

[0100] Step S303: Process continuous in-vehicle voice interaction requests initiated by the same user based on the voice synthesis channel; the voice synthesis channel is the voice synthesis channel after performing the target preheating operation, and the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation.

[0101] It should be noted that in-vehicle voice interaction can be triggered by different users of the vehicle. That is, different users can trigger the activation sequence event to generate different in-vehicle voice interaction requests. In addition, the same user of the vehicle can initiate continuous in-vehicle voice interaction requests while activating in-vehicle voice interaction. For example, after pressing the voice button and initiating an in-vehicle voice interaction request about navigation, the driver can continue to issue voice control commands about the vehicle's smart cockpit, thus initiating a new in-vehicle voice interaction request about cockpit control. In this way, the two in-vehicle voice interaction requests are considered as continuous in-vehicle voice interaction requests initiated by the driver.

[0102] After the vehicle terminal responds to the user-triggered startup sequence event and performs any one of the first-level, second-level, or third-level preheating operations to preheat the speech synthesis channel, it can process continuous vehicle voice interaction requests initiated by the user based on the preheated speech synthesis channel, thereby supporting the reuse of preheated resources for continuous voice interaction requests from the same user.

[0103] In some embodiments, when the vehicle terminal processes consecutive vehicle voice interaction requests initiated by the same user based on the preheated voice synthesis channel, it can reuse the established WebSocket connection for the consecutive voice interaction requests of the same user.

[0104] In some embodiments, the in-vehicle terminal can also reuse the loaded TTS model for in-vehicle voice interaction requests with the same timbre based on the preheated speech synthesis channel.

[0105] In some embodiments, the in-vehicle terminal can also reuse an initialized audio decoder instance for in-vehicle voice interaction requests of the same audio format, based on the preheated speech synthesis channel.

[0106] In some embodiments, the startup sequence event includes a first startup sequence event triggered by a first user and a second startup sequence event triggered by a second user; the speech synthesis channel includes a first speech synthesis channel preheated based on a first preheating resource in response to the first startup sequence event, and a second speech synthesis channel preheated based on a second preheating resource in response to the second startup sequence event; the in-vehicle voice interaction request includes a first in-vehicle voice interaction request generated based on the first startup sequence event, and a second in-vehicle voice interaction request generated based on the second startup sequence event; the first speech synthesis channel is used to process the first in-vehicle voice interaction request, and the second speech synthesis channel is used to process the second in-vehicle voice interaction request.

[0107] When different users of the vehicle trigger a startup sequence event to initiate an in-vehicle voice interaction request, the in-vehicle terminal can perform multi-user concurrent warm-up. That is, in response to the first startup sequence event triggered by the first user, the first voice synthesis channel is warmed up, and the first in-vehicle voice interaction request generated based on the first startup sequence event is processed based on the first voice synthesis channel. Also, in response to the second startup sequence event triggered by the second user, the second voice synthesis channel is warmed up, and the second in-vehicle voice interaction request generated based on the second startup sequence event is processed based on the second voice synthesis channel.

[0108] In some embodiments, the vehicle terminal can manage concurrent preheating for multiple users through a concurrent controller, reasonably allocate preheating resource quotas to different users, and resolve resource conflicts.

[0109] In some embodiments, during the preheating process of the voice synthesis channel, if a preheating failure occurs at any preheating level, the vehicle terminal can also execute a fast recovery and degradation processing mechanism.

[0110] For example, the vehicle terminal can be pre-designed with the state management components shown in the table below:

[0111] In this way, the vehicle terminal can manage the state transition logic of the preheating state machine using a state machine controller, manage the allocation and reclamation of preheating resources using a resource lifecycle manager, monitor the timeout status of each preheating state using a timeout monitor, and manage multi-user concurrent preheating using a concurrency controller during the preheating of the voice synthesis channel.

[0112] In this embodiment, a complete preheating state lifecycle management system (including state transition, timeout recovery, and resource reuse) is designed using an intelligent preheating state management mechanism in the vehicle terminal. Timeout management is implemented for the preheating speech synthesis channel, with independent timeout periods set for each preheating level to prevent long-term resource occupation. Preheated resources can be reused for consecutive requests from the same user, and concurrent preheating by multiple users is supported for reasonable resource allocation. Furthermore, based on an anomaly recovery strategy, a rapid recovery and degradation mechanism is implemented when any operation in the preheating speech synthesis channel fails. This further improves the utilization rate of preheating resources used for the preheating speech synthesis channel, reduces resource waste, and lowers anomaly recovery time (e.g., reducing anomaly recovery time to <100ms).

[0113] Please refer to Figure 5 , Figure 5 Figure 1 A detailed flowchart of step S103.

[0114] like Figure 5As shown, in some embodiments, step S103 above: performing scene perception processing on the in-vehicle voice interaction request to obtain the target voice broadcast scene to which the in-vehicle voice interaction request belongs may include steps S501 to S505 as shown below.

[0115] Step S501: Obtain the source of the vehicle voice interaction request; the source of the request includes the vehicle driving control system, navigation system, vehicle cockpit control system and vehicle entertainment system.

[0116] When the vehicle terminal automatically identifies the target voice broadcast scenario based on the source of the vehicle voice interaction request, it first determines which of the following is the source of the vehicle voice interaction request: the vehicle driving control system, the navigation system, the vehicle cockpit control system, or the vehicle entertainment system.

[0117] In some embodiments, the in-vehicle terminal can also determine the source of the in-vehicle voice interaction request by parsing the field information contained in the request. For example, after parsing and determining the request ID, sender domain ID (such as smart cockpit), sender module ID (such as voice assistant), etc. of the in-vehicle voice interaction request, the in-vehicle terminal can determine that the source of the in-vehicle voice interaction request is the vehicle cockpit control system based on the content of these fields, such as "smart cockpit" and "voice assistant".

[0118] Step S502: If the source of the generation is the vehicle driving control system, determine that the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is an emergency vehicle control broadcast scenario.

[0119] The vehicle terminal can pre-establish a mapping relationship between the vehicle voice interaction request generated by the vehicle driving control system and the emergency vehicle control broadcast scenario. Thus, if it is determined that the source of the vehicle voice interaction request generated based on the startup sequence event is the vehicle driving control system, the target voice broadcast scenario to which the vehicle voice interaction request belongs can be directly determined as the emergency vehicle control broadcast scenario.

[0120] Step S503: If the source of the generation is the navigation system, determine that the target voice broadcasting scenario to which the in-vehicle voice interaction request belongs is the navigation broadcasting scenario.

[0121] The vehicle terminal can also pre-establish a mapping relationship between the vehicle voice interaction requests generated by the navigation system and the navigation broadcast scenario. Thus, if it is determined that the source of the vehicle voice interaction request generated based on the startup sequence event is the navigation system, the target voice broadcast scenario to which the vehicle voice interaction request belongs can be directly determined as the navigation broadcast scenario.

[0122] Step S504: If the source of the generation is the vehicle cockpit control system, determine that the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is the vehicle control feedback scenario.

[0123] The vehicle terminal can also pre-establish a mapping relationship between the vehicle voice interaction request generated by the vehicle cockpit control system and the vehicle control feedback scenario. Thus, if it is determined that the source of the vehicle voice interaction request generated based on the startup sequence event is the vehicle cockpit control system, the target voice broadcast scenario to which the vehicle voice interaction request belongs can be directly determined as the vehicle control feedback scenario.

[0124] Step S505: If the generation source is the in-vehicle entertainment system, determine that the target voice broadcasting scenario to which the in-vehicle voice interaction request belongs is an entertainment dialogue scenario.

[0125] The vehicle terminal can pre-establish a mapping relationship between the vehicle voice interaction requests generated by the vehicle entertainment system and the entertainment dialogue scenario. Thus, if it is determined that the source of the vehicle voice interaction request generated based on the startup sequence event is the vehicle entertainment system, the target voice broadcast scenario to which the vehicle voice interaction request belongs can be directly determined to be the entertainment dialogue scenario.

[0126] Please refer to Figure 6 , Figure 6 for Figure 1 A detailed flowchart of step S104.

[0127] like Figure 6 As shown, in some embodiments, step S104 above: allocating inference resources for the voice synthesis channel to process the in-vehicle voice interaction request based on the target voice broadcast scenario, may include steps S601 to S604 as shown below.

[0128] Step S601: When the target voice broadcast scenario is the emergency vehicle control broadcast scenario, according to the first response priority corresponding to the emergency vehicle control broadcast scenario, allocate inference resources from the first reserved inference resources for the voice synthesis channel to process the vehicle voice interaction request.

[0129] It should be noted that the first response priority can be adapted to the urgency of the emergency vehicle control broadcast scenario. That is, if the urgency of the emergency vehicle control broadcast scenario is the highest compared to the urgency of the navigation broadcast scenario, vehicle control feedback scenario, and entertainment dialogue scenario, then the first response priority will also be the highest. In addition, the first reserved inference resource can be 40% of the inference resources that the in-vehicle terminal reserves for the emergency vehicle control broadcast scenario and the navigation broadcast scenario respectively.

[0130] After the vehicle terminal automatically identifies the target voice broadcast scenario based on the source of the vehicle voice interaction request, it can allocate inference resources from the first reserved inference resources to process the vehicle voice interaction request if the target voice broadcast scenario is an emergency vehicle control broadcast scenario with the highest urgency level.

[0131] Step S602: When the target voice broadcast scenario is the navigation broadcast scenario, according to the second response priority corresponding to the navigation broadcast scenario, inference resources are allocated from the first reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request; the second response priority is lower than the first response priority.

[0132] It should be noted that the second response priority can be adapted to the urgency of the navigation broadcast scenario. Since the urgency of the navigation broadcast scenario is lower than that of the emergency vehicle control broadcast scenario, the second response priority is lower than the first response priority.

[0133] When the target voice broadcast scenario is a navigation broadcast scenario, the vehicle terminal can allocate inference resources from the first reserved inference resources according to the second response priority corresponding to the navigation broadcast scenario, and also allocate inference resources for the voice synthesis channel to process the vehicle voice interaction request.

[0134] Step S603: When the target voice broadcast scenario is the vehicle control feedback scenario, according to the third response priority corresponding to the vehicle control feedback scenario, inference resources are allocated from the second reserved inference resources for the voice synthesis channel to process the vehicle voice interaction request; the third response priority is lower than the second response priority, and the second reserved inference resources are greater than the first reserved inference resources.

[0135] It should be noted that the third response priority can be adapted to the urgency of the vehicle control feedback scenario. Since the urgency of the vehicle control feedback scenario is lower than that of the navigation broadcast scenario, the third response priority is lower than the second response priority. In addition, the second reserved inference resource can be 40% of the inference resources reserved by the in-vehicle terminal for the vehicle control feedback scenario and the entertainment dialogue scenario respectively.

[0136] When the target voice broadcast scenario is a vehicle control feedback scenario, the vehicle terminal can allocate inference resources from the second reserved inference resources according to the third response priority corresponding to the vehicle control feedback scenario to process the vehicle voice interaction request for the voice synthesis channel.

[0137] Step S604: When the target voice broadcast scenario is the entertainment dialogue scenario, according to the fourth response priority corresponding to the entertainment dialogue scenario, inference resources are allocated from the second reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request; the fourth response priority is lower than the third response priority.

[0138] It should be noted that the fourth response priority can be adapted to the urgency of the entertainment dialogue scenario. Since the urgency of the entertainment dialogue scenario is lower than that of the vehicle control feedback scenario, the fourth response priority is lower than the third response priority.

[0139] When the target voice broadcast scenario is an entertainment dialogue scenario, the vehicle terminal can allocate inference resources from the second reserved inference resources according to the fourth response priority corresponding to the entertainment dialogue scenario to process the vehicle voice interaction request for the voice synthesis channel.

[0140] For example, the vehicle-mounted terminal can adopt such as Figure 7 The scene-aware inference resource hierarchical scheduling algorithm shown here, after an in-vehicle voice interaction request arrives, performs scene awareness to automatically identify the target voice broadcast scene to which the request belongs. Then, it performs priority allocation to dynamically allocate inference resource priorities to the speech synthesis channel handling the request based on the urgency of the target voice broadcast scene. This, in turn, schedules the corresponding inference board resources for the speech synthesis channel to process the request. The scene priority is defined as follows:

[0141] Among them, priorities P0 to P4 can be the first response priority to the fourth response priority mentioned above. P0+P1 share 40% of the reserved inference resources mentioned above as the first reserved inference resources, and share the remaining 60% as the second reserved inference resources mentioned above.

[0142] Furthermore, based on the aforementioned scenario priority definition, when allocating inference resources for the speech synthesis channel to process in-vehicle voice interaction requests, the in-vehicle terminal can employ the resource scheduling algorithm shown in the table below:

[0143] That is, when the target voice broadcast scenario of the in-vehicle voice interaction request is an emergency vehicle control broadcast scenario / navigation broadcast scenario, and the inference resource priority is dynamically allocated to the in-vehicle voice interaction request as P0 / P1, the speech synthesis channel uses reserved resources (40%) to process the in-vehicle voice interaction request. However, when the target voice broadcast scenario is a vehicle control feedback scenario / entertainment dialogue scenario, and the inference resource priority is dynamically allocated to the in-vehicle voice interaction request as P2 / P3, the speech synthesis channel uses shared remaining resources (60%) to process the in-vehicle voice interaction request.

[0144] In some embodiments, when the vehicle terminal executes the above-mentioned resource scheduling algorithm, it can check the availability of reserved resources to trigger a preemption mechanism when a high-priority request arrives. That is, it performs resource evaluation based on the detection of reserved resources: if the reserved resources are insufficient, it evaluates the currently running low-priority tasks. When the preemption rule indicates that the low-priority task can be preempted, it interrupts the low-priority task, releases resources to the high-priority task, and further restores the interrupted low-priority task after the high-priority task is completed.

[0145] In this embodiment, by adopting a scene-aware hierarchical scheduling mechanism for inference resources in the vehicle terminal, the priority of inference resources is dynamically allocated according to the urgency of the broadcast scenario to which the vehicle voice interaction request belongs, and high-priority preemption is supported. This can ensure that the voice synthesis response latency in critical safety scenarios such as emergency vehicle control and navigation broadcast is greatly reduced (e.g., response time <300ms, latency reduced by 90% in critical scenarios), and can make full use of inference resources, avoiding the phenomenon of wasted inference resources (e.g., inference resource utilization rate reaches 90%).

[0146] Please refer to Figure 8 , Figure 8 The following is a flowchart illustrating the steps of the resource scheduling method for vehicle-mounted speech synthesis provided in this application embodiment in some other embodiments.

[0147] like Figure 8 As shown, in some embodiments, the resource scheduling method for vehicle-mounted voice synthesis provided in this application may further include steps S801 and S802 as shown below.

[0148] Step S801: Perform delay monitoring on multiple preset key nodes to obtain delay information for each of the multiple key nodes; the multiple key nodes include a step of receiving the vehicle voice interaction request, a step of allocating inference resources to the voice synthesis channel, a step of performing voice synthesis processing on the vehicle voice interaction request, and a step of playing the synthesized audio.

[0149] The vehicle terminal can design an end-to-end latency monitoring and adaptive prediction strategy and a resource allocation adjustment mechanism for the entire link of preheating the speech synthesis channel and allocating inference resources to the speech synthesis channel based on scene awareness to process vehicle voice interaction requests. First, by establishing full-link latency monitoring from request to playback, latency monitoring is performed on multiple key nodes, such as the stage of receiving vehicle voice interaction requests, the stage of allocating inference resources to the speech synthesis channel, the stage of speech synthesis processing of vehicle voice interaction requests, and the stage of playing synthesized audio, to obtain the latency information of each of these key nodes.

[0150] In some embodiments, the vehicle terminal can embed delay points at multiple key nodes, such as the stage of receiving vehicle voice interaction requests, the stage of allocating inference resources to the voice synthesis channel, the stage of performing voice synthesis processing on vehicle voice interaction requests, and the stage of playing synthesized audio, so as to perform delay monitoring processing on these multiple key nodes and obtain the delay information of each of the multiple key nodes.

[0151] Step S802: Adjust the warm-up strategy and / or resource allocation strategy based on the delay information; the warm-up strategy is used to warm up the speech synthesis channel, and the resource allocation strategy is used to allocate inference resources to the speech synthesis channel.

[0152] After obtaining the latency information of any one or more key nodes among multiple key nodes, the vehicle terminal compares the latency information with the latency budget allocation corresponding to the node, thereby triggering an early warning when the node exceeds the budget, and adaptively and dynamically adjusting the warm-up strategy of the preheating speech synthesis channel and / or the resource allocation strategy for allocating inference resources to the speech synthesis channel based on the historical latency data of the node.

[0153] In some embodiments, the vehicle terminal can allocate differentiated delay budgets for key nodes such as request reception, resource allocation, model loading, synthesis completion, and playback start based on the different scenarios mentioned above (emergency vehicle control broadcast scenario, navigation broadcast scenario, vehicle control feedback scenario, and entertainment dialogue scenario).

[0154] In some embodiments, the vehicle terminal can trigger an alert and initiate a degradation strategy if the budget is exceeded at any stage.

[0155] In some embodiments, the vehicle terminal can also periodically statistically analyze the latency distribution of each scenario based on the latency information of multiple key nodes, thereby optimizing the hot strategy and resource allocation strategy.

[0156] For example, the vehicle terminal can be configured as follows: Figure 9The end-to-end latency monitoring and adaptive adjustment pipeline shown establishes end-to-end latency monitoring from request to playback for the entire link between warming up the speech synthesis channel and allocating inference resources to the speech synthesis channel based on scene awareness to process in-vehicle voice interaction requests. It also dynamically adjusts the warm-up strategy and inference resource scheduling strategy based on real-time latency. The latency tracking design can be illustrated in the table below.

[0157]

[0158] In addition, when the vehicle terminal adjusts the preheating strategy and inference resource scheduling strategy based on the real-time latency monitored by the embedded points, it can adopt a preset adaptive adjustment mechanism to dynamically adjust the latency budget. That is, adjust the latency budget of each link according to historical data, and optimize the preheating strategy and inference resource scheduling strategy. That is, adjust the triggering timing and preheating content according to the preheating success rate, and adjust the resource reservation ratio according to the usage frequency of the scenario.

[0159] In this embodiment, by adopting an end-to-end latency monitoring and adaptive strategy adjustment mechanism in the vehicle terminal device, end-to-end latency monitoring from request to playback is established. The scheduling strategy is dynamically adjusted according to the real-time latency, which enables end-to-end latency observation in the entire process of vehicle speech synthesis, reduces the speech synthesis response timeout rate, and automatically optimizes the warm-up strategy and inference resource scheduling strategy.

[0160] Next, we present an embodiment of the vehicle speech synthesis resource scheduling method provided in this application in a TTS resource scheduling system based on multi-level warm-up and scene priority.

[0161] Please refer to Figure 10 , Figure 10 The system architecture diagram of the resource scheduling method for vehicle speech synthesis provided in the embodiments of this application is shown in some embodiments.

[0162] like Figure 10 As shown, in some embodiments, a TTS resource scheduling system based on multi-level preheating and scene priority may include: a preheating management layer, a resource scheduling layer, and an execution monitoring layer. These layers work collaboratively to achieve multi-level preheating and scene priority scheduling. Among these, The preheating management layer includes: a preheating trigger, used to listen for voice key press (L0), VAD detection (L1), and ASR completion (L2) events; a state machine manager, used to manage the state transitions from NONE to L0 to L1 to L2 to ACTIVE; a resource pre-allocator, used to pre-allocate corresponding resources for different preheating levels; and a timeout reclaimer, used to manage the lifecycle of preheating resources and reclaim timeout resources in a timely manner. The resource scheduling layer includes: a scene recognizer, used to automatically identify the scene type (P0-P3) based on the request source; a priority queue manager, used to maintain a four-level priority queue and support preemptive scheduling; a resource pool manager, used to manage the reserved resource pool and the shared resource pool; and a degradation decision-maker, used to decide whether to degrade to the local TTS when resources are scarce. The execution monitoring layer includes: a latency tracking collector, used to collect timestamps at key nodes; a performance monitor, used to monitor latency and resource usage in various scenarios in real time; and an adaptive adjuster, used to dynamically adjust scheduling strategies based on monitoring data.

[0163] In this embodiment, a three-level preheating mechanism (L0 / L1 / L2) is adopted, which utilizes the timing of user voice interaction to achieve overlapping preheating times. Based on a four-level scenario priority scheduling system (P0-P3), it supports resource preemption for low-priority requests by high-priority requests. A resource management strategy of dynamic reservation and intelligent preemption is adopted, reserving 40% of resources to ensure core scenarios. Furthermore, based on a complete preheating state machine and end-to-end latency monitoring, full lifecycle management and full-link observable operation of resources are achieved. This reduces the voice broadcast latency of P0 / P1 core scenarios to less than 300ms, with a scenario latency reduction of over 90%. The utilization rate of related inference resources can also be increased to 90%, an increase of about 80%. This ensures the real-time nature of driving safety-related broadcasts and solves the problem of idle and wasted resources.

[0164] It should be noted that the TTS resource scheduling system based on multi-level preheating and scene priority, which implements multi-level preheating and scene priority scheduling based on the preheating management layer, resource scheduling layer and execution monitoring layer, is the same as the various embodiments of the resource scheduling method for vehicle speech synthesis provided in the embodiments of this application for the above-mentioned vehicle terminal equipment, and will not be described again here.

[0165] Please refer to Figure 11 This application also provides a resource scheduling system for in-vehicle speech synthesis, which can implement the above-mentioned resource scheduling method for in-vehicle speech synthesis.

[0166] like Figure 11 As shown in the embodiments of this application, the in-vehicle speech synthesis resource scheduling system may include: The acquisition module is used to acquire the startup sequence events of the in-vehicle voice system; A preheating module is used to preheat the speech synthesis channel in response to the start sequence event, and to generate an in-vehicle voice interaction request based on the start sequence event; the speech synthesis channel is used to process the in-vehicle voice interaction request; The scene perception module is used to perform scene perception processing on the vehicle voice interaction request to obtain the target voice broadcast scene to which the vehicle voice interaction request belongs. The resource scheduling module is used to allocate inference resources for the speech synthesis channel to process the in-vehicle voice interaction request based on the target voice broadcast scenario.

[0167] In some embodiments, the startup sequence events include a voice start button trigger event, a voice activity detection event, and a voice end prediction event. The warm-up module is further configured to perform a first-level warm-up operation on the voice synthesis channel in response to the voice start button trigger event; the first-level warm-up operation is used to establish a communication connection for the voice synthesis channel; a second-level warm-up operation is performed on the voice synthesis channel in response to the voice activity detection event; the second-level warm-up operation is used to establish an inference context for the voice synthesis channel; and a third-level warm-up operation is performed on the voice synthesis channel in response to the voice end prediction event; the third-level warm-up operation is used to establish an audio output context for the voice synthesis channel.

[0168] In some embodiments, the preheating module is further configured to, in response to successful execution of a target preheating operation, control a preset preheating state machine to switch the preheating state of the speech synthesis channel; the target preheating operation includes any one of a first-level preheating operation, a second-level preheating operation, and a third-level preheating operation, and the preheating state includes at least a first-level preheating ready state, a second-level preheating ready state, and a third-level preheating ready state; in response to the execution duration of the target preheating operation exceeding a preset duration threshold, reclaim the preheating resources allocated for executing the target preheating operation; the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation; process continuous in-vehicle voice interaction requests initiated by the same user based on the speech synthesis channel; the speech synthesis channel is the speech synthesis channel after executing the target preheating operation, and the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation.

[0169] In some embodiments, the startup sequence event includes a first startup sequence event triggered by a first user and a second startup sequence event triggered by a second user; the speech synthesis channel includes a first speech synthesis channel preheated based on a first preheating resource in response to the first startup sequence event, and a second speech synthesis channel preheated based on a second preheating resource in response to the second startup sequence event; the in-vehicle voice interaction request includes a first in-vehicle voice interaction request generated based on the first startup sequence event, and a second in-vehicle voice interaction request generated based on the second startup sequence event; the first speech synthesis channel is used to process the first in-vehicle voice interaction request, and the second speech synthesis channel is used to process the second in-vehicle voice interaction request.

[0170] In some embodiments, the scene perception module is further configured to obtain the generation source of the in-vehicle voice interaction request; the generation source includes a vehicle driving control system, a navigation system, a vehicle cockpit control system, and an in-vehicle entertainment system; when the generation source is the vehicle driving control system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be an emergency vehicle control broadcast scenario; when the generation source is the navigation system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be a navigation broadcast scenario; when the generation source is the vehicle cockpit control system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be a vehicle control feedback scenario; and when the generation source is the in-vehicle entertainment system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be an entertainment dialogue scenario.

[0171] In some embodiments, the resource scheduling module is further configured to: when the target voice broadcast scenario is the emergency vehicle control broadcast scenario, allocate inference resources from the first reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the first response priority corresponding to the emergency vehicle control broadcast scenario; when the target voice broadcast scenario is the navigation broadcast scenario, allocate inference resources from the first reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the second response priority corresponding to the navigation broadcast scenario; the second response priority is lower than the first response priority; when the target voice broadcast scenario is the vehicle control feedback scenario, allocate inference resources from the second reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the third response priority corresponding to the vehicle control feedback scenario; the third response priority is lower than the second response priority, and the second reserved inference resources are greater than the first reserved inference resources; when the target voice broadcast scenario is the entertainment dialogue scenario, allocate inference resources from the second reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the fourth response priority corresponding to the entertainment dialogue scenario; the fourth response priority is lower than the third response priority.

[0172] In some embodiments, the vehicle-mounted speech synthesis resource scheduling system provided in this application may further include: The latency monitoring and strategy adjustment module is used to perform latency monitoring on multiple preset key nodes to obtain latency information for each of the multiple key nodes. The multiple key nodes include a stage for receiving the in-vehicle voice interaction request, a stage for allocating inference resources to the voice synthesis channel, a stage for performing voice synthesis processing on the in-vehicle voice interaction request, and a stage for playing the synthesized audio. Based on the latency information, the module adjusts a warm-up strategy and / or a resource allocation strategy. The warm-up strategy is used to warm up the voice synthesis channel, and the resource allocation strategy is used to allocate inference resources to the voice synthesis channel.

[0173] It should be noted that the specific implementation of the vehicle-mounted speech synthesis resource scheduling system provided in this application is basically the same as the specific implementation of the above-described vehicle-mounted speech synthesis resource scheduling method, and will not be repeated here.

[0174] Please see Figure 12 This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned resource scheduling method for vehicle-mounted voice synthesis.

[0175] In some embodiments, the electronic device may be any smart terminal such as a vehicle, an in-vehicle terminal, an in-vehicle hardware platform (e.g., an in-vehicle computer), a tablet computer, a smartphone, or a wearable device; or, the electronic device may be a vehicle including a memory and a processor.

[0176] like Figure 12 As shown, the electronic device provided in this application embodiment may include: The processor 1201 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1202 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1202 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1202 and is called and executed by the processor 1201 to execute the resource scheduling method for vehicle-mounted voice synthesis according to the embodiments of this application. The input / output interface 1203 is used to implement information input and output; The communication interface 1204 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1205 transmits information between various components of the device (e.g., processor 1201, memory 1202, input / output interface 1203, and communication interface 1204); The processor 1201, memory 1202, input / output interface 1203 and communication interface 1204 are connected to each other within the device via bus 1205.

[0177] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described resource scheduling method for in-vehicle speech synthesis.

[0178] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0179] This application also provides a computer program product, including a computer program. The steps implemented by the computer program when executed by a processor are basically the same as those in the specific embodiments of the above-described resource scheduling method for in-vehicle speech synthesis, and will not be repeated here.

[0180] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0181] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0182] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0184] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0185] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding factor is divided by the following factor, or that the related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0186] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0187] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0189] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A resource scheduling method for vehicle-mounted speech synthesis, characterized in that, The method includes: Obtain the startup sequence events of the in-vehicle voice system; In response to the launch sequence event, the speech synthesis channel is warmed up, and an in-vehicle voice interaction request is generated based on the launch sequence event; the speech synthesis channel is used to process the in-vehicle voice interaction request. The in-vehicle voice interaction request is processed for scene perception to obtain the target voice broadcast scene to which the in-vehicle voice interaction request belongs. Based on the target voice broadcast scenario, inference resources are allocated to the voice synthesis channel to process the in-vehicle voice interaction request.

2. The method according to claim 1, characterized in that, The startup sequence events include a voice start button trigger event, a voice activity detection event, and a voice end prediction event. The preheating of the voice synthesis channel in response to the startup sequence events includes: In response to the voice start button trigger event, a first-level warm-up operation is performed on the voice synthesis channel; the first-level warm-up operation is used to establish the communication connection of the voice synthesis channel; In response to the speech activity detection event, a second-level warm-up operation is performed on the speech synthesis channel; the second-level warm-up operation is used to construct the inference context of the speech synthesis channel. In response to the speech end prediction event, a third-level warm-up operation is performed on the speech synthesis channel; the third-level warm-up operation is used to construct the audio output context of the speech synthesis channel.

3. The method according to claim 2, characterized in that, The method further includes at least one of the following: In response to the successful execution of the target preheating operation, the preset preheating state machine is controlled to switch the preheating state of the speech synthesis channel; the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation, and the preheating state includes at least the first-level preheating ready state, the second-level preheating ready state, and the third-level preheating ready state. In response to the fact that the duration of the target preheating operation exceeds a preset duration threshold, the preheating resources allocated for performing the target preheating operation are reclaimed; the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation. The voice synthesis channel is used to process continuous in-vehicle voice interaction requests initiated by the same user; the voice synthesis channel is a voice synthesis channel after performing a target preheating operation, and the target preheating operation includes any one of the first-level preheating operation, the second-level preheating operation, and the third-level preheating operation.

4. The method according to claim 1, characterized in that, The startup sequence events include a first startup sequence event triggered by a first user and a second startup sequence event triggered by a second user; the speech synthesis channels include a first speech synthesis channel that is preheated based on a first preheating resource in response to the first startup sequence event, and a second speech synthesis channel that is preheated based on a second preheating resource in response to the second startup sequence event; the in-vehicle voice interaction request includes a first in-vehicle voice interaction request generated based on the first startup sequence event, and a second in-vehicle voice interaction request generated based on the second startup sequence event; the first speech synthesis channel is used to process the first in-vehicle voice interaction request, and the second speech synthesis channel is used to process the second in-vehicle voice interaction request.

5. The method according to claim 1, characterized in that, The step of performing scene-aware processing on the in-vehicle voice interaction request to obtain the target voice broadcasting scene to which the in-vehicle voice interaction request belongs includes: Obtain the source from which the in-vehicle voice interaction request is generated; the source includes the vehicle driving control system, navigation system, vehicle cockpit control system, and in-vehicle entertainment system; When the source of the generation is the vehicle driving control system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be an emergency vehicle control broadcast scenario; When the source of the request is the navigation system, the target voice broadcasting scenario to which the in-vehicle voice interaction request belongs is determined to be a navigation broadcasting scenario; In the case where the generation source is the vehicle cockpit control system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be the vehicle control feedback scenario; When the source of the generation is the in-vehicle entertainment system, the target voice broadcast scenario to which the in-vehicle voice interaction request belongs is determined to be an entertainment dialogue scenario.

6. The method according to claim 5, characterized in that, The allocation of inference resources for the speech synthesis channel to process the in-vehicle voice interaction request based on the target voice broadcast scenario includes: When the target voice broadcast scenario is the emergency vehicle control broadcast scenario, inference resources are allocated from the first reserved inference resources for the voice synthesis channel to process the vehicle voice interaction request according to the first response priority corresponding to the emergency vehicle control broadcast scenario; When the target voice broadcast scenario is the navigation broadcast scenario, inference resources are allocated from the first reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the second response priority corresponding to the navigation broadcast scenario; the second response priority is lower than the first response priority; When the target voice broadcast scenario is the vehicle control feedback scenario, inference resources are allocated from the second reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the third response priority corresponding to the vehicle control feedback scenario; the third response priority is lower than the second response priority, and the second reserved inference resources are greater than the first reserved inference resources; When the target voice broadcast scenario is the entertainment dialogue scenario, inference resources are allocated from the second reserved inference resources for the voice synthesis channel to process the in-vehicle voice interaction request according to the fourth response priority corresponding to the entertainment dialogue scenario; the fourth response priority is lower than the third response priority.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Delay monitoring is performed on multiple preset key nodes to obtain delay information for each of the multiple key nodes; the multiple key nodes include a stage for receiving the vehicle voice interaction request, a stage for allocating inference resources to the voice synthesis channel, a stage for performing voice synthesis processing on the vehicle voice interaction request, and a stage for playing the synthesized audio. The warm-up strategy and / or resource allocation strategy are adjusted based on the delay information; the warm-up strategy is used to warm up the speech synthesis channel, and the resource allocation strategy is used to allocate inference resources to the speech synthesis channel.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the resource scheduling method for vehicle-mounted voice synthesis as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the resource scheduling method for in-vehicle speech synthesis as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the resource scheduling method for in-vehicle speech synthesis as described in any one of claims 1 to 7.

Images