Multi-speech synthesis model bearing method and device based on virtual GPU

Abstract

The invention provides a multi-speech synthesis model bearing method and device based on a virtual GPU, and relates to the technical field of graphics processing units, and the method comprises the steps: carrying out the virtualization processing of a physical graphics processing unit, dividing the physical graphics processing unit into a plurality of virtual processing units with independent video memories and calculation quotas, and combining a resource scheduling mechanism, and deploying the speech synthesis language model instances in a plurality of service containers, and constructing a plurality of speech synthesis model bearing units. After a voice synthesis request is accessed, the scheduling module carries out load balancing according to the request connection number of each bearing unit, the request is distributed to a target bearing unit with the minimum connection number, and a voice generation task is completed by a virtual processing unit bound with the target bearing unit. According to the invention, resource division can be carried out on the physical graphic processing unit, and efficient operation of the multi-speech synthesis model is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of a graphics processing unit, and in particular to a multi-speech synthesis model carrying method and device based on a virtual GPU. BACKGROUND

[0002] The deployment of computing resources for speech synthesis services usually uses a whole physical graphics processing unit as a basic running unit. In a multi-model service structure based on a container platform, each speech synthesis model instance usually exclusively occupies the resources of a whole physical graphics processing unit, so as to ensure that the display memory calling, computing operation and acoustic modeling task have a stable running environment. In the scenario of parallel deployment of multiple speech synthesis models, the resources are usually scheduled by a container orchestration system such as Kubernetes, and the speech synthesis task is dynamically scheduled to an available service container instance according to the resource request parameters. However, since the scheduling granularity depends on the resource boundary of the physical graphics processing unit, the resource utilization rate is greatly limited. In addition, the display memory and computing quota required by a single model instance in the speech synthesis service are usually much lower than the load capacity of a whole graphics processing unit, resulting in low efficiency of physical resource use.

[0003] In the existing deployment structure, a speech synthesis model instance usually occupies part of the resources of a graphics processing unit as needed to perform front-end text analysis, acoustic feature prediction and waveform synthesis and other computationally intensive tasks. These tasks usually only require 2 to 4 GB of display memory and a medium-strength computing quota, which is far lower than the processing capacity of a whole graphics processing unit. Limited by the whole-card level strategy of the graphics processing unit resource scheduling of the container platform, the resource scheduling module cannot achieve fine division and allocation at the display memory granularity or operation core granularity, resulting in that each speech synthesis model instance needs to be bound to exclusively occupy a whole graphics processing unit, which in turn magnifies the computing resource cost and reduces the number of deployable model instances per node, thereby severely limiting the deployment density and resource utilization efficiency of the speech synthesis service. Therefore, a method is needed to divide the resources of a physical graphics processing unit to achieve efficient operation of multiple speech synthesis models. SUMMARY

[0004] The present application provides a multi-speech synthesis model carrying method and device based on a virtual GPU, which can divide the resources of a physical graphics processing unit to achieve efficient operation of multiple speech synthesis models.

[0005] In a first aspect of the present application, a multi-speech synthesis model carrying method based on a virtual GPU is provided, which comprises: performing virtualization processing on a physical graphics processing unit, and dividing the physical graphics processing unit into a plurality of virtual processing units; deploying a speech synthesis instance constructed based on a speech synthesis language model in a plurality of service containers, the service containers binding the virtual processing units through a resource scheduling mechanism, constructing a plurality of carrying units for the speech synthesis model, each of the carrying units binding one of the virtual processing units and providing a speech generation service based on the speech synthesis instance; When receiving a speech synthesis request initiated by a terminal user, distributing the speech synthesis request to a target carrying unit with the least current connection number among the plurality of carrying units based on a current request connection number of each of the carrying units; Performing speech generation processing on the speech synthesis request based on a target processing unit bound to the target carrying unit, and outputting a speech synthesis result to the terminal user.

[0006] On the basis of the above technical solutions, preferably, the virtualization processing of the physical graphics processing unit includes: Accessing a video memory resource mapping table and a core execution unit configuration register of the physical graphics processing unit to obtain current global resources of the physical graphics processing unit; Dividing the global resources into a plurality of video memory partitions and operation unit partitions, and configuring a virtual processing unit for each of the video memory partitions or each of the operation unit partitions, wherein the video memory partitions and the operation unit partitions correspond to each other, and each of the virtual processing units has independent video memory resources and a calculation quota; By constructing a mapping table to record the binding relationship between the video memory partitions and the operation unit partitions and the virtual processing units, the resource scheduling mechanism responds to resource application requests based on the mapping table during operation, and ensures that the isolation constraint of the video memory resources and the calculation quota is always met during the process in which the service container binds the virtual processing units.

[0007] On the basis of the above technical solutions, preferably, the resource scheduling mechanism responds to resource application requests based on the mapping table during operation, and ensures that the isolation constraint of the video memory resources and the calculation quota is always met during the process in which the service container binds the virtual processing units, and specifically includes: When the service container is initialized, the resource scheduling mechanism obtains resource states of the virtual processing units in the mapping table, the resource states including video memory resource states and calculation quota states; Constraining and matching deployment requirement parameters of a speech synthesis model declared by the service container and the resource states, the deployment requirement parameters including video memory capacity requirements and calculation quota requirements; determining a target processing unit when the deployment requirement parameter is successfully matched with the graphics memory resource state and the computing quota state, establishing a binding relationship between the service container and the target processing unit, updating the mapping table to record the binding relationship, and writing the unique identifier of the service container into the running context of the target processing unit to limit the graphics memory access and operation scheduling boundary of the speech synthesis model running process, thereby ensuring that the binding process meets the isolation constraint.

[0008] On the basis of the above technical solutions, preferably, the speech synthesis instances constructed based on the speech synthesis language models are respectively deployed in a plurality of service containers, and the service containers bind the virtual processing units through a resource scheduling mechanism to construct a plurality of bearing units for the speech synthesis model, each bearing unit binds a virtual processing unit and provides a speech generation service based on the speech synthesis instance, and specifically includes: a service container image is constructed for each speech synthesis language model, which includes an execution environment, a speech synthesis language model component, and a calling interface; after the service container is started, a binding request is initiated to the resource scheduling mechanism, and the binding request includes the deployment requirement parameter; the binding request and the resource state are matched, and a target virtual processing unit that meets the condition is bound to the service container, and the service container identifier is written into the running context of the target virtual processing unit to construct a bearing unit with graphics memory resource isolation and computing quota isolation.

[0009] On the basis of the above technical solutions, preferably, when a speech synthesis request initiated by a terminal user is received, the speech synthesis request is distributed to a target bearing unit with the least number of current connections among a plurality of bearing units based on the current number of request connections of each bearing unit, and specifically includes: the request content of the speech synthesis request is parsed into a standardized structure including text input data, a request identifier, and a priority parameter; a connection state table of the bearing unit is constructed based on the mapping table, which records the real-time request connection number of the bearing unit and the load state of the virtual processing unit; a connection number comparison operation is performed based on the connection state table to select a target bearing unit that is currently in a running state and has the least number of connections as a target processing instance of the speech synthesis request.

[0010] On the basis of the above technical solutions, preferably, the target processing unit bound to the target bearing unit performs speech generation processing on the speech synthesis request and outputs a speech synthesis result to the terminal user, and specifically includes: The target bearing unit receives a speech synthesis request structure containing text input data, calls the structure to generate a pronunciation unit sequence, and performs an acoustic modeling task in the computing quota and video memory resources allocated by the target processing unit, and outputs an intermediate acoustic representation vector; The intermediate acoustic representation vector is used to complete speech waveform data generation in the target processing unit; The speech waveform data is encapsulated into a data packet conforming to a transmission protocol, and is sent to an access gateway through the network interface of the speech synthesis model bearing unit, and the speech synthesis result is responded to the terminal user by the access gateway.

[0011] On the basis of the above technical scheme, preferably, when receiving a speech synthesis request initiated by a terminal user, based on the current number of request connections of each bearing unit, the speech synthesis request is distributed to a target bearing unit with the least number of current connections among a plurality of bearing units, and the method further comprises: In the process of distributing the speech synthesis request to the target bearing unit, the connection count of the target bearing unit is increased by one in real time through the connection state table after each request forwarding, and the request identification and timestamp are recorded; After the target bearing unit completes the speech generation process and outputs the speech synthesis result, the corresponding target bearing unit is determined in the connection state table according to the request identification, and the connection count is reduced by one; When the resource occupation of the virtual processing unit exceeds a preset threshold, the non-schedulable flag of the target bearing unit is set in the connection state table.

[0012] In a second aspect of the application, a virtual GPU-based multi-speech synthesis model bearing device is provided, which is used to execute any one of the virtual GPU-based multi-speech synthesis model bearing methods described above. The device comprises an acquisition module, a processing module and an output module, wherein: The acquisition module is used to perform virtualization processing on a physical graphics processing unit, and divide the physical graphics processing unit into a plurality of virtual processing units; The processing module is used to deploy a speech synthesis instance constructed based on a speech synthesis language model in a plurality of service containers, the service containers are bound to the virtual processing units through a resource scheduling mechanism, a plurality of bearing units for speech synthesis models are constructed, each bearing unit is bound to a virtual processing unit and provides speech generation services based on the speech synthesis instance; The processing module is configured to, when receiving a voice synthesis request initiated by a terminal user, distribute the voice synthesis request to a target bearing unit with the least current connection number among a plurality of bearing units based on a current request connection number of each bearing unit. The output module is configured to perform voice generation processing on the voice synthesis request based on a target processing unit bound to the target bearing unit, and output a voice synthesis result to the terminal user.

[0013] On the basis of the above technical solutions, preferably, the obtaining module is configured to access a video memory resource mapping table and a core execution unit configuration register of the physical graphics processing unit to obtain current global resources of the physical graphics processing unit. The processing module is configured to divide the global resources into a plurality of video memory partitions and operation unit partitions, and configure a virtual processing unit for each video memory partition or each operation unit partition, wherein the video memory partitions and the operation unit partitions correspond to each other, and each virtual processing unit has independent video memory resources and a computing quota. The processing module is configured to record a binding relationship between the video memory partitions and the operation unit partitions and the virtual processing units by constructing a mapping table, and a resource scheduling mechanism responds to a resource application request based on the mapping table during operation to ensure that the service container always satisfies the isolation constraint of the video memory resources and the computing quota during the process of binding the virtual processing units.

[0014] On the basis of the above technical solutions, preferably, the obtaining module is configured to, when the service container is initialized, obtain resource states of the virtual processing units in the mapping table by a resource scheduling mechanism, wherein the resource states include video memory resource states and computing quota states. The processing module is configured to perform constraint matching between deployment requirement parameters of a voice synthesis model declared by the service container and the resource states, wherein the deployment requirement parameters include video memory capacity requirements and computing quota requirements. The processing module is configured to, when the constraint matching between the deployment requirement parameters and the video memory resource states and the computing quota states is successful, determine a target processing unit, establish a binding relationship between the service container and the target processing unit, update the mapping table to record the binding relationship, and write a unique identifier of the service container into a running context of the target processing unit, so as to limit video memory access and operation scheduling boundaries of a voice synthesis model running process, and ensure that the binding process satisfies the isolation constraint.

[0015] On the basis of the above technical scheme, preferably, the processing module is configured to construct, for each of the speech synthesis language models, a service container image containing an execution environment, a speech synthesis language model component, and a calling interface; The processing module is configured to initiate a binding request to the resource scheduling mechanism after starting the service container, wherein the binding request includes the deployment requirement parameter; The processing module is configured to perform a matching operation according to the binding request and the resource state, and establish a binding relationship between a target virtual processing unit that meets a condition and the service container, and write the service container identifier into a running context of the target virtual processing unit, thereby constructing a bearing unit with display memory resource isolation and computing quota isolation.

[0016] On the basis of the above technical scheme, preferably, the processing module is configured to parse the request content of the speech synthesis request into a standardized structure containing text input data, a request identifier, and a priority parameter; The processing module is configured to construct a connection state table of the bearing unit based on the mapping table, wherein the connection state table records a real-time request connection number of the bearing unit and a load state of the virtual processing unit; The processing module is configured to perform a connection number comparison operation based on the connection state table, and filter out a target bearing unit that is currently in a running state and has the least connection number as a target processing instance of the speech synthesis request.

[0017] On the basis of the above technical scheme, preferably, the processing module is configured to receive, by the target bearing unit, a speech synthesis request structure containing text input data, invoke the structure to generate a pronunciation unit sequence, and perform an acoustic modeling task in the computing quota and display memory resource allocated by the target processing unit, thereby outputting an intermediate acoustic representation vector; The processing module is configured to complete speech waveform data generation in the target processing unit based on the intermediate acoustic representation vector; The processing module is configured to encapsulate the speech waveform data into a data packet conforming to a transmission protocol, and send the data packet to an access gateway through a network interface of the speech synthesis model bearing unit, and respond, by the access gateway, the speech synthesis result to the terminal user.

[0018] On the basis of the above technical scheme, preferably, the processing module is configured to, in the process of distributing the speech synthesis request to the target bearing unit, increment a connection count of the target bearing unit by one in real time through the connection state table after each request forwarding, and record a request identifier and a timestamp; The processing module is configured to, after the target bearing unit completes voice generation processing and outputs a voice synthesis result, determine a corresponding target bearing unit in the connection state table according to the request identifier, and reduce the connection count by one. The processing module is configured to, when resource occupation of the virtual processing unit exceeds a preset threshold, set an unschedulable flag of the target bearing unit in the connection state table.

[0019] In a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, a user interface and a network interface, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of the above aspects.

[0020] In a fourth aspect of the present application, a computer readable storage medium is provided, which stores instructions, when the instructions are executed, the method of any one of the above aspects is performed.

[0021] In summary, the one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The present application introduces a graphics processing resource virtualization mechanism, divides the physical graphics processing unit into multiple virtual processing units with independent resource boundaries according to the video memory and computing quota, and combines the container deployment method to realize the dynamic binding of service containers and virtual processing units through the resource scheduling mechanism, thereby constructing multiple isolated voice synthesis model bearing units; at the same time, based on the minimization strategy of request connection number, the task scheduling is combined with the resource state of the virtual processing unit for dynamic load balancing, to ensure that each voice synthesis task is efficiently executed while the resources are fully utilized, thereby significantly improving the resource utilization rate of the physical graphics processing unit and the concurrent running capability of the voice synthesis model.

[0022] 2. The physical graphics processing unit is virtually processed and divided into multiple virtual processing units with independent video memory resources and computing quotas, so that the resources that can only be exclusively occupied by the whole card can be called in parallel by multiple tasks, thereby improving the resource utilization efficiency and providing a hardware basis for subsequent fine-grained scheduling of service containers, significantly improving the deployment density and running concurrency of the voice synthesis model.

[0023] 3. The mapping relationship between the resource scheduling mechanism and the virtual processing unit realizes the resource state matching and running context writing of the service container in the process of binding the virtual processing unit, ensures that the boundary of the video memory resource and the computing quota is not accessed out of bounds in the task allocation process, strengthens the task isolation, and improves the scheduling safety and running stability of the system in the concurrent deployment of multiple voice synthesis models.

[0024] 4. Introduce the active declaration mechanism of deployment requirement parameters and the running context binding mechanism in the voice synthesis language model instance deployment process, fundamentally guarantee the resource isolation and execution independence of each voice synthesis model bearing unit, and effectively support the large-scale voice synthesis service deployment in the resource sharing environment.

[0025] 5. Construct a scheduling decision model based on the real-time request connection number and resource load state, dynamically select the target bearing unit through the least connection number principle, realize the balanced allocation of requests among the voice synthesis model bearing units, effectively avoid resource hotspots, and improve the throughput capacity of the overall voice service link and the timeliness of request response.

[0026] 6. Real-time update the request connection number in the connection state table, and accurately recycle the resource record after the task is completed, combine the resource occupation dynamic monitoring of the virtual processing unit and the non-scheduling flag setting mechanism, realize the real-time linkage of the scheduling strategy and the resource state, effectively prevent the repeated invocation of the resource overload node, and guarantee the scheduling stability and task queuing orderliness of the system under high concurrency requests. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a flow diagram of a virtual GPU-based multi-voice synthesis model bearing method disclosed by the embodiment of the present application; Figure 2 is a module diagram of a virtual GPU-based multi-voice synthesis model bearing device disclosed by the embodiment of the present application; Figure 3 is a structural diagram of an electronic device disclosed by the embodiment of the present application.

[0028] Mark explanation: 201, acquisition module; 202, processing module; 203, output module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0029] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be described clearly and completely in conjunction with the drawings in the embodiment of the specification. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments.

[0030] In the description of the embodiments of the present application, the words such as "for example" or "for instance" are used to represent an example, an illustration, or a description. Any embodiment or design scheme described as "for example" or "for instance" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words such as "for example" or "for instance" are intended to present the relevant concept in a specific manner.

[0031] In the description of the embodiments of the present application, the term "a plurality of" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly indicating the technical features indicated. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more features. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0032] The computing resource deployment of the existing voice synthesis service generally relies on a whole physical graphics processing unit as a running unit. In a multi-model structure based on a container platform, due to the inability to finely schedule the graphics processing unit at the video memory level or the calculation quota level, each voice synthesis model instance must exclusively occupy the whole graphics processing unit resource, although it only needs a small amount of video memory and moderate calculation capacity for actual operation, thereby causing low resource utilization, limited deployment density, and significantly increased calculation cost. Therefore, there is an urgent need for a mechanism for dividing the physical graphics processing unit into fine-grained resources to support efficient parallel operation of multiple voice synthesis model instances.

[0033] The embodiment discloses a multi-voice synthesis model carrying method based on a virtual GPU, referring to Figure 1 , comprising the following steps S110-S140: S110, performing virtualization processing on a physical graphics processing unit, and dividing the physical graphics processing unit into a plurality of virtual processing units.

[0034] The multi-voice synthesis model carrying method based on a virtual GPU disclosed by the embodiment of the present application is applied to a server. The server includes but is not limited to electronic devices such as mobile phones, tablet computers, wearable devices, PC (Personal Computer), and the like, and can also be a background server running a multi-voice synthesis model carrying method based on a virtual GPU. The server can be realized by an independent server or a server cluster composed of multiple servers.

[0035] In a possible implementation, the virtualization processing of the physical graphics processing unit includes: accessing a memory resource mapping table and a core execution unit configuration register of the physical graphics processing unit, and obtaining global resources of the physical graphics processing unit; dividing the global resources into a plurality of memory partitions and a plurality of operation unit partitions, and configuring a virtual processing unit for each memory partition or each operation unit partition, wherein the memory partitions and the operation unit partitions are in one-to-one correspondence, and each virtual processing unit has independent memory resources and a computing quota; and constructing a mapping table to record a binding relationship between the memory partitions and the operation unit partitions and the virtual processing units, and ensuring that the service container always satisfies an isolation constraint of the memory resources and the computing quota in a process of binding the virtual processing units, by using a resource scheduling mechanism to respond to a resource application request based on the mapping table in a running process.

[0036] Specifically, the physical graphics processing unit is initialized by loading a bottom-layer driver interface supporting virtualization extension. In a driver initialization stage, a memory resource mapping table and a core execution unit configuration register stored in the physical graphics processing unit are accessed by calling a bottom-layer device control instruction. The memory resource mapping table is used to describe a distribution structure of a physical memory address space of the graphics processing unit, and each entry contains information such as a physical address, an allocation state, and an available capacity. The core execution unit configuration register is used to represent an organization form and a distribution structure of a computing core in the graphics processing unit, including a current state of configuration items such as a streaming processor (Streaming Processor) and a computing unit cluster (SM, Streaming Multiprocessor). By reading the above structures, a total available memory of the current graphics processing unit, a usage of each address block, and a running distribution of the computing core can be obtained, thereby forming a global resource view of the graphics processing unit.

[0037] According to the global resource view of the graphics processing unit, the addressable video memory space is divided into multiple logical partitions by a video memory division strategy, each video memory partition has a determined start address and capacity boundary, and the access permission of only binding virtual processing units is set through a memory protection mechanism; in terms of operation resources, the core execution unit is proportionally divided into several subsets according to the load capacity of the computing unit and the number of parallel processing units, each subset is used as an operation unit partition, which includes a fixed number of stream processors and local shared memory resources, ensuring that each partition has the ability to execute tasks independently. The video memory partition and the operation unit partition are bound in a one-to-one manner to ensure that each virtual processing unit can access its exclusive video memory space in its exclusive computing core when scheduling and executing tasks, avoiding resource conflicts. Taking a graphics processing unit with 24GB of video memory and 84 stream processors as an example, it can be divided into 6 virtual processing units, each virtual processing unit is bound with 4GB of video memory and 14 stream processors, forming a virtual execution unit with complete computing and storage capabilities.

[0038] A unique identifier is allocated to each virtual processing unit and a resource mapping table is constructed, which records the video memory partition boundary, computing core number, current binding state and service container identifier of each virtual processing unit. After receiving the virtual processing unit resource application request initiated by the service container, the resource scheduling mechanism reads the resource mapping table to obtain all virtual processing units in the schedulable state, and filters and matches the remaining capacity of the video memory and the computing quota state, when a virtual processing unit meets the video memory threshold and computing quota threshold requested by the service container, the resource scheduling mechanism marks its binding state in the resource mapping table and writes the current service container identifier, completing the binding process of the service container and the virtual processing unit. In this process, all bound virtual processing units remain invisible to other service container requests, ensuring that the scheduling operation has isolation and uniqueness in the resource allocation dimension, thereby avoiding the problem of video memory preemption or computing core conflict between multiple speech synthesis model instances when executing speech synthesis tasks.

[0039] In a possible implementation, the resource scheduling mechanism responds to resource application requests based on the mapping table during operation, and ensures that the isolation constraint of the display memory resource and the calculation quota is always met during the process of binding the service container to the virtual processing unit. Specifically, the resource scheduling mechanism obtains the resource state of each virtual processing unit in the mapping table when the service container is initialized, and the resource state includes the display memory resource state and the calculation quota state; the deployment requirement parameters of the speech synthesis model declared by the service container are matched with the resource state, and the deployment requirement parameters include the display memory capacity requirement and the calculation quota requirement; when the deployment requirement parameters are successfully matched with the display memory resource state and the calculation quota state, a target processing unit is determined, and the service container is bound to the target processing unit, and the mapping table is updated to record the binding relationship, and the unique identifier of the service container is written into the running context of the target processing unit, which is used to limit the display memory access and operation scheduling boundary of the speech synthesis model running process, and ensure that the binding process meets the isolation constraint.

[0040] Specifically, the service container initiates a resource binding request to the resource scheduling mechanism in the initialization phase started by the container orchestration platform, the resource scheduling mechanism queries the pre-stored virtual processing unit resource mapping table, and obtains the current state information of each virtual processing unit recorded therein. The display memory resource state indicates the remaining display memory capacity that can be allocated for each virtual processing unit, the current display memory allocation upper limit, and the corresponding display memory access permission setting; the calculation quota state reflects the number of calculation cores currently occupied by the virtual processing unit, the running frequency range, and the upper limit of the number of concurrent tasks supported. The resource state information is regularly updated by monitoring the scheduling context and resource access statistical data of the virtual processing unit in the running time, to ensure the accuracy and real-time performance of reflecting the current allocable capacity.

[0041] When the service container image is constructed, the resource requirement parameters in the speech synthesis model configuration file are pre-set, which are submitted to the resource scheduling mechanism as structured resource request information by the container runtime in the container deployment process. The parameters include fields such as the required minimum display memory capacity, the minimum number of calculation cores, and the expected running frequency. The resource scheduling mechanism performs bidirectional constraint matching according to the above resource request parameters and the virtual processing unit resource state obtained from the mapping table, eliminates the virtual processing units that do not meet the conditions, and only retains the candidate virtual processing units that have corresponding remaining display memory capacity and calculation quota, to construct a target matching set. The matching logic filters based on the constraint condition that the display memory capacity and the calculation quota are met at the same time, to avoid the problem of false scheduling caused by the fact that part of the resources are sufficient but the whole is not available.

[0042] The resource scheduling mechanism selects one from the candidate virtual processing unit set as a target processing unit according to a preset strategy (such as the current lowest load or the smallest request response delay), triggers a binding operation, updates the target processing unit state to the "bound" state, marks the unique identifier of the current service container of the virtual processing unit, the binding timestamp and the task category information in the mapping table, and synchronously writes the unique identifier into the running context identifier area of the virtual processing unit, so that the identifier becomes the basis for boundary judgment of all GPU calls and computing scheduling. In this way, it is ensured that the speech synthesis model can only call resources within the bound virtual processing unit range during execution, preventing crossing of GPU boundaries or preemption of computing cores of other tasks, and achieving logical isolation and physical limitation of resources. For example, if the service container declares a GPU requirement of 3 GB and a computing quota of 8 stream processor cores, the scheduling mechanism will match the virtual processing unit with a remaining GPU of ≥ 3 GB and a computing core of ≥ 8 cores that can be allocated in the current mapping table, and after binding is completed, all speech generation tasks of the service container will be executed within the specified resource boundary, ensuring controllability of scheduling behavior and resource utilization efficiency of the system as a whole.

[0043] S120, respectively deploying a speech synthesis instance constructed based on a speech synthesis language model in a plurality of service containers, the service container binding a virtual processing unit through a resource scheduling mechanism, constructing a plurality of carrying units for the speech synthesis model, each carrying unit binding a virtual processing unit and providing a speech generation service based on the speech synthesis instance.

[0044] In one possible implementation, the speech synthesis instance constructed based on the speech synthesis language model is respectively deployed in a plurality of service containers, the service container binds the virtual processing unit through the resource scheduling mechanism, and a plurality of carrying units for the speech synthesis model are constructed, each of the carrying units binds a virtual processing unit and provides a speech generation service based on the speech synthesis instance, and specifically includes: constructing a service container image containing an execution environment, a speech synthesis language model component and an invocation interface for each speech synthesis language model; after the service container is started, initiating a binding request to the resource scheduling mechanism, the binding request including the deployment requirement parameter; performing a matching operation according to the binding request and the resource state, and establishing a binding relationship between the target virtual processing unit meeting the condition and the service container, and writing the service container identifier into the running context of the target virtual processing unit, to construct a carrying unit with GPU resource isolation and computing quota isolation.

[0045] Specifically, during the service container image building process of each speech synthesis language model, the underlying execution environment, middleware components, and inference framework of the speech synthesis language model need to be integrated, specifically including text preprocessing modules related to speech synthesis tasks, acoustic feature modeling networks, and neural vocoder modules, while integrating standardized external calling interfaces for receiving speech synthesis requests and returning speech data. The execution environment usually includes an operating system base image, a driver module supporting graphics processing instructions, and speech synthesis dependent libraries such as Python interpreter, CUDA computing library, and TensorRT acceleration engine, etc., to ensure that the speech synthesis language model has reproducible, deployable, and migratable running capabilities in the container.

[0046] After the service container completes instantiation and starts, the internal scheduling access module is called to generate a structured binding request, which encapsulates the resource parameters required for the speech synthesis language model to run, including minimum video memory capacity, minimum computing quota, model category identification, and expected concurrent processing capability. The binding request is transmitted to the resource scheduling mechanism processing module through the scheduling channel between the service container and the resource scheduling mechanism, which is used to trigger the selection process of the target virtual processing unit. The scheduling channel uses standard API interfaces or message queue mechanisms to ensure the transmissibility of resource requests and the horizontal expansion capability of the scheduling system.

[0047] The resource scheduling mechanism matches the deployment requirement parameters in the binding request with the resource states of all virtual processing units in real time, and selects the target virtual processing unit according to the preset priority strategy in the virtual processing unit set that meets all the constraint conditions. After the binding operation is executed, the resource scheduling mechanism writes the unique identification code of the current service container in the running context of the virtual processing unit, which serves as the verification basis for the video memory access range and operation scheduling boundary, and is used to limit the access behavior of the speech synthesis language model during execution to non-binding resources. At the same time, the mapping table records the binding relationship, including service container identification, virtual processing unit number, resource occupancy, and allocation timestamp, etc., to ensure that the resource scheduling module can be controlled during the entire life cycle. The above process builds a speech synthesis model carrying unit with resource isolation properties, and each carrying unit independently completes the speech generation task within the range of its bound virtual processing unit resources, ensuring that tasks do not interfere with each other and improving the overall concurrent processing capability. For example, if speech synthesis language model A requires 3GB of video memory and 6-core computing cores, the scheduling mechanism will bind it to virtual processing unit B with ≥3GB of video memory and ≥6-core resources, and write the identification of container A in its context to form a completed speech synthesis model carrying unit.

[0048] S130, when receiving a terminal user initiated voice synthesis request, based on the current request connection number of each bearing unit, the voice synthesis request is distributed to the target bearing unit with the least current connection number in the plurality of bearing units.

[0049] In a possible implementation, when receiving a terminal user initiated voice synthesis request, based on the current request connection number of each bearing unit, the voice synthesis request is distributed to the target bearing unit with the least current connection number in the plurality of bearing units, specifically comprising: parsing the request content of the voice synthesis request into a standardized structure containing text input data, request identification and priority parameters; based on the mapping table, a connection state table of the bearing unit is constructed, which records the real-time request connection number of the bearing unit and the load state of the virtual processing unit; based on the connection state table, a connection number comparison operation is performed to screen out the target bearing unit with the least connection number in the current running state as the target processing instance of the voice synthesis request.

[0050] Specifically, after the voice synthesis request is submitted to the access gateway by the terminal user through the access port, the request is first structured by the pre-analysis module, and the text input data, request identification and priority parameters contained therein are extracted and uniformly constructed into a standardized structure. The text input data is a natural language text sequence converted into voice by the user request, the request identification is a unique task identification code for tracking the whole life cycle of the voice synthesis task, and the priority parameter is used to distinguish the importance level of different requests to support the priority sorting logic in the subsequent scheduling. The structure is used as a standardized scheduling input, which maintains structural consistency in the subsequent scheduling stage, facilitating efficient transmission and scheduling module uniform processing.

[0051] The resource scheduling mechanism extracts the information of all voice synthesis model bearing units from the resource mapping table according to the voice synthesis service running state, and constructs a real-time connection state table. In the connection state table, the current request connection number, the bound virtual processing unit number and the load state of the bound virtual processing unit of each voice synthesis model bearing unit are recorded. The request connection number is the number of voice synthesis tasks currently being processed or waiting to be processed by the voice synthesis model bearing unit, and the load state includes dynamic indicators such as video memory occupation ratio, calculation quota usage rate and concurrent task scheduling queue length. The construction process of the connection state table adopts a dual mechanism of periodic update and event triggering: on the one hand, the scheduling mechanism regularly obtains the latest indicators from the running state monitoring module, and on the other hand, the update is triggered immediately when the request is completed or the binding state changes, ensuring the real-time and accuracy of the data.

[0052] The connection number comparison operation is performed in the scheduling module, and is used to determine a target voice synthesis model carrying unit currently in a running state and having a scheduling qualification according to a request connection number recorded in a connection state table. The running state is jointly determined by a service container life cycle and a virtual processing unit resource state, and only when the service container of the voice synthesis model carrying unit is in an active state and the virtual processing unit bound to the voice synthesis model carrying unit is not marked as an unusable state, the voice synthesis model carrying unit has a scheduling qualification. The scheduling module sorts all voice synthesis model carrying units satisfying the running condition according to the request connection number from small to large, and selects a voice synthesis model carrying unit with the smallest request connection number as a target processing instance of the current voice synthesis request, so as to implement a load balancing strategy based on the least connection number. For example, if the voice synthesis request arrives, the carrying units A, B and C have 8, 4 and 6 request connection numbers respectively, and are all in the running state, the scheduling module allocates the request to the carrying unit B with the connection number of 4. This way effectively avoids centralized scheduling of resources, and improves the overall request processing performance and the stability of the voice response.

[0053] In a possible implementation, after the voice synthesis request is allocated to the target carrying unit with the least current connection number based on the current request connection number of each of the carrying units when the terminal user initiates the voice synthesis request, the method further includes: in the process of allocating the voice synthesis request to the target carrying unit, the connection count of the target carrying unit is increased by one in real time through the connection state table after each request forwarding, and the request identification and the time stamp are recorded; after the target carrying unit completes the voice generation processing and outputs the voice synthesis result, the corresponding target carrying unit is determined in the connection state table according to the request identification, and the connection count is reduced by one; and when the resource occupation of the virtual processing unit exceeds a preset threshold, an unschedulable flag of the target carrying unit is set in the connection state table.

[0054] Specifically, after the voice synthesis request is allocated to the target voice synthesis model carrying unit by the scheduling module, the scheduling module immediately performs an increment operation on the connection count of the voice synthesis model carrying unit in the connection state table, and writes the current request identification and the corresponding time stamp into the connection record list of the voice synthesis model carrying unit. The connection count represents the number of requests currently in the active processing state of the voice synthesis model carrying unit, and is a core index of scheduling load balancing; the request identification is used for task tracking and result backtracking, and the time stamp is used for subsequent analysis of timeout, exception and scheduling delay and the like. The connection state table is organized in a key-value structure, each voice synthesis model carrying unit corresponds to a group of state records, and the update operation is completed through an atomic lock mechanism or a consistent writing strategy, so as to ensure the accuracy of state synchronization under high concurrency.

[0055] The voice synthesis model carrying unit returns the voice synthesis result to the access gateway through the service container interface after completing the voice generation processing, and the access gateway returns a task completion event to the scheduling module immediately after confirming the end of the task, and attaches the original request identifier. The scheduling module looks up the target voice synthesis model carrying unit in the connection state table according to the request identifier, confirms the task attribution, performs a self-decrement operation of the connection count, and at the same time clears the request identifier and timestamp in the record, and releases the connection entry. This operation ensures that the connection number obtained by the scheduling module always reflects the actual running state, avoiding resource misjudgment or scheduling deviation caused by connection count lag. For example, the request "REQ123" is processed by the voice synthesis model carrying unit X, and after the task end signal is confirmed, the scheduling module immediately reduces the connection number of X from 5 to 4, and deletes the "REQ123" entry.

[0056] The scheduling module monitors the resource occupation state of each virtual processing unit in real time, including the display memory occupation ratio, the calculation quota usage rate and the current queued task number. When any index exceeds the preset threshold (such as the display memory usage rate exceeds 90% or the number of concurrent tasks exceeds the maximum concurrent configuration), the scheduling module sets the corresponding voice synthesis model carrying unit to the "unschedulable" state in the connection state table, that is, marks it as an unallocable target, and the voice synthesis model carrying unit will be skipped in subsequent voice synthesis request scheduling. The flag is periodically refreshed by the dynamic resource monitoring module, and is automatically cleared by the scheduling module when the resource occupation falls back to the safe range, restoring the schedulable qualification of the voice synthesis model carrying unit. By introducing this mechanism, the risk of repeated scheduling of resource overload nodes can be effectively avoided, and the overall throughput rate and response stability of the voice synthesis service can be improved.

[0057] S140, performing voice generation processing on the voice synthesis request based on the target processing unit bound to the target carrying unit, and outputting a voice synthesis result to an end user.

[0058] In a possible implementation, the performing voice generation processing on the voice synthesis request based on the target processing unit bound to the target carrying unit, and outputting a voice synthesis result to the end user specifically includes: the target carrying unit receives a voice synthesis request structure body containing text input data, calls the structure body to generate a pronunciation unit sequence, and performs an acoustic modeling task in the calculation quota and display memory resources allocated by the target processing unit to output an intermediate acoustic representation vector; the voice waveform data generation is completed on the intermediate acoustic representation vector in the target processing unit; the voice waveform data is packaged into a data packet conforming to a transmission protocol, and is sent to an access gateway through a network interface of the voice synthesis model carrying unit, and the voice synthesis result is responded to the end user by the access gateway.

[0059] Specifically, after the speech synthesis request structure is received by the target speech synthesis model carrying unit, the text front-end module is first called by the built-in speech synthesis language model instance to process the text input data in the structure, perform language standardization, punctuation cleaning, and syllable segmentation, and further identify the part of speech, stress information, and prosodic annotation, thereby generating a pronunciation unit sequence for modeling. Subsequently, the model calls the acoustic modeling module based on the deep neural network, combines the pronunciation unit sequence, schedules the required computing cores within the computing quota allocated by the target virtual processing unit, loads the network parameters and input tensors within the available video memory capacity, completes the encoding calculation of phonemes, stress, duration, and other features, and outputs multi-dimensional intermediate acoustic representation vectors, commonly including mel-spectrogram, fundamental frequency trajectory, and energy distribution sequence, etc., for subsequent waveform reconstruction.

[0060] The intermediate acoustic representation vector is input to the neural vocoder module, which serves as the post-modeling backend to generate continuous speech waveform data based on the vector. In this phase, the target virtual processing unit continues to provide the required computing power and video memory resources to support the vocoder network structure to complete the time domain reconstruction task. The neural vocoder can use WaveNet, HiFi-GAN, etc. structure to convert frequency domain features into time-continuous audio signals in multi-layer convolution or attention network, output high-quality speech waveform with sampling rate of 16kHz or higher, and ensure the naturalness and clarity of speech synthesis.

[0061] The generated speech waveform data is then processed by the audio packaging module. The audio packaging module converts the original waveform into the target format such as PCM, AAC or Opus format according to the system communication protocol, and constructs data packets according to task identification, frame number and timestamp, etc. to form complete speech data units with protocol header and payload field. The data packet is sent to the access gateway through the container network interface of the speech synthesis model carrying unit. After receiving and verifying the data packet integrity and task identification consistency, the access gateway returns the speech synthesis result to the terminal user according to the request channel, completing a closed-loop processing process from speech request to speech response. For example, if the user request is "What's the weather today", the system generates the corresponding pronunciation unit, completes acoustic modeling and neural vocoder processing, and outputs the waveform packaged as PCM format and sent to the access gateway, and finally played to the user terminal in the form of audio. The whole process is based on virtual processing unit resource scheduling and isolation mechanism to ensure task independence, resource stability and speech output quality.

[0062] The embodiment also discloses a virtual GPU-based multi-speech synthesis model carrying device, referring to Figure 2 , comprising an acquisition module 201, a processing module 202, and an output module 203, and the device is used for executing any one of the above virtual GPU-based multi-speech synthesis model carrying methods, wherein: The acquisition module 201 is configured to perform virtualization processing on the physical graphics processing unit, and divide the physical graphics processing unit into a plurality of virtual processing units.

[0063] The processing module 202 is configured to deploy a speech synthesis instance constructed based on a speech synthesis language model in each of the plurality of service containers, and bind the virtual processing units by a resource scheduling mechanism to construct a plurality of bearing units for the speech synthesis model, and provide speech generation services based on the speech synthesis instance by each bearing unit binding a virtual processing unit.

[0064] The processing module 202 is configured to, when receiving a speech synthesis request initiated by a terminal user, distribute the speech synthesis request to a target bearing unit with the least current connection number among the plurality of bearing units based on the current request connection number of each bearing unit.

[0065] The output module 203 is configured to perform speech generation processing on the speech synthesis request based on a target processing unit bound by the target bearing unit, and output a speech synthesis result to the terminal user.

[0066] In a possible implementation, the acquisition module 201 is configured to access a video memory resource mapping table of the physical graphics processing unit and a core execution unit configuration register to obtain global resources of the physical graphics processing unit.

[0067] The processing module 202 is configured to divide the global resources into a plurality of video memory partitions and operation unit partitions, and configure a virtual processing unit for each video memory partition or each operation unit partition, wherein the video memory partitions and the operation unit partitions correspond to each other, and each virtual processing unit has independent video memory resources and a calculation quota.

[0068] The processing module 202 is configured to record a binding relationship between the video memory partitions and the operation unit partitions and the virtual processing units by constructing a mapping table, and the resource scheduling mechanism responds to a resource application request based on the mapping table during running to ensure that the isolation constraint of the video memory resources and the calculation quota is always met during the process of binding the virtual processing units by the service container.

[0069] In a possible implementation, the acquisition module 201 is configured to, when the service container is initialized, the resource scheduling mechanism obtains resource states of the virtual processing units in the mapping table, and the resource states include video memory resource states and calculation quota states.

[0070] The processing module 202 is configured to perform constraint matching on deployment requirement parameters of the speech synthesis model declared by the service container and the resource states, and the deployment requirement parameters include video memory capacity requirements and calculation quota requirements.

[0071] The processing module 202 is used to determine the target processing unit when the deployment requirement parameters are successfully matched with the status of video memory resources and the status of computing quotas. It also establishes a binding relationship between the service container and the target processing unit, updates the mapping table to record the binding relationship, and writes the unique identifier of the service container into the running context of the target processing unit. This is used to restrict the boundaries of video memory access and computation scheduling of the speech synthesis model running process and ensure that the binding process meets the isolation constraints.

[0072] In one possible implementation, the processing module 202 is configured to build a service container image for each speech synthesis language model, which includes an execution environment, speech synthesis language model components, and calling interfaces.

[0073] The processing module 202 is used to initiate a binding request to the resource scheduling mechanism after the service container starts. The binding request includes deployment requirement parameters.

[0074] The processing module 202 is used to perform matching operations based on the binding request and resource status, and to establish a binding relationship between the target virtual processing unit that meets the conditions and the service container. At the same time, the service container identifier is written into the running context of the target virtual processing unit to construct a bearer unit with memory resource isolation and computing quota isolation.

[0075] In one possible implementation, the processing module 202 is used to parse the request content of the speech synthesis request into a standardized structure containing text input data, request identifier and priority parameters.

[0076] The processing module 202 is used to construct a connection status table for the bearer unit based on the mapping table. The connection status table records the real-time request connection number of the bearer unit and the load status of the virtual processing unit.

[0077] The processing module 202 is used to perform a connection count comparison operation based on the connection status table, and filter out the target bearer unit that is currently running and has the fewest connections, as the target processing instance for the speech synthesis request.

[0078] In one possible implementation, the processing module 202 is used to receive a speech synthesis request structure containing text input data from the target bearer unit, call the structure to generate a sequence of pronunciation units, and perform an acoustic modeling task within the computing quota and video memory resources allocated to the target processing unit, and output an intermediate acoustic representation vector.

[0079] The processing module 202 is used to generate speech waveform data by converting the intermediate acoustic representation vector within the target processing unit.

[0080] The processing module 202 is configured to encapsulate the voice waveform data into a data packet conforming to a transmission protocol, and send the data packet to an access gateway through a network interface of the voice synthesis model bearing unit, so that the access gateway responds the voice synthesis result to a terminal user.

[0081] In a possible implementation, the processing module 202 is configured to, in the process of assigning the voice synthesis request to the target bearing unit, increase the connection count of the target bearing unit by one in real time through the connection state table after each request forwarding, and record the request identifier and the time stamp.

[0082] The processing module 202 is configured to, after the target bearing unit completes the voice generation processing and outputs the voice synthesis result, determine the corresponding target bearing unit in the connection state table according to the request identifier, and decrease the connection count by one.

[0083] The processing module 202 is configured to, when the resource occupation of the virtual processing unit exceeds a preset threshold, set an unschedulable flag of the target bearing unit in the connection state table.

[0084] It should be noted that, in implementing the functions, the apparatus provided in the above embodiments is only exemplified by the above division of functional modules, and in actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0085] The embodiment also discloses an electronic device, which refers to Figure 3 The electronic device can include at least one processor 301, at least one communication bus 302, a user interface 303, a network interface 304, and at least one memory 305.

[0086] The communication bus 302 is configured to realize the connection and communication between the components.

[0087] The user interface 303 can include a display screen (Display) and a camera (Camera), and the optional user interface 303 can further include a standard wired interface and a wireless interface.

[0088] The network interface 304 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0089] The processor 301 can include one or more processing cores. The processor 301 connects various parts within the server through various interfaces and lines, performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Alternatively, the processor 301 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 301 can integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs. The GPU is responsible for rendering and drawing the content needed to be displayed on the display screen. The modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 301, but be realized by a separate chip.

[0090] The memory 305 can include a random access memory (RAM) and a read-only memory (ROM). Alternatively, the memory includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 305 can also be at least one storage device located away from the aforementioned processor 301. As a computer storage medium, the memory 305 can include an operating system, a network communication module, a user interface 303 module, and an application program of a virtual GPU-based multi-voice synthesis model carrying method.

[0091] In Figure 3In the electronic device shown, the user interface 303 is mainly used to provide an interface for user input, and obtain data input by the user. The processor 301 can be used to invoke an application program stored in the memory 305, which is a virtual GPU-based multi-voice synthesis model carrying method, and when executed by one or more processors 301, causes the electronic device to perform the method of one or more of the above embodiments.

[0092] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the described actions, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0093] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0094] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner for actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different parts can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical or other forms.

[0095] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0096] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0097] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory 305 and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory 305 includes: a U disk, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0098] The present application also discloses a computer readable storage medium, which stores instructions. When executed by one or more processors 301, the instructions cause an electronic device to perform one or more methods as described in the above embodiments.

[0099] The above are only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the true principles of the present disclosure. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the art that are not described in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for supporting multiple speech synthesis models based on virtual GPUs, characterized in that, The method includes: The physical graphics processing unit is virtualized and divided into multiple virtual processing units. Speech synthesis instances based on speech synthesis language models are deployed in multiple service containers. The service containers bind the virtual processing units through a resource scheduling mechanism to build multiple carrier units for speech synthesis models. Each carrier unit is bound to one of the virtual processing units and provides speech generation services based on the speech synthesis instances. When a speech synthesis request initiated by an end user is received, the speech synthesis request is allocated to the target bearer unit with the fewest current connections among the multiple bearer units based on the current number of requested connections for each bearer unit. The target processing unit bound to the target bearer unit performs speech generation processing on the speech synthesis request and outputs the speech synthesis result to the terminal user.

2. The method for supporting a multi-speech synthesis model based on a virtual GPU according to claim 1, characterized in that, The virtualization process of the physical graphics processing unit, which divides the physical graphics processing unit into multiple virtual processing units, specifically includes: Access the video memory resource mapping table and core execution unit configuration register of the physical graphics processing unit to obtain the current global resources of the physical graphics processing unit; The global resources are divided into multiple video memory partitions and computing unit partitions, and a virtual processing unit is configured for each video memory partition or each computing unit partition. The video memory partitions and computing unit partitions correspond one-to-one, and each virtual processing unit has independent video memory resources and computing quotas. By constructing a mapping table to record the binding relationship between the video memory partition and the computing unit partition to the virtual processing unit, the resource scheduling mechanism responds to resource request requests based on the mapping table during operation, ensuring that the isolation constraint between the video memory resources and the computing quota is always satisfied during the process of the service container binding to the virtual processing unit.

3. The method for supporting a multi-speech synthesis model based on a virtual GPU according to claim 2, characterized in that, The resource scheduling mechanism responds to resource request requests based on the mapping table during operation, ensuring that the isolation constraint between the video memory resources and the computing quota is always satisfied during the process of the service container binding to the virtual processing unit. Specifically, this includes: During the initialization of the service container, the resource scheduling mechanism obtains the resource status of each virtual processing unit in the mapping table, including the video memory resource status and the computing quota status. The deployment requirement parameters of the speech synthesis model declared by the service container are matched with the resource status, and the deployment requirement parameters include the video memory capacity requirement and the computing quota requirement. When the deployment requirement parameters are successfully matched with the memory resource status and the computing quota status, the target processing unit is determined, and the service container is bound to the target processing unit. At the same time, the mapping table is updated to record the binding relationship, and the unique identifier of the service container is written into the running context of the target processing unit to restrict the memory access and computation scheduling boundaries of the speech synthesis model running process, ensuring that the binding process satisfies the isolation constraints.

4. The method for supporting a multi-speech synthesis model based on a virtual GPU according to claim 3, characterized in that, The process involves deploying speech synthesis instances based on a speech synthesis language model in multiple service containers. Each service container binds to the virtual processing unit via a resource scheduling mechanism, constructing multiple bearer units for the speech synthesis model. Each bearer unit is bound to one of the virtual processing units and provides speech generation services based on the speech synthesis instance. Specifically, this includes: For each of the speech synthesis language models, a service container image is constructed, which includes an execution environment, speech synthesis language model components, and calling interfaces. After the service container starts, a binding request is initiated to the resource scheduling mechanism, and the binding request includes the deployment requirement parameters; A matching operation is performed based on the binding request and the resource status, and a binding relationship is established between the target virtual processing unit that meets the conditions and the service container. At the same time, the service container identifier is written into the running context of the target virtual processing unit to construct a bearer unit with memory resource isolation and computing quota isolation.

5. The method for supporting a multi-speech synthesis model based on a virtual GPU according to claim 2, characterized in that, When a speech synthesis request initiated by a terminal user is received, the speech synthesis request is allocated to the target bearer unit with the fewest current connections among the multiple bearer units based on the current number of requested connections for each bearer unit. Specifically, this includes: The request content of the speech synthesis request is parsed into a standardized structure containing text input data, request identifier and priority parameters; Based on the mapping table, a connection status table for the bearer unit is constructed. The connection status table records the real-time request connection count of the bearer unit and the load status of the virtual processing unit. Based on the connection status table, a connection count comparison operation is performed to filter out the target bearer unit that is currently running and has the fewest connections, which is then used as the target processing instance for the speech synthesis request.

6. The method for supporting a multi-speech synthesis model based on a virtual GPU according to claim 1, characterized in that, The process of the target processing unit bound to the target bearer unit performing speech generation processing on the speech synthesis request and outputting the speech synthesis result to the terminal user specifically includes: The target bearer unit receives a speech synthesis request structure containing text input data, calls the structure to generate a sequence of pronunciation units, and performs an acoustic modeling task within the computing quota and video memory resources allocated by the target processing unit, outputting an intermediate acoustic representation vector. The intermediate acoustic representation vector is used to generate speech waveform data within the target processing unit; The speech waveform data is encapsulated into data packets conforming to the transmission protocol and sent to the access gateway through the network interface of the speech synthesis model carrier unit. The access gateway then responds to the end user with the speech synthesis result.

7. The method for supporting a multi-speech synthesis model based on a virtual GPU according to claim 5, characterized in that, After receiving a speech synthesis request initiated by a terminal user, and allocating the speech synthesis request to the target bearer unit with the fewest current connections among the multiple bearer units based on the current number of requested connections for each bearer unit, the method further includes: During the process of allocating speech synthesis requests to the target bearer unit, the connection count of the target bearer unit is incremented by one in real time through the connection status table after each request is forwarded, and the request identifier and timestamp are recorded. After the target bearer unit completes the speech generation process and outputs the speech synthesis result, the corresponding target bearer unit is determined in the connection status table according to the request identifier, and the connection count is decremented by one. When the resource usage of the virtual processing unit exceeds a preset threshold, the unschedulable flag of the target bearer unit is set in the connection status table.

8. A multi-speech synthesis model carrier device based on a virtual GPU, characterized in that, The apparatus is used to execute a method for carrying out a multi-speech synthesis model based on a virtual GPU as described in any one of claims 1-7. The apparatus includes an acquisition module (201), a processing module (202), and an output module (203), wherein: The acquisition module (201) is used to perform virtualization processing on the physical graphics processing unit and divide the physical graphics processing unit into multiple virtual processing units; The processing module (202) is used to deploy speech synthesis instances based on speech synthesis language models in multiple service containers. The service containers bind the virtual processing units through a resource scheduling mechanism to build multiple carrier units for speech synthesis models. Each carrier unit is bound to one of the virtual processing units and provides speech generation services based on the speech synthesis instances. The processing module (202) is used to, when receiving a speech synthesis request initiated by a terminal user, allocate the speech synthesis request to the target bearer unit with the fewest current connections among the multiple bearer units based on the current number of requested connections of each bearer unit; The output module (203) is used to perform speech generation processing on the speech synthesis request based on the target processing unit bound to the target bearer unit, and output the speech synthesis result to the terminal user.

9. An electronic device, characterized in that, The device includes a processor (301), a communication bus (302), a user interface (303), a network interface (304), and a memory (305). The memory (305) is used to store instructions. The user interface (303) and the network interface (304) are both used to communicate with other devices. The communication bus (302) is used to realize the connection and communication between the components within the electronic device. The processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device performs the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images