Voice processing method, device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a voice processing method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring aliasing voice data to be segmented; acquiring reference voice data of a specified object in the at least two objects; extracting voiceprint characteristics of a specified object from the reference voice data, and segmenting voice signals matched with the voiceprint characteristics of the specified object from the aliasing voice data based on an attention mechanism; and generating a voice file of the specified object based on the voice signals obtained by segmentation. By adopting the embodiment of the invention, the pure voice signal of any specified object can be segmented from the aliasing voice data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to the field of artificial intelligence, and specifically to a speech processing method, a speech processing device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Aliased voice data (or aliased voice) is voice data that is a mixture of voice signals generated by multiple sound sources (i.e., objects that generate sound). For example, in a conference scene, the aliased voice data recorded by a recording device from a physical environment may include voice signals generated by multiple participants, and may also include voice signals generated by certain devices in the physical environment (such as devices that play conference videos).

[0003] At present, the source separation methods provided for aliased speech data include: 1. Separating aliased speech data by human ears. This artificial listening method results in a long segmentation process and low efficiency. 2. Relying on timbre frequency to separate aliased speech data. When there are multiple objects with similar timbre frequencies, accurate segmentation cannot be achieved. 3. Separating aliased speech data based on the distance of the sound source will limit the speech segmentation to the different distances of each sound source. 4. Using a dedicated speech segmentation model for a specified object to separate aliased speech data. This method is not portable and cannot be universal. Summary of the invention

[0004] The embodiments of the present application provide a speech processing method, apparatus, device, medium and program product. The method can separate the pure speech signal of any specified object from the mixed speech data and has universality.

[0005] On the one hand, an embodiment of the present application provides a speech processing method, the method comprising:

[0006] Acquire aliased speech data to be segmented, where the aliased speech data contains speech signals of at least two objects;

[0007] Acquire reference voice data of a designated object among at least two objects; the designated object refers to any one of the at least two objects; the reference voice data includes a voice signal of the designated object;

[0008] Extract the voiceprint characteristics of the specified object from the reference voice data, and segment the voice signal matching the voiceprint characteristics of the specified object from the aliased voice data based on the attention mechanism;

[0009] Based on the segmented speech signal, a speech file of a designated object is generated.

[0010] On the other hand, an embodiment of the present application provides a speech processing device, the device comprising:

[0011] An acquisition unit, used for acquiring aliased speech data to be segmented, wherein the aliased speech data includes speech signals of at least two objects;

[0012] The acquisition unit is further used to acquire reference voice data of a designated object among the at least two objects; the designated object refers to any one of the at least two objects; and the reference voice data includes a voice signal of the designated object;

[0013] A processing unit, configured to extract a voiceprint characteristic of a specified object from the reference voice data, and segment a voice signal matching the voiceprint characteristic of the specified object from the aliased voice data based on an attention mechanism;

[0014] The processing unit is further used to generate a voice file of a designated object based on the segmented voice signal.

[0015] In one implementation, the voiceprint characteristic of the specified object is represented by a voiceprint representation vector; the processing unit is used to segment the voice signal matching the voiceprint characteristic of the specified object from the aliased voice data based on the attention mechanism, specifically to:

[0016] Convert the aliased speech data from the time domain to the frequency domain to obtain speech spectrum features corresponding to the aliased speech data;

[0017] Based on the attention mechanism, the correlation between the voiceprint representation vector and the speech spectrum features is calculated to obtain the speech spectrum feature segmentation that matches the voiceprint characteristics;

[0018] The speech spectrum feature segments are converted from the frequency domain to the time domain to obtain a speech signal that matches the voiceprint characteristics.

[0019] In one implementation, the correlation calculation is implemented by a speech segmentation model; the speech segmentation model includes a feature extraction subnetwork and an upsampling subnetwork, and the feature extraction subnetwork and the upsampling subnetwork are connected by a convolutional connection layer;

[0020] The feature extraction subnetwork and the upsampling subnetwork are symmetrical; the feature extraction subnetwork contains m convolutional layers distributed in a hierarchy, and the upsampling subnetwork contains an upsampling layer corresponding to each convolutional layer, where m is a positive integer; the convolutional layer, the convolutional connection layer, and the upsampling layer include multiple convolutional networks connected in sequence;

[0021] Among them, the attention mechanism is integrated into all or part of the network layers in the speech segmentation model, and the fusion position of the attention mechanism in multiple convolutional networks corresponding to the network layer is not fixed; the network layer includes a convolutional layer, an upsampling layer and a convolutional connection layer.

[0022] In one implementation, each network layer in the speech segmentation model is integrated with an attention mechanism; the processing unit is used to calculate the correlation between the voiceprint representation vector and the speech spectrum feature based on the attention mechanism to obtain the speech spectrum feature segmentation matching the voiceprint characteristics, specifically for:

[0023] Input the voiceprint representation vector into each network layer in the speech segmentation network;

[0024] According to the attention mechanism integrated in each network layer, the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer is calculated to obtain the second feature map output by the corresponding network layer; the voiceprint characteristics represented by the second feature map output by the corresponding network layer match the voiceprint characteristics represented by the voiceprint representation vector;

[0025] The second feature map output by the 2m+1th network layer in the speech segmentation model is used as a speech spectrum feature segment that matches the voiceprint characteristics represented by the voiceprint representation vector.

[0026] In one implementation, any network layer in the speech segmentation model that is integrated with the attention mechanism is represented as a target network layer; the target network layer is a convolutional layer or a convolutional connection layer in the speech segmentation model; the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the target network layer is: the first convolutional network in the multiple convolutional networks connected in sequence, and the position between the convolutional networks that are adjacent to the first convolutional network and located after the first convolutional network in the multiple convolutional networks connected in sequence;

[0027] The processing unit is used to calculate the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer according to the attention mechanism integrated in each network layer to obtain the second feature map output by the corresponding network layer, specifically for:

[0028] The first convolutional network in the target network layer is used to perform feature extraction processing on the first feature map of the target network layer to obtain the third feature map of the target network layer; wherein, when the target network layer is the first convolutional layer of the hierarchical distribution in the feature extraction subnetwork, the first feature map of the target network layer is the speech spectrum feature; when the target network layer is a convolutional layer other than the first convolutional layer in the speech segmentation model, the first feature map of the target network layer is obtained by pooling the feature map output by the upper-level network layer adjacent to the target network layer;

[0029] According to the attention mechanism integrated in the target network layer, the correlation between the voiceprint representation vector and the third feature map of the target network layer is calculated to obtain the fourth feature map of the target network layer; the feature dimension of the third feature map is the same as the feature dimension of the fourth feature map;

[0030] The fourth feature map is subjected to feature extraction processing by using other convolutional networks except the first convolutional network in the target network layer to obtain a second feature map output by the target network layer.

[0031] In one implementation, any network layer in the speech segmentation model that is integrated with the attention mechanism is represented as a target network layer; the target network layer is an upsampling layer in the speech segmentation model; the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the target network layer is: the position after the tail convolutional network in the multiple convolutional networks connected sequentially;

[0032] The processing unit is used to calculate the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer according to the attention mechanism integrated in each network layer to obtain the second feature map output by the corresponding network layer, specifically for:

[0033] Using multiple convolutional networks connected sequentially in the target network layer, feature extraction processing is performed on the target feature map to obtain the first feature map of the target network layer; the target feature map is obtained by feature concatenating the feature map output by the convolutional layer corresponding to the target network layer and the feature map output by the upper-level network layer of the target network layer;

[0034] The attention mechanism integrated in the target network layer is used to calculate the correlation between the voiceprint representation vector and the first feature map of the target network layer to obtain the second feature map output by the target network layer; the feature dimension of the second feature map is the same as that of the first feature map.

[0035] In one implementation, the processing unit is further configured to:

[0036] Performing dimension transformation on the voiceprint representation vector to obtain a dimension-transformed voiceprint representation vector;

[0037] Among them, the feature dimension of the voiceprint representation vector after dimension transformation is the same as the feature dimension of the feature map of the attention mechanism to be input into the corresponding network layer for fusion.

[0038] In one implementation, if the number of network layers integrated with the attention mechanism in the speech segmentation model is greater than a quantity threshold, the processing unit is further configured to:

[0039] Performing model distillation on the speech segmentation model to obtain a speech segmentation model after model distillation;

[0040] Among them, the correlation calculation is realized by the speech segmentation model after model distillation.

[0041] In one implementation, the voiceprint characteristic of the specified object is represented by a voiceprint representation vector; and the processing unit, when used to extract the voiceprint characteristic of the specified object from the reference speech data, is specifically used to:

[0042] Segmenting the reference voice data to obtain a plurality of voice data segments corresponding to the reference voice data;

[0043] Convert the reference speech data from the time domain to the frequency domain to obtain reference speech spectrum features corresponding to the reference speech data, and segment the reference speech spectrum features to obtain reference speech spectrum feature segments corresponding to each speech data segment;

[0044] Based on each speech data segment and the corresponding reference speech spectrum feature segment, short-term correlation analysis is performed on each speech data segment to obtain a voiceprint semantic feature vector corresponding to each speech data segment; the voiceprint semantic feature vector is used to represent the semantic characteristics of the speech data segment;

[0045] A long-term correlation analysis is performed on the voiceprint semantic feature vector sequence to obtain a voiceprint representation vector of a specified object; the voiceprint semantic feature vector sequence includes a voiceprint semantic feature vector corresponding to each speech data segment.

[0046] In one implementation, any one of the multiple voice data segments is represented as a target voice data segment; the processing unit is used to perform short-term correlation analysis on each voice data segment based on each voice data segment and the corresponding reference voice spectrum feature segment, and obtain the voiceprint semantic feature vector corresponding to each voice data segment, specifically for:

[0047] Perform feature extraction processing on the target speech data segments to obtain a time domain feature map;

[0048] Performing feature extraction processing on the reference speech spectrum feature segments corresponding to the target speech data segments to obtain a frequency domain feature map;

[0049] The time domain feature map and the frequency domain feature map are fused to generate the voiceprint semantic feature vector corresponding to the target speech data segment.

[0050] In one implementation, the number of feature extraction processes is k, where k is an integer greater than 1; any feature extraction process is represented as the i-th feature extraction process; and the processing unit is used to fuse the time domain feature map and the frequency domain feature map to generate a voiceprint semantic feature vector corresponding to the target speech data segment, specifically for:

[0051] When i=1, the intermediate time domain feature map and the intermediate frequency domain feature map obtained by the first feature extraction process are fused to generate the first intermediate feature vector after the first feature extraction process;

[0052] When 1 < i ≤ k, the intermediate time-domain feature map and the intermediate frequency-domain feature map obtained by the i-th feature extraction process, and the (i - 1)-th intermediate feature vector obtained by the (i - 1)-th feature extraction process are fused to generate the i-th intermediate feature vector after the i-th feature extraction process;

[0053] Based on the k-th intermediate feature vector after the k-th feature extraction process when i = k, a voiceprint semantic feature vector corresponding to the segmented target voice data is generated.

[0054] On the other hand, an embodiment of the present application provides a computer device, which includes:

[0055] A processor for loading and executing a computer program;

[0056] A computer-readable storage medium storing a computer program, which when executed by the processor, implements the above-mentioned voice processing method.

[0057] On the other hand, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and the computer program is suitable for being loaded and executed by the processor to implement the above-mentioned voice processing method.

[0058] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by the processor, the above-mentioned voice processing method is implemented.

[0059] In the embodiment of the present application, the aliased voice data to be segmented is obtained, and the aliased voice data contains the voice signals of at least two objects; if there is a need to segment the voice signal generated by a specified object among the at least two objects, then a section of reference voice data of the specified object (such as a few seconds of voice generated by the specified object) can be obtained; the specified object can be any one of the at least two objects. Then, the voiceprint characteristics of the specified object are extracted from the reference voice data, and the voiceprint characteristics are unique and can characterize the identity of the specified object. Further, the voice signal matching the voiceprint characteristics of the specified object can be segmented from the aliased voice data based on the attention mechanism, so that a separate voice file of the specified object can be generated based on the segmented voice signal. It can be seen from this that, on the one hand, the embodiment of the present application only needs to obtain the reference voice data of the specified object to achieve the segmentation of the voice signal of the specified object from the aliased voice data; if you want to obtain the voice data of other objects, you only need to change the voiceprint characteristics of the object to be segmented, and there is no need to train a dedicated network for each object, which can make the scheme convenient and transferable, making the scheme highly versatile. On the other hand, the embodiment of the present application realizes the segmentation of aliased speech data based on the attention mechanism, which can more clearly and accurately calculate and extract the speech signal of the specified object in the mixed speech data, ensure the purity of the extracted speech signal, and achieve a more accurate and pure speech separation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 is a schematic diagram of the architecture of a speech processing system provided by an exemplary embodiment of the present application;

[0062] Figure 2 It is a schematic diagram of the architecture of a speech processing scenario provided by an exemplary embodiment of the present application;

[0063] Figure 3 is a flowchart of a speech processing method provided by an exemplary embodiment of the present application;

[0064] Figure 4 It is a schematic diagram of an interface for a user to input reference voice data of a specified object provided by an exemplary embodiment of the present application;

[0065] Figure 5 It is a schematic diagram of the structure of an existing Unet network;

[0066] Figure 6 It is a structural diagram of a speech segmentation model constructed by adding an Attention mechanism to each network layer in a Unet network, provided by an exemplary embodiment of the present application;

[0067] Figure 7a It is a schematic diagram of speech segmentation when a target network layer is a convolutional layer or a convolutional connection layer provided by an exemplary embodiment of the present application;

[0068] Figure 7b It is a schematic diagram of speech segmentation when a target network layer is an upsampling layer provided by an exemplary embodiment of the present application;

[0069] Figure 8 is a flowchart of another speech processing method provided by an exemplary embodiment of the present application;

[0070] Fig. 9 It is a schematic diagram of a process of voiceprint vector extraction provided by an exemplary embodiment of the present application;

[0071] Fig.10 is a schematic structural diagram of an improved PANNS provided by an exemplary embodiment of the present application;

[0072] Fig.11 is a schematic diagram of the structure of a transformer network provided by an exemplary embodiment of the present application;

[0073] Fig.12 is a structural schematic diagram of a speech processing device provided by an exemplary embodiment of the present application;

[0074] Fig.13 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0076] In an embodiment of the present application, a speech processing scheme is provided, specifically, a speech separation scheme for performing source separation on aliased speech data is provided. Among them, aliased speech data can be referred to as aliased speech or mixed audio signal, which is an audio mixed with multiple speech signals (or called audio signals). Among them, multiple speech signals can be generated by different objects (also called sound sources), and the objects here can include but are not limited to: humans, animals or physical devices (such as cars), etc.; the embodiment of the present application does not limit the sources of the multiple speech signals contained in the aliased speech data. For example, in a conference scene where multiple people participate in the discussion, the collected speech data usually contains speech signals generated by different participants; of course, if the conference scene also includes a device for playing audio and video, the collected speech data also includes the speech signal emitted by the device; at this time, the collected speech data containing speech signals generated by multiple objects can be called aliased speech data.

[0077] Furthermore, the process of separating the voice signal of a specified object from the aliased voice data can be called source separation. In other words, source separation can be simply understood as: separating the aliased voice data mentioned above through signal processing or other algorithms to achieve the technology of segmenting the voice signal of a specified object from the aliased voice data, and finally generating a separate audio file (or voice file) of the specified object. For example, after collecting a section of aliased voice data in a noisy outdoor scene, the voice signal generated by a specified object can be extracted from the aliased voice data through the source separation technology to generate a voice file of the specified object; in this way, when the voice file is played, only the voice generated by the specified object exists, so as to achieve the purpose of identifying the voice generated by a specified object.

[0078] Based on the above brief introduction to the concepts of aliased speech data and source separation, an embodiment of the present application provides a new speech processing solution, which mainly includes: obtaining aliased speech data to be segmented, wherein the aliased speech data contains speech signals of at least two objects; if the user wants to extract the speech signal generated by a specified object (such as any one of the at least two objects) from the aliased speech data, a reference speech data containing the speech signal of the specified object can be obtained. In this way, the voiceprint characteristics of the specified object can be extracted based on the reference speech data. Then, based on the attention mechanism, a speech signal matching the voiceprint characteristics is segmented and extracted from the aliased speech data to generate a separate speech file for the specified object based on the speech signal.

[0079] It can be seen that, on the one hand, the embodiment of the present application relies on the uniqueness of the voiceprint characteristics of each user. It only needs to provide a section of reference voice data of any specified object to extract the voiceprint characteristics of the specified object, and then the voice signal of any specified object can be separated and extracted from the aliased voice data based on the voiceprint characteristics; not only can the purpose of accurately separating the voice signal of the specified object from the aliased voice data be achieved, but also the source separation can be achieved for the object to which any voice signal contained in the mutually aliased voice data belongs, so it is highly reusable and portable, reduces the complexity of user input operations, and makes the entire system more universal. On the other hand, the embodiment of the present application calculates the voiceprint characteristics and aliased voice data of the specified object based on the attention mechanism, which can more clearly and accurately calculate and extract the voice signal of the specified object in the aliased voice data, avoid too much noise in the extracted voice signal, and achieve a more accurate and pure voice separation effect.

[0080] The embodiment of the present application mainly implements the speech processing solution provided by the embodiment of the present application through a reusable designated speaker speech segmentation system based on voiceprint vector embedding; that is, the system deploys the speech processing solution provided by the embodiment of the present application; in this way, when any user has the need to separate the speech signal of the aliased speech data, the system can be called to automatically separate and extract the speech file corresponding to the designated object from the aliased speech data. Among them, the exemplary architecture diagram of the system can be seen in Figure 1 ;like Figure 1 As shown in the figure, the system mainly includes two modules: voiceprint vector extraction model and speech segmentation model. The following is a brief introduction to these two modules:

[0081] (1) A voiceprint vector extraction model, which can be called a voiceprint vector extractor or a voiceprint recognition network, etc. The voiceprint vector extraction model is mainly used to identify the identity of the specified object to be segmented and extract the identity semantic vector of the specified object; the identity semantic vector here is called a voiceprint representation vector (or simply a voiceprint vector) in the embodiment of the present application, which is used to represent the voiceprint characteristics of the specified object.

[0082] See also Figure 1It can be seen that the voiceprint vector extraction model is constructed based on the improved audio neural network (Pretrained AudioNeural Networks, PANNS) network and the transformer network. The voiceprint vector extraction model is fully trained using an open source large-scale speaker data set (a data set containing rich voice data). The trained voiceprint vector extraction model has the ability to fully characterize the voiceprint of the object; in this way, the voiceprint vector extraction model can be used as the voiceprint vector extractor of the entire system. In the inference stage, after loading the model parameters trained in advance with large-scale data, the trained voiceprint vector extraction model can be used to calculate the voiceprint representation vector of the specified object for the reference voice data of the specified object (such as a short segment (such as a few seconds or more than ten seconds) of speech). The voiceprint representation vector is used to characterize the voiceprint characteristics of the specified object. It can be seen that the method of training the voiceprint vector extraction model through a large-scale speaker data set does not need to specifically collect relevant training data for each object in the aliased voice data, and gets rid of the reliance on a model exclusive to a certain object by extracting data from objects in the aliased voice data.

[0083] Among them: ① The improved PANNS in the voiceprint vector extraction model is an improvement on the traditional PANNS; the improvement is mainly reflected in: designing the information exchange link between the time domain link and the frequency domain link, so that there are multiple information exchanges in the time domain and frequency domain during the voiceprint vector extraction process, thereby achieving information complementarity between the time domain and the frequency domain, enabling the high-level network to fully perceive the underlying network information and improve the accuracy of voiceprint vector extraction. Among them, PANNS is an audio neural network trained based on a large audio data set (including speech data of a large number of speakers); it is usually used for audio pattern recognition or audio frame-level vectorization (embedding) as the encoding network at the front end of the model. ② The transformer network is a conversion model that relies on the attention mechanism to calculate input and output; the Transformer network abandons the convolutional model structure and achieves good performance only through the attention mechanism and the feed forward neural network, without the need to use a sequence alignment loop architecture.

[0084] (2) A speech segmentation model, which can be called a semantic segmentation network or a segmentation network. The speech segmentation network is mainly used to receive the voiceprint characteristics of a specified object (specifically, the voiceprint representation vector) input by the voiceprint vector extraction model, and extract a speech signal that matches the voiceprint characteristics of the specified object from the aliased speech data using an attention mechanism based on the voiceprint representation vector.

[0085] See also Figure 1It can be seen that the speech segmentation model is a segmentation model that integrates the attention mechanism. By introducing the attention mechanism into the segmentation network for transformation, it is possible to realize that in the process of feature processing of the segmentation network for the aliased speech data, the speech signal related to the voiceprint feature of the specified object can be calculated by combining the attention mechanism, and finally the speech signal of the specified object can be segmented from the aliased speech data; in this way, the speech signal of the specified object in the aliased speech data can be calculated and extracted more clearly and accurately, and separated purely, so as to achieve a more accurate and pure speech separation effect. Among them, the attention mechanism is a solution proposed by imitating human attention; in short, it is to imitate human attention to quickly filter out the information you want to pay attention to from a large amount of information. It is mainly used to solve the problem that it is difficult to obtain a reasonable vector representation when the input sequence of the time series model is long. The method is to retain the intermediate results of the time series model, learn it with a new model and associate it with the output, so as to achieve the purpose of information screening.

[0086] In summary, the system provided by the embodiment of the present application includes two modules. After the voiceprint vector extraction model extracts the reference voice data of the specified object to obtain the voiceprint characteristics used to characterize the identity of the specified object, the voiceprint characteristics can be embedded into the voice segmentation model; in this way, the voice segmentation model can extract and separate the voice signal matching the voiceprint characteristics from the aliased voice data based on the attention mechanism, thereby achieving a better signal separation effect. On the one hand, see Figure 1 It can be seen that the system provided by the embodiment of the present application is a fully automatic segmentation system constructed based on multiple deep learning neural networks (such as an improved PANNS network, a conversion network, and a segmentation network integrated with an attention mechanism, etc.); for the fully automatic segmentation system, only a small segment of reference speech data of the specified object and the aliased speech data to be segmented need to be input, and the speech signal of the specified object can be automatically and quickly extracted from the aliased speech data, which can greatly improve the efficiency of speech segmentation, completely get rid of manual participation, and form a rapid standardization. On the other hand, by innovatively embedding the voiceprint characteristics extracted by the voiceprint vector extraction model into the model architecture of the speech segmentation model, the speech separation model in the system can be reused; that is, each time it is used, it is only necessary to replace the voiceprint characteristics of the extracted object, and there is no need to train a separate segmentation network for each object, which can achieve convenient migration of the network, making the entire system highly versatile.

[0087] Figure 1The system shown can be deployed in a computer device, specifically in an application running in the computer device (such as deployed in the application in the form of a plug-in); that is, the solution is provided by the application running in the computer device. Among them: ① The application may refer to a computer program for completing one or more specific tasks. By classifying the application according to different dimensions (such as the operation mode and function of the application), the types of the same application in different dimensions can be obtained. For example: according to the operation mode of the application, the application may include but is not limited to: a client installed in the terminal, a small program that can be used without downloading and installing (as a subprogram of the client), a web (World Wide Web, Global Wide Area Network) application opened through a browser, etc. For another example: according to the functional type of the application, the application may include but is not limited to: IM (Instant Messaging) application, content interaction application, audio application or video application, etc. Among them, the instant messaging application refers to an application for instant communication of messages and social interaction based on the Internet, and the instant messaging application may include but is not limited to: a social application with communication functions, a map application with social interaction functions, a game application, etc. Content interaction application refers to an application that can realize content interaction, such as online banking, sharing platform, personal space, news and other applications. Audio applications refer to applications that implement audio functions based on the Internet. Audio applications may include but are not limited to: music applications with music playback and editing capabilities, radio applications with radio playback capabilities, or live broadcast applications with live broadcast capabilities, etc. Video applications refer to applications that can play pictures. Video applications may include but are not limited to: applications with short videos (videos are often short, such as a few seconds or minutes), applications with long videos (such as movies or TV series that often have a long playback time), etc.

[0088] ②Computer equipment may include terminals and / or servers. The terminals may include but are not limited to: smartphones (such as smartphones with Android systems or smartphones with Internet working operating systems (IOS)), tablet computers, portable personal computers, mobile Internet devices (MIDs), vehicle-mounted devices, head-mounted devices, smart TVs or smart homes, etc. The present application does not limit the types of terminals, which is explained here. The terminal is deployed with Figure 1The system shown or the application (or plug-in) providing the system, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0089] It can be seen that the embodiments of the present application can be executed by a terminal or a server, or by a terminal and a server together. Figure 2 ;like Figure 2 As shown, terminal 201 is a device held by a user who has a need for voice separation. When the user has a need to separate the voice file of a specified object from the aliased voice data, the user can send the aliased voice data to be segmented and the reference voice data of the specified object to the server 202 through the terminal 201. In this way, after receiving the reference voice data of the specified object and the aliased voice data to be segmented, the server 202 can first identify the reference voice data through the voiceprint vector extraction model in the system to obtain a voiceprint representation vector for representing the identity of the specified object, and then embed the voiceprint representation vector into the voice segmentation model in the system, so that the voice segmentation model can extract a pure voice signal matching the voiceprint characteristics represented by the voiceprint representation vector from the aliased voice data based on the attention mechanism, thereby generating a voice file of the specified object based on the voice signal. Finally, the server 202 returns the voice file of the specified object to the terminal 201, so that the user can play the voice file containing only the voice data of the specified object through the terminal 201.

[0090] It should be understood that the above briefly introduces the process of the voice processing solution by taking the computer device as a terminal and a server as an example; however, when the computer device is a terminal or a server, the idea of ​​the computer device executing the voice processing solution is similar to the above-described process, but the execution subject is different, which will not be repeated here. In addition, Figure 2 The terminal 201 and the server 202 shown may be directly or indirectly connected via wired or wireless communication, which is not limited in the present application.

[0091] Furthermore, the speech processing solution provided in the embodiment of the present application can be applied to any application scenario with speech separation requirements; the computer device providing the solution may also be different depending on the application scenario, and this is not limited to this. The application scenario may include but is not limited to at least one of the following: film and television drama scenario, audio and video creation scenario, and conversation scenario.

[0092] Optionally, the application scenario is a film and television drama scenario. Exemplarily, the film and television drama scenario is a dubbing scenario for a character in a film and television drama; specifically: in the film and television drama production stage, a dubbing actor is often required to dub a certain character in the film and television drama (such as after the aliased voice data recorded by the radio is submitted for review, there are some lines that do not meet the regulations and need to be re-recorded). However, the voice data obtained by the radio recording during the filming process or post-production process of the film and television drama is usually aliased voice data containing multiple voice signals. Therefore, before dubbing, it is necessary to extract the voice signals of other actors in the aliased voice data except the voice signals of the designated actors to be re-dubbed, so as to mix the voice signals of the designated actors after dubbing and the extracted pure voice signals of other actors, generate new aliased voice data, and add them to the film and television drama. It can be seen that in the film and television drama scenario, accurate voice segmentation can be provided by the embodiment of the present application, and there is no need to use a large amount of data of all actors to train a dedicated segmentation network, and the pure actor voice can be extracted and segmented quickly and efficiently.

[0093] Optionally, the application scenario is an audio and video creation scenario. Exemplarily, the audio and video creation scenario is a secondary creation scenario for audio and video (i.e., re-creation for an existing audio and video); specifically: in the secondary creation scenario, the user likes to extract part of the lines of a specified actor in multiple audio and video for line dialogue editing, that is, to edit the voice data of the specified actor in different audio and video into the same audio and video. This will involve extracting the pure voice signal of the specified actor in multiple audio and video; considering that the lines in each audio and video carry background music or voice signals of other objects, it is necessary to remove the background music from the audio and video to obtain the pure voice signal of the specified actor, so as to facilitate the fusion of the extracted multiple pure voice signals and generate a clipped voice file corresponding to the specified actor.

[0094] Optionally, the application scenario is a conversation scenario. Exemplarily, the conversation scenario is an online conference scenario; specifically: in online conference scenarios, there is often a need to transcribe speech into text, that is, to convert recorded speech data into text form; however, in online conference scenarios with multiple participants, the transcription of mixed speech data containing speech signals of multiple people has always been a difficult problem. By using the embodiment of the present application, the speech signal of each object participating in the online conference can be segmented and extracted from the mixed speech data according to the voiceprint characteristics of each object, and then the speech signal of each object can be input into the speech recognition system to achieve text transcription, which can greatly improve the accuracy of the mixed speech transcription.

[0095] It is worth noting that the application scenarios applicable to the voice processing solution provided in the embodiment of the present application are not limited to the above-mentioned ones; and as the application scenarios vary, the application or platform that carries the voice processing solution may vary.

[0096] It should also be noted that the collection and processing of relevant data in the embodiments of this application should be strictly in accordance with the requirements of the laws and regulations of relevant countries. The acquisition of personal information requires the knowledge or consent of the individual subject (or the legal basis for obtaining the information), and the subsequent use and processing of data shall be carried out within the scope of authorization of laws and regulations and the subject of personal information. For example, when the embodiments of this application are applied to specific products or technologies, such as obtaining the reference voice data of a specified object, it is necessary to obtain the permission or consent of the specified object, and the collection, use and processing of relevant data (such as the collection and release of the barrage posted by the object, etc.) need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0097] Based on the speech processing scheme described above, the embodiment of the present application proposes a more detailed speech processing method. The speech processing method proposed in the embodiment of the present application will be described in detail below in conjunction with the accompanying drawings.

[0098] Figure 3 A flow chart of a speech processing method provided by an exemplary embodiment of the present application is shown; the speech processing method may be executed by a computer device in the aforementioned system, such as a terminal and / or a server; the speech processing method may include but is not limited to steps S301-S304:

[0099] S301: Acquire aliased speech data to be segmented.

[0100] S302: Acquire reference speech data of a designated object among at least two objects.

[0101] In steps S301-S302, the aliased voice data to be segmented contains voice signals of at least two objects. For example, a heavy metal music includes "lyrics" generated by a "singer", a melody generated by a "guitar", and a melody generated by a "drum set", etc. Therefore, it is determined that the heavy metal music is aliased voice data, and the objects contained in the aliased voice data are: a singer, a guitar, and a drum set, and the voice signals contained in the aliased voice data are respectively a voice signal generated by the "singer", a voice signal generated by the "guitar", and a voice signal generated by the "drum set".

[0102] Furthermore, if the user has a need to separate and extract the voice signal of a certain object from the mixed voice data, the reference voice data of the certain object can be obtained first. At this time, the certain specified object is referred to as the specified object among the at least two objects, and the reference voice data of the specified object contains the voice signal of the specified object. The following is a brief introduction to the specified object and reference voice data mentioned above, wherein: ① The specified object can be any object among the at least two objects mentioned above from which the user wants to extract the voice signal; as can be seen from the foregoing description, the object can refer to a human, an animal or a physical device; for ease of explanation, the object type of the specified object is human as an example for introduction, which is specially explained here. For example, in the above-mentioned heavy metal music example, if the user wants to extract the "lyrics" produced by the "singer" from the noisy heavy metal music, then the "singer" is determined to be the specified object. At this time, it is necessary to separate the voice signals produced by the instruments in the heavy metal music and the voice signals produced by the "singer", and extract the pure voice signal of the "singer".

[0103] ② The reference voice data refers to a segment of voice data containing the voice signal of the specified object. In order to ensure that the relatively pure voiceprint characteristics of the specified object can be extracted from the reference voice data, the reference voice data obtained should be a segment of relatively pure voice data containing the specified object. For example, the reference voice data may only contain the voice signal of the specified object; for another example, the reference voice data may contain the voice signal of the specified object and other voice signals at the same time, but it is necessary to ensure that the voice signal of the specified object can be easily extracted from the reference voice data mixed with other voice signals (such as the signal frequency of other voice signals is low, while the signal frequency of the voice signal of the specified object is relatively high, etc.), which is conducive to analyzing the pure reference voice data to extract the relatively accurate voiceprint characteristics of the specified object. The embodiment of the present application does not limit the type, duration and source of the reference voice data; illustratively: the type of reference voice data may include but is not limited to: a segment of audio generated by the reading of the article by the specified object, a segment of audio generated when the specified object speaks, or a segment of audio generated when the specified object sings a cappella, etc. The duration of the reference voice data can be a few seconds or more than ten seconds, etc. The source of the reference voice data may include but is not limited to: when the designated object and the user with the voice separation requirement are different users, the reference voice data may be sent to the user by the designated object, or downloaded or recorded by the user through certain channels (such as historical voice information); when the designated object and the user with the voice separation requirement are the same user, the reference voice data may be input in real time by the designated object, that is, collected in real time through a microphone deployed in the terminal held by the user.

[0104] An exemplary schematic diagram of an interface for a user to input reference voice data of a specified object can be seen in Figure 4 ;like Figure 4 As shown, a voice acquisition interface 401 is displayed in the terminal screen of the terminal held by the user, and the voice acquisition interface 401 includes an acquisition area 402 for reference voice data. In detail, at least two voice acquisition entrances can be displayed in the acquisition area 402, such as a collection entrance 4021 and an upload entrance 4022. When the collection entrance 4021 is triggered, it means that the user wants to input the reference voice data of the specified object (at this time, the specified object is the user, or the specified object and the user are in the same physical environment) by real-time collection, then the microphone of the terminal is turned on, so that the voice signal in the physical environment where the user is located can be collected in real time as reference voice data. When the upload entrance 4022 is triggered, it means that the user wants to input the reference voice data of the specified object by uploading a file, then the user can upload the reference voice data about the specified object from the storage space (such as the local storage space of the terminal, the cloud storage space or the server storage space, etc.).

[0105] It should be understood that the interface elements (such as the interface content contained in the interface) and interface styles of the voice acquisition interface are not limited to Figure 4 As shown. For example, the voice acquisition interface may also display an upload entry for aliased voice data, through which the user can replace the aliased voice data to be segmented. For another example, a text conversion control (or component, button, option, etc.) may be added to the voice acquisition interface, so that the user can trigger the text conversion control before or after voice separation to convert the separated voice signal into text form with one click, shortening the text conversion path to a certain extent, thereby improving the text conversion efficiency.

[0106] S303: extracting the voiceprint characteristics of the specified object from the reference voice data, and segmenting the voice signal matching the voiceprint characteristics of the specified object from the aliased voice data based on the attention mechanism.

[0107] S304: Generate a voice file of a designated object based on the segmented voice signal.

[0108] In steps S303-S304, voiceprint is a sound wave spectrum that carries voice information, and is a biometric feature composed of multiple characteristic dimensions such as wavelength, frequency, and intensity; voiceprint has the characteristics of stability, measurability, and uniqueness, and can be used to uniquely identify the sound characteristics of an object, that is, voiceprint can be used to characterize the identity of an object. Therefore, after obtaining relatively pure reference voice data of a specified object, the embodiment of the present application supports extracting the voiceprint characteristics of the specified object from the reference voice data, so as to facilitate the subsequent separation and extraction of voice signals based on the unique voiceprint characteristics.

[0109] By the aforementioned Figure 1It can be seen from the system shown that by analyzing the reference voice data through the voiceprint vector extraction model, the voiceprint characteristics used to characterize the identity of the specified object can be extracted from the reference voice data. In practical applications, the voiceprint vector extraction model outputs the voiceprint characterization vector (or simply referred to as voiceprint vector) of the specified object, that is, the voiceprint characteristics of the specified object are characterized by the voiceprint characterization vector. Further, after extracting the voiceprint characterization vector of the specified object, the voiceprint vector extraction model can innovatively use vector embedding as the transmission of voiceprint information representation, and input it into the voice segmentation model to participate in the calculation of the attention mechanism, so as to segment the voice signal matching the voiceprint characteristics of the specified object from the aliased voice data. This innovative vector embedding mechanism enables the voice segmentation model to be independent of any object's historical voice data for additional training, and only a small amount of reference voice data is needed to extract the voiceprint vector, which can get rid of the dependence on large-scale voice data of the object, so as to achieve high reusability and portability of the system, thereby making the entire system more efficient, reducing the complexity of user input operations, and making the entire system more universal.

[0110] The speech segmentation model provided in the embodiment of the present application is obtained by improving the traditional speech segmentation network by using the attention mechanism; specifically, it is obtained by integrating the attention mechanism into the traditional speech segmentation network. Among them, the traditional speech segmentation network involved in the embodiment of the present application is a Unet (or written as U-net, U-Net, etc.) network, which is one of the algorithms for semantic segmentation using a fully convolutional network; it mainly uses a symmetrical U-shaped structure including a compression path and an expansion path.

[0111] A schematic diagram of an exemplary Unet network structure can be found in Figure 5 .like Figure 5 As shown in the figure, the Unet network is a U-shaped symmetrical network structure, which includes a left-right symmetrical feature extraction subnetwork and an upsampling subnetwork, and the feature extraction subnetwork and the upsampling subnetwork are connected through a convolution connection layer. Among them: ① The feature extraction subnetwork can be simply understood as a downsampling layer or encoding network, which contains a hierarchical distribution (the feature map output by the convolution layer of the previous level is used as the input of the convolution layer of the next level) m( Figure 5 There are m = 4 convolutional layers, where m is a positive integer. Figure 5As shown, a pooling function can be deployed after each convolutional layer; by first using the convolutional network in the convolutional layer to perform feature extraction on the aliased speech data, and then using the pooling function (pool) to further extract higher-order features, the features that are desired to be highlighted in the aliased speech data can be effectively retained; wherein, the embodiment of the present application does not limit the type of pooling function, such as when the pooling function is maximum pooling (max pool), it tends to be the maximum feature within the pooling window (such as a window size of 2*2) in the feature map output by the convolutional layer. ② Accordingly, the feature extraction subnetwork and the upsampling subnetwork are symmetrical, and the upsampling subnetwork can be simply understood as a decoding network, which includes an upsampling layer (up sampling layer) corresponding to each convolutional layer in the feature extraction subnetwork. As shown Figure 5 As shown in the figure, a transposed convolution (up-Conv) with a convolution kernel of 2*2 is deployed after the convolution connection layer and each upsampling layer to achieve the upsampling function. It can be seen that the symmetrical network structure of the Unet network can either realize the network from scratch and initialize the weights, and then train the model; or borrow the convolution layer structure of some existing networks (such as vgg (a convolutional network) in resnet (residual neural network)) and the corresponding trained weight files, and then add the subsequent upsampling layer for training calculations, etc.; in this way, using the existing weight model files in the deep learning model training can greatly speed up the model training speed.

[0112] Furthermore, each convolutional layer, convolutional connection layer and upsampling layer includes multiple convolutional networks connected in sequence; Figure 5As shown in the figure, the feature extraction subnetwork, the convolution connection layer and the upsampling layer can all contain three convolution networks with a convolution kernel of 3*3. Among them, the convolution network or convolutional neural network (CNN) belongs to the field of artificial intelligence (AI). Among them, artificial intelligence is the theory, method, technology and application system of using digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. Furthermore, convolutional neural networks belong to the field of machine learning in the field of artificial intelligence; the convolutional neural network is a feedforward neural network, mainly composed of one or more convolutional layers and a fully connected layer at the top, as well as associated weights and a pooling layer. Figure 5 As shown, an activation function can be deployed after each convolutional network to add nonlinear factors to the model through the activation function, so that the trained model can solve problems that the linear model cannot solve; the embodiment of the present application does not limit the type of activation function, such as the activation function can be a ReLu function (ReLuSigmoidTanh, linear rectification function) and the like.

[0113] Furthermore, the Unet network can also effectively combine high-level feature maps and low-level feature maps through skip connections (also called copy and crop) to obtain the final feature map. The specific process of the skip connection may include: the feature map obtained by each convolutional layer in the feature extraction subnetwork will be concatenated to the corresponding upsampling layer in the upsampling subnetwork; thereby achieving the effective use of each layer of feature maps in subsequent calculations. Compared with the network structure that does not implement skip connections, this method of skip-connecting feature maps of different dimensions can effectively avoid direct supervision and loss calculation in high-level feature maps, and effectively combine the features in low-level feature maps, so that the final feature map contains both high-dimensional features and many low-dimensional features, realizing the fusion of features at different scales and improving the accuracy of the model results.

[0114] Above Figure 5 The network structure of the traditional Unet network is introduced in detail, and the speech segmentation model provided in the embodiment of the present application is obtained by improving the network structure of the Unet network. The improvement of the network structure of the Unet network in the embodiment of the present application mainly includes: integrating the attention mechanism in all or part of the network layers (such as convolutional layers, convolutional connection layers and upsampling layers) in the network structure of the Unet network. In other words, the attention mechanism is integrated in all or part of the networks in the speech segmentation model obtained by improving the Unet network based on the attention mechanism. For the convenience of explanation, taking the addition of the Attention mechanism to each network layer in the Unet network as an example of a speech segmentation model, by adding the attention mechanism to each network layer, the voiceprint representation vector of the specified object can be embedded in each network layer in the Unet network, so that each network layer in the network can deeply feel the voiceprint information or voiceprint characteristics represented by the voiceprint representation vector, so that the final output voice signal is closer to the specified object, ensuring that the extracted voice signal is purer.

[0115] For example, the structural diagram of the speech segmentation model constructed by adding the Attention mechanism to each network layer in the Unet network can be seen in Figure 6 .like Figure 6 As shown, compared with the Unet network, the basic network architecture of the improved speech segmentation model is the same as the original Unet network architecture, but an attention mechanism is added to each layer in the Unet network architecture, and the input information of the attention mechanism is: the voiceprint representation vector of the specified object and the feature map output by the previous layer of the attention mechanism. By embedding the voiceprint representation vector of the specified object into each network layer, mainly performing attention calculations with the feature map of each network layer, the entire model can deeply perceive and learn the extracted voiceprint representation vector, so that each level calculation can move closer to the voiceprint representation vector, ensuring that the final extracted speech signal matches the voiceprint characteristics represented by the voiceprint representation vector. It is worth noting that the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the network layer is not fixed. Figure 6 The fusion locations shown are exemplary.

[0116] Based on the above Figure 6 The network structure of the speech segmentation model is introduced below. Taking the example that each network layer in the speech segmentation model is integrated with the attention mechanism, the speech segmentation model is based on the attention mechanism to segment the speech signal matching the voiceprint characteristics of the specified object from the aliased speech data, and generate the speech file of the specified object based on the speech signal. The process may include but is not limited to steps (1)-(4), wherein:

[0117] (1) Convert the aliased speech data from the time domain to the frequency domain to obtain the speech spectrum features corresponding to the aliased speech data.

[0118] Considering that the input information of the Unet network belongs to the frequency domain, it is necessary to convert the aliased speech data to obtain the speech spectrum features corresponding to the aliased speech data; in this way, the speech spectrum features can be input into the speech segmentation network as an image. Among them, time domain and frequency domain are two commonly used concepts in audio applications, and are also two dimensional concepts for measuring audio features; time domain is to display and process the sampling points of the speech signal in time, that is, to bind with time; frequency domain is a characteristic representation of the energy distribution of the speech signal in each frequency band; through conversion formulas (such as Fourier Transform, Laplace Transform or ZTransform, etc.), the speech signal can be converted from time domain to frequency domain, or from frequency domain to time domain.

[0119] (2) Based on the attention mechanism, the correlation between the voiceprint representation vector and the speech spectrum features is calculated to obtain the speech spectrum feature segmentation that matches the voiceprint characteristics.

[0120] like Figure 6 As shown, after obtaining the voiceprint representation vector of the specified object extracted by the voiceprint vector extraction model, the voiceprint representation vector will be input into each network layer in the speech segmentation network, specifically the attention mechanism integrated in each network layer. In this way, according to the attention mechanism integrated in each network layer, the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer can be calculated to obtain the second feature map output by the corresponding network layer; it is worth noting that according to the different positions of the network layer in the speech segmentation network, the first feature map of the network layer is also different, which will be introduced in the subsequent embodiments. Then, the second feature map output by the 2m+1th network layer (i.e., the last upsampling layer in the upsampling subnetwork) in the speech segmentation model can be used as the speech spectrum feature segmentation matching the voiceprint representation vector. It can be seen that by embedding the voiceprint representation vector used to represent the voiceprint characteristics of the specified object into each network layer in the speech segmentation model to participate in the feature extraction process, each network layer in the speech segmentation network can deeply feel the voiceprint information of the specified object, so that the final segmented output voice signal is closer to the voiceprint characteristics of the specified object, ensuring that the extracted voice signal is purer and more accurate.

[0121] It is not difficult to understand that, depending on the different fusion positions of the attention mechanism in the speech segmentation model, the specific implementation process of the network layers in the feature extraction subnetwork and the upsampling subnetwork performing correlation calculation on the voiceprint representation vector and the first feature map output by the previous layer of the corresponding network layer according to the fused attention mechanism is different. The following takes any network layer fused with the attention mechanism in the speech segmentation model as an example to represent the target network layer, and the target network layer performs correlation calculation according to the attention mechanism for exemplary explanation, where:

[0122] In one implementation, Figure 7a As shown, assuming that the target network layer is a convolutional layer or a convolutional connection layer in a speech segmentation model, and the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the target network layer is: the first convolutional network in the multiple convolutional networks connected in sequence, and the position between the convolutional networks adjacent to the first convolutional network and located after the first convolutional network in the multiple convolutional networks connected in sequence. In this implementation, the target network layer calculates the correlation between the voiceprint representation vector and the first feature map of the target network layer to obtain the specific implementation process of the second feature map output by the target network layer may include: first, using the first convolutional network in the target network layer, performing feature extraction processing on the first feature map of the target network layer to obtain the third feature map of the target network layer. Among them, when the target network layer is the first convolution layer 701 of the hierarchical distribution in the speech segmentation model (specifically, the feature extraction subnetwork), the first feature map of the target network layer is the speech spectrum feature obtained by frequency domain conversion of the aliased speech data; when the target network layer is other convolution layers (such as convolution layer 702) in the speech segmentation model except the first convolution layer 701, the first feature map of the target network layer is obtained by pooling the feature map output by the upper-level network layer adjacent to the target network layer (such as convolution layer 701). Then, according to the attention mechanism integrated in the target network layer, the correlation between the voiceprint representation vector and the third feature map of the target network layer is calculated to obtain the fourth feature map of the target network layer; the feature dimension of the third feature map of the target network layer is the same as the feature dimension of the fourth feature map, that is, the feature dimension before and after the attention mechanism calculation is the same. Finally, the fourth feature map is subjected to feature extraction processing using other convolution networks in the target network layer except the first convolution network to obtain the second feature map output by the target network layer.

[0123] In other implementations, such as Figure 7bAs shown, it is assumed that the target network layer is the upsampling layer in the speech segmentation model, and the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the target network layer is: the position after the tail convolutional network in the multiple sequentially connected convolutional networks. In this implementation, the target network layer calculates the correlation between the voiceprint representation vector and the first feature map of the target network layer to obtain the second feature map output by the target network layer. The specific implementation process may include: first, using multiple sequentially connected convolutional networks in the target network layer to perform feature extraction processing on the target feature map to obtain the first feature map of the target network layer. The target feature map here is obtained by feature splicing the feature map output by the convolutional layer corresponding to the target network layer and the feature map output by the upper-level network layer of the target network layer; as shown Figure 7b As shown, the input information of the first upsampling layer 703 in the upsampling subnetwork is the target feature map, which is obtained by concatenating the feature map output by the convolution connection layer 704 of the previous level of the first upsampling layer 703 and the feature map output by the convolution layer 705 corresponding to the first upsampling layer 703. Then, the attention mechanism fused in the target network layer is used to calculate the correlation between the voiceprint representation vector and the first feature map of the target network layer to obtain the second feature map output by the target network layer; the feature dimension of the second feature map is the same as that of the first feature map.

[0124] It should be understood that the above Figure 7a and Figure 7b It is just an exemplary process of the target network layer performing correlation calculation when the attention mechanism is respectively fused to an exemplary fusion position in the feature extraction module, the convolution connection layer and the upsampling sub-network layer; when the attention mechanism is fused to different fusion positions in the target network layer, the specific implementation process of the target network layer performing the correlation calculation is different.

[0125] (3) The speech spectrum feature segments matching the voiceprint characteristics are converted from the frequency domain to the time domain to obtain a speech signal matching the voiceprint characteristics.

[0126] After extracting the speech spectrum feature segment matching the voiceprint characteristics of the specified object from the speech spectrum feature corresponding to the aliased speech data based on the above steps, it is also necessary to convert the speech spectrum feature segment from the frequency domain to the time domain to obtain a speech signal that conforms to the data transmission format. The method of converting the speech spectrum feature segment from the frequency domain to the time domain may include but is not limited to the aforementioned Fourier Transform, Laplace Transform or ZTransform, etc., and is not limited to this.

[0127] (4) Generate a voice file of a specified object based on a voice signal matching the voiceprint characteristics. The file format of the voice file can be set according to the personalized needs of the user, and the embodiment of the present application does not limit the file format of the voice file.

[0128] Based on the speech signal extraction process described in the above steps (1)-(4), it can be seen that the embodiment of the present application supports the calculation of the Attention mechanism for each network layer in the voiceprint representation vector and the speech segmentation model; however, considering that the feature dimensions of each network layer in the speech segmentation network are different, specifically, the feature dimensions of each convolutional network in each network layer are different. Therefore, before inputting the voiceprint representation vector of the specified object into each network layer in the speech segmentation network, it is necessary to first use a network layer (such as Figure 6 The network layer 601 shown in the figure performs dimension transformation on the voiceprint representation vector to obtain the voiceprint representation vector after dimension transformation. The feature dimension of the voiceprint representation vector after dimension transformation is the same as the feature dimension of the feature map of the attention mechanism to be input into the corresponding network layer for fusion; the feature map of the attention mechanism to be input into the corresponding network layer for fusion here may refer to the third feature map described above. In addition, the attention mechanism may be inserted between any two convolutional networks in the multiple convolutional networks connected sequentially in the network layer; the first feature map of the network layer may refer to the feature map output by the convolutional network in the network layer that is adjacent to the attention mechanism and located before the attention mechanism.

[0129] In summary, the embodiment of the present application innovatively constructs a fully automatic speech processing solution, which realizes the segmentation of speech signals based on the voiceprint vector embedding method; for users, they only need to input a small segment of reference speech data of the specified object and the aliased speech data to be segmented, so as to realize automatic and rapid separation of the speech signal of the specified object from the aliased speech data, which can greatly improve the efficiency of speech segmentation, completely get rid of manual participation, and form rapid standardization. In this solution, on the one hand, the voiceprint vector embedding method is adopted, and the voiceprint representation vector used to characterize the voiceprint characteristics of the specified object is input into the speech segmentation model to participate in the Attention calculation, which can make the speech segmentation model independent of any object's historical speech data for additional training, and can get rid of the dependence on the object's large-scale speech data, so as to make the system highly reusable and portable, and make the whole system more universal. On the other hand, by improving the Unet network based on the attention mechanism, the input voiceprint representation vector can be calculated with the Attention mechanism of each network layer in the Unet network, thereby ensuring that the feature map output by the speech segmentation model and the voiceprint characteristics represented by the voiceprint representation vector of the specified object are more consistent, avoiding the extracted speech signal of the specified object containing too much noise, improving the purity of the speech signal, and improving the segmentation accuracy of the speech segmentation model.

[0130] See also Figure 8 , Figure 8 A flow chart of another speech processing method provided by an exemplary embodiment of the present application is shown; the speech processing method may be executed by a computer device in the aforementioned system, such as a terminal and / or a server; the speech processing method may include but is not limited to steps S801-S806:

[0131] S801: Acquire aliased speech data to be segmented.

[0132] S802: Acquire reference speech data of a designated object among at least two objects.

[0133] It should be noted that the specific implementation process shown in steps S801-S802 can be found in the aforementioned Figure 3 The relevant description of the specific implementation process shown in steps S301-S302 in the illustrated embodiment is not repeated here.

[0134] S803: Perform short-term correlation analysis on the reference speech data of the designated object.

[0135] S804: Perform long-term correlation analysis on the reference speech data of the designated object to obtain a voiceprint representation vector of the designated object.

[0136] In steps S803-S804, in order to learn the clearer voiceprint characteristics of the specified object from the reference voice data of the specified object, the embodiment of the present application supports the analysis of the reference voice data in combination with short-time correlation and long-time correlation to extract the voiceprint representation vector that can fully express the voiceprint characteristics of the specified object. Among them, the short-time correlation analysis of the reference voice data can be simply understood as: the process of feature analysis of a shorter (such as 20 milliseconds) voice signal in the reference voice data; considering that the voice signal in the reference voice data usually does not change in a short time, after the reference voice data is discretized, the information distribution of each shorter voice signal in the time domain and frequency domain can be used to extract the characteristics of the voice signal in the short time, thereby realizing the feature analysis of each voice signal in the reference voice data. In short, the short-time correlation analysis focuses on the feature analysis of the segmented voice signals in the reference voice data. The difference is that the long-time correlation analysis of the reference voice data can be simply understood as: the process of feature analysis of the entire reference voice data; that is, the long-time correlation analysis focuses on the semantic expression of the entire signal sequence of the reference voice data.

[0137] The short-term correlation analysis and long-term correlation analysis mentioned above mainly rely on Figure 1 The voiceprint vector extraction model in the system shown is implemented. As described above, the voiceprint vector extraction model includes an improved audio neural network (PANNS) network and a transformer network; wherein: the improved audio neural network (PANNS) network can be referred to as improved PANNS, which is mainly used for short-term correlation analysis of reference voice data; the transformer network can be referred to as transformer network, which is mainly used for long-term correlation analysis of reference voice data.

[0138] An exemplary schematic diagram of using the improved PANNS and transformer network to perform short-term correlation analysis and long-term correlation analysis on reference speech data can be found in Fig. 9 .like Fig. 9As shown, first, the reference speech data is converted from the time domain to the frequency domain to obtain the reference speech spectrum characteristics corresponding to the reference speech data; wherein, the conversion formula from the time domain to the frequency domain can refer to the aforementioned related description, which will not be repeated here. Then, in order to realize short-time correlation analysis, the reference speech spectrum characteristics can also be segmented to obtain the reference speech spectrum characteristics segment corresponding to each speech data segment; the reference speech spectrum characteristics segment involved in the embodiment of the present application can be a logarithmic Mel spectrum (Log-mel or Logmel). Among them, the mel spectrum is a nonlinear frequency scale based on the human ear's sensory judgment of equidistant pitch changes; when performing signal processing, it can be more artificially set to cater to the changes in the auditory perception threshold of the human ear.

[0139] Then, the reference speech spectrum feature segment corresponding to each speech data segment is input into the improved PANNS network by segmented input, and the complete reference speech data is input into the improved PANNS; in this way, the improved PANNS can segment the received reference speech data according to the segmentation rules followed when the reference speech spectrum feature is segmented, and obtain multiple speech data segments corresponding to the reference speech data. In addition, the improved PANNS will perform short-term correlation analysis on each speech data segment based on each speech data segment and the corresponding reference speech spectrum feature segment, and obtain the voiceprint semantic feature vector corresponding to each speech data segment; the voiceprint semantic feature vector is used to characterize the semantic characteristics of the corresponding speech data segment.

[0140] Finally, the vector sequence composed of the voiceprint semantic feature vectors corresponding to each speech data segment (or called voiceprint semantic feature vector sequence) is input into the transformer network, so that the transformer network can perform long-term correlation analysis on the voiceprint semantic feature vector sequence to obtain the voiceprint representation vector of the specified object, where the voiceprint semantic feature vector sequence includes the voiceprint semantic feature vector corresponding to each speech data segment. Among them, the transformer network is a sequence network, and its input is an overall vector sequence, and its output is also a sequence (that is, the voiceprint representation vector of the specified object is a sequence); after the transformer network outputs a sequence, the last vector in the sequence can be called state, which contains the semantic fusion of the entire sequence; in this way, the overall sequence output by the transformer network can be averaged (mean), and the average result and the state can be superimposed to generate a voiceprint representation vector that integrates the semantic features expressed by the entire sequence. The above-mentioned method of obtaining the voiceprint representation vector by combining the speech features represented by the average result of the transformer network output sequence and the speech features represented by the overall sequence can effectively ensure that the voiceprint representation vector can more fully reflect the voiceprint characteristics of the specified object and ensure the accuracy of voiceprint feature extraction.

[0141] Based on the above overall introduction to the voiceprint vector extraction model including the improved PANNS and transformer network, the following introduces the structure and process of the improved PANNS and transformer network respectively, where:

[0142] (1) Improved PANNS.

[0143] An exemplary structural diagram of the improved PANNS can be found in Fig.10 ;like Fig.10As shown, the input information of the improved PANNS is the reference speech data, that is, the input of the improved PANNS uses the original speech sampling point sequence, that is, the original sequence of the audio signal. The improved PANNS can be divided into two branches, namely the time domain branch (or called the time domain processing branch) and the frequency domain branch (or called the frequency domain processing branch); wherein: the input information of the time domain branch is the reference speech data, and the input information of the frequency domain branch is the reference speech spectrum feature corresponding to the reference speech data; the reference speech spectrum feature is obtained by converting the time domain signal "reference speech data" from the time domain to the frequency domain. Further, it can be seen from the above description that in order to fully extract and characterize the voiceprint characteristics of the specified object, the improved PANNS focuses on short-term correlation analysis of the reference speech data, that is, it supports processing using the improved PANNS by means of segmented input; specifically, the improved PANNS only processes a segment of speech data in the reference speech data and the reference speech spectrum feature corresponding to the speech data each time.

[0144] That is to say, the input information of each input time domain branch is a segment of speech data in the reference speech data; similarly, the input information of each input frequency domain branch is a reference speech spectrum feature segment in the frequency domain corresponding to a segment of speech data in the reference speech data. After the reference speech data is input into the improved PANNS, the reference speech data can be converted from the time domain to the frequency domain to obtain the reference speech spectrum feature corresponding to the reference speech data, and the reference speech spectrum feature is segmented to obtain the reference speech spectrum feature segment corresponding to each speech data segment; and the reference speech data is segmented to obtain multiple speech data segments corresponding to the reference speech data. It is worth noting that the segmentation rules followed by the reference speech data for segmentation processing are the same as the segmentation rules followed by the reference speech spectrum feature for segmentation processing; for example, when the reference speech data is segmented, 20 milliseconds of speech data are periodically collected as a segment. Then, when the reference speech spectrum feature is segmented, each reference speech spectrum feature segment is converted to the time domain and the corresponding speech data segment length is 20 milliseconds; through this corresponding method of time domain segmentation and frequency domain segmentation, the accuracy of each characteristic analysis is ensured.

[0145] Continue to see Fig.10 In the case of segmented input, the improved PANNS can perform short-term correlation analysis on each speech data segment based on each speech data segment and the corresponding reference speech spectrum feature segment, and obtain the voiceprint semantic feature vector corresponding to each speech data segment; the voiceprint semantic feature vector is used to characterize the semantic characteristics of the corresponding speech data segment. For the sake of convenience, taking any speech data segment among multiple speech data segments as the target speech data segment as an example, the specific process of the short-term correlation analysis described above is introduced, where:

[0146] ① The time domain branch contains multiple one-dimensional convolutional layers (Conv 1D) and maximum pooling layers (Max pooling); Conv is the abbreviation of Convolutional, D is the abbreviation of Dimension, and the dimension of the maximum pooling layer is 1, and the stride (abbreviated as s) is a one-dimensional vector with a length of 4. Fig.10 As shown, it is assumed that the time domain branch includes in sequence: one-dimensional convolution layer → one-dimensional convolution block (Conv 1D block, composed of one or more one-dimensional convolution layers) → maximum pooling layer (dimension is 1, and step size s is 4) → one-dimensional convolution block → maximum pooling layer → one-dimensional convolution block → maximum pooling layer; then after the target speech data segment is subjected to feature extraction processing by the convolution layer, the feature map after feature extraction processing is selected by the adjacent maximum pooling layer after the convolution layer, which is conducive to further extracting the features that are desired to be focused on in the feature map. Through the progressive manner between multiple one-dimensional convolution layers and maximum pooling layers, the reference speech data (specifically the target speech data segment) can be subjected to multiple feature extractions to obtain a one-dimensional sequence resize, and the resize is subjected to dimensional transformation (Reshape) to obtain multiple two-dimensional time domain feature maps (or two-dimensional spectrum Wavegram); the purpose of the dimensional transformation here is to enable the converted time domain feature map to be fused with the frequency domain feature map output by the frequency domain branch. In the above process, by using a large number of one-dimensional convolutional layers in the time domain branch, the time domain characteristics of the speech signal in the reference speech data (such as information such as audio loudness and sampling point amplitude) can be directly learned when the reference speech data is subjected to feature extraction processing through the time domain branch.

[0147] ② The frequency domain branch contains multiple two-dimensional convolution layers (Conv 2D) and maximum pooling layers (Max pooling); the dimension of the maximum pooling layer of the frequency domain branch is 2. Fig.10 As shown, it is assumed that the frequency domain branch includes in sequence: a two-dimensional convolution block (composed of one or more two-dimensional convolution layers) → a maximum pooling layer (dimension is 2) → a two-dimensional convolution block → a maximum pooling layer → a two-dimensional convolution block; through the multiple two-dimensional convolution layers and the maximum pooling layer, the reference speech spectrum feature segment (Logmel) corresponding to the target speech data segment can be subjected to feature extraction processing to obtain multiple frequency domain feature maps (Feature maps). It is worth noting that the feature dimension of the frequency domain feature map here is the same as the feature dimension of the time domain feature map output by the time domain branch. In the above process, by using a large number of two-dimensional convolution layers in the frequency domain branch, when the reference speech spectrum feature segment is subjected to feature extraction processing through the frequency domain branch, the frequency domain characteristics of the speech signal in the reference speech data can be directly learned.

[0148] ③ After obtaining the time domain feature map (Wavegram) output by the time domain branch and the frequency domain feature map (Feature map) output by the frequency domain branch based on the above steps, the time domain feature map and the frequency domain feature map can be fused to generate the voiceprint semantic feature vector corresponding to the target speech data segment. Fig.10 As shown, there are multiple information exchanges between the time domain branch and the frequency domain branch, which are to transform the dimension (Reshape) of the information features of the time domain branch, and then fuse (concat) with the features of the frequency domain branch, and then convolve the fusion results through a two-dimensional convolution block (Conv 2D block) and input them into a higher-level fusion module for fusion. Considering that the correlation of the reference speech data in time sequence can be obtained in time domain processing, and the correlation of the reference speech data in different frequencies can be obtained in frequency domain processing, the two belong to different fields; then through the information interaction between the two domains (time domain and frequency domain) described above, the time domain and frequency domain can maintain information complementarity, so that the high-level network can perceive the underlying network information, thereby achieving full learning of the reference speech data.

[0149] Based on this, assume that the number of times of feature extraction processing performed in the time-domain branch and the frequency-domain branch is k, where k is an integer greater than 1, and any feature extraction processing is represented as the i-th feature extraction processing. Then the fusion between the time-domain feature map output by the time-domain branch and the frequency-domain feature map output by the frequency-domain branch mentioned above can specifically include: First, when i = 1 (i.e., the first feature extraction processing), fuse the intermediate time-domain feature map obtained by the time-domain branch in the first feature extraction processing and the intermediate frequency-domain feature map obtained by the frequency-domain branch in the first feature extraction processing to generate the first intermediate feature vector after the first feature extraction processing. Then, when 1 < i ≤ k, fuse the intermediate time-domain feature map obtained by the time-domain branch in the i-th feature extraction processing, the intermediate frequency-domain feature map obtained by the frequency-domain branch in the i-th feature extraction processing, and the (i - 1)-th intermediate feature vector obtained by the (i - 1)-th feature extraction processing to generate the i-th intermediate feature vector after the i-th feature extraction processing. Finally, based on the k-th intermediate feature vector after the k-th feature extraction processing when i = k, generate the voiceprint semantic feature vector corresponding to the target semantic data segment. Among them, the specific generation process here can include: input the k-th intermediate feature vector into a two-dimensional convolutional neural network (2D CNN layers) for feature extraction to obtain a vector sequence, and perform an average operation (mean) and a maximum operation (max) on this vector sequence to find the average value and the maximum value. Then add the obtained average value and the maximum value (sum) and then pass through an activation function (Rule) to obtain a feature vector (vector), and use a normalization function (softmax) to normalize this feature vector to convert the feature vector into a voiceprint semantic feature vector representing a probability distribution, that is, the voiceprint semantic feature vector corresponding to the target speech data segment.

[0150] To sum up, the entire reference speech data is framed and separately input into the improved PANNS. The improved PANNS selects to fuse the last vector of the entire sequence and the mean value of the entire sequence together to generate the final voiceprint representation vector, and can obtain a multi-band semantic feature vector sequence representing the entire reference speech data (including the voiceprint semantic feature vector corresponding to each speech data segment). Through this improved PANNS, short-term correlation analysis of the reference speech data can be realized, the voiceprint characteristics of each segmented speech signal in the reference speech data can be fully learned, and the extracted voiceprint representation vector can fully express the voiceprint information of the specified object.

[0151] (2) Transformer network.

[0152] Considering that the improved PANNS focuses on short-term correlation, that is, the features are calculated in segments; therefore, in order to fully learn the voiceprint characteristics of the specified object from the reference speech data, the embodiment of the present application also introduces a transformer network to learn the long-term correlation information of the reference speech data. The attention mechanism of the transformer network can better pay attention to the voiceprint characteristics of the reference language data about the specified object, and better realize the extraction of global feature information for the reference language data.

[0153] For example, the network structure diagram of the transformer network can be found in Fig.11 ;like Fig.11 As shown in Figure 1, the transformer network uses an encoder (encoding)-decoder (decoding) architecture. The encoder side and the decoder side are both composed of N encoder layers (i.e. Fig.11 The encoder layer mainly consists of two sublayers, among which: the first sublayer contains a multi-head attention mechanism (Multi-Head Attention), residual and normalization (Add&Norm); the second sublayer contains a feedforward neural network (Feed Forward), residual and normalization layer. Among them, the multi-head attention mechanism contained in the first sublayer can help obtain the contextual semantics of the voiceprint semantic feature vector corresponding to the voice data segment. The decoder layer mainly consists of three sublayers, among which: the first sublayer contains a masked multi-head attention mechanism (MaskedMulti-Head Attention), residual and normalization layer, the first sublayer contains a multi-head attention mechanism (Multi-Head Attention), residual and normalization (Add&Norm) layer, the first sublayer contains a feedforward neural network (Feed Forward), residual and normalization layer; the decoding layer can help obtain the key content that needs attention through this three-layer structure.

[0154] In the specific implementation, the input information of the encoder side in the transformer network is the voiceprint semantic feature vector sequence (Input Embedding) output by the improved PANNS, and the voiceprint semantic feature vector sequence contains the voiceprint semantic feature vector corresponding to each semantic data segment. Then, the input voiceprint semantic feature vector sequence is positionally encoded (Positional Encoding) to realize the preprocessing of the voiceprint semantic feature vector sequence; wherein, positional encoding is a method of secondary representation of each word in the vector sequence using the position information of the word, so that the data input to the encoding side can carry the position information of the word. Further, the position-encoded data is input to the encoding side, and each layer in the encoding side (such as the above description of the encoding side structure) encodes the input data to obtain the encoding result. Then, the decoding side will combine the encoding result and the output feature (shifted right) of the decoding side at the previous moment as the input data of the decoding side at this time, and the decoding side will decode the input data. Finally, the output features of the decoding side are linearly transformed (Linear) and classified (softmax) to obtain the voiceprint representation vector (Output Probabilities) of the specified object. Through the above process, the voiceprint semantic feature vector sequence can be calculated through the transformer network, which can make the long-term semantic expression of the entire reference speech data of the entire voiceprint semantic feature vector sequence clearer, that is, the final output voiceprint representation vector can fully and clearly express the voiceprint characteristics of the specified object.

[0155] S805: Based on the attention mechanism, a speech signal matching the voiceprint characteristics of the specified object is segmented from the aliased speech data.

[0156] It should be noted that the specific implementation process shown in step S805 can be found in the aforementioned Figure 3 In the illustrated embodiment, the description of the specific implementation process of segmenting the speech signal matching the voiceprint characteristics of the specified object from the aliased speech data based on the attention mechanism in step S302 is not repeated here.

[0157] As can be seen from the relevant description shown in the aforementioned step S302, the embodiment of the present application mainly adopts the improved Unet network (i.e., the speech segmentation network) to implement the segmentation of the aliased speech data based on the attention mechanism to obtain a speech signal that matches the voiceprint characteristics of the specified object. In addition, the speech segmentation network mainly integrates the attention mechanism into the traditional Unet network, such as adding the attention mechanism to each network layer in the traditional Unet network to achieve the improvement of the traditional Unet network. Considering that more Attention mechanisms may be introduced in the speech segmentation system, resulting in a large number of parameters of the entire model, the embodiment of the present application also supports model distillation of the speech segmentation model integrated with the attention mechanism to obtain a speech segmentation model after model distillation; in this way, the aforementioned correlation calculation can be implemented by the speech segmentation model after model distillation, thereby reducing the scale of the entire system, reducing the overall parameter amount and time consumption. Among them, model distillation is a method of learning a large model (teacher model) with a large number of parameters to obtain a more compact small model (student model) with a small number of parameters; the model distillation for the large model is mainly achieved through pruning (pruning) and knowledge distillation and other technologies. The embodiment of the present application does not limit the specific implementation method of model distillation.

[0158] S806: Generate a voice file of a designated object based on the segmented voice signal.

[0159] It should be noted that the specific implementation process shown in step S806 can be found in the aforementioned Figure 3 The relevant description of the specific implementation process shown in step S304 in the illustrated embodiment is not repeated here.

[0160] In summary, the embodiment of the present application provides a reusable voice segmentation method for a specified object with voiceprint vector embedding. The method uses a voiceprint vector extraction model composed of an improved PANNS and a transformer network to extract voiceprints from a small section of reference voice data of the specified object. By combining the short-time correlation analysis of the improved PANNS and the long-time correlation analysis of the transformer network, the voiceprint representation vector of the specified object extracted can more fully and clearly express the voiceprint characteristics of the specified object. In addition, the embodiment of the present application also integrates the attention mechanism into the Unet network, so that the Unet network (i.e., the voice segmentation model) integrated with the attention mechanism can realize the segmentation of aliased voice data based on the attention mechanism, and can more clearly and accurately calculate and extract the voice signal of the specified object in the mixed voice data, ensure the purity of the extracted voice signal, and achieve a more accurate and pure voice separation effect. In addition, the embodiment of the present application only needs to obtain the reference voice data of the specified object to segment the voice signal of the specified object from the mixed voice data; if you want to obtain the voice data of other objects, you only need to change the voiceprint characteristics of the object to be segmented. There is no need to train a dedicated network for each object, which makes the solution convenient and transferable, making the solution extremely versatile.

[0161] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0162] Fig.12 A schematic diagram of the structure of a speech processing device provided by an exemplary embodiment of the present application is shown; the speech processing device can be used to perform Figure 3 or Figure 8 Some or all of the steps in the method embodiment shown. Fig.12 , the speech processing device comprises the following units:

[0163] An acquiring unit 1201 is configured to acquire aliased speech data to be segmented, wherein the aliased speech data includes speech signals of at least two objects;

[0164] The acquisition unit 1201 is further configured to acquire reference speech data of a designated object among at least two objects; the designated object refers to any one of the at least two objects; and the reference speech data includes a speech signal of the designated object;

[0165] The processing unit 1202 is used to extract the voiceprint characteristics of the specified object from the reference voice data, and segment the voice signal matching the voiceprint characteristics of the specified object from the aliased voice data based on the attention mechanism;

[0166] The processing unit 1202 is further configured to generate a voice file of a designated object based on the segmented voice signal.

[0167] In one implementation, the voiceprint characteristic of the specified object is represented by a voiceprint representation vector; the processing unit 1202 is used to segment the voice signal matching the voiceprint characteristic of the specified object from the aliased voice data based on the attention mechanism, specifically to:

[0168] Convert the aliased speech data from the time domain to the frequency domain to obtain speech spectrum features corresponding to the aliased speech data;

[0169] Based on the attention mechanism, the correlation between the voiceprint representation vector and the speech spectrum features is calculated to obtain the speech spectrum feature segmentation that matches the voiceprint characteristics;

[0170] The speech spectrum feature segments are converted from the frequency domain to the time domain to obtain a speech signal that matches the voiceprint characteristics.

[0171] In one implementation, the correlation calculation is implemented by a speech segmentation model; the speech segmentation model includes a feature extraction subnetwork and an upsampling subnetwork, and the feature extraction subnetwork and the upsampling subnetwork are connected by a convolutional connection layer;

[0172] The feature extraction subnetwork and the upsampling subnetwork are symmetrical; the feature extraction subnetwork contains m convolutional layers distributed in a hierarchy, and the upsampling subnetwork contains an upsampling layer corresponding to each convolutional layer, where m is a positive integer; the convolutional layer, the convolutional connection layer, and the upsampling layer include multiple convolutional networks connected in sequence;

[0173] Among them, the attention mechanism is integrated into all or part of the network layers in the speech segmentation model, and the fusion position of the attention mechanism in multiple convolutional networks corresponding to the network layer is not fixed; the network layer includes a convolutional layer, an upsampling layer and a convolutional connection layer.

[0174] In one implementation, each network layer in the speech segmentation model is integrated with an attention mechanism; the processing unit 1202 is used to calculate the correlation between the voiceprint representation vector and the speech spectrum feature based on the attention mechanism to obtain the speech spectrum feature segmentation matching the voiceprint characteristics, specifically for:

[0175] Input the voiceprint representation vector into each network layer in the speech segmentation network;

[0176] According to the attention mechanism integrated in each network layer, the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer is calculated to obtain the second feature map output by the corresponding network layer; the voiceprint characteristics represented by the second feature map output by the corresponding network layer match the voiceprint characteristics represented by the voiceprint representation vector;

[0177] The second feature map output by the 2m+1th network layer in the speech segmentation model is used as a speech spectrum feature segment that matches the voiceprint characteristics represented by the voiceprint representation vector.

[0178] In one implementation, any network layer in the speech segmentation model that is integrated with the attention mechanism is represented as a target network layer; the target network layer is a convolutional layer or a convolutional connection layer in the speech segmentation model; the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the target network layer is: the first convolutional network in the multiple convolutional networks connected in sequence, and the position between the convolutional networks that are adjacent to the first convolutional network and located after the first convolutional network in the multiple convolutional networks connected in sequence;

[0179] The processing unit 1202 is used to calculate the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer according to the attention mechanism integrated in each network layer to obtain the second feature map output by the corresponding network layer, specifically for:

[0180] The first convolutional network in the target network layer is used to perform feature extraction processing on the first feature map of the target network layer to obtain the third feature map of the target network layer; wherein, when the target network layer is the first convolutional layer of the hierarchical distribution in the feature extraction subnetwork, the first feature map of the target network layer is the speech spectrum feature; when the target network layer is a convolutional layer other than the first convolutional layer in the speech segmentation model, the first feature map of the target network layer is obtained by pooling the feature map output by the upper-level network layer adjacent to the target network layer;

[0181] According to the attention mechanism integrated in the target network layer, the correlation between the voiceprint representation vector and the third feature map of the target network layer is calculated to obtain the fourth feature map of the target network layer; the feature dimension of the third feature map is the same as the feature dimension of the fourth feature map;

[0182] The fourth feature map is subjected to feature extraction processing by using other convolutional networks except the first convolutional network in the target network layer to obtain a second feature map output by the target network layer.

[0183] In one implementation, any network layer in the speech segmentation model that is integrated with the attention mechanism is represented as a target network layer; the target network layer is an upsampling layer in the speech segmentation model; the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the target network layer is: the position after the tail convolutional network in the multiple convolutional networks connected sequentially;

[0184] The processing unit 1202 is used to calculate the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer according to the attention mechanism integrated in each network layer to obtain the second feature map output by the corresponding network layer, specifically for:

[0185] Using multiple convolutional networks connected sequentially in the target network layer, feature extraction processing is performed on the target feature map to obtain the first feature map of the target network layer; the target feature map is obtained by feature concatenating the feature map output by the convolutional layer corresponding to the target network layer and the feature map output by the upper-level network layer of the target network layer;

[0186] The attention mechanism integrated in the target network layer is used to calculate the correlation between the voiceprint representation vector and the first feature map of the target network layer to obtain the second feature map output by the target network layer; the feature dimension of the second feature map is the same as that of the first feature map.

[0187] In one implementation, the processing unit 1202 is further configured to:

[0188] Performing dimension transformation on the voiceprint representation vector to obtain a dimension-transformed voiceprint representation vector;

[0189] Among them, the feature dimension of the voiceprint representation vector after dimension transformation is the same as the feature dimension of the feature map of the attention mechanism to be input into the corresponding network layer for fusion.

[0190] In one implementation, if the number of network layers integrated with the attention mechanism in the speech segmentation model is greater than the number threshold, the processing unit 1202 is further configured to:

[0191] Performing model distillation on the speech segmentation model to obtain a speech segmentation model after model distillation;

[0192] Among them, the correlation calculation is realized by the speech segmentation model after model distillation.

[0193] In one implementation, the voiceprint characteristics of the specified object are represented by a voiceprint representation vector; the processing unit 1202 is used to extract the voiceprint characteristics of the specified object from the reference speech data, specifically to:

[0194] Segmenting the reference voice data to obtain a plurality of voice data segments corresponding to the reference voice data;

[0195] Convert the reference speech data from the time domain to the frequency domain to obtain reference speech spectrum features corresponding to the reference speech data, and segment the reference speech spectrum features to obtain reference speech spectrum feature segments corresponding to each speech data segment;

[0196] Based on each speech data segment and the corresponding reference speech spectrum feature segment, short-term correlation analysis is performed on each speech data segment to obtain a voiceprint semantic feature vector corresponding to each speech data segment; the voiceprint semantic feature vector is used to represent the semantic characteristics of the speech data segment;

[0197] Perform long-term correlation analysis on the sequence of voiceprint semantic feature vectors to obtain the voiceprint representation vector of the specified object; the sequence of voiceprint semantic feature vectors includes the voiceprint semantic feature vectors corresponding to each speech data segment.

[0198] In one implementation, any one of the multiple speech data segments is represented as the target speech data segment; the processing unit 1202, when performing short-term correlation analysis on each speech data segment based on each speech data segment and the corresponding reference speech spectrum feature segment respectively to obtain the voiceprint semantic feature vector corresponding to each speech data segment, is specifically used for:

[0199] Perform feature extraction processing on the target speech data segment to obtain a time-domain feature map;

[0200] Perform feature extraction processing on the reference speech spectrum feature segment corresponding to the target speech data segment to obtain a frequency-domain feature map;

[0201] Fuse the time-domain feature map and the frequency-domain feature map to generate the voiceprint semantic feature vector corresponding to the target speech data segment.

[0202] In one implementation, the number of times of feature extraction processing is k times, where k is an integer greater than 1; any one time of feature extraction processing is represented as the i-th feature extraction processing; the processing unit 1202, when fusing the time-domain feature map and the frequency-domain feature map to generate the voiceprint semantic feature vector corresponding to the target speech data segment, is specifically used for:

[0203] When i = 1, fuse the intermediate time-domain feature map and the intermediate frequency-domain feature map obtained by the first feature extraction processing to generate the first intermediate feature vector after the first feature extraction processing;

[0204] When 1 < i ≤ k, fuse the intermediate time-domain feature map and the intermediate frequency-domain feature map obtained by the i-th feature extraction processing, and the (i - 1)-th intermediate feature vector obtained by the (i - 1)-th feature extraction processing to generate the i-th intermediate feature vector after the i-th feature extraction processing;

[0205] Based on the k-th intermediate feature vector after the k-th feature extraction processing when i = k, generate the voiceprint semantic feature vector corresponding to the target speech data segment.

[0206] According to an embodiment of the present application, Fig.12The various units in the speech processing device shown can be individually or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the speech processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, it can be executed by running on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 3 and Figure 8 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Fig.12 The speech processing device shown in the embodiment of the present application is implemented by the speech processing method. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0207] In the embodiment of the present application, the aliased voice data to be segmented is obtained, and the aliased voice data contains voice signals of at least two objects; if there is a need to segment the voice signal generated by a specified object among the at least two objects, then a section of reference voice data of the specified object (such as a few seconds of voice generated by the specified object) can be obtained; the specified object can be any one of the at least two objects. Then, the voiceprint characteristics of the specified object are extracted from the reference voice data, and the voiceprint characteristics are unique and can characterize the identity of the specified object. Further, a voice signal matching the voiceprint characteristics of the specified object can be segmented from the aliased voice data based on the attention mechanism, so that a separate voice file of the specified object can be generated based on the segmented voice signal. It can be seen from this that, on the one hand, the embodiment of the present application only needs to obtain the reference voice data of the specified object to achieve the segmentation of the voice signal of the specified object from the aliased voice data; if you want to obtain the voice data of other objects, you only need to change the voiceprint characteristics of the object to be segmented, and there is no need to train a dedicated network for each object, which can make the scheme convenient and transferable, making the scheme highly versatile. On the other hand, the embodiment of the present application realizes the segmentation of aliased speech data based on the attention mechanism, which can more clearly and accurately calculate and extract the speech signal of the specified object in the mixed speech data, ensure the purity of the extracted speech signal, and achieve a more accurate and pure speech separation effect.

[0208] Fig.13 FIG. 1 shows a schematic diagram of a computer device provided by an exemplary embodiment of the present application. Fig.13 , the computer device includes a processor 1301, a communication interface 1302 and a computer-readable storage medium 1303. The processor 1301, the communication interface 1302 and the computer-readable storage medium 1303 may be connected via a bus or other means. The communication interface 1302 is used to receive and send data. The computer-readable storage medium 1303 may be stored in the memory of the computer device, the computer-readable storage medium 1303 is used to store computer programs, and the processor 1301 is used to execute the computer programs stored in the computer-readable storage medium 1303. The processor 1301 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for implementing one or more computer programs, and is specifically suitable for loading and executing one or more computer programs to implement corresponding method flows or corresponding functions.

[0209] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the processing system of the computer device. In addition, one or more computer programs suitable for being loaded and executed by the processor 1301 are also stored in the storage space. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory, such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.

[0210] In one embodiment, the computer device may be the terminal or server mentioned in the aforementioned embodiment; one or more computer programs are stored in the computer-readable storage medium; the processor 1301 loads and executes the one or more computer programs stored in the computer-readable storage medium to implement the corresponding steps in the above-mentioned speech processing method embodiment; in a specific implementation, the one or more computer programs in the computer-readable storage medium are loaded and executed by the processor 1301 to execute the steps of each embodiment of the present application; wherein, the steps of each embodiment of the present application can refer to the relevant description of the aforementioned embodiments, and will not be repeated here.

[0211] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the computer device in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems provided by the speech processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0212] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned speech processing method is implemented.

[0213] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0214] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes a computer program (one or more). When a computer program is loaded and executed on a computer device, the computer program executes the above-mentioned process or function of the embodiment of the present application. The computer device can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer program can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer program can be transmitted from a website site, a computer device, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer device, server or data center. The computer-readable storage medium can be any available medium that a computer device can access or a data storage device such as a server or a data center that contains one or more available media integrations. Available media can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (Solid State Disk, SSD)) and the like.

[0215] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A speech processing method, characterized in that: include: Acquire aliased speech data to be segmented, wherein the aliased speech data contains speech signals of at least two objects; Acquire reference speech data of a designated object among the at least two objects; the designated object refers to any one of the at least two objects; The reference voice data includes a voice signal of the designated object; Extracting the voiceprint characteristics of the designated object from the reference voice data, and segmenting a voice signal matching the voiceprint characteristics of the designated object from the aliased voice data based on an attention mechanism; A voice file of the designated object is generated based on the segmented voice signal.

2. The method according to claim 1, characterized in that The voiceprint characteristic of the designated object is represented by a voiceprint representation vector; and the speech signal matching the voiceprint characteristic of the designated object is segmented from the mixed speech data based on the attention mechanism, including: Convert the mixed speech data from the time domain to the frequency domain to obtain speech spectrum features corresponding to the mixed speech data; Based on the attention mechanism, the correlation between the voiceprint representation vector and the speech spectrum feature is calculated to obtain a speech spectrum feature segment matching the voiceprint characteristic; The speech spectrum feature segments are converted from the frequency domain to the time domain to obtain a speech signal matching the voiceprint characteristics.

3. The method according to claim 2, characterized in that The correlation calculation is implemented by a speech segmentation model; the speech segmentation model includes a feature extraction subnetwork and an upsampling subnetwork, and the feature extraction subnetwork and the upsampling subnetwork are connected via a convolutional connection layer; The feature extraction subnetwork and the upsampling subnetwork are symmetrical; the feature extraction subnetwork includes m convolutional layers distributed in a hierarchy, and the upsampling subnetwork includes an upsampling layer corresponding to each convolutional layer, where m is a positive integer; the convolutional layer, the convolutional connection layer, and the upsampling layer include a plurality of convolutional networks connected sequentially; Among them, the attention mechanism is integrated into all or part of the network layers in the speech segmentation model, and the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the network layer is not fixed; the network layer includes the convolutional layer, the upsampling layer and the convolutional connection layer.

4. The method according to claim 3, characterized in that Each network layer in the speech segmentation model is integrated with an attention mechanism; the correlation between the voiceprint representation vector and the speech spectrum feature is calculated based on the attention mechanism to obtain a speech spectrum feature segment matching the voiceprint feature, including: Inputting the voiceprint representation vector into each network layer in the speech segmentation network; According to the attention mechanism integrated in each network layer, the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer is calculated to obtain a second feature map output by the corresponding network layer; the voiceprint characteristics represented by the second feature map output by the corresponding network layer match the voiceprint characteristics represented by the voiceprint representation vector; The second feature map output by the 2m+1th network layer in the speech segmentation model is used as a speech spectrum feature segment that matches the voiceprint characteristics represented by the voiceprint representation vector.

5. The method according to claim 4, characterized in that Any network layer in the speech segmentation model that is integrated with the attention mechanism is represented as a target network layer; the target network layer is a convolutional layer or a convolutional connection layer in the speech segmentation model; the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the target network layer is: the first convolutional network in the multiple convolutional networks connected in sequence, and the position between the convolutional networks adjacent to the first convolutional network and located after the first convolutional network in the multiple convolutional networks connected in sequence; The step of calculating the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer according to the attention mechanism integrated in each network layer to obtain the second feature map output by the corresponding network layer includes: The first convolutional network in the target network layer is used to perform feature extraction processing on the first feature map of the target network layer to obtain a third feature map of the target network layer; wherein, when the target network layer is the first convolutional layer of the hierarchical distribution in the feature extraction subnetwork, the first feature map of the target network layer is the speech spectrum feature; when the target network layer is a convolutional layer other than the first convolutional layer in the speech segmentation model, the first feature map of the target network layer is obtained by pooling the feature map output by the upper-level network layer adjacent to the target network layer; According to the attention mechanism integrated in the target network layer, the correlation between the voiceprint representation vector and the third feature map of the target network layer is calculated to obtain a fourth feature map of the target network layer; the feature dimension of the third feature map is the same as the feature dimension of the fourth feature map; The fourth feature map is subjected to feature extraction processing by using other convolutional networks in the target network layer except the first convolutional network to obtain a second feature map output by the target network layer.

6. The method according to claim 4, characterized in that Any network layer in the speech segmentation model that is integrated with the attention mechanism is represented as a target network layer; the target network layer is an upsampling layer in the speech segmentation model; the fusion position of the attention mechanism in the multiple convolutional networks corresponding to the target network layer is: the position after the tail convolutional network in the multiple convolutional networks connected in sequence; The step of calculating the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer according to the attention mechanism integrated in each network layer to obtain the second feature map output by the corresponding network layer includes: Using multiple convolutional networks connected sequentially in the target network layer, feature extraction processing is performed on the target feature map to obtain a first feature map of the target network layer; the target feature map is obtained by feature splicing the feature map output by the convolutional layer corresponding to the target network layer and the feature map output by the upper-level network layer of the target network layer; The attention mechanism integrated in the target network layer is used to calculate the correlation between the voiceprint representation vector and the first feature map of the target network layer to obtain a second feature map output by the target network layer; the feature dimension of the second feature map is the same as the feature dimension of the first feature map.

7. The method according to claim 4, characterized in that Before calculating the correlation between the voiceprint representation vector and the first feature map of the corresponding network layer according to the attention mechanism integrated in each network layer to obtain the second feature map output by the corresponding network layer, the method further includes: Performing dimension transformation on the voiceprint representation vector to obtain the voiceprint representation vector after dimension transformation; Among them, the feature dimension of the voiceprint representation vector after dimension transformation is the same as the feature dimension of the feature map of the attention mechanism to be input into the corresponding network layer for fusion.

8. The method according to claim 3, characterized in that If the number of network layers integrated with the attention mechanism in the speech segmentation model is greater than the number threshold, the method further includes: Performing model distillation on the speech segmentation model to obtain a speech segmentation model after model distillation; The correlation calculation is implemented by the speech segmentation model after model distillation.

9. The method according to claim 1, characterized in that The voiceprint characteristic of the designated object is represented by a voiceprint representation vector; and extracting the voiceprint characteristic of the designated object from the reference voice data includes: Segmenting the reference voice data to obtain a plurality of voice data segments corresponding to the reference voice data; Converting the reference speech data from the time domain to the frequency domain to obtain reference speech spectrum features corresponding to the reference speech data, and segmenting the reference speech spectrum features to obtain reference speech spectrum feature segments corresponding to each speech data segment; Based on each of the speech data segments and the corresponding reference speech spectrum feature segments, short-term correlation analysis is performed on each of the speech data segments to obtain a voiceprint semantic feature vector corresponding to each of the speech data segments; the voiceprint semantic feature vector is used to characterize the semantic characteristics of the speech data segment; A long-term correlation analysis is performed on the voiceprint semantic feature vector sequence to obtain a voiceprint representation vector of the specified object; the voiceprint semantic feature vector sequence includes the voiceprint semantic feature vector corresponding to each speech data segment.

10. The method according to claim 9, characterized in that Any one of the multiple voice data segments is represented as a target voice data segment; and the short-term correlation analysis is performed on each voice data segment based on each voice data segment and a corresponding reference voice spectrum feature segment to obtain a voiceprint semantic feature vector corresponding to each voice data segment, including: Performing feature extraction processing on the target speech data segments to obtain a time domain feature map; Performing feature extraction processing on the reference speech spectrum feature segments corresponding to the target speech data segments to obtain a frequency domain feature graph; The time domain feature map and the frequency domain feature map are fused to generate a voiceprint semantic feature vector corresponding to the target speech data segment.

11. The method according to claim 10, characterized in that The number of feature extraction processes is k, where k is an integer greater than 1; any feature extraction process is represented as the i-th feature extraction process; The fusing the time domain feature map and the frequency domain feature map to generate the voiceprint semantic feature vector corresponding to the target speech data segment includes: When i=1, the intermediate time domain feature map and the intermediate frequency domain feature map obtained by the first feature extraction process are fused to generate a first intermediate feature vector after the first feature extraction process; When 1 < i ≤ k, the intermediate time-domain feature map and the intermediate frequency-domain feature map obtained by the i-th feature extraction process, and the (i - 1)-th intermediate feature vector obtained by the (i - 1)-th feature extraction process are fused to generate the i-th intermediate feature vector after the i-th feature extraction process; Based on the k-th intermediate feature vector after the k-th feature extraction process when i = k, the voiceprint semantic feature vector corresponding to the segmented target voice data is generated.

12. A speech processing device, characterized in that: Including: An acquisition unit, configured to acquire the aliased voice data to be segmented, where the aliased voice data contains voice signals of at least two objects; The acquisition unit is further configured to acquire the reference voice data of a specified object among the at least two objects; the specified object refers to any one of the at least two objects; the reference voice data contains the voice signal of the specified object; A processing unit, configured to extract the voiceprint characteristics of the specified object from the reference voice data, and based on the attention mechanism, segment the voice signal in the aliased voice data that matches the voiceprint characteristics of the specified object; The processing unit is further configured to generate the voice file of the specified object based on the segmented voice signal.

13. A computer device, characterized in that A processor, adapted to execute a computer program; A computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by the processor, the voice processing method according to any one of claims 1 - 11 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the voice processing method according to any one of claims 1 - 11.

15. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by the processor, the voice processing method according to any one of claims 1 - 11 is implemented.

Images