Method for constructing silence detection model, electronic device, storage medium and program product

Abstract

The present disclosure provides a method for constructing a silence detection model, which may include: performing silence detection on training samples by an initial detection model to obtain silence detection results; analyzing the silence detection results by using at least two loss functions to obtain simulated insertion loss values ​​and silence segment loss values; and adjusting the weights of the initial detection model according to the simulated insertion loss values ​​and silence segment loss values ​​to obtain a silence detection model. The construction method of the present disclosure uses a small amount of training samples to complete the end-to-end training process of the silence detection model, and at the same time realizes the construction of a silence detection model with high silence detection capabilities. The present disclosure also provides an electronic device, a storage medium, and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent voice technology, and in particular to a method, device, electronic device, storage medium, program product for constructing a silence detection model, and a silence detection method and device. Background Art

[0002] The end-to-end speech recognition method based on CTC (Connectionist Temporal Classification) is a common method for speech recognition. This method realizes the alignment between the input audio and the output text by introducing the spacer blank, and obtains the posterior probability of the previous frame based on the context of the whole sentence, and then calculates the loss to realize the end-to-end training process of converting the speech sequence into the text sequence. However, in a real continuous conversation scenario, there will be a large number of silent segments in the input audio, such as multiple short pauses, long silence at the beginning and end of the sentence, etc. A large number of silent segments will inevitably affect the accuracy of the speech recognition results, so a silence detection model is usually used to detect the silent segments of the input audio.

[0003] However, some silence detection models in related technologies mainly use non-end-to-end training methods with frame alignment, which requires first training an alignment model. The design and training process of the alignment model is relatively difficult, and the overall training cost is high. In addition, the non-end-to-end training method is less effective than the end-to-end training method. The end-to-end training method based on CTC cannot distinguish between silence and non-silence, and the training set does not provide annotations for the silence segments of the input audio. If the annotations for silence segments are added, the annotation cost will inevitably increase. In addition, the existing training data is mostly single sentences, which cannot fully match the real continuous dialogue scene, and the silence detection time period will be delayed. Summary of the invention

[0004] In order to solve at least one of the above technical problems, the present disclosure provides a method, device, electronic device and storage medium for constructing a silence detection model, so as to complete the end-to-end training process of the silence detection model with a small number of training samples, and at the same time realize the construction of a silence detection model with high silence detection capability. The present disclosure also provides a silence detection method and device, so as to realize more accurate silence detection of target speech by using the silence detection model.

[0005] One aspect of the present disclosure provides a method for constructing a silence detection model, which may include: performing silence detection on a training sample using an initial detection model to obtain a silence detection result; analyzing the silence detection result using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value; and adjusting the weight of the initial detection model according to the simulated insertion loss value and the silence segment loss value to obtain a silence detection model.

[0006] In some embodiments, performing silence detection on training samples by an initial detection model to obtain silence detection results may include: performing silence endpoint recognition on the training samples by the initial detection model, and respectively determining the speech positions of multiple silence starting points and multiple silence ending points of the training samples; determining the time length between sequentially adjacent silence starting points and silence ending points, and taking the speech range between the silence starting point and the silence ending point whose time length meets the silence threshold as a silence segment; calculating the splicing probability that the silence segment belongs to silence splicing data; and integrating the speech position of the silence starting point and the speech position of the silence ending point of each silence segment, the speech duration of each silence segment, and the splicing probability of each silence segment to generate a silence detection result for the training samples.

[0007] In some embodiments, at least two loss functions are used to analyze the silence detection results to obtain simulated insertion loss values ​​and silence segment loss values, which may include: using a cross entropy loss function to compare the splicing probability of the silence segment in the silence detection result with the actual splicing data to obtain a simulated insertion loss value that characterizes the degree of fit between the splicing probability and the actual splicing data; and using a CTC loss function to compare the speech position of the silence start point and the speech position of the silence end point of the silence segment in the silence detection result as well as the speech duration of the silence segment with the actual silence segment data of the silence segment, to obtain a silence segment loss value that characterizes the degree of fit between the speech position of the silence start point and the speech position of the silence end point and the speech duration of the silence segment and the actual silence segment data.

[0008] In some embodiments, after analyzing the silence detection results using at least two loss functions to obtain simulated insertion loss values ​​and silent segment loss values, it may also include: integrating the simulated insertion loss values ​​and silent segment loss values ​​to obtain a total loss value used to characterize the degree of deviation of the silence detection results from the actual values.

[0009] In some embodiments, before the initial detection model performs silence detection on the training sample and obtains the silence detection result, the method may further include: obtaining silence splicing data in the training sample. Obtaining silence splicing data in the training sample may include: segmenting the original speech segment and the original silence segment in the original speech data to obtain a speech set and a silence set; and segmenting the silence set into a plurality of simulated silence segments with a preset time length, and inserting each simulated silence segment into the speech set, respectively, to obtain the training sample spliced ​​by the speech set and a plurality of simulated silence segments.

[0010] In some embodiments, the silence set is cut into multiple simulated silence segments with preset time lengths, and each simulated silence segment is inserted into the speech set respectively to obtain the training sample composed of the speech set and the multiple simulated silence segments, which may include: dividing at least two simulated silence segments with preset time lengths from the silence set, and inserting at least two simulated silence segments with preset time lengths into the tail of the head speech set of the speech set respectively.

[0011] In some embodiments, the silence set is cut into multiple simulated silence segments with preset time lengths, and each simulated silence segment is inserted into the speech set respectively, to obtain the training sample formed by the speech set and the multiple simulated silence segments. It may also include: dividing the speech set into multiple speech segments with preset time lengths, and inserting the multiple simulated silence segments with preset time lengths divided from the silence set between two adjacent speech segments in sequence, to obtain the training sample formed by the speech set and the multiple simulated silence segments.

[0012] Another aspect of the present disclosure provides such a device for constructing a silence detection model, which may include an initial result generation module, a loss value acquisition module and a silence detection model generation module. The initial result generation module is used to perform silence detection on training samples using an initial detection model to obtain silence detection results. The loss value acquisition module is used to analyze the silence detection results using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value. The silence detection model generation module is used to adjust the weight of the initial detection model according to the simulated insertion loss value and the silence segment loss value to obtain a silence detection model.

[0013] Yet another aspect of the present disclosure provides a silence detection method, which may include: decomposing a target speech into multiple speech packets and inputting the packets into a silence detection model, wherein the silence detection model is constructed by the method for constructing a silence detection model of any of the above-mentioned embodiments; judging by the silence detection model whether each speech packet is a silence segment; and generating a decoding instruction in response to the judgment result that the speech packet is a silence segment, wherein the decoder performs content recognition on the speech packet.

[0014] In some embodiments, a method for constructing a silence detection model may include: performing silence detection on a training sample using an initial detection model to obtain a silence detection result; analyzing the silence detection result using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value; and adjusting the weight of the initial detection model according to the simulated insertion loss value and the silence segment loss value to obtain a silence detection model.

[0015] Another aspect of the present disclosure provides such a silence detection device, which may include: an input module, a detection module and a decoding module. The input module is used to decompose the target speech into multiple voice packets and input them into a silence detection model, wherein the silence detection model is constructed by the construction method of the silence detection model of any of the above-mentioned embodiments. The detection module is used to determine whether each voice packet is a silence segment by the silence detection model. The decoding module is used to generate a decoding instruction in response to the judgment result that the voice packet is a silence segment, and the decoder performs content recognition on the voice packet.

[0016] In some embodiments, a model building module is also included, and its execution steps are: using an initial detection model to perform silence detection on training samples to obtain silence detection results; using at least two loss functions to analyze the silence detection results to obtain simulated insertion loss values ​​and silence segment loss values; and according to the simulated insertion loss values ​​and silence segment loss values, adjusting the weights of the initial detection model to obtain a silence detection model.

[0017] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0018] A memory storing execution instructions;

[0019] A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the method for constructing a silence detection model of any embodiment of the present disclosure and / or executes the silence detection method of any embodiment of the present disclosure.

[0020] According to another aspect of the present disclosure, a storage medium is provided, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the method for constructing a silence detection model of any embodiment of the present disclosure and / or implement the silence detection method of any embodiment of the present disclosure.

[0021] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, it implements the construction method of any embodiment of the present disclosure and / or implements the silence detection method of any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0023] Figure 1 It is a flow chart of a method for constructing a silence detection model according to an embodiment of the present disclosure.

[0024] Figure 2 It is a schematic diagram of the specific process of the method for constructing a silence detection model.

[0025] Figure 3 It is a schematic diagram of the splicing of silent splicing data.

[0026] Figure 4 The figure is a structural block diagram of a device for constructing a silence detection model according to an embodiment of the present invention.

[0027] Figure 5 It is a flow chart of a silence detection method according to an embodiment of the present disclosure.

[0028] Figure 6 is a schematic diagram of an electronic device according to an embodiment of the present disclosure.

[0029] Figure 7 is a schematic diagram of a storage medium according to an embodiment of the present disclosure.

[0030] Description of Reference Numerals

[0031] 200 Build Device

[0032] 210 Initial result generation module

[0033] 220 Loss value acquisition module

[0034] 230 Silence detection model generation module

[0035] 301 Bus

[0036] 302 CPU

[0037] 303 Read-only memory

[0038] 304 Random Access Memory

[0039] 305 Communication port

[0040] 306 Output Components

[0041] 307 Hard Drive

[0042] 308 User Interface

[0043] 400 storage media. DETAILED DESCRIPTION

[0044] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0045] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0046] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0047] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0048] When a component is referred to as being "on" or "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0049] The terms used herein are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0050] Figure 1 is a flow chart of a method for constructing a silence detection model according to an embodiment of the present disclosure; Figure 2 It is a schematic diagram of the specific process of the method for constructing a silence detection model.

[0051] like Figure 1 As shown, one aspect of the present disclosure provides a method for constructing a silence detection model, which may include: step S110, performing silence detection on training samples by an initial detection model to obtain silence detection results; step S120, analyzing the silence detection results using at least two loss functions to obtain simulated insertion loss values ​​and silence segment loss values; and step S130, adjusting the weights of the initial detection model according to the simulated insertion loss values ​​and silence segment loss values ​​to obtain a silence detection model. It should be noted that due to Figure 2 is Figure 1 On the basis of Figure 1 A more detailed description of the method shown in the figure also provides a more intuitive construction process of the silence detection model, so the following text will combine Figure 1 and Figure 2 This method is described in more detail.

[0052] In some embodiments, firstly, a plurality of original voice data are collected, and the original voice data may be the content of the conversation in the actual conversation scene, or the content recorded by the user according to the specified text. It should be noted that the voice duration and content of each original voice data, as well as the number of original voice data collected can be set according to the requirements, and there is no restriction here. Of course, the voice duration of the original voice needs to meet the provisions of the preset original voice duration threshold. If the voice duration of the original voice is too short, it is impossible or difficult to insert the simulated silence segment; if the voice duration of the original voice is too long, it will lead to the increase of the insertion cost of the simulated silence segment and the increase of the training time cost of the initial detection model. Based on the above, the original voice data that does not meet the requirements can be preprocessed, that is, the original voice data with too short voice duration is filled, or the original voice data with too long voice duration is cut to ensure that the voice duration of the original voice data meets the requirements of the training sample. Further, when preprocessing the original voice, the clarity of the original voice data can also be screened, and the original voice data that cannot be recognized by voice content in multiple original voice data can be cleaned out to eliminate noise interference and ensure the validity of the original voice data.

[0053] Furthermore, the original speech data after preprocessing is segmented. At this stage, the segmentation purpose is to separate the effective original speech segments and the original silence segments in the original speech data, that is, to extract the original silence segments in the original speech data, and the original silence segments include the original head silence part, the original tail silence part, and the pause silence part in the original middle position in the original speech data. Furthermore, the original head silence part, the original tail silence part, and the pause silence part in the original middle position in the original speech data are integrated into a silence set, which can reflect the real silence situation in the original speech data. In the real silence situation, it may contain certain noise or non-voicing subject speech content. Therefore, the silence set extracted by using the real original speech data is more in line with the actual situation. The silence detection model trained based on this will be more in line with the real speech scene, which is conducive to improving the silence recognition accuracy of the silence detection model. Furthermore, the original speech segments in the original speech data separated from the original silence segment are integrated, that is, the speech segments are integrated according to the order of the speech content in the original speech data to obtain the speech set. It should be noted that the word order of the speech content in the speech set is consistent with the word order of the original speech segment, and of course the content and meaning expressed are also consistent.

[0054] In some embodiments, after obtaining the voice set and the silence set, the silence set is divided into a plurality of simulated silence segments according to a preset time length, and then inserted into various positions of the voice set to obtain silence splicing data formed by splicing the voice set and the plurality of simulated silence segments. Specifically, according to the different insertion positions of the simulated silence segments in the voice set, the splicing methods are also divided into two types, including: a splicing method of inserting the simulated silence segments at the head and tail of the voice set, and a splicing method of inserting the simulated silence segments at the middle position of the voice set.

[0055] In some embodiments, when inserting a simulated silent segment at the head and tail of a speech set, since the silence duration of the pronunciation subject before starting to speak or after finishing speaking in the real scene is relatively long, in order to fit the real scene, a simulated silent segment with a longer time length can be cut out in the silent set. For example, the time length of the simulated silent segment in this scene can be in the range of 0.3 seconds to 1 second, and of course it can be appropriately extended according to actual needs. Further, the simulated silent segment with a time length between 0.3 seconds and 1 second cut out is respectively inserted into the head and tail of the speech set, and the time length and voice position of the inserted simulated silent segment are recorded. It should be noted that the time length of at least two simulated silent segments inserted into the head and tail of the speech set is randomly cut within the set time length range, and the time length of the simulated silent segment inserted into the head of the speech set and the simulated silent segment inserted into the tail of the speech set does not need to be the same. Furthermore, the speech set with simulated silence segments at the head and tail can be directly used as silence splicing data, and on this basis, a simulated silence segment with a shorter time length can be inserted into the middle position of the speech set to imitate the pause between sentences in real scenes. Of course, if the speech set of the pronunciation subject has more content, a simulated silence segment with a longer time length can also be inserted in the middle position to represent the intermission time of real scenes, such as the interval between meetings and speeches. In addition, it should be noted that the cutting of silence segments of the original speech data and the insertion of simulated silence segments will not affect other text annotations of the speech set in the original speech data.

[0056] In some embodiments, when inserting a simulated silent segment in the middle of a speech set, since the speaker in the real scene will pause when speaking, and the pause time is usually short, in order to fit the real scene, a simulated silent segment with a shorter time length can be cut out in the silent set. For example, the time length of the simulated silent segment in this scene can be controlled to be about 0.2 seconds, and of course it can be appropriately extended or shortened according to actual needs, but if the time length of the inserted simulated silent segment is too long, it does not meet the pause scene during speaking; if the time length of the inserted simulated silent segment is too short, it can be ignored in the process of speech recognition of the input speech. Further, the speech set is randomly cut into multiple speech segments, in order to fit the speaking habits of the speaker in the real scene, that is, at least a pause will be made after finishing a sentence, the time length of each speech segment should be greater than 0.5 seconds, and of course other lengths can be set as required. Further, a simulated silent segment with a time length of about 0.2 seconds is inserted between two adjacent speech segments, and the time length and speech position of the inserted simulated silent segment are recorded. It should be noted that the time lengths of the multiple simulated silence segments inserted in the middle of the voice set are randomly cut within the set time length range, and the time lengths of the various simulated silence segments inserted in the middle of the voice set do not need to be exactly the same. Based on the above operations, the silence splicing data formed by splicing the voice set and the multiple simulated silence segments can be obtained. It should be noted that the word order of each voice segment in the silence splicing data is the same as the word order of the voice set in the original voice data, and the expression meaning is the same; in other words, only the simulated silence segments are inserted into the voice set, and the word order is not adjusted. In addition, the cutting of silence segments in the original voice data and the insertion of simulated silence segments will not affect other text annotations of the voice set in the original voice data.

[0057] In some embodiments, when inserting a simulated silence segment into the middle position of a speech set, the insertion object may be the original speech data, or may be silence concatenation data after inserting the simulated silence segment into the head and tail, so that the final silence concatenation data meets the position requirements for various simulated silence segments. Due to the different insertion positions of the silence segments, multiple different silence concatenation data may be generated, thereby increasing the number of training samples.

[0058] Figure 3 This is a schematic diagram of the silent splicing data. Figure 3, more specifically, the splicing situation of the silent splicing data having simulated silent segments at the head, tail and middle positions is explained. Specifically, the original voice data is divided into 1 to n voice segments, and the original silent data is integrated into a silent set. Further, a simulated silent segment with a longer time length (i.e., the long silence shown in the figure) is inserted at the head and tail of the original voice data, and a simulated silent segment with a shorter time length (i.e., the short silence shown in the figure) is inserted between two adjacent voice segments in sequence. For example, long silence and short silence are respectively provided before and after the first voice segment, short silence is provided before and after the i-th voice segment, and short silence and long silence are respectively provided before and after the n-th voice segment. It should be noted that the time length of the i-th voice segment should be greater than 0.5 seconds, the time length of the short silence should be about 0.2 seconds, and the time length of the long silence should be between 0.3 seconds and 1 second. Further, after the long silence and the short silence are inserted, the silent splicing data with completed splicing is obtained.

[0059] In some embodiments, since the time lengths of simulated silent segments at different positions in the silent splicing data all meet the pre-set time length requirements, it can only meet most real scenes, but cannot characterize the silence of special scenes. Based on this, in order to be able to better fit the real scene, when training the initial detection model, the unprocessed original voice data and the silent splicing data are also mixed in a certain proportion as training samples. Specifically, any floating-point number between 0 and 1 that meets the requirements is used as the mixing ratio of the original voice data and the silent splicing data, which is not limited here. The training samples mixed with the original voice data and the silent splicing data can increase the number of training samples and improve the degree of fit with the real scene.

[0060] In some embodiments, after obtaining the training samples, the training samples are used as input data for the initial detection model. Furthermore, firstly, features of multiple dimensions of the training samples are extracted for training, such as 80-dimensional features. Furthermore, based on the analysis and processing of the extracted features, the silence endpoints of the training samples can be identified, and the speech positions of multiple silence starting points and multiple silence ending points of the training samples can be determined respectively; further, the time length between the silence starting points and silence ending points that are adjacent in sequence is calculated. Furthermore, the time length between the silence starting point and the silence ending point is compared with the silence threshold. If the time length between the silence starting point and the silence ending point is greater than the silence threshold, the speech range between the silence starting point and the silence ending point is determined to be a silence segment. Of course, when the pronunciation subject produces speech, there will inevitably be short silent gaps between each word, and it is impossible to achieve a completely seamless connection. Therefore, if the time length between the silence start point and the silence end point is less than or equal to the silence threshold, there is no need to identify the speech range between the silence start point and the silence end point as a silence segment, and generate a speech recognition instruction for the speech segment including the speech range between the silence start point and the silence end point to instruct the decoder to recognize this part of the content.

[0061] In some implementations, since the training sample is a mixed sample of silence splicing data and original speech data, after obtaining the training sample, the initial detection model also detects the splicing probability of the training sample to obtain a splicing probability used to characterize the possibility that the training sample belongs to silence splicing data. Specifically, the initial detection model determines whether there are splicing traces in the silence segment, and obtains splicing state information including the splicing position and the time length of the simulated silence segment, and then analyzes the splicing probability of the possibility that the training sample belongs to silence splicing data based on the above splicing information. Furthermore, since the silence splicing data has a head simulated silence segment and a tail simulated silence segment with a longer time length, when the splicing probability of a certain training sample is detected to be higher than the preset splicing threshold, the corresponding weight of the silence detection model can be adjusted to enable the silence detection model to adapt to the silence splicing data with a head simulated silence segment and a tail simulated silence segment with a longer time length, so as to avoid the problem of timestamp delay caused by the inability to determine the time length of the silence segment, the speech position of the starting point of the silence, and the speech position of the end point of the silence when calculating the CTC loss function.

[0062] In some embodiments, the speech positions of multiple silence start points and multiple silence end points of the training sample, the time length between the silence start point and the silence end point, and the splicing probability are integrated as the silence detection result of the initial detection model for the training sample.

[0063] In some embodiments, the CE-Loss (CrossEntropy Loss) function is used to measure the difference between two probability distributions. The larger the final information entropy value, the greater the deviation between the splicing probability output by the silence detection model and the actual splicing situation. Conversely, the splicing probability output by the silence detection model and the actual splicing situation deviate less. In the present disclosure, the CE-Loss function is used to compare the splicing probability output by the initial detection model to characterize the possibility that the training sample belongs to the silence splicing data with the actual splicing situation, and obtain a simulated insertion loss value (i.e., information entropy) that characterizes the degree of agreement between the splicing probability and the actual splicing situation.

[0064] In some implementations, the CTC-Loss (CTC loss, connection temporal classification criterion loss) function is a loss function that only considers the word order and does not overly care about whether the speech and text are aligned. Since it only considers the label order of the word order and does not require absolute alignment of the input audio and text, the loss value calculation for silence detection is simpler and less costly. Based on this, the present disclosure uses the CTC loss function to compare the speech position of the silence start point and the speech position of the silence end point of the silence segment in the silence detection result and the speech duration of the silence segment with the actual silence segment data of the silence segment (i.e., the speech position of the silence start point of the actual silence segment in the original speech data or the silence splicing data, the speech position of the silence end point and the time length of the silence segment), and obtains a silence segment loss value that characterizes the degree of consistency between the speech position of the silence start point and the speech position of the silence end point of the silence segment and the speech duration of the silence segment and the actual silence segment data.

[0065] In some embodiments, since only one loss value can be input when adjusting the weight of the initial detection model, the simulation insertion loss value and the silent segment loss value are integrated to obtain a total loss value. Furthermore, the total loss value including the weight adjustment amount of the initial detection model is fed back to the initial detection model, and the initial detection model adjusts its own weight to obtain a silent detection model that can accurately detect the information of the silent segment. Of course, if a single weight adjustment fails to obtain a silent detection model that can accurately detect the information of the silent segment, the above steps can be repeated until a silent detection model that can accurately detect the information of the silent segment is output.

[0066] According to a method for constructing a silence detection model disclosed in the present invention, the silence detection result of the initial detection model is simulated and inserted with loss value and silence segment loss value respectively using the cross entropy loss function and the CTC loss function. This solves the problem of timestamp delay caused by a long silence segment when the CTC loss function is used alone for model training, and also solves the problem of a large number of training samples required when the cross entropy loss function is used alone. At the same time, it ensures the fit between the output result of the silence detection model and the real scene, and improves the silence segment recognition capability of the constructed silence detection model.

[0067] Figure 4 It is a structural block diagram of a device for constructing a silence detection model according to another embodiment of the present disclosure.

[0068] like Figure 4 As shown, the present disclosure also provides such a device 200 for constructing a silence detection model, which may include: an initial result generation module 210, a loss value acquisition module 220 and a silence detection model generation module 230. The initial result generation module 210 is used to perform silence detection on the training sample by the initial detection model to obtain a silence detection result. The loss value acquisition module 220 is used to analyze the silence detection result using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value. The silence detection model generation module 230 is used to adjust the weight of the initial detection model according to the simulated insertion loss value and the silence segment loss value to obtain a silence detection model.

[0069] In some embodiments, the execution steps of the initial result generation module 210 may be: performing silence endpoint recognition on the training sample by the initial detection model, and determining the speech positions of multiple silence starting points and multiple silence ending points of the training sample respectively; determining the time length between the silence starting points and silence ending points that are adjacent in sequence, and taking the speech range between the silence starting point and the silence ending point whose time length meets the silence threshold as the silence segment; calculating the splicing probability that the silence segment belongs to the silence splicing data; and integrating the speech position of the silence starting point and the speech position of the silence ending point of each silence segment, the speech duration of each silence segment, and the splicing probability of each silence segment to generate a silence detection result for the training sample.

[0070] In some embodiments, the execution steps of the loss value acquisition module 220 may include: using the cross entropy loss function to compare the splicing probability in the silence detection result with the actual splicing data, and obtaining a simulated insertion loss value that characterizes the degree of fit between the splicing probability and the actual splicing data; and using the CTC loss function to compare the speech position of the silence start point and the speech position of the silence end point of the silence segment in the silence detection result as well as the speech duration of the silence segment with the actual silence segment data of the silence segment, and obtaining a silence segment loss value that characterizes the degree of fit between the speech position of the silence start point and the speech position of the silence end point of the silence segment as well as the speech duration of the silence segment and the actual silence segment data.

[0071] In some embodiments, the silence detection model construction device 200 also includes: a total loss value acquisition module (not shown), which is used to integrate the simulated insertion loss value and the silence segment loss value to obtain a total loss value used to characterize the degree of deviation of the silence detection result from the actual value.

[0072] In some embodiments, the device 200 for constructing a silence detection model further includes: a silence splicing data generating module (not shown) for obtaining silence splicing data in a training sample. The execution steps of the silence splicing data generating module may include: segmenting the original speech segment and the original silence segment in the original speech data to obtain a speech set and a silence set; and cutting the silence set into a plurality of simulated silence segments with a preset time length, and inserting each simulated silence segment into the speech set, respectively, to obtain the training sample spliced ​​by the speech set and a plurality of simulated silence segments.

[0073] In some embodiments, the execution steps of the silence splicing data generation module also include: segmenting at least two simulated silence segments with preset time lengths from the silence set, and inserting at least two simulated silence segments with preset time lengths into the tail of the head speech set of the speech set; and / or segmenting the speech set into multiple speech segments with preset time lengths, and inserting the multiple simulated silence segments with preset time lengths segmented from the silence set between two adjacent speech segments in sequence, to obtain a training sample spliced ​​by the speech set and multiple simulated silence segments.

[0074] In some embodiments, the silence detection model construction device 200 also includes an application module (not shown), and its execution steps are: decomposing the target speech into multiple voice packets and inputting them into the silence detection model; judging by the silence detection model whether each voice packet is a silence segment; and generating a decoding instruction in response to the judgment result that the voice packet is a silence segment, and the decoder performs content recognition on the voice packet.

[0075] According to a device for constructing a silence detection model disclosed in the present invention, the silence detection result of the initial detection model is simulated and inserted with loss value and silence segment loss value respectively calculated by using the cross entropy loss function and the CTC loss function. This solves the problem of timestamp delay caused by a long silence segment when the CTC loss function is used alone for model training, and also solves the problem of a large number of training samples required when the cross entropy loss function is used alone. At the same time, it ensures the fit between the output result of the silence detection model and the real scene, and improves the silence segment recognition capability of the constructed silence detection model.

[0076] Yet another aspect of the present disclosure provides a silence detection method, which may include: decomposing a target speech into multiple speech packets and inputting the packets into a silence detection model, wherein the silence detection model is constructed by the method for constructing a silence detection model of any of the above-mentioned embodiments; judging by the silence detection model whether each speech packet is a silence segment; and generating a decoding instruction in response to the judgment result that the speech packet is a silence segment, wherein the decoder performs content recognition on the speech packet.

[0077] Figure 5 FIG. 1 is a flow chart of a silence detection method according to an embodiment of the present disclosure. Figure 5 As shown, after obtaining a silence detection model that can accurately detect information about silence segments, in a real conversation scenario, the target speech of the pronunciation subject can be divided into multiple voice packages, and the multiple voice packages are input into the silence detection model, and the silence detection model determines whether each voice package is a silence segment; if the voice package is not a silence segment, a decoding instruction is generated to control the decoder to identify the content of the voice package, and the decoder performs content recognition on the voice package. Of course, if the voice package is a silence segment, a non-decoding instruction is generated to prevent the decoder from decoding it, thereby improving the decoding efficiency of the decoder for the target speech. It should be noted that the decoder can be a speech recognition model.

[0078] In some embodiments, a method for constructing a silence detection model may include: performing silence detection on a training sample using an initial detection model to obtain a silence detection result; analyzing the silence detection result using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value; and adjusting the weight of the initial detection model according to the simulated insertion loss value and the silence segment loss value to obtain a silence detection model.

[0079] Another aspect of the present disclosure provides such a silence detection device (not shown), which may include: an input module (not shown), a detection module (not shown) and a decoding module (not shown). The input module is used to decompose the target speech into multiple voice packets and input them into a silence detection model, wherein the silence detection model is constructed by the construction method of the silence detection model of any of the above-mentioned embodiments. The detection module is used to determine whether each voice packet is a silence segment by the silence detection model. The decoding module is used to generate a decoding instruction in response to the judgment result that the voice packet is a silence segment, and the decoder performs content recognition on the voice packet.

[0080] In some embodiments, a model building module (not shown) is also included, and its execution steps are: performing silence detection on training samples by an initial detection model to obtain silence detection results; analyzing the silence detection results using at least two loss functions to obtain simulated insertion loss values ​​and silence segment loss values; and adjusting the weights of the initial detection model according to the simulated insertion loss values ​​and silence segment loss values ​​to obtain a silence detection model.

[0081] Figure 6 is a schematic diagram of an electronic device according to an embodiment of the present disclosure.

[0082] like Figure 6 As shown, according to another aspect of the present disclosure, an electronic device is also provided. The electronic device may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, the method for constructing a silence detection model as described above may be executed.

[0083] The method or device according to the embodiment of the present disclosure can also be used by Figure 6 The electronic device architecture shown in FIG. Figure 6 As shown, the electronic device may include a bus 301, one or more CPUs 302, a read-only memory (ROM) 303, a random access memory (RAM) 304, a communication port 305 connected to a network, an input / output component 306, a hard disk 307, etc. The storage device in the electronic device, such as ROM 303 or hard disk 307, can store various data or files used for processing and communication of the method for constructing a silence detection model provided by the present disclosure, as well as program instructions executed by the CPU. The method for constructing a silence detection model may, for example, include performing silence detection on a training sample by an initial detection model to obtain a silence detection result; analyzing the silence detection result using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value; and adjusting the weight of the initial detection model according to the simulated insertion loss value and the silence segment loss value to obtain a silence detection model. Further, the electronic device may also include a user interface 308. Of course, Figure 6The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 6 One or more components of an electronic device are shown.

[0084] Figure 7 is a schematic diagram of a storage medium according to one embodiment of the present disclosure.

[0085] like Figure 7 As shown, a storage medium 400 according to an embodiment of the present disclosure is shown. Computer-readable instructions are stored on the storage medium 400. When the computer-readable instructions are executed by the processor, the method for constructing a silence detection model according to an embodiment of the present disclosure described with reference to the above figures can be executed. The storage medium 400 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0086] In addition, according to an embodiment of the present disclosure, the process described with reference to the flowchart above can be implemented as a computer software program. For example, the present disclosure provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided by the present disclosure, for example: performing silence detection on training samples by an initial detection model to obtain silence detection results; analyzing the silence detection results using at least two loss functions to obtain simulated insertion loss values ​​and silence segment loss values; and adjusting the weights of the initial detection model according to the simulated insertion loss values ​​and silence segment loss values ​​to obtain a silence detection model. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present disclosure are executed.

[0087] According to the technical solution of the above-mentioned implementation mode, at least one of the following beneficial effects can be obtained.

[0088] The construction method, device, electronic device, storage medium and program product of the silence detection model disclosed in the present invention use the cross entropy loss function and the CTC loss function to respectively calculate the simulation insertion loss value and the silence segment loss value of the silence detection result of the initial detection model, solving the problem of timestamp delay caused by encountering a long silence segment when the CTC loss function is used alone for model training, and also solving the problem of requiring a large number of training samples when the cross entropy loss function is used alone. At the same time, it ensures the fit between the output result of the silence detection model and the real scene, and improves the silence segment recognition ability of the constructed silence detection model. The silence detection method and device disclosed in the present invention use the silence detection model to achieve more accurate silence detection of the target speech.

[0089] The method, apparatus, and device of the present disclosure may be implemented in many ways. For example, the method, apparatus, and device of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above sequence of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the sequence specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0090] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present disclosure that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0091] The above description is only an implementation method of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of protection involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the technical concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A method for constructing a silence detection model, characterized in that: include: The initial detection model performs silence detection on the training samples to obtain silence detection results; The silence detection result is analyzed by using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value, including: using a cross entropy loss function to compare the splicing probability in the silence detection result with the actual splicing data to obtain a simulated insertion loss value that characterizes the degree of fit between the splicing probability and the actual splicing data; using a CTC loss function to compare the speech position of the silence start point and the speech position of the silence end point of the silence segment in the silence detection result and the speech duration of the silence segment with the actual silence segment data of the silence segment, to obtain a silence segment loss value that characterizes the degree of fit between the speech position of the silence start point and the speech position of the silence end point of the silence segment and the speech duration of the silence segment and the actual silence segment data; and The initial detection model is weighted according to the simulated insertion loss value and the silent segment loss value to obtain a silent detection model.

2. The method for constructing a silence detection model according to claim 1, characterized in that: The initial detection model performs silence detection on the training sample to obtain the silence detection result, including: The initial detection model performs silence endpoint recognition on the training sample, and determines the speech positions of multiple silence starting points and multiple silence ending points of the training sample respectively; Determine the time length between the silence start point and the silence end point that are adjacent in sequence, and take the speech range between the silence start point and the silence end point whose time length meets the silence threshold as a silence segment; Calculating the splicing probability that the silence segment belongs to the silence splicing data; and The speech position of the silence start point and the speech position of the silence end point of each silence segment, the speech duration of each silence segment and the splicing probability of each silence segment are integrated to generate a silence detection result for the training sample.

3. The method for constructing a silence detection model according to claim 1, characterized in that: After analyzing the silence detection result by using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value, the method further includes: The simulated insertion loss value and the silent segment loss value are integrated to obtain a total loss value used to characterize the degree of deviation between the silent detection result and the actual value.

4. The method for constructing a silence detection model according to claim 1, characterized in that: Before the initial detection model performs silence detection on the training sample to obtain the silence detection result, the method further includes: Acquiring the silent splicing data in the training sample, including: Segmenting the original speech segments and the original silence segments in the original speech data to obtain a speech set and a silence set; and The silence set is cut into a plurality of simulated silence segments with preset time lengths, and each of the simulated silence segments is respectively inserted into the speech set to obtain the training sample formed by splicing the speech set and the plurality of simulated silence segments.

5. The method for constructing a silence detection model according to claim 4, characterized in that: The step of cutting the silence set into a plurality of simulated silence segments having a preset time length, and inserting each of the simulated silence segments into the speech set to obtain the training sample formed by splicing the speech set and the plurality of simulated silence segments comprises: At least two simulated silence segments with preset time lengths are segmented from the silence set, and the at least two simulated silence segments with preset time lengths are respectively inserted into the head of the voice set and the tail of the voice set.

6. The method for constructing a silence detection model according to claim 4 or 5, characterized in that: The step of cutting the silence set into a plurality of simulated silence segments having a preset time length, and inserting each of the simulated silence segments into the speech set to obtain the training sample formed by splicing the speech set and the plurality of simulated silence segments comprises: The speech set is divided into a plurality of speech segments with a preset time length, and a plurality of simulated silence segments with a preset time length divided from the silence set are inserted between two adjacent speech segments in sequence to obtain the training sample composed of the speech set and the plurality of simulated silence segments.

7. A silence detection method, characterized in that: include: Decomposing the target speech into a plurality of speech packets and inputting the packets into the silence detection model, wherein the silence detection model is constructed by the method for constructing the silence detection model according to any one of claims 1 to 6; Determining whether each of the voice packets is a silence segment by the silence detection model; as well as In response to the judgment result that the voice packet is the silent segment, a decoding instruction is generated, and the decoder performs content recognition on the voice packet.

8. The silence detection method according to claim 7, characterized in that: The method for constructing the silence detection model includes: The initial detection model performs silence detection on the training samples to obtain silence detection results; Analyzing the silence detection result using at least two loss functions to obtain a simulated insertion loss value and a silence segment loss value; and The initial detection model is weighted according to the simulated insertion loss value and the silent segment loss value to obtain a silent detection model.

9. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the method for constructing a silence detection model according to any one of claims 1 to 6 and / or executes the silence detection method according to claim 7 or 8.

10. A storage medium, characterized in that: The storage medium stores execution instructions, which, when executed by the processor, are used to implement the method for constructing a silence detection model according to any one of claims 1 to 6 and / or the silence detection method according to claim 7 or 8.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the construction method described in any one of claims 1 to 6 and / or the silence detection method described in claim 7 or 8 are implemented.

Images