Audio recall method, model training method, device and electronic equipment

Abstract

The present disclosure provides an audio recall method, a model training method, a device and an electronic device, relates to the technical field of cloud computing, in particular to the technical field of deep learning, intelligent search and voice technology, and the audio recall method comprises: acquiring a first audio; segmenting the first audio to obtain N audio segments, any two adjacent audio segments in the N audio segments partially coincide, and N is an integer greater than 1; recalling a second audio corresponding to each of the N audio segments from a sample pool.

Description

Technical Field

[0001] This disclosure relates to the field of cloud computing technology, and more particularly to the fields of deep learning, intelligent search and voice technology, specifically to an audio retrieval method, a model training method, a device and an electronic device. Background Technology

[0002] With the continuous development of audio recognition technology, its application scope is becoming increasingly wide. Audio recognition technology can be applied to the identification of similar audio. Currently, it is usually done by recognizing the melody of the audio and then recalling audio with similar melody. Summary of the Invention

[0003] This disclosure provides an audio recall method, a model training method, an apparatus, and an electronic device.

[0004] According to a first aspect of this disclosure, an audio recall method is provided, comprising:

[0005] Get the first audio;

[0006] The first audio is segmented to obtain N audio segments. Any two adjacent audio segments in the N audio segments partially overlap, where N is an integer greater than 1.

[0007] Retrieve the second audio from the sample pool corresponding to each of the N audio segments.

[0008] According to a second aspect of this disclosure, a method for training an audio recall model is provided, comprising:

[0009] Obtain training audio from the sample pool;

[0010] The training audio is defined as a sample group comprising positive and negative sample audio.

[0011] The sample group is input into the model to be trained for audio recall training;

[0012] When the model to be trained meets the preset conditions, the model to be trained is determined as an audio retrieval model. The audio retrieval model is used to retrieve the second audio corresponding to each of the N audio segments from the sample pool. The N audio segments are audio segments obtained by segmenting the first audio.

[0013] According to a third aspect of this disclosure, an audio recall device is provided, comprising:

[0014] The first acquisition module is used to acquire the first audio;

[0015] The segmentation module is used to segment the first audio to obtain N audio segments, wherein any two adjacent audio segments in the N audio segments partially overlap, and N is an integer greater than 1;

[0016] The recall module is used to recall a second audio from the sample pool corresponding to each of the N audio segments.

[0017] According to a fourth aspect of this disclosure, an audio recall model training apparatus is provided, comprising:

[0018] The second acquisition module is used to acquire training audio from the sample pool;

[0019] The first determining module is used to determine the sample group included in the training audio, wherein the sample group includes positive sample audio and negative sample audio;

[0020] The training module is used to input the sample group into the model to be trained for audio recall training;

[0021] The second determining module is used to determine the model to be trained as an audio recall model when the model to be trained meets the preset conditions. The audio recall model is used to recall a second audio corresponding to each of the N audio segments from the sample pool. The N audio segments are audio segments obtained by segmenting the first audio.

[0022] According to a fifth aspect of this disclosure, an electronic device is provided, comprising:

[0023] At least one processor; and

[0024] A memory that is communicatively connected to at least one processor; wherein,

[0025] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform either the first aspect or the second aspect.

[0026] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform any one of the methods of the first or second aspect.

[0027] According to a seventh aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any one of the methods of the first or second aspect.

[0028] In this embodiment of the disclosure, the first audio is segmented to obtain N audio segments. Any two adjacent audio segments in the N audio segments partially overlap in content. This can improve the correlation and matching degree between any two adjacent audio segments in the N audio segments, thereby making the correlation and matching degree between the second audio recalled based on each audio segment higher, that is, improving the recall rate and precision of the second audio.

[0029] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0030] Figure 1 This is a flowchart of an audio recall method provided in an embodiment of this disclosure;

[0031] Figure 2 This is a schematic diagram of a first audio segmentation provided in an embodiment of this disclosure;

[0032] Figure 3 This is a flowchart of an audio recall model training method provided in an embodiment of this disclosure;

[0033] Figure 4 This is a schematic diagram of a sample group provided in an embodiment of this disclosure;

[0034] Figure 5 This is a schematic diagram of the structure of an audio recall device provided in an embodiment of this disclosure;

[0035] Figure 6 This is a schematic diagram of another audio recall device provided in an embodiment of this disclosure;

[0036] Figure 7 This is a schematic diagram of the structure of an audio recall model training device provided in an embodiment of this disclosure;

[0037] Figure 8 This is a schematic diagram of another audio recall model training device provided in this embodiment of the present disclosure;

[0038] Figure 9 This is a schematic block diagram of an example electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0039] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0040] With the continuous development of audio recognition technology, its application scope is becoming increasingly wide, including the identification of similar audio. Currently, this is usually done by recognizing the melody of the audio and then recalling audio with similar melody. However, this method is not suitable in certain scenarios, such as risk control scenarios, where the audio contains a lot of content, resulting in low accuracy of audio recall using this method.

[0041] To address the aforementioned issues, the following solutions are proposed. In other words, the implementation methods disclosed herein can be applied to risk control scenarios, and of course, they can also be applied to other scenarios, such as preventing cache breakdown and detecting new silences. Specific applications are not limited here.

[0042] See Figure 1 , Figure 1 A flowchart of an audio recall method provided in this disclosure embodiment is shown below. Figure 1 As shown, the audio retrieval method includes the following steps:

[0043] Step S101: Obtain the first audio.

[0044] The first audio segment can also be referred to as the recorded audio or the audio to be recorded.

[0045] Step S102: Segment the first audio to obtain N audio segments. Any two adjacent audio segments in the N audio segments partially overlap, where N is an integer greater than 1.

[0046] The specific method of segmenting the first audio is not limited here. As an optional implementation method, the first audio can be randomly segmented to obtain N audio segments.

[0047] As another alternative implementation, the first audio can be divided evenly, meaning that each audio segment has the same duration.

[0048] For example, the first audio segment can be divided into segments with a hop duration of 1 second and a duration of 2 seconds. Furthermore, any two adjacent audio segments within the N segments will have overlapping content, meaning there will be 1 second of overlapping content between any two adjacent audio segments. See details... Figure 2 The first audio 201 can be divided into 5 audio segments. The encoding information corresponding to the 5 audio segments can be t, and the encoding information of the 5 audio segments can be t=1, t=2, t=3, t=4 and t=5 respectively. The duration of each audio segment can be 2 seconds, and two adjacent audio segments can include 1 second of overlapping content.

[0049] Step S103: Retrieve the second audio corresponding to each of the N audio segments from the sample pool.

[0050] Depending on the application scenario, the audio content included in the sample pool will also be different. For example, the audio content included in the sample pool may include sensitive content, which may also be referred to as sensitive information.

[0051] In this embodiment of the disclosure, through steps S101 to S103, since the content of any two adjacent audio segments among the N audio segments partially overlaps, the correlation and matching degree of the N audio segments are both high. This results in a high correlation and matching degree between the second audio segments recalled based on each audio segment, thereby improving the recall rate and precision of the second audio segments.

[0052] For example, if the content of the first audio segment is "I am A, B, C", then the first audio segment can be divided into 4 audio segments, and the content of the 4 audio segments can be "I am", "I am A", "A B" and "B C" respectively. That is, the content of two adjacent audio segments in the 4 audio segments includes some overlapping content, and the content of the 4 audio segments has a high degree of correlation. The content of the second audio segment recalled based on the 4 audio segments can be "You are", "You are A", "AB" and "BC" respectively. This makes the correlation between the second audio segments recalled from the sample library based on the above 4 audio segments also high, thereby improving the recall rate and precision of the second audio segment.

[0053] Other segmentation methods can prevent the content of adjacent audio segments from overlapping. For example, the content of the first audio segment can be divided into three audio segments with the content of "I am", "A and B" and "C" in sequence. The content of the second audio segment recalled by the three audio segments obtained by the other segmentation methods can be "You are", "12" and "Okay" in sequence. It can be seen that the correlation of the second audio segment recalled by the three audio segments obtained by other segmentation methods is lower than that of the second audio segment recalled by the present invention. That is, the recall rate and precision of the second audio segment recalled by the present invention are higher.

[0054] Each audio segment can recall one or more second audio segments; the specifics are not limited here.

[0055] In addition, when recalling the second audio corresponding to each audio segment, the content of each audio segment can be matched with the content of the audio to be recalled. When the matching degree is higher than a preset threshold, the audio to be recalled can be identified as the second audio corresponding to that audio segment.

[0056] Alternatively, the second audio corresponding to each audio segment can be determined in other ways. For example, each audio segment and the audio to be recalled can include tag information. When the tag information of the two matches, the audio to be recalled can be determined as the second audio corresponding to that audio segment.

[0057] It should be noted that the specific method for determining the second audio corresponding to the audio segment is not limited here; the above is merely an example.

[0058] As an optional implementation, the step of recalling the second audio corresponding to each of the N audio segments from the sample pool includes...

[0059] Based on the target parameters, recall the second audio corresponding to each of the N audio segments from the sample pool;

[0060] The target parameters include at least one of the following: a similarity score threshold, an encoding information set, a tag information set, and an interval threshold. The encoding information set includes the encoding information of N audio segments, the tag information set includes the tag information of N audio segments, and the interval threshold is a preset interval threshold for the N audio segments.

[0061] This involves calculating the similarity between each audio segment and the audio to be recalled in the sample pool, and identifying the audio to be recalled with a similarity greater than the similarity score threshold as the second audio for recall.

[0062] Each audio segment can correspond to encoding information and tag information. Each audio segment to be recalled in the sample pool can also correspond to encoding information and tag information. When the encoding information of the audio segment to be recalled matches the encoding information of the audio segment, the audio segment to be recalled can be identified as the second audio segment. Similarly, when the tag information of the audio segment to be recalled matches the tag information of the audio segment, the audio segment to be recalled can be identified as the second audio segment.

[0063] In this context, any two adjacent audio segments can be referred to as the first audio segment and the second audio segment, respectively. The encoding information of the second audio segment corresponding to the first audio segment can be the first code. The second audio segment can correspond to two audio segments to be recalled, and the encoding information of the two audio segments to be recalled can be the second code and the third code, respectively. When the difference between the second code and the first code is less than the interval threshold, the audio segment to be recalled corresponding to the second code can be identified as the second audio segment corresponding to the second audio segment. When the difference between the second code and the first code is greater than or equal to the interval threshold, the audio segment to be recalled corresponding to the second code can be uncertain as the second audio segment corresponding to the second audio segment, that is, the audio segment to be recalled corresponding to the second code is not recalled. Similarly, when the difference between the third code and the first code is less than the interval threshold, the audio segment to be recalled corresponding to the third code can be identified as the second audio segment corresponding to the second audio segment. When the difference between the third code and the first code is greater than or equal to the interval threshold, the audio segment to be recalled corresponding to the third code can be uncertain as the second audio segment corresponding to the second audio segment, that is, the audio segment to be recalled corresponding to the third code is not recalled.

[0064] In this embodiment of the disclosure, since the target parameters include at least one of the following: similarity score threshold, encoding information set, tag information set and interval threshold, the methods for recalling the second audio corresponding to each audio segment are increased, that is, the diversity and flexibility of the recall methods for the second audio are increased.

[0065] It should be noted that the more content included in the target parameters, the more accurate the recall results of the second audio.

[0066] As an optional implementation, the target parameter includes the similarity score threshold, and the step of recalling the second audio corresponding to each of the N audio segments from the sample pool according to the target parameter includes:

[0067] Feature extraction is performed on each of the N audio segments to obtain N first features;

[0068] N second features are determined in the sample pool, wherein the cosine distance between each second feature and its corresponding first feature is greater than the similarity score threshold.

[0069] Recall the second audio from the sample pool corresponding to each of the N second features.

[0070] The similarity between each second feature and its corresponding first feature can be calculated by calculating the cosine distance between each second feature and its corresponding first feature. The cosine distance can also be called the Cosine distance.

[0071] In addition, the fact that the cosine distance between each second feature and its corresponding first feature is greater than the similarity score threshold can also be understood as: the cosine distance between each first feature and multiple candidate features in the sample pool can be calculated, and the candidate features whose cosine distance is greater than the similarity score threshold are determined as the second features of the first feature.

[0072] The sample pool can store the second audio and the second feature corresponding to the second audio, which can improve the recall efficiency of the second audio; or, the sample pool can only store the second audio. After obtaining the first feature, the second audio can be used to extract features to obtain the second feature, which can save the storage resources of the sample pool.

[0073] In this embodiment of the disclosure, the similarity between the audio segment corresponding to the first feature and the second audio segment corresponding to the second feature is calculated by calculating the cosine distance between the first feature and the second feature. This simplifies the similarity calculation method and improves the accuracy and efficiency of the similarity calculation results.

[0074] It should be noted that the value of the similarity score threshold is not limited here; however, different values ​​of the similarity score threshold will result in different recall and precision for the second audio.

[0075] See Table 1. Table 1 is a statistical table of the precision and recall rates of the second audio when using different similarity score thresholds. As shown in Table 1, when the similarity score threshold is 0.5, the recall rate of the second audio is 95% and the precision rate is 82%. When the similarity score threshold is 0.6, the recall rate of the second audio is 93% and the precision rate is 79%. When the similarity score threshold is 0.7, the recall rate of the second audio is 93% and the precision rate is 80%. When the similarity score threshold is 0.8, the recall rate of the second audio is 88% and the precision rate is 90%.

[0076] Score threshold 0.5 0.6 0.7 0.8 Accuracy 75％ 79％ 80％ 90％ Recall rate 95％ 93％ 93％ 88％

[0077] Table 1

[0078] As an optional implementation, the N audio segments are arranged in a preset order, wherein the preset order is the order in which the encoded information gradually increases;

[0079] The step of recalling the second audio corresponding to each of the N second features from the sample pool includes:

[0080] The N second features are divided into multiple feature sequence groups, each of which includes multiple second features, and the encoded information of the multiple second features within the same feature sequence group gradually increases;

[0081] Determine a target feature sequence group among a plurality of feature sequence groups, wherein the target feature sequence group is the feature sequence group that includes the most second features among the plurality of feature sequence groups;

[0082] Retrieve the second audio corresponding to the second feature included in the target feature sequence group from the sample pool.

[0083] Among them, the number of second features included in multiple feature sequence groups can be the same or different. When the number of second features included in multiple feature sequence groups is different, the above target feature sequence group can be called the longest feature sequence group or the longest subsequence.

[0084] In this embodiment, since the N audio segments are arranged in order of increasing encoded information, and the encoded information of multiple second features within the feature sequence group gradually increases, second features whose encoded information is significantly smaller than that of the previous second feature can be excluded. This reduces the occurrence of false recall of second audio corresponding to second features that are less related to the previous second feature, thereby further improving the accuracy of recalling second audio corresponding to the second feature.

[0085] In addition, the target feature sequence group is the feature sequence group that includes the most second features among multiple feature sequence groups. This ensures that the number of recalled second audios is as large as possible, thereby ensuring a good recall effect for the second audios.

[0086] As an optional implementation, each of the N audio segments includes target label information, and the label information of the second feature included in the target feature sequence group matches the target label information.

[0087] The first audio may include target tag information, and each audio segment includes the same target tag information. The target tag information included in each audio segment may also be the same as the target tag information included in the first audio. In other words, the target tag information mentioned above can be used to indicate the source of the first audio and audio segments. The target tag information of audio or audio segments from different sources may be different.

[0088] In this embodiment of the disclosure, since the label information of the second feature included in the target feature sequence group matches the target label information, the accuracy (also known as precision) of the second audio corresponding to the recalled second feature can be further improved, and the occurrence of other audio being falsely recalled can be reduced.

[0089] For example, the target label information of the first audio and each audio segment can be Audio1, and the encoding information can be expressed in the form of "ID-number". For example, the encoding information can be represented in the form of ID-1, ID-2, ID-3, etc. The encoding information can be used to represent the numbering information (i.e., the order) of the audio segments in the first audio, and the similarity can be called the score.

[0090] When the duration of the first audio is 10 seconds, 10 second features are retrieved from the sample library, and the target parameters of the 10 second features can be as follows:

[0091] Audio1-ID-1, score: 0.82;

[0092] Audio2-ID-1, score: 0.82;

[0093] Audio1-ID-3, score: 0.85;

[0094] Audio1-ID-4, score: 0.87;

[0095] Audio1-ID-5, score: 0.83;

[0096] Audio2-ID-4, score: 0.85;

[0097] Audio1-ID-6, score: 0.84;

[0098] Audio1-ID-2, score: 0.84;

[0099] Audio2-ID-5, score: 0.84;

[0100] Audio1-ID-7, score: 0.86.

[0101] Then, based on the target parameters, the following target feature sequence group can be selected from the above 10 second features. The target parameters of the second features included in the target feature sequence group are as follows:

[0102] Audio1-ID-1, score: 0.82;

[0103] Audio1-ID-3, score: 0.85;

[0104] Audio1-ID-4, score: 0.87;

[0105] Audio1-ID-5, score: 0.83;

[0106] Audio1-ID-6, score: 0.84;

[0107] Audio1-ID-7, score: 0.86.

[0108] As can be seen, the target label information for the first audio and each audio segment can be "Audio1", and the label information for the second feature mentioned above is also "Audio1", that is, the second feature whose label information matches the target label information is selected; at the same time, the encoding information of the second feature is sequentially increasing, that is: ID-1, ID-3, ID-4, ID-5, ID-6, ID-7; in addition, the similarity score threshold can be 0.8, so only the second features with similarity greater than 0.8 can be selected (that is, the second features with scores of 0.82, 0.85, 0.87, 0.83, 0.84 and 0.86 are selected), which can make the recall accuracy of the second audio higher, that is, improve the recall accuracy of the second audio.

[0109] It should be noted that, referring to Table 2, which is a statistical table of the recall precision rate of the second audio when using this embodiment and not using this embodiment, as shown in Table 2, when not using this embodiment, the recall precision rate of the second audio is 82%, and when using this embodiment, the recall precision rate of the second audio is 90%.

[0110] Use yes no Accuracy 90％ 82％

[0111] Table 2

[0112] As an optional implementation, the difference in encoded information between any two adjacent second features included in the target feature sequence group is less than the interval threshold.

[0113] In this embodiment of the disclosure, when the difference in encoded information between any two adjacent second features included in the target feature sequence group is greater than or equal to the interval threshold, it can be considered that the correlation between the two adjacent second features is small, so the latter second feature can be deleted from the target feature sequence group, that is, the second audio corresponding to the latter second feature is not recalled, thereby further improving the accuracy of the recalled second audio.

[0114] For example, when the encoded information of the second feature in the determined target feature sequence group is ID-1, ID-3, ID-4, ID-5, ID-6, ID-10, and ID-20, and the interval threshold can be 5, it can be seen from the above that the interval difference between the last second feature (i.e., the encoded information is ID-20) and the previous second feature (i.e., the encoded information is ID-10) is 10, which is significantly greater than the interval threshold of 5. Therefore, it can be considered that the correlation between the last second feature (i.e., the encoded information is ID-20) and the previous second feature (i.e., the encoded information is ID-10) is small. Therefore, the last second feature (i.e., the encoded information is ID-20) can be deleted from the target feature sequence group, and only the second audio corresponding to ID-1, ID-3, ID-4, ID-5, ID-6, and ID-10 can be recalled, thereby further improving the accuracy of the second audio recall.

[0115] See Table 3, which is a statistical table of the recall precision of the second audio with and without this embodiment. As shown in Table 3, when this embodiment is not used, the recall precision of the second audio is 88%, and when this embodiment is used, the recall precision of the second audio is 90%.

[0116]

[0117] Table 3

[0118] As an optional implementation, the step of recalling the second audio corresponding to each of the N audio segments from the sample pool includes:

[0119] The second audio corresponding to each of the N audio segments is recalled from the sample pool using an audio recall model;

[0120] The audio recall model is a pre-trained model used to recall audio.

[0121] In this embodiment of the disclosure, the second audio can be recalled through a pre-trained audio recall model, which makes the recall efficiency of the second audio higher and the recall of the second audio more convenient.

[0122] It should be noted that the training method for the audio recall model can be found in [reference needed]. Figure 3 The embodiments shown, i.e., the embodiments of this disclosure, can be applied to using... Figure 3 The audio recall model trained using the training method shown is used.

[0123] See Figure 3 , Figure 3 A flowchart of an audio recall model training method provided in this disclosure embodiment is shown below. Figure 3 As shown, the audio recall model training method includes the following steps:

[0124] Step S301: Obtain training audio from the sample pool.

[0125] The training audio can be referred to as the sample audio, and the training audio can be from the same source as the first audio. This allows the audio recall model trained based on the training audio to have higher accuracy in recognizing the first audio, thereby further improving the recall and precision of the second audio.

[0126] The aforementioned audio from the same source can refer to audio from the same scene or audio from the same speaker, etc.

[0127] Step S302: Determine the sample group included in the training audio, wherein the sample group includes positive sample audio and negative sample audio.

[0128] Positive sample audio can be referred to as positive audio, and negative sample audio can be referred to as negative audio.

[0129] It should be noted that the method and number of sample groups are not limited here.

[0130] As an optional implementation, determining the sample group included in the training audio includes:

[0131] The audio obtained by randomly breaking down the training audio is determined as the positive sample audio.

[0132] The portion of the training audio other than the positive sample audio is identified as the negative sample audio.

[0133] The method of randomly interrupting the training audio can be called random shuffle, and the audio obtained by randomly interrupting the training audio can be called positive sample audio, or anchor audio.

[0134] All audio in the training audio except for the positive sample audio can be negative sample audio. It should be noted that there can be multiple negative sample audio.

[0135] That is, in the same sample group, there can be one positive sample audio and multiple negative sample audio.

[0136] Among them, see Figure 4 , Figure 4 This is a schematic diagram illustrating the positive and negative audio samples included in the sample group, as shown below. Figure 4As shown, a horizontal column or a vertical column can be a sample group 400, and a sample group 400 can include a positive sample audio 401 and multiple negative sample audio 402.

[0137] It should be noted that, see Figure 4 Different sample groups 400 can share the same positive sample audio 401.

[0138] In this embodiment of the disclosure, the method of obtaining sample groups can be more flexible and diversified, thereby increasing the number of sample groups and making the content of different sample groups different, thus improving the training effect of the model to be trained.

[0139] As an optional implementation, the number of sample groups is multiple.

[0140] In this embodiment of the disclosure, using multiple sample groups to train the model can improve the robustness of the model.

[0141] Step S303: Input the sample group into the model to be trained for audio recall training.

[0142] In this case, the positive and negative audio samples included in the sample group can be input into the model to be trained simultaneously for audio recall training, or the positive and negative audio samples included in the sample group can be input into the model to be trained separately for audio recall training. The specific method is not limited here.

[0143] Step S304: When the model to be trained meets the preset conditions, the model to be trained is determined as an audio recall model. The audio recall model is used to recall the second audio corresponding to each of the N audio segments from the sample pool. The N audio segments are audio segments obtained by segmenting the first audio.

[0144] The specific content of the preset conditions is not limited here.

[0145] As an optional implementation, the preset conditions include: when the similarity between the audio recalled by the training model and the second audio corresponding to the audio segment is greater than a preset threshold, it can be considered that the precision and recall of the audio recalled by the training model meet the requirements, that is, the training model has converged, and the training model at this time can be determined as the audio recall model.

[0146] It should be noted that the various technical features in the embodiments of this disclosure can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0147] In this embodiment of the present disclosure, an audio recall model can be trained through steps S301 to S304, and the audio recall model has good precision and recall rate for recalling the second audio, that is, the audio recall model trained through this embodiment has good performance.

[0148] See Figure 5 , Figure 5 This is a schematic diagram of the structure of an audio recall device provided in an embodiment of this disclosure, as shown below. Figure 5 As shown, the audio recall device 500 includes:

[0149] The first acquisition module 501 is used to acquire the first audio;

[0150] The segmentation module 502 is used to segment the first audio to obtain N audio segments, wherein any two adjacent audio segments in the N audio segments partially overlap, and N is an integer greater than 1;

[0151] The recall module 503 is used to recall a second audio from the sample pool corresponding to each of the N audio segments.

[0152] As an optional implementation, the recall module 503 is further configured to recall a second audio corresponding to each of the N audio segments from the sample pool based on the target parameters;

[0153] The target parameters include at least one of the following: a similarity score threshold, an encoding information set, a tag information set, and an interval threshold. The encoding information set includes the encoding information of N audio segments, the tag information set includes the tag information of N audio segments, and the interval threshold is a preset interval threshold for the N audio segments.

[0154] As an optional implementation, see [link to implementation details]. Figure 6 The target parameters include the similarity score threshold, and the recall module 503 includes:

[0155] The feature extraction submodule 5031 is used to extract features from the N audio segments respectively to obtain N first features;

[0156] The first determining submodule 5032 is used to determine N second features in the sample pool, wherein the cosine distance between each second feature and its corresponding first feature is greater than the similarity score threshold.

[0157] The recall submodule 5033 is used to recall the second audio corresponding to each of the N second features from the sample pool.

[0158] As an optional implementation, the N audio segments are arranged in a preset order, wherein the preset order is an order in which the encoded information gradually increases; the recall submodule 5033 is specifically used for:

[0159] The N second features are divided into multiple feature sequence groups, each of which includes multiple second features, and the encoded information of the multiple second features within the same feature sequence group gradually increases;

[0160] Determine a target feature sequence group among a plurality of feature sequence groups, wherein the target feature sequence group is the feature sequence group that includes the most second features among the plurality of feature sequence groups;

[0161] Retrieve the second audio corresponding to the second feature included in the target feature sequence group from the sample pool.

[0162] As an optional implementation, each of the N audio segments includes target label information, and the label information of the second feature included in the target feature sequence group matches the target label information.

[0163] As an optional implementation, the difference in encoded information between any two adjacent second features included in the target feature sequence group is less than the interval threshold.

[0164] As an optional implementation, the recall module 503 is further configured to recall a second audio corresponding to each of the N audio segments from the sample pool using an audio recall model;

[0165] The audio recall model is a pre-trained model used to recall audio.

[0166] The audio recall device 500 provided in this disclosure can implement all the processes implemented in the audio recall method embodiments and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0167] See Figure 7 , Figure 7 This is a schematic diagram of the structure of an audio recall model training device provided in an embodiment of this disclosure, as shown below. Figure 7 As shown, the audio recall model training device 700 includes:

[0168] The second acquisition module 701 is used to acquire training audio from the sample pool;

[0169] The first determining module 702 is used to determine the sample group included in the training audio, wherein the sample group includes positive sample audio and negative sample audio;

[0170] Training module 703 is used to input the sample group into the model to be trained for audio recall training;

[0171] The second determining module 704 is used to determine the model to be trained as an audio recall model when the model to be trained meets the preset conditions. The audio recall model is used to recall a second audio corresponding to each of the N audio segments from the sample pool. The N audio segments are audio segments obtained by segmenting the first audio.

[0172] As an optional implementation, see [link to implementation details]. Figure 8 The first determining module 702 includes:

[0173] The second determining submodule 7021 is used to determine the part of the audio obtained by randomly interrupting the training audio as the positive sample audio;

[0174] The third determining submodule 7022 is used to determine the portion of the training audio other than the positive sample audio as the negative sample audio.

[0175] As an optional implementation, the number of sample groups is multiple.

[0176] The audio recall model training device 700 provided in this disclosure can implement all the processes implemented in the audio recall model training method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described in detail here.

[0177] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0178] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0179] like Figure 9As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0180] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0181] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as audio recall methods or audio recall model training methods. For example, in some embodiments, the audio recall method or audio recall model training method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the audio recall method or audio recall model training method described above can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform an audio recall method or an audio recall model training method by any other suitable means (e.g., by means of firmware).

[0182] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0184] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0186] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0187] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0188] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0189] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An audio recall method, comprising: Get the first audio; The first audio is segmented to obtain N audio segments. Any two adjacent audio segments in the N audio segments partially overlap, where N is an integer greater than 1. Retrieve the second audio corresponding to each of the N audio segments from the sample pool; The step of recalling the second audio corresponding to each of the N audio segments from the sample pool includes: Based on the target parameters, recall the second audio corresponding to each of the N audio segments from the sample pool; The target parameter includes a similarity score threshold. The step of recalling the second audio corresponding to each of the N audio segments from the sample pool based on the target parameter includes: Feature extraction is performed on each of the N audio segments to obtain N first features; N second features are determined in the sample pool, wherein the cosine distance between each second feature and its corresponding first feature is greater than the similarity score threshold. Retrieve the second audio corresponding to each of the N second features from the sample pool; The N audio segments are arranged in a preset order, wherein the preset order is the order in which the encoded information gradually increases; The step of recalling the second audio corresponding to each of the N second features from the sample pool includes: The N second features are divided into multiple feature sequence groups, each of which includes multiple second features, and the encoded information of the multiple second features within the same feature sequence group gradually increases; Determine a target feature sequence group among the plurality of feature sequence groups, wherein the target feature sequence group is the feature sequence group that includes the most second features among the plurality of feature sequence groups; Retrieve the second audio corresponding to the second feature included in the target feature sequence group from the sample pool.

2. The method according to claim 1, wherein, The target parameters also include at least one of the following: an encoding information set, a tag information set, and an interval threshold. The encoding information set includes the encoding information of N audio segments, the tag information set includes the tag information of N audio segments, and the interval threshold is a preset interval threshold for the N audio segments.

3. The method of claim 1, wherein, Each of the N audio segments includes target label information, and the label information of the second feature included in the target feature sequence group matches the target label information.

4. The method of claim 1, wherein, The difference in encoded information between any two adjacent second features in the target feature sequence group is less than the interval threshold.

5. An audio recall device, comprising: The first acquisition module is used to acquire the first audio; The segmentation module is used to segment the first audio to obtain N audio segments, wherein any two adjacent audio segments in the N audio segments partially overlap, and N is an integer greater than 1; The recall module is used to recall a second audio from the sample pool corresponding to each of the N audio segments; The recall module is further configured to recall a second audio corresponding to each of the N audio segments from the sample pool based on the target parameters; The target parameters include a similarity score threshold, and the recall module includes: The feature extraction submodule is used to extract features from the N audio segments respectively to obtain N first features; The first determining submodule is used to determine N second features in the sample pool, wherein the cosine distance between each second feature and its corresponding first feature is greater than the similarity score threshold. The recall submodule is used to recall the second audio corresponding to each of the N second features from the sample pool; The N audio segments are arranged in a preset order, where the preset order is an order in which the encoded information gradually increases; the recall submodule is specifically used for: The N second features are divided into multiple feature sequence groups, each of which includes multiple second features, and the encoded information of the multiple second features within the same feature sequence group gradually increases; Determine a target feature sequence group among a plurality of feature sequence groups, wherein the target feature sequence group is the feature sequence group that includes the most second features among the plurality of feature sequence groups; Retrieve the second audio corresponding to the second feature included in the target feature sequence group from the sample pool.

6. The apparatus of claim 5, wherein, The target parameters also include at least one of the following: an encoding information set, a tag information set, and an interval threshold. The encoding information set includes the encoding information of N audio segments, the tag information set includes the tag information of N audio segments, and the interval threshold is a preset interval threshold for the N audio segments.

7. The apparatus of claim 5, wherein, Each of the N audio segments includes target label information, and the label information of the second feature included in the target feature sequence group matches the target label information.

8. The apparatus of claim 5, wherein, The difference in encoded information between any two adjacent second features in the target feature sequence group is less than the interval threshold.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

10. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-4.

Images