Audio keyword retrieval method, intelligent outbound method and related devices

Abstract

The application provides a method for retrieving an audio keyword, a method for intelligent outbound call and related devices. The method for retrieving an audio keyword comprises: performing feature processing on a preset audio segment according to a preset rule to obtain a first feature data set, the first feature data set containing at least one first feature vector sequence, wherein the preset audio contains an audio keyword; obtaining a to-be-retrieved audio segment, and performing feature processing on the to-be-retrieved audio segment according to a preset rule to obtain a second feature data set, the second feature data set containing at least one second feature vector sequence; intercepting a third feature vector sequence with the same length as the first feature vector sequence from the second feature vector sequence corresponding to the first feature vector sequence; and judging whether the to-be-retrieved audio segment contains the audio keyword by calculating the similarity between the third feature vector sequence and the first feature vector sequence. The method is simple to implement, has low computational complexity, and is efficient and fast.

Description

Technical Field

[0001] This invention relates to the field of audio retrieval technology, and in particular to an audio keyword retrieval method, an intelligent outbound calling method, and related devices. Background Technology

[0002] Artificial intelligence voice services are becoming a new hot research and industrialization direction in the field of machine learning, with the mainstream technologies being speech-to-text and text-to-speech. However, in some application scenarios, complex speech recognition or synthesis models are not required. The server only needs to know whether a recording or online audio contains certain keywords to meet subsequent business needs. This technology is called voice keyword retrieval.

[0003] Keyword retrieval typically involves extracting audio segments containing keywords and then searching for them within the audio file using methods such as retrieval and matching. If a match is found, the location of the match is returned.

[0004] In the patent application No. 201810096472.X, "Method and Apparatus for Recognizing Speech Keywords", a feature model is trained based on the original audio signal, and then the obtained features are used for keyword recognition. It has strong generalization ability, but the overall design, training and development costs are high, making it unsuitable for small and medium-sized enterprises to develop applications, and it is difficult to guarantee sufficient accuracy for preset applications.

[0005] In the patent application No. 201811524225.0, entitled "A Method and Apparatus for Voice Keyword Detection," the voice data is first converted into text data using speech-to-text technology. Then, a text keyword matching method is used to determine whether there are keyword voices in the audio. This method is costly, requiring in-house development or the purchase of speech-to-text technology services; and its effectiveness heavily depends on the recognition results of the speech-to-text technology. If the error rate of the speech-to-text technology is high, the keyword detection is likely to fail. In addition, it is not suitable for use in online service environments because online audio, such as telephone calls, is transmitted in real-time segments, and the audio content may not be able to form complete sentences; while general speech-to-text technology relies heavily on the naturalness and completeness of the spoken content, otherwise, typos are highly likely to occur.

[0006] In the patent "A Keyword-Based Speech Recognition Method" with an application date of 201811636670.7, it is similar to the second patent, but it first converts the speech signal into Chinese Pinyin and then performs the search.

[0007] In the patent application No. 201410266543.6, entitled "A Speech Keyword Retrieval Method Based on Audio Templates", a probability distribution sequence is used as the features of speech and sample speech templates. The DTW (Dynamic Time Warping) algorithm is used to perform speech search, and finally the scores of different templates are sorted to determine the retrieval results.

[0008] The main drawbacks of Patent 1 are its high research and development costs, the need for a large number of meticulously labeled samples, and the high hardware costs required to ensure real-time performance after service deployment. Patents 2 and 3 suffer from the significant dependence of their recognition and retrieval effectiveness on the quality of intermediate technologies (speech-to-text or audio-to-pinyin conversion) or the service itself. Patent 3, by using probability distributions for speech features, requires an extremely large volume of relevant audio samples to be effective, and the effectiveness of using simple probability distributions for audio features cannot be guaranteed. Furthermore, the DTW algorithm itself is computationally complex and resource-intensive, making it unsuitable for real-time service deployment.

[0009] Therefore, existing technologies need to be improved. Summary of the Invention

[0010] This invention provides an audio keyword retrieval method, an intelligent outbound calling method, and related devices. The method is simple to implement, has low computational complexity, and is highly efficient and fast.

[0011] To solve the above-mentioned technical problems, the first technical solution adopted by the present invention is as follows: A first feature dataset is obtained by performing feature processing on a preset audio segment according to preset rules, the first feature dataset containing at least one first feature vector sequence, wherein the preset audio segment contains audio keywords; an audio segment to be retrieved is obtained, and the audio segment to be retrieved is feature processed according to the preset rules to obtain a second feature dataset, the second feature dataset containing at least one second feature vector sequence; a third feature vector sequence with the same length as the first feature vector sequence is correspondingly extracted from the second feature vector sequence; and the similarity between the third feature vector sequence and the first feature vector sequence is calculated to determine whether the audio segment to be retrieved contains the audio keywords.

[0012] Before the step of extracting a third feature vector sequence with the same length as the first feature vector sequence from the second feature vector sequence, the method further includes: obtaining the length of a second feature vector sequence corresponding to any of the first feature vector sequences; if the length of the second feature vector sequence is greater than or equal to the length of the first feature vector sequence, then determining whether the audio segment to be retrieved contains the audio keyword by calculating the similarity between the third feature vector sequence and the first feature vector sequence; if the length of the second feature vector sequence is less than the length of the first feature vector sequence, then the audio segment to be retrieved does not contain the audio keyword.

[0013] The step of performing feature processing on a preset audio segment according to preset rules to obtain a first feature dataset, wherein the first feature dataset contains at least one first feature vector sequence, and wherein the preset audio segment contains audio keywords, specifically includes: segmenting the preset audio segment into at least one preset audio segment of preset duration according to a set window duration and a preset movement step size, and calculating a first short-time average amplitude value and a first short-time average zero-crossing rate for any preset audio segment of preset duration, to obtain a first short-time average amplitude value feature vector and a first short-time average zero-crossing rate feature vector, wherein the first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector constitute the first feature vector sequence. The step of acquiring the audio segment to be retrieved and performing feature processing on the audio segment to be retrieved according to the preset rules to obtain a second feature dataset, wherein the second feature dataset contains at least one second feature vector sequence, specifically includes: segmenting the audio segment to be retrieved into at least one audio segment to be retrieved of a preset duration according to the window duration and the preset movement step size, and calculating the second short-time average amplitude value and the second short-time average zero-crossing rate of any audio segment to be retrieved of the preset duration, so as to obtain the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector, wherein the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector constitute the second feature vector sequence.

[0014] The step of extracting a third feature vector sequence with the same length as the first feature vector sequence from the second feature vector sequence corresponding to the first feature vector sequence includes: extracting at least one third feature vector sequence with a length equal to the length of the first feature vector sequence from the second feature vector sequence using a preset step length.

[0015] The step of determining whether the audio segment to be retrieved contains the audio keyword by calculating the similarity between the third feature vector sequence and the first feature vector sequence specifically includes: determining whether the value of the similarity is greater than or equal to a preset threshold; if the value of the similarity is greater than or equal to the preset threshold, then the audio segment to be retrieved contains the audio keyword; if the value of the cosine similarity is less than the preset threshold, then the audio segment to be retrieved does not contain the audio keyword.

[0016] The step of determining whether the audio segment to be retrieved contains the audio keyword by calculating the similarity between the third feature vector sequence and the first feature vector sequence specifically includes: calculating the first similarity between a first short-time average amplitude value feature vector in any of the first feature vector sequences and a second short-time average amplitude value feature vector in the corresponding third feature vector sequence; calculating the second similarity between a first short-time average zero-crossing rate feature vector in any of the first feature vector sequences and a second short-time average zero-crossing rate feature vector in the corresponding third feature vector sequence; calculating the product of the first similarity and the second similarity to obtain the similarity value, and determining whether the similarity value is greater than or equal to a preset threshold.

[0017] The step of determining that the audio segment to be retrieved contains the audio keyword if the similarity value is greater than or equal to the preset threshold further includes: recording the offset value of the third feature vector sequence; determining the time position of the audio keyword in the audio segment to be retrieved based on the offset value of the third feature vector sequence and the preset moving step size; the step of determining that the audio segment to be retrieved does not contain the audio keyword if the similarity value is less than the preset threshold further includes: determining whether the third feature vector sequence is the last feature vector sequence in the second feature vector sequence; if so, the audio segment to be retrieved does not contain the audio keyword; if not, continuing to execute the step of calculating the similarity between the third feature vector sequence and the first feature vector sequence.

[0018] To solve the above-mentioned technical problems, the second technical solution adopted by the present invention is: to provide an intelligent outbound calling method, comprising: receiving early media audio from a communication server as an audio segment to be retrieved, wherein the audio segment to be retrieved is used to provide feedback on the current communication status of the communication terminal of the user to be automatically called; retrieving the audio segment to be retrieved according to the keyword retrieval method described above, and obtaining retrieval results; determining the current communication status of the communication terminal according to the retrieval results; and determining an automatic outbound calling strategy according to the determined current communication status.

[0019] To solve the above-mentioned technical problems, the third technical solution adopted by the present invention is: providing an audio keyword retrieval device, comprising: a first feature processing module, configured to perform feature processing on a preset audio segment according to a preset rule to obtain a first feature dataset, the first feature dataset containing at least one first feature vector sequence, wherein the preset audio contains audio keywords; a second feature processing module, configured to acquire an audio segment to be retrieved, and perform feature processing on the audio segment to be retrieved according to the preset rule to obtain a second feature dataset, the second feature dataset containing at least one second feature vector sequence; a feature extraction module, configured to extract a third feature vector sequence from the second feature vector sequence with the same length as the first feature vector sequence; and a similarity calculation module, configured to determine whether the audio segment to be retrieved contains the audio keywords by calculating the similarity between the third feature vector sequence and the first feature vector sequence.

[0020] To solve the above-mentioned technical problems, the fourth technical solution adopted by the present invention is: to provide an intelligent outbound calling device, comprising: a receiving module, used to receive early media audio from a communication server as an audio segment to be retrieved, wherein the audio segment to be retrieved is used to provide feedback on the current communication status of the automatic outbound calling user's communication terminal; a keyword retrieval module, used to retrieve the audio segment to be retrieved according to any of the keyword retrieval methods described above, and obtain retrieval results; a status detection module, used to determine the current communication status of the communication terminal according to the retrieval results; and an execution module, used to determine an automatic outbound calling strategy according to the determined current communication status.

[0021] To solve the above-mentioned technical problems, the fifth technical solution adopted by the present invention is: to provide an audio keyword retrieval device, comprising: a processor and a memory coupled together, wherein the memory stores program instructions; the processor is used to retrieve the program instructions stored in the memory to execute the audio keyword retrieval method described in any one of the above-mentioned methods.

[0022] To solve the above-mentioned technical problems, the sixth technical solution adopted by the present invention is to provide a storage medium storing a program file, wherein the program file can be executed to implement the audio keyword retrieval method described in any one of the above-mentioned methods.

[0023] The beneficial effects of this invention are as follows: Unlike existing technologies, this invention obtains a first feature dataset by performing feature processing on preset audio segments containing audio keywords according to preset rules. The first feature dataset contains at least one first feature vector sequence, and the length of any first feature vector sequence is a preset length. The invention then obtains the audio segment to be retrieved and performs feature processing on it according to preset rules to obtain a second feature dataset, which contains at least one second feature vector sequence. A third feature vector sequence with the same length as the first feature vector sequence is extracted from the second feature vector sequence. The similarity between the first and third feature vector sequences is calculated, and the similarity result determines whether the audio segment to be retrieved contains audio keywords. This invention's audio keyword retrieval method is simple to implement, has low computational complexity, and is highly efficient and fast. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the first embodiment of the audio keyword retrieval method of the present invention;

[0025] Figure 2 This is a flowchart illustrating the second embodiment of the audio keyword retrieval method of the present invention;

[0026] Figure 3 This is a flowchart illustrating the third embodiment of the audio keyword retrieval method of the present invention;

[0027] Figure 4 This is a flowchart illustrating the fourth embodiment of the audio keyword retrieval method of the present invention;

[0028] Figure 5 This is a flowchart illustrating an embodiment of the intelligent outbound calling method of the present invention;

[0029] Figure 6 This is a schematic diagram of the structure of the first embodiment of the audio keyword retrieval device of the present invention;

[0030] Figure 7 This is a flowchart illustrating an embodiment of the intelligent outbound calling device of the present invention;

[0031] Figure 8 This is a schematic diagram of the structure of the second embodiment of the audio keyword retrieval device of the present invention;

[0032] Figure 9 This is a schematic diagram of the structure of the storage medium of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0034] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] Please see Figure 1 This is a flowchart illustrating the first embodiment of the audio keyword retrieval method of the present invention. It includes:

[0036] Step S11: Perform feature processing on the preset audio segment according to preset rules to obtain a first feature dataset. The first feature dataset contains at least one first feature vector sequence, wherein the preset audio contains audio keywords.

[0037] Specifically, a preset audio segment containing audio keywords is obtained, and the preset audio segment is processed according to preset rules to obtain a first feature dataset. The first feature dataset contains at least one first feature vector sequence.

[0038] Step S12: Obtain the audio segment to be retrieved, and perform feature processing on the audio segment to be retrieved according to the preset rules to obtain a second feature dataset, wherein the second feature dataset contains at least one second feature vector sequence.

[0039] A segment of audio to be retrieved is acquired and characterized according to preset rules to obtain a second feature dataset. The second feature dataset includes at least one sequence of second feature vectors.

[0040] Step S13: Extract a third feature vector sequence from the second feature vector sequence, which has the same length as the first feature vector sequence.

[0041] Extract a third feature vector sequence with the same length as the first feature vector sequence from the second feature vector sequence corresponding to the first feature vector sequence.

[0042] Step S14: Determine whether the audio segment to be retrieved contains audio keywords by calculating the similarity between the third feature vector sequence and the first feature vector sequence.

[0043] Specifically, in this embodiment, the cosine similarity between the third feature vector sequence and the first feature vector sequence is calculated, and the value of the calculated cosine similarity is used to determine whether the audio segment to be retrieved contains audio keywords.

[0044] In this embodiment, preset audio segments containing audio keywords and the audio segment to be retrieved are characterized to obtain a first feature dataset and a second feature dataset. The first feature dataset contains at least one first feature vector sequence, and the second feature dataset contains at least one second feature vector sequence. A third feature vector sequence with the same length as the first feature vector sequence is extracted from the second feature vector sequence. The cosine similarity between the third feature vector sequence and the first feature vector sequence is calculated to determine whether the audio segment to be retrieved contains audio keywords. This achieves the goal of a simple retrieval method with low computational complexity and high efficiency.

[0045] Please see Figure 2 This is a flowchart illustrating a second embodiment of the audio keyword retrieval method of the present invention. The difference from the first embodiment is that, before step S13, the method further includes:

[0046] Step S21: Obtain the length of the second feature vector sequence corresponding to any of the first feature vector sequences.

[0047] Specifically, when calculating similarity, the lengths of the vectors need to be consistent. Understandably, if a third feature vector sequence with the same length as the first feature vector sequence is to be extracted from the second feature vector sequence, the length of the second feature vector sequence must be greater than or equal to the length of the first feature vector sequence.

[0048] In this application, the first feature dataset includes a first feature vector sequence of the short-time average zero-crossing rate feature and a first feature vector sequence of the short-time average amplitude value feature. The second feature dataset includes a second feature vector sequence of the short-time average zero-crossing rate feature and a second feature vector sequence of the short-time average amplitude value feature. When performing length comparisons, the corresponding feature vector sequences need to be obtained first. That is, the lengths of the first feature vector sequence of the short-time average zero-crossing rate feature and the second feature vector sequence of the short-time average zero-crossing rate feature are compared, and the lengths of the first feature vector sequence of the short-time average amplitude value feature and the second feature vector sequence of the short-time average amplitude value feature are also compared.

[0049] Step S22: If the length of the second feature vector sequence is greater than or equal to the length of the first feature vector sequence, then the similarity between the third feature vector sequence and the first feature vector sequence is calculated to determine whether the audio segment to be retrieved contains the audio keyword.

[0050] In this embodiment, the cosine similarity between the third feature vector sequence and the first feature vector sequence is calculated to determine whether the audio segment to be retrieved contains audio keywords. If the length of the second feature vector sequence is greater than or equal to the length of the first feature vector sequence, the third feature vector sequence is extracted, and the cosine similarity between the third feature vector sequence and the first feature vector sequence is calculated. The cosine similarity value is then used to determine whether the audio segment to be retrieved contains audio keywords.

[0051] Step S23: If the length of the second feature vector sequence is less than the length of the first feature vector sequence, the audio segment to be retrieved does not contain audio keywords.

[0052] Specifically, if the length of the second feature vector sequence is less than the length of the first feature vector sequence, then it is impossible to obtain a third feature vector sequence with the same length as the first feature vector sequence from the second feature vector sequence, and thus it is impossible to calculate the similarity. In this case, it is determined that the audio segment to be retrieved does not contain audio keywords.

[0053] In this embodiment, similar to the first embodiment described above, preset audio segments containing audio keywords and the audio segment to be retrieved are characterized to obtain a first feature dataset and a second feature dataset. The first feature dataset contains at least one first feature vector sequence, and the second feature dataset contains at least one second feature vector sequence. The difference lies in that before extracting a third feature vector sequence with the same length as the first feature vector sequence from the second feature vector sequence, it is necessary to determine the length relationship between the second and first feature vector sequences. If the length of the second feature vector sequence is greater than or equal to the length of the first feature vector sequence, then a third feature vector sequence with the same length as the first feature vector sequence is extracted from the second feature vector sequence. The cosine similarity between the third feature vector sequence and the first feature vector sequence is calculated to determine whether the audio segment to be retrieved contains audio keywords. If the length of the second feature vector sequence is less than the length of the first feature vector sequence, then the audio segment to be retrieved is determined not to contain audio keywords. This achieves the goal of a simple retrieval method with low computational complexity and high efficiency.

[0054] Please see Figure 3 This is a flowchart illustrating the third embodiment of the audio keyword retrieval method of the present invention. Compared to the first embodiment, the difference lies in that step S11 includes:

[0055] Step S21: According to the set window duration and preset movement step size, the preset audio segment is divided into at least one preset audio segment of preset duration, and the first short-time average amplitude value and the first short-time average zero-crossing rate of any preset audio segment of preset duration are calculated to obtain the first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector. The first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector constitute the first feature vector sequence.

[0056] In this embodiment, a preset audio segment is segmented according to a set window duration and a preset movement step size to vectorize the preset audio segment. In this embodiment, the length of the preset audio segment can be 1 second, but it can also be of other lengths. For example, in one embodiment, the length of the preset audio segment is 1 second. In this case, the set window duration is 0.1 seconds and the movement step size is 0.025 seconds. When vectorizing the preset audio segment, the first segment is 0 to 0.1 seconds, the second segment is 0.025 to 0.125 seconds, the third segment is 0.05 to 0.15 seconds, and so on, until the last segment is 0.975 to 1 second. After obtaining multiple segments based on the window duration and movement step size, the first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector of each segment are calculated. For example, after vectorizing the preset audio segment, a segment of preset length is obtained, and the first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector of preset length are calculated. The first short-time average amplitude feature vector is denoted as Ve = (V1, V2, V3...VL), and the first short-time average zero-crossing rate feature vector is denoted as Ue = (U1, U2, U3...UL). Ve = (V1, V2, V3...VL) and Ue = (U1, U2, U3...UL) are both sequences of the first feature vector.

[0057] In this embodiment, the preset audio segments with audio keywords are characterized in this way, which can greatly reduce the amount of computation and is simple to implement.

[0058] Step S12 includes: Step S22: Segment the audio segment to be retrieved into at least one audio segment to be retrieved with a preset duration according to the window duration and the preset movement step size, and calculate the second short-time average amplitude value and the second short-time average zero-crossing rate of any audio segment to be retrieved with a preset duration to obtain the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector, wherein the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector constitute the second feature vector sequence.

[0059] In this embodiment, the audio segment to be retrieved is segmented according to the same window duration and movement step size as in step S21 to vectorize the audio segment. For example, in one embodiment, the length of the audio segment to be retrieved is 5s, the window duration is set to 0.1s, and the movement step size is set to 0.025s. When performing vector feature generation on the preset audio segment, the first segment is 0-0.1s, the second segment is 0.025-0.125s, the third segment is 0.05-0.15s, and so on, until the last segment is 4.975-5s. After obtaining multiple segments based on the window duration and movement step size, the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector are calculated for each segment. For example, if the audio segment to be retrieved is vectorized to obtain a segment of length S, then S second short-time average amplitude value feature vectors and second short-time average zero-crossing rate feature vectors are calculated. The second short-time average amplitude feature vector is denoted as Vue = (V1, V2, V3…VL…VS), and the second short-time average zero-crossing rate feature vector is denoted as Uue = (U1, U2, U3…UL…US). Both Vue = (V1, V2, V3…VL…VS) and Uue = (U1, U2, U3…UL…US) are sequences of the second feature vector.

[0060] In this embodiment, the audio segment to be retrieved is characterized in this way, which can greatly reduce the amount of computation and is simple to implement.

[0061] Step S13 includes: Step S33: Extract at least one third feature vector sequence from the second feature vector sequence with a length equal to the length of the first feature vector sequence, using a preset step length.

[0062] Specifically, in this embodiment, at least one third feature vector sequence with a length equal to the length of the first feature vector sequence is extracted from the second feature vector sequence using a preset step length. Specifically, if the first feature vector sequence is a first short-time average amplitude value feature vector, then the second short-time average amplitude value feature vector in the second feature vector sequence is extracted to form a third feature vector sequence with the same length as the first feature vector sequence.

[0063] If the first feature vector sequence is the first short-time average zero-crossing rate feature vector, then the second short-time average zero-crossing rate feature vector in the second feature vector sequence is truncated, and a third feature vector sequence with the same length as the first feature vector sequence is truncated.

[0064] In one specific embodiment, when the third feature vector sequence is obtained, it is truncated according to a preset step length. To ensure the accuracy of the retrieval, the second feature dataset is divided into multiple third feature datasets of length L, with a minimum unit length as the step length. Specifically, assuming the minimum unit length is 1, the step length is 1. The first third feature vector sequence with a preset length is SubVue = (V1, V2, V3…VL) and SubUue = (U1, U2, U3…UL), the second third feature vector sequence with a preset length is SubVue = (V2, V3, V4…VL, VL+1) and SubUue = (U2, U3, U4…UL, UL+1), and so on, until the nth third feature vector sequence with a preset length is SubVue = (…VS) and SubUue = (…US).

[0065] Please see Figure 4 This is a flowchart illustrating the fourth embodiment of the keyword retrieval method of the present invention, and is consistent with the above. Figure 1 Compared to the first embodiment shown, the difference lies in that step S14 specifically includes:

[0066] Step S40: Calculate the first similarity between the first short-time average amplitude value feature vector in any first feature vector sequence and the second short-time average amplitude value feature vector in the corresponding third feature vector sequence.

[0067] Specifically, the first short-time average amplitude feature vector in the first feature vector sequence is Ve = (V1, V2, V3...VL), and the second short-time average amplitude feature vector in the third feature vector sequence is SubVue = (V1, V2, V3...VL), (V2, V3, V4...VL, VL+1), (...VS). During this process, the first similarity between the first short-time average amplitude feature vector and any second short-time average amplitude feature vector in the third feature vector sequence is calculated. For example, in one embodiment, the first similarity is cosine(Ve, SubVue).

[0068] Step S41: Calculate the second similarity between the first short-time average zero-crossing rate feature vector in any first feature vector sequence and the second short-time average zero-crossing rate feature vector in the corresponding third feature vector sequence.

[0069] Specifically, the first short-time average zero-crossing rate eigenvector is denoted as Ue = (U1, U2, U3...UL), and the second short-time average zero-crossing rate eigenvector in the third eigenvector sequence is denoted as SubUue = (U1, U2, U3...UL), SubUue = (U2, U3, U4...UL, UL+1), and SubUue = (...US). During this process, the second similarity between the first short-time average zero-crossing rate eigenvector and any second short-time average zero-crossing rate eigenvector in the third eigenvector sequence is calculated. For example, in one embodiment, the second similarity is cosine(Ve, SubVue).

[0070] Step S42: Calculate the product of the first similarity and the second similarity to obtain the similarity value.

[0071] Specifically, the similarity value is the first similarity multiplied by the second similarity, i.e., cosine(Ve, SubVue)×cosine(Ve, SubVue).

[0072] Step S43: Determine whether the similarity value is greater than or equal to the preset threshold.

[0073] Specifically, it determines whether the value of cosine(Ve, SubVue) × cosine(Ve, SubVue) is greater than or equal to a preset threshold, and determines whether the audio segment to be searched contains audio keywords based on the judgment result.

[0074] If the similarity value is greater than or equal to the preset threshold, then step S44: The audio segment to be retrieved contains audio keywords.

[0075] After determining that the audio segment to be retrieved contains audio keywords, it is also necessary to confirm the location of the audio keywords. For details, please refer to step S45: Record the offset value of the third feature vector sequence.

[0076] In this embodiment, if there are a total of 3 extracted third feature vector sequences, since the location of the audio keyword needs to be confirmed starting from the start time, the three third feature vector sequences are represented by offset values ​​of 0, 1, and 2. If the audio keyword is retrieved when searching the second third feature vector sequence, the offset value is recorded as 1. If there are a total of n extracted third feature vector sequences, the n third feature vector sequences are represented by offset values ​​of 0, 1, 2, 3, ..., n-2, n-1. If the audio keyword is retrieved when searching the nth third feature vector sequence, the offset value is recorded as n-1.

[0077] Step S46: Determine the temporal position of the audio keyword in the audio segment to be retrieved based on the offset value and the moving step size of the third feature vector sequence.

[0078] In this application, continuing the example above, when characterizing the preset audio segment in steps S21 and S22, the set movement step size is 0.025s. If an audio keyword is found when searching the second third feature vector sequence, the time position of the audio keyword in the audio segment to be searched is obtained by multiplying the movement step size by the offset value. That is, 1 × 0.025 = 0.025, so the audio keyword is at 0.025 seconds in the audio segment to be searched. If an audio keyword is found when searching the nth third feature vector sequence, the time position of the audio keyword in the audio segment to be searched is obtained by multiplying the movement step size by the offset value. That is, (n-1) × 0.5, so the audio keyword is at (n-1) × 0.5 seconds in the audio segment to be searched.

[0079] If the similarity value is less than the preset threshold, then step S47: The audio segment to be retrieved does not contain audio keywords.

[0080] Specifically, to confirm that the audio segment to be retrieved does not contain audio keywords, step S48 is required: determine whether the third feature vector sequence is the last feature vector sequence in the second feature vector sequence.

[0081] Specifically, it determines whether the third feature vector sequence extracted from the second feature vector sequence has reached its last value. If so, step S49 states: the audio segment to be retrieved does not contain audio keywords. If it is determined that the third feature vector sequence has not reached its last value, then step S50 states: return to step S14 and recalculate the similarity between the first and third feature vector sequences. In other words, if it is determined that the currently calculated second third feature vector sequence is not the last feature vector sequence, then return to step S14 and begin calculating the similarity between the third third feature vector sequence and the first feature vector sequence. If the calculated result contains audio keywords, the retrieval is complete. If no result contains audio keywords, then the next third feature vector sequence is calculated again until the last value is reached.

[0082] The retrieval method described above, after feature processing, greatly reduces the computational steps. It is simple to implement, has low computational complexity, is highly efficient and fast, and is suitable for the needs of audio retrieval application scenarios.

[0083] Specifically, please see Figure 5 This is a flowchart illustrating the intelligent outbound calling method of the present invention. Specifically, it includes:

[0084] Step S51: Receive early media audio from the communication server as an audio segment to be retrieved. The audio segment to be retrieved is used to provide feedback on the current communication status of the automatic outbound call user's communication terminal.

[0085] Specifically, after the intelligent outbound calling service dials a target phone number, it receives early media audio returned by the called phone from the communication server. This early media audio is a voice segment reflecting the status of the called party. In this embodiment, the communication server is a device connected to FreeSwitch, i.e., a voice gateway. For example, if the other party is already on a call, the caller will receive a voice segment such as "Sorry, the number you dialed is currently busy, please try again later."

[0086] Step S52: Retrieve the audio segment to be retrieved using the keyword retrieval method and obtain the retrieval results.

[0087] After the caller receives the early media audio, it uses the early media audio as the audio segment to be retrieved. According to preset rules, the audio segment to be retrieved is characterized to obtain a second feature dataset. The second feature dataset contains at least one second feature vector sequence. In this embodiment, the caller typically has a preset audio segment containing at least one audio keyword that reflects the status of the called party, such as "phone off," "in a call," or "service paused." The caller's terminal performs feature processing on the preset audio segment to obtain a first feature dataset, which includes at least one first feature vector sequence.

[0088] Specifically, if the preset rules set the window duration to 0.1 seconds and the movement step size to 0.025 seconds, when performing vector feature generation on the preset audio segments, the first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector for each segment are obtained. The first short-time average amplitude value feature vector is denoted as Ve = (V1, V2, V3…VL), and the first short-time average zero-crossing rate feature vector is denoted as Ue = (U1, U2, U3…UL). The caller performs feature generation on the audio segments to be retrieved (i.e., early media audio) in the same manner, calculating the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector for each segment. The second short-time average amplitude value feature vector is denoted as Vue = (V1, V2, V3…VL…VS), and the second short-time average zero-crossing rate feature vector is denoted as Uue = (U1, U2, U3…UL…US).

[0089] When determining whether an audio segment to be retrieved contains keywords, it is necessary to compare the similarity between the audio segment to be retrieved and the preset audio segment. When performing the similarity comparison, it is necessary to extract at least one third feature vector sequence with a length equal to the length of the first feature vector sequence from the second feature vector sequence. Therefore, the length of the second feature dataset needs to be greater than the length of the first feature dataset.

[0090] Specifically, with a preset step length, at least one third feature vector sequence with a length equal to the length of the first feature vector sequence is extracted from the second feature vector sequence. For example, assuming the minimum unit length is 1, the step length is 1. The first extracted third feature vector sequence with a preset length is SubVue = (V1, V2, V3…VL) and SubUue = (U1, U2, U3…UL), the second is SubVue = (V2, V3, V4…VL, VL+1) and SubUue = (U2, U3, U4…UL, UL+1), and so on, until the nth third feature vector sequence with a preset length is SubVue = (…VS) and SubUue = (…US).

[0091] Calculate the first similarity between the first short-time average amplitude feature vector and the second short-time average amplitude feature vector in any third feature vector sequence. For example, in one embodiment, the first similarity = cosine(Ve, SubVue); calculate the second similarity between the first short-time average zero-crossing rate feature vector and the second short-time average zero-crossing rate feature vector in any third feature vector sequence. For example, in one embodiment, the second similarity = cosine(Ve, SubVue); then calculate the product of the first similarity and the second similarity to obtain the similarity value, i.e., cosine(Ve, SubVue) × cosine(Ve, SubVue).

[0092] If the similarity value is greater than the preset threshold, it indicates that the early media audio contained audio keywords.

[0093] Step S53: Based on the search results, confirm the current communication status of the communication terminal.

[0094] For example, if the audio keyword "power off" is detected from early media audio, it means that the called party's phone is powered off. If the keyword "currently on a call" is found, it means that the called party is currently on a call. If the keyword "service suspended" is found, it means that the called party's service has been suspended.

[0095] Step S54: Determine the automatic outbound calling strategy based on the current communication status.

[0096] If no keywords are found using the keyword search method described above, it indicates that the recipient is using the service normally or their status is unknown. The caller can choose to continue calling or not. If audio keywords are found, it indicates that the recipient is unable to receive telephone signals, such as being "on a call" or "phone off." The caller can adjust their calling strategy based on the prompts.

[0097] Please see Figure 6This is a schematic diagram of the structure of the first embodiment of the audio keyword retrieval device of the present invention. It includes: a first feature processing module 61, a second feature processing module 62, a feature extraction module 63, and a similarity comparison module 64.

[0098] The first feature processing module 61 is used to perform feature processing on a preset audio segment according to a preset rule to obtain a first feature dataset. The first feature dataset contains at least one first feature vector sequence, wherein the preset audio contains audio keywords. Specifically, the first feature processing module 61 is also used to divide the preset audio segment into at least one preset audio segment of preset duration according to a set window duration and a preset movement step size, and calculate the first short-time average amplitude value and the first short-time average zero-crossing rate of any preset audio segment of preset duration to obtain the first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector. The first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector constitute the first feature vector sequence.

[0099] The second feature processing module 62 is used to acquire the audio segment to be retrieved and perform feature processing on the audio segment to be retrieved according to the preset rules to obtain a second feature dataset. The second feature dataset contains at least one second feature vector sequence. Specifically, the second feature processing module 62 is also used to segment the audio segment to be retrieved into at least one audio segment to be retrieved with a preset duration according to the window duration and the preset movement step size, and calculate the second short-time average amplitude value and the second short-time average zero-crossing rate of any audio segment to be retrieved with a preset duration to obtain the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector. The second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector constitute the second feature vector sequence.

[0100] The feature extraction module 63 is used to extract a third feature vector sequence with the same length as the first feature vector sequence from the second feature vector sequence corresponding to the first feature vector sequence. Specifically, the feature extraction module 63 obtains the length of the second feature vector sequence corresponding to any first feature vector sequence. If the length of the second feature vector sequence is greater than or equal to the length of the first feature vector sequence, the feature extraction module 63 extracts at least one third feature vector sequence with a length equal to the length of the first feature vector sequence from the second feature vector sequence with a preset step length.

[0101] The similarity comparison module 64 is used to determine whether the audio segment to be retrieved contains the audio keyword by calculating the similarity between the third feature vector sequence and the first feature vector sequence. Specifically, the similarity comparison module 64 is used to calculate the first similarity between the first short-time average amplitude value feature vector in any first feature vector sequence and the second short-time average amplitude value feature vector in the corresponding third feature vector sequence; calculate the second similarity between the first short-time average zero-crossing rate feature vector in any first feature vector sequence and the first short-time average zero-crossing rate feature vector in the corresponding third feature vector sequence; calculate the product of the first similarity and the second similarity to obtain the similarity value, and determine whether the similarity value is greater than or equal to a preset threshold. If the similarity value is greater than or equal to the preset threshold, the audio segment to be retrieved contains the audio keyword; if the similarity value is less than the preset threshold, the audio segment to be retrieved does not contain the audio keyword.

[0102] Furthermore, the similarity comparison module 64 is also used to record the offset value of the third feature vector sequence; and to determine the temporal position of the audio keyword in the audio segment to be retrieved based on the offset value and the moving step size of the third feature vector sequence. It is then determined whether the third feature vector sequence is the last feature vector sequence in the second feature vector sequence. If so, the audio segment to be retrieved does not contain the audio keyword; otherwise, the similarity between the third feature vector sequence and the first feature vector sequence is calculated.

[0103] Please see Figure 7 This is a schematic diagram of the structure of an embodiment of the intelligent outbound calling device of the present invention. It includes: a receiving module 71, a keyword retrieval module 72, a status detection module 73, and an execution module 74.

[0104] The receiving module 71 receives early media audio from the communication server as an audio segment to be retrieved. This audio segment is used to reflect the current communication status of the automatic outbound calling user's communication terminal. The keyword retrieval module 72 retrieves the audio segment to be retrieved according to any of the keyword retrieval methods described above and obtains the retrieval results. The status detection module 73 determines the current communication status of the communication terminal based on the retrieval results. Specifically, if the retrieval results contain audio keywords, the called party is currently in the state indicated by the keywords; if no audio keywords are found, the current status of the called party is unknown. The execution module 74 determines the automatic outbound calling strategy based on the determined current communication status. If the called party is currently in the state indicated by the keywords, the execution module 74 executes a pause call command or a continue call command.

[0105] Please see Figure 8 This is a schematic diagram of the structure of a second embodiment of the audio keyword retrieval device of the present invention. The audio keyword retrieval device includes a memory 81 and a processor 82 connected to each other.

[0106] Memory 81 is used to store program instructions.

[0107] The processor 82 is used to retrieve program instructions stored in the memory 81 to execute the audio keyword retrieval method as described in any of the above.

[0108] The processor 82 can also be referred to as a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0109] The memory 81 can be a memory module, TF card, etc., and can store all information in the audio keyword retrieval device, including the input raw data, computer program, intermediate running results, and final running results. It stores and retrieves information according to the location specified by the controller. With the memory, the audio keyword retrieval device has a memory function and can ensure normal operation. The memory in the audio keyword retrieval device can be classified according to its purpose into main memory (RAM) and auxiliary memory (external storage), or it can be classified into external memory and internal memory. External storage is usually magnetic media or optical discs, which can store information for a long time. RAM refers to the storage components on the motherboard, used to store currently executing data and programs, but it is only used for temporary storage of programs and data; the data will be lost when the power is turned off.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, system server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application.

[0114] Please see Figure 9 This is a schematic diagram of the structure of the storage medium of the present invention. The storage medium of this application stores a program file 91 capable of implementing all the above-mentioned intelligent input methods. The program file 91 can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage device includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.

[0115] The above are merely embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for retrieving audio keywords, characterized in that, include: A first feature dataset is obtained by performing feature processing on a preset audio segment according to a preset rule. The first feature dataset contains at least one first feature vector sequence, wherein the preset audio contains audio keywords. The audio segment to be retrieved is obtained, and the audio segment to be retrieved is characterized according to the preset rules to obtain a second feature dataset. The second feature dataset contains at least one second feature vector sequence. The characteristic processing includes segmenting the audio segment to be retrieved according to the window duration and the preset movement step to obtain at least one audio segment to be retrieved with a preset duration, and calculating the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector. Extract a third feature vector sequence of the same length as the first feature vector sequence from the second feature vector sequence; The similarity between the third feature vector sequence and the first feature vector sequence is calculated to determine whether the audio segment to be retrieved contains the audio keywords. The step of determining whether the audio segment to be retrieved contains the audio keyword by calculating the similarity between the third feature vector sequence and the first feature vector sequence specifically includes: Calculate the first similarity between the first short-time average amplitude value feature vector in any of the first feature vector sequences and the second short-time average amplitude value feature vector in the corresponding third feature vector sequence; Calculate the second similarity between the first short-time average zero-crossing rate feature vector in any of the first feature vector sequences and the second short-time average zero-crossing rate feature vector in the corresponding third feature vector sequence; Calculate the product of the first similarity and the second similarity to obtain the value of the similarity, and determine whether the value of the similarity is greater than or equal to a preset threshold.

2. The keyword retrieval method according to claim 1, characterized in that, Before the step of extracting a third feature vector sequence from the second feature vector sequence that has the same length as the first feature vector sequence, the method further includes: Obtain the length of the second feature vector sequence corresponding to any of the first feature vector sequences; If the length of the second feature vector sequence is greater than or equal to the length of the first feature vector sequence, then the similarity between the third feature vector sequence and the first feature vector sequence is calculated to determine whether the audio segment to be retrieved contains the audio keyword. If the length of the second feature vector sequence is less than the length of the first feature vector sequence, then the audio segment to be retrieved does not contain the audio keyword.

3. The keyword retrieval method according to claim 2, characterized in that, The step of performing feature processing on a preset audio segment according to a preset rule to obtain a first feature dataset, wherein the first feature dataset contains at least one first feature vector sequence, and wherein the preset audio segment contains audio keywords, specifically includes: The preset audio segment is divided into at least one preset audio segment of preset duration according to the set window duration and preset movement step size, and the first short-time average amplitude value and the first short-time average zero-crossing rate of any preset audio segment of preset duration are calculated to obtain the first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector. The first short-time average amplitude value feature vector and the first short-time average zero-crossing rate feature vector constitute the first feature vector sequence. The steps of obtaining the audio segment to be retrieved and performing feature processing on the audio segment to be retrieved according to the preset rules to obtain a second feature dataset, wherein the second feature dataset contains at least one second feature vector sequence, specifically include: The audio segment to be retrieved is segmented into at least one audio segment of a preset duration according to the window duration and the preset movement step size. The second short-time average amplitude value and the second short-time average zero-crossing rate of any audio segment of the preset duration are calculated to obtain the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector. The second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector constitute the second feature vector sequence.

4. The retrieval method according to claim 3, characterized in that, The step of extracting a third feature vector sequence from the second feature vector sequence that has the same length as the first feature vector sequence includes: At a preset step length, at least one third feature vector sequence with a length equal to the length of the first feature vector sequence is extracted from the second feature vector sequence.

5. The retrieval method according to claim 4, characterized in that, The step of determining whether the audio segment to be retrieved contains the audio keyword by calculating the similarity between the third feature vector sequence and the first feature vector sequence specifically includes: Determine whether the similarity value is greater than or equal to a preset threshold; If the similarity value is greater than or equal to the preset threshold, then the audio segment to be retrieved contains the audio keywords; If the similarity value is less than the preset threshold, then the audio segment to be retrieved does not contain the audio keywords.

6. The retrieval method according to claim 5, characterized in that, The step of determining that if the similarity value is greater than or equal to the preset threshold, the audio segment to be retrieved contains the audio keyword further includes: Record the offset value of the third feature vector sequence; The temporal position of the audio keyword in the audio segment to be retrieved is determined based on the offset value of the third feature vector sequence and the preset movement step size. The step of determining that if the similarity value is less than the preset threshold, the audio segment to be retrieved does not contain the audio keyword further includes: Determine whether the third feature vector sequence is the last feature vector sequence in the second feature vector sequence. If so, the audio segment to be retrieved does not contain the audio keyword. If not, then continue with the step of calculating the similarity between the third feature vector sequence and the first feature vector sequence.

7. An intelligent outbound calling method, characterized in that, include: Receive early media audio from the communication server as an audio segment to be retrieved. The audio segment to be retrieved is used to provide feedback on the current communication status of the automatic outbound call user's communication terminal. The keyword retrieval method according to any one of claims 1-6 retrieves the audio segment to be retrieved and obtains the retrieval results; Based on the search results, determine the current communication status of the communication terminal; Based on the determined current communication status, an automatic outbound calling strategy is determined.

8. An audio keyword retrieval device, characterized in that, include: The first feature processing module is used to perform feature processing on a preset audio segment according to a preset rule to obtain a first feature dataset. The first feature dataset contains at least one first feature vector sequence, wherein the preset audio contains audio keywords. The second feature processing module is used to acquire the audio segment to be retrieved and perform feature processing on the audio segment to be retrieved according to the preset rules to obtain a second feature dataset. The second feature dataset contains at least one second feature vector sequence. The feature processing includes segmenting the audio segment to be retrieved according to the window duration and the preset movement step to obtain at least one audio segment to be retrieved with a preset duration, and calculating the second short-time average amplitude value feature vector and the second short-time average zero-crossing rate feature vector. The feature extraction module is used to extract a third feature vector sequence from the second feature vector sequence, which has the same length as the first feature vector sequence. The similarity calculation module is used to determine whether the audio segment to be retrieved contains the audio keywords by calculating the similarity between the third feature vector sequence and the first feature vector sequence; The similarity calculation module is further configured to calculate the first similarity between a first short-time average amplitude value feature vector in any first feature vector sequence and a second short-time average amplitude value feature vector in the corresponding third feature vector sequence; calculate the second similarity between a first short-time average zero-crossing rate feature vector in any first feature vector sequence and a second short-time average zero-crossing rate feature vector in the corresponding third feature vector sequence; calculate the product of the first similarity and the second similarity to obtain the similarity value, and determine whether the similarity value is greater than or equal to a preset threshold.

9. An intelligent outbound calling device, characterized in that, include: The receiving module is used to receive early media audio from the communication server as an audio segment to be retrieved. The audio segment to be retrieved is used to provide feedback on the current communication status of the automatic outbound call user's communication terminal. The keyword retrieval module is used to retrieve the audio segment to be retrieved by the keyword retrieval method according to any one of claims 1-6, and obtain retrieval results; A status detection module is used to determine the current communication status of the communication terminal based on the search results; The execution module is used to determine the automatic outbound calling strategy based on the determined current communication status.

10. An audio keyword retrieval device, characterized in that, include: Intercoupled processors and memory, among which, The memory stores program instructions; The processor is used to retrieve the program instructions stored in the memory to execute the audio keyword retrieval method as described in any one of claims 1-6 and the intelligent outbound calling method as described in claim 7.

11. A storage medium, characterized in that, The system contains program files that can be executed to implement the audio keyword retrieval method as described in any one of claims 1-6 and the intelligent outbound calling method as described in claim 7.

Images