Intelligent audio audit method and system based on speech recognition

Abstract

The present invention relates to the technical field of audio auditing, and in particular to an intelligent audio auditing method and system based on speech recognition. First, a sensitive pinyin group and a similar pinyin group corresponding to a sensitive word set are determined, and then speech recognition is performed to obtain a corresponding text segment. Then, the text segment is split and reorganized to obtain a text phrase, and a text pinyin group of the text phrase is obtained. Then, it is determined whether there is a text pinyin group consistent with the sensitive pinyin group or the similar pinyin group. If so, the text pinyin group consistent with the sensitive pinyin group or the similar pinyin group is marked as a target pinyin group, and the target pinyin group and corresponding position information are sent to an audit terminal for review, thereby ensuring the audit accuracy. Auditors only need to review the target pinyin group and the corresponding position information. The audit efficiency is improved, and the pinyin comparison method is used to make the audit accuracy higher.

Description

Technical Field

[0001] The present invention relates to the technical field of audio auditing, and in particular to an intelligent audio auditing method and system based on speech recognition. Background Art

[0002] With the continuous development of information technology, Internet audio live broadcast and on-demand services have attracted widespread attention for their novel forms and rich content. However, the audio information in audio services may be mixed with illegal content, so effective review and supervision of audio information is required.

[0003] In the prior art, the review and supervision of audio information is mainly manual review, that is, reviewers listen to the audio information. Although this method can ensure the accuracy of the review, it is very time-consuming and labor-intensive, and it is difficult to conduct efficient and accurate review of the massive amount of audio information on the Internet. Summary of the invention

[0004] The main purpose of the present invention is to provide an intelligent audio review method and system based on speech recognition, aiming to solve the problem that the existing audio information review scheme is time-consuming and labor-intensive, and it is difficult to conduct efficient and accurate review of the massive amount of audio information on the Internet.

[0005] The technical solution proposed by the present invention is:

[0006] An intelligent audio audit method based on speech recognition is applied to an intelligent audio audit system based on speech recognition; the system comprises a server and an audit terminal which are connected to each other in communication; the server stores a sensitive word set, the sensitive word set comprises a plurality of sensitive words which are different from each other, and each sensitive word comprises at least 2 characters; the method comprises:

[0007] The server determines a sensitive pinyin group and a similar pinyin group corresponding to the sensitive word set, wherein the sensitive pinyin group is a pinyin group of each sensitive word in the sensitive word set, and the similar pinyin group is a pinyin group with a pronunciation similar to the sensitive pinyin group;

[0008] After the audit terminal obtains the audio to be audited, it sends the audio to be audited to the server;

[0009] The server performs speech recognition on the received audio to be reviewed to obtain the corresponding text segment;

[0010] The server splits and reorganizes the text segment based on the sensitive pinyin group to obtain a text phrase, and marks the pinyin of the text phrase as a text pinyin group;

[0011] The server determines whether there is a text pinyin group that is consistent with the sensitive pinyin group or the similar pinyin group;

[0012] If so, the server marks the text pinyin group that is consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, generates location information for describing the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed, sends the target pinyin group and the corresponding location information to the review terminal, and marks the audio to be reviewed as reviewed audio;

[0013] If not, the server generates review pass information to indicate that the audio to be reviewed does not contain sensitive words, sends the review pass information to the review terminal, and marks the audio to be reviewed as reviewed audio.

[0014] Preferably, the server performs speech recognition on the received audio to be reviewed to obtain a corresponding text segment, including:

[0015] The server determines whether the received audio to be reviewed meets the segmentation condition: the duration of the audio to be reviewed is greater than the segmentation duration;

[0016] If the requirements are met, the server divides the audio to be reviewed to obtain multiple audio segments;

[0017] The server performs speech recognition on the audio segment to obtain multiple text segments;

[0018] If not satisfied, the server performs speech recognition on the audio to be reviewed to obtain a text segment.

[0019] Preferably, the server splits and reorganizes the text segments based on the sensitive pinyin groups to obtain text phrases, and marks the pinyin of the text phrases as text pinyin groups, including:

[0020] The server obtains the total number of characters S in the text segment;

[0021] The server obtains the most sensitive words, and marks the most sensitive words as the longest word number C;

[0022] The server sets N=1;

[0023] The server sets i=1;

[0024] The server combines the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a character phrase;

[0025] The server sets i=i+1 and determines whether i+N≥S holds;

[0026] If so, let N=N+1, and determine whether N≥C holds;

[0027] If N ≥ C, obtain the pinyin of all the obtained text phrases and use them as text pinyin groups;

[0028] If N≥C is not true, execute the server to set i=1 and the following steps;

[0029] If not, the server is executed to combine the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a character phrase, and the following steps.

[0030] Preferably, it also includes:

[0031] The server marks the moment when the audit terminal receives the audited audio as the first moment, marks the moment when the audit terminal receives the target pinyin group and the corresponding position information, or the audit pass information as the second moment, and uses the duration between the first moment and the second moment as the audit duration corresponding to the audited audio;

[0032] The server establishes a review time estimation model;

[0033] The server obtains historical review information corresponding to all reviewed audios within a preset time period in the past, wherein the historical review information includes the duration of the reviewed audios, the total number of words in all text segments corresponding to the reviewed audios, and the review duration corresponding to the reviewed audios;

[0034] The server uses the duration of the audited audio in the historical audit information and the total number of words in all text segments corresponding to the audited audio as input parameters of the audit time estimation model, and uses the audit time corresponding to the audited audio in the historical audit information as the output parameter of the audit time estimation model for training, so as to obtain a fully trained audit time estimation model;

[0035] The server obtains the duration of the audio to be reviewed and the total number of characters in all text segments corresponding to the audio to be reviewed, and inputs the trained review duration estimation model to obtain the output estimated review duration corresponding to the audio to be reviewed;

[0036] The server sends the estimated review time corresponding to the audio to be reviewed to the review terminal.

[0037] Preferably, if the above conditions are met, the server segments the audio to be audited to obtain multiple audio segments, including:

[0038] The server determines the overlap duration based on the longest word number C , where the overlap duration Less than split time ;

[0039] The server determines the start and end times of each audio segment, including:

[0040] The server sets j=1;

[0041] The server uses the start time of the audio to be reviewed as the start time of the jth audio segment, and the interval between the start time of the jth audio segment and the start time of the jth audio segment is the split duration. The moment of is taken as the end moment of the jth audio segment;

[0042] The server sets j=j+1;

[0043] The server will be before the end time of the j-1th audio segment, and the interval between the end time of the j-1th audio segment and the end time of the j-1th audio segment will be the overlap time. The moment of is taken as the starting moment of the jth audio segment;

[0044] The server determines whether the interval between the end time of the audio to be reviewed and the start time of the jth audio segment is less than or equal to the split time. ;

[0045] If yes, the server uses the end time of the audio to be reviewed as the end time of the jth audio segment;

[0046] If not, the server will be after the start time of the jth audio segment, and the interval between the start time of the jth audio segment is the split duration. As the end time of the j-th audio segment, and execute the server setting j=j+1, and the subsequent steps;

[0047] The server divides the audio to be reviewed based on the start time and the end time of each audio segment to obtain multiple audio segments.

[0048] Preferably, the server determines the overlap duration based on the longest word number C ,include:

[0049] The server obtains the normal reading speed , and based on normal reading speed Calculate the overlap time with the longest word C :

[0050] ,

[0051] In the formula, is the overlap duration; A is the first redundant duration, which takes a positive value.

[0052] Preferably, the generating of the position information for expressing the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed includes:

[0053] The server marks the text segment corresponding to the text phrase to which the target pinyin group belongs as the target text segment;

[0054] The server obtains the sequence number of the first character in the character phrase to which the target pinyin group belongs in the target text segment , and the sequence number of the last character in the character phrase to which the target pinyin group belongs in the target text segment ;

[0055] The server uses the duration between the estimated appearance time of the first character in the character phrase to which the target pinyin group belongs in the audio to be reviewed and the start time of the audio to be reviewed as the first duration;

[0056] The server uses the estimated duration of the text phrase to which the target pinyin group belongs in the audio to be reviewed as the second duration, wherein the calculation formula for the second duration is:

[0057] ,

[0058] In the formula, is the second duration; B is the second redundant duration, which takes a positive value; is the unit reading speed;

[0059] The server determines whether the audio to be reviewed corresponding to the target text segment includes multiple audio segments;

[0060] If yes, the server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be reviewed. ;

[0061] The server calculates the first duration according to the following formula:

[0062] ,

[0063] In the formula, For the first duration; The total number of characters in the target text segment of the target pinyin group;

[0064] If not, the server calculates the first duration according to the following formula:

[0065] ,

[0066] The server uses the first duration and the second duration as position information for expressing the position of the text phrase belonging to the target pinyin group in the audio to be reviewed.

[0067] Preferably, the server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be reviewed. , and later:

[0068] The total number of characters in the target text segment based on the target pinyin group of the server and split duration Calculate the average reading speed of the target audio segment of the target pinyin group;

[0069] ,

[0070] In the formula, The average reading speed of the target audio segment for the target pinyin group;

[0071] The server determines whether the following conditions are met: the ratio of the average reading speed of the target audio segment of the target pinyin group to the normal reading speed is less than a preset lower limit;

[0072] If yes, the server divides the target audio segment of the target pinyin group from the middle time to obtain a first audio segment in front and a second audio segment in the back;

[0073] The server performs speech recognition on the first audio segment to obtain a corresponding first text segment, and performs speech recognition on the second audio segment to obtain a corresponding second text segment;

[0074] The server based on the total number of characters in the first text segment and the total number of characters in the second text segment Calculate the first duration.

[0075] Preferably, the server is based on the total number of characters in the first text segment. and the total number of characters in the second text segment Calculate the first duration, including:

[0076] If the total number of characters in the first text segment is 0, the server calculates the first duration according to the following formula:

[0077] ,

[0078] If the total number of characters in the second text segment is 0, the server calculates the first duration according to the following formula:

[0079] ,

[0080] If the total number of characters in the first text segment and the total number of characters in the second text segment are both greater than 0, the server calculates the first duration according to the following formula:

[0081] .

[0082] The present invention also proposes an intelligent audio audit system based on speech recognition, which applies an intelligent audio audit method based on speech recognition; the system includes a server and an audit terminal that are communicatively connected to each other; the server stores multiple sensitive word sets, the sensitive word sets include multiple sensitive words that are different from each other, and each sensitive word includes at least 2 characters.

[0083] Through the above technical solution, the following beneficial effects can be achieved:

[0084] The intelligent audio audit method based on speech recognition proposed in the present invention can audit the audio for sensitive words more efficiently and accurately; first determine the sensitive pinyin group and similar pinyin group corresponding to the sensitive word set, and then perform speech recognition on the received audio to be audited to obtain the corresponding text segment; then split and reorganize the text segment to obtain the text phrase, and obtain the text pinyin group of the text phrase; then compare the text pinyin group with the sensitive pinyin group and the similar audio group to determine whether there is a text pinyin group consistent with the sensitive pinyin group or the similar pinyin group; if so, it means that there is a text phrase that is consistent or similar in pronunciation to the sensitive word, indicating that the sensitive word may appear in the audio to be audited, then directly mark the text pinyin group consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, and then send the target pinyin group and the corresponding position information to the audit terminal, so that the operator of the audit terminal can review the recognition result, and further ensure the audit accuracy; this method no longer requires the auditor to manually listen to all the voices of the audio to be audited, and only needs to review the target pinyin group and the corresponding position information; this scheme improves the audit efficiency, and the pinyin comparison method makes the audit accuracy higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0086] Figure 1 A schematic diagram of the process steps of the first embodiment of an intelligent audio review method based on speech recognition proposed by the present invention. DETAILED DESCRIPTION

[0087] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0088] The present invention provides an intelligent audio audit method and system based on speech recognition.

[0089] As attached Figure 1As shown in the first embodiment of an intelligent audio review method based on speech recognition proposed by the present invention, this method is applied to an intelligent audio review system based on speech recognition; the system includes a server and a review terminal (a terminal operated by a reviewer) that are communicatively connected to each other; the server stores a set of sensitive words, the set of sensitive words includes a plurality of different sensitive words, and each sensitive word includes at least 2 characters; this embodiment includes the following steps:

[0090] Step S110: The server determines a sensitive pinyin group and a similar pinyin group corresponding to the set of sensitive words, where the sensitive pinyin group is the pinyin group of each sensitive word in the set of sensitive words, and the similar pinyin group is a pinyin group with a pronunciation similar to that of the sensitive pinyin group.

[0091] Specifically, for example, the set of sensitive words includes the following sensitive words "A C" (not an actual sensitive word, only for example) and "A B C D"; then the corresponding sensitive pinyin groups are "jiabing" and "jiayibingding", and correspondingly, the similar pinyin groups of the sensitive pinyin group "jiabing" include but are not limited to "jiabin", "jiapin", and "gabin"; the similar pinyin groups of the sensitive pinyin group "jiayibingding" include but are not limited to "jiayibinding", "gayibingding", and "jiaxibingding".

[0092] Step S120: After the review terminal obtains the audio to be reviewed, it sends the audio to be reviewed to the server.

[0093] Step S130: The server performs speech recognition on the received audio to be reviewed to obtain the corresponding text segment.

[0094] Specifically, the text segment here is the corresponding text segment formed after the speech in the audio to be reviewed is converted.

[0095] Step S140: The server splits and reorganizes the text segment based on the sensitive pinyin group to obtain text phrases, and marks the pinyin of the text phrases as the text pinyin group.

[0096] Specifically, after splitting the text segment, all possible combinations of text phrases in the text segment can be obtained, and these text phrases are the phrases that need to be compared with the sensitive words.

[0097] Step S150: The server determines whether there is a text pinyin group that is the same as the sensitive pinyin group or the similar pinyin group.

[0098] Specifically, during the speech recognition process, each character in the recognized text segment may differ from the actual speech in the audio to be reviewed. For example, "door god" may be recognized as "muffled sound", etc. Therefore, directly comparing the character phrases split from the text segment obtained by speech recognition with sensitive words will result in errors. In this embodiment, instead of directly comparing the character phrases with sensitive words, the pinyin groups of the character phrases are compared with the sensitive pinyin groups, which can more accurately identify sensitive words and thus improve the review accuracy. In addition, comparing the pinyin groups of the character phrases with similar pinyin groups can conduct a more comprehensive review of the audio to be reviewed, avoiding missing sensitive words due to similar pronunciations, thereby ensuring the final review effect.

[0099] If so, execute step S160: The server marks the pinyin group of the character that is consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, generates position information for expressing the position of the character phrase to which the target pinyin group belongs in the audio to be reviewed, sends the target pinyin group and the corresponding position information to the review terminal, and marks the audio to be reviewed as the reviewed audio.

[0100] Specifically, sending the target pinyin group and the corresponding position information to the review terminal facilitates the operator of the review terminal to review the recognition result, further ensuring the review accuracy.

[0101] If not, execute step S170: The server generates a review passed information for expressing that the audio to be reviewed does not contain sensitive words, sends the review passed information to the review terminal, and marks the audio to be reviewed as the reviewed audio.

[0102] The intelligent audio review method based on speech recognition proposed by the present invention can more efficiently and accurately review sensitive words in the audio. First, determine the sensitive pinyin groups and similar pinyin groups corresponding to the sensitive word set, and then perform speech recognition on the received audio to be reviewed to obtain the corresponding text segment. Then, split and reorganize the text segment to obtain character phrases, and obtain the pinyin groups of the character phrases. After that, compare the pinyin groups of the character phrases with the sensitive pinyin groups and similar audio groups to determine whether there is a pinyin group of the character that is consistent with the sensitive pinyin group or the similar pinyin group. If so, it means that there is a character phrase with the same or similar pronunciation as the sensitive word, indicating that a sensitive word may appear in the audio to be reviewed. Then, directly mark the pinyin group of the character that is consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, and then send the target pinyin group and the corresponding position information to the review terminal, facilitating the operator of the review terminal to review the recognition result, further ensuring the review accuracy. This method no longer requires the reviewer to manually listen to all the speech of the audio to be reviewed, and only needs to review the target pinyin group and the corresponding position information. This solution improves the review efficiency, and the use of pinyin comparison makes the review accuracy higher.

[0103] In the second embodiment of an intelligent audio review method based on speech recognition proposed by the present invention, based on the first embodiment, step S130 includes the following steps:

[0104] Step S210: The server determines whether the audio to be reviewed received meets the segmentation condition: the duration of the audio to be reviewed is greater than the segmentation duration (for example, 1 minute).

[0105] Step S220: If it meets the condition, the server segments the audio to be reviewed to obtain multiple audio segments.

[0106] Step S230: The server performs speech recognition on the audio segments to obtain multiple text segments.

[0107] Step S240: If it does not meet the condition, the server performs speech recognition on the audio to be reviewed to obtain one text segment.

[0108] Specifically, the audio to be reviewed is segmented to obtain audio segments, so that the server can perform speech recognition on each audio segment simultaneously to improve efficiency. In addition, subsequently, the position of the target pinyin group can be directly locked in a single audio segment (rather than the entire audio to be reviewed), thereby generating more accurately and quickly the position information indicating the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed.

[0109] In the third embodiment of an intelligent audio review method based on speech recognition proposed by the present invention, based on the first embodiment, step S140 includes the following steps:

[0110] Step S310: The server obtains the total number S of characters in the text segment.

[0111] Specifically, for example, in this embodiment, the text segment is "I am Jia Yi Bing Ding ah". Then the corresponding total number S of characters is 7.

[0112] Step S320: The server obtains the sensitive word with the most characters and marks the number of characters of the sensitive word with the most characters as the longest number of characters C.

[0113] Specifically, as in the above embodiment, the sensitive word with the most characters is "Jia Yi Bing Ding", then the longest number of characters C is 4.

[0114] Step S330: The server sets N = 1.

[0115] Step S340: The server sets i = 1.

[0116] Step S350: The server combines the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a text phrase.

[0117] Step S360: The server sets i=i+1 and determines whether i+N≥S holds.

[0118] If yes, execute step S370: set N=N+1, and determine whether N≥C holds.

[0119] Step S380: If N≥C holds, obtain the pinyin of all obtained text phrases and use them as text pinyin groups.

[0120] Step S390: If N≥C is not established, execute step S340 and subsequent steps.

[0121] If not, execute step S350 and subsequent steps.

[0122] Specifically, this embodiment provides detailed steps for splitting a text segment to obtain multiple text phrases and obtaining the pinyin of the text phrases; through the above steps, the text segment "I am Jia, Yi, Bing, Ding, ah" can be split to obtain the following text phrases: "I am", "I am Jia", "Jia Yi", "Yi Bing", "Cing Ding", "Ding ah", "I am Jia", "I am Jia Yi", "Jia Yi Bing", "Yi Bing Ding", "Cing Ding ah", "I am Jia Yi", "I am Jia Yi Bing", "Jia Yi Bing Ding", "Jia Yi Bing Ding" and "Yi Bing Ding ah". In this way, all the word combinations in the text segment can be exhausted to obtain a more complete number of text phrases, thereby ensuring the accuracy of subsequent audits.

[0123] In a fourth embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the first embodiment, this embodiment further includes the following steps:

[0124] Step S410: The server marks the moment when the audio to be reviewed sent by the review terminal is received as the first moment, marks the moment when the review terminal receives the target pinyin group and corresponding location information, or the review pass information as the second moment, and uses the duration between the first moment and the second moment as the review duration corresponding to the audio to be reviewed.

[0125] Step S420: The server establishes a review time estimation model.

[0126] Step S430: The server obtains the historical review information corresponding to all reviewed audios within a preset time period in the past (for example, 3 months), wherein the historical review information includes the duration of the reviewed audios, the total number of characters in all text segments corresponding to the reviewed audios, and the review duration corresponding to the reviewed audios.

[0127] Step S440: The server uses the duration of the reviewed audio in the historical review information and the total number of characters in all text segments corresponding to the reviewed audio as input parameters of the review time estimation model, and uses the review duration corresponding to the reviewed audio in the historical review information as the output parameter of the review time estimation model for training, so as to obtain a trained review time estimation model.

[0128] Step S450: The server obtains the duration of the audio to be reviewed and the total number of characters in all text segments corresponding to the audio to be reviewed, and inputs the trained review duration estimation model to obtain the output estimated review duration corresponding to the audio to be reviewed.

[0129] Step S460: The server sends the estimated review duration corresponding to the audio to be reviewed to the review terminal.

[0130] Specifically, the solution of this embodiment establishes a review time estimation model, through which the review time of the audio to be reviewed can be estimated, so that the reviewer can know the estimated review time of the audio to be reviewed in time, which increases practicality.

[0131] In a fifth embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the third embodiment, step S220 includes the following steps:

[0132] Step S510: The server determines the overlap duration based on the longest word number C (e.g. 6 seconds), where the overlap duration Less than split time .

[0133] Specifically, the overlap time here That is, when the audio to be reviewed is segmented, the length of the overlapping period between the beginning and the end of two adjacent audio segments; for example, if the total length of the audio to be reviewed is 180 minutes, the starting time of the first audio segment is the starting time of the audio to be reviewed, and the ending time of the first audio segment is the 60th second of the audio to be reviewed; the starting time of the second audio segment is the 54th second of the audio to be reviewed (the length between the starting time of the second audio segment and the ending time of the first audio segment is the overlapping time, that is, 6 seconds), and the ending time of the second audio segment is the 104th second of the audio to be reviewed.

[0134] The reason for setting the overlap duration in this embodiment is:

[0135] If the audio to be reviewed is directly segmented, it will result in the loss of some word phrases. For example, if the text recognized from the voice at the last moment of the first audio segment is "A", and the text recognized from the voice at the beginning moment of the second audio segment is "BCD", since the first audio segment and the second audio segment are separately subjected to speech recognition, the coherent phrase "ABCD" cannot be recognized. However, in fact, the phrase "ABCD" exists in the audio to be reviewed. By setting the overlapping duration, both the end of the first audio segment and the start of the second audio segment can recognize complete word phrases, avoiding the above situation of losing word phrases.

[0136] Step S520: The server determines the start time and end time of each audio segment, which specifically includes the following steps:

[0137] Step S521: The server sets j = 1.

[0138] Step S522: The server takes the start time of the audio to be reviewed as the start time of the j-th audio segment, and takes the time that is after the start time of the j-th audio segment and has an interval duration equal to the segmentation duration from the start time of the j-th audio segment as the end time of the j-th audio segment.

[0139] Step S523: The server sets j = j + 1.

[0140] Step S524: The server takes the time that is before the end time of the (j - 1)-th audio segment and has an interval duration equal to the overlapping duration from the end time of the (j - 1)-th audio segment as the start time of the j-th audio segment.

[0141] Step S525: The server determines whether the interval duration between the end time of the audio to be reviewed and the start time of the j-th audio segment is less than or equal to the segmentation duration .

[0142] If so, execute Step S526: The server takes the end time of the audio to be reviewed as the end time of the j-th audio segment.

[0143] If not, execute Step S527: The server takes the time that is after the start time of the j-th audio segment and has an interval duration equal to the segmentation duration from the start time of the j-th audio segment as the end time of the j-th audio segment, and executes Step S523 and subsequent steps.

[0144] Specifically, through the above steps, after segmenting the audio to be reviewed with a total duration of 180 minutes, the following audio segments can be obtained:

[0145] The starting time of the first audio segment is the starting time of the audio to be reviewed, and the ending time of the first audio segment is the 60th second of the audio to be reviewed; the starting time of the second audio segment is the 54th second of the audio to be reviewed, and the ending time of the second audio segment is the 104th second of the audio to be reviewed; the starting time of the third audio segment is the 98th second of the audio to be reviewed, and the ending time of the third audio segment is the 158th second of the audio to be reviewed; the starting time of the fourth audio segment is the 152th second of the audio to be reviewed, and the ending time of the fourth audio segment is the 180th second of the audio to be reviewed.

[0146] Step S530: The server divides the audio to be reviewed based on the start time and the end time of each audio segment to obtain multiple audio segments.

[0147] Specifically, this embodiment provides a specific solution for how to segment the audio to be reviewed to obtain multiple audio segments.

[0148] In a sixth embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the fifth embodiment, step S510 includes the following steps:

[0149] Step S610: The server obtains the normal reading speed (in words per second, e.g. 1 word per second) and based on normal reading speed Calculate the overlap time with the longest word C :

[0150] ,

[0151] In the formula, is the overlap duration; A is the first redundant duration, which takes a positive value, such as 1 second.

[0152] Specifically, the overlap time here needs to take into account the maximum number of words C, and the overlap time is set to at least ensure that the text with the longest number of words can be read at a normal reading speed within the overlap time; The calculation formula is set as: .

[0153] In the seventh embodiment of the intelligent audio audit method based on speech recognition proposed by the present invention, based on the sixth embodiment, the step of generating position information for expressing the position of the text phrase to which the target pinyin group belongs in the audio to be audited in step S160 includes the following steps:

[0154] Step S710: The server marks the text segment corresponding to the text phrase to which the target pinyin group belongs as the target text segment.

[0155] Step S720: The server obtains the sequence number of the first character in the character phrase to which the target pinyin group belongs in the target text segment. , and the sequence number of the last character in the character phrase to which the target pinyin group belongs in the target text segment .

[0156] Specifically, under normal circumstances, the speech in the audio to be reviewed is continuous, so the position of the text phrase to which the target pinyin group belongs in the corresponding text segment is also the position of the target pinyin group in the audio to be reviewed; therefore, the arrangement sequence number of the first character in the text phrase to which the target pinyin group belongs in the target text segment is , and the sequence number of the last character in the character phrase to which the target pinyin group belongs in the target text segment It can reflect the position of the text phrase belonging to the target pinyin group in the audio to be reviewed.

[0157] Step S730: The server uses the duration between the estimated appearance time of the first character in the text phrase to which the target pinyin group belongs in the audio to be reviewed and the start time of the audio to be reviewed as the first duration.

[0158] Step S740: The server uses the estimated duration of the text phrase to which the target pinyin group belongs in the audio to be reviewed as the second duration, wherein the calculation formula for the second duration is:

[0159] ,

[0160] In the formula, is the second duration; B is the second redundant duration, which takes a positive value, for example, 2 seconds; The unit reading speed is 1 word per second, which is used to unify the units on both sides of the formula.

[0161] Specifically, by setting the second redundant time length, the estimated time period for the text phrases belonging to the target pinyin group to appear in the audio to be reviewed can be increased, thereby increasing the probability of the text phrases belonging to the target pinyin group appearing in the audio to be reviewed within the estimated time period, so as to facilitate reviewers to conduct review.

[0162] Specifically, the above calculation formula means: based on the number of sensitive words and the normal reading speed, the estimated duration of the sensitive words in the audio to be reviewed is determined.

[0163] For example, if the target pinyin group of the sensitive word "jiayibingding" (i.e., the Chinese characters "甲乙丙丁") that appears in the audio to be audited is "jiayibingding", and the text phrase to which the target pinyin group belongs is "甲乙丙丁"; the duration between the estimated appearance time of the character "甲" in the sensitive word "甲乙丙丁" and the start time of the audio to be audited is the first duration, and the estimated duration of the sensitive word "甲乙丙丁" is the second duration; that is, the first duration can locate the starting position of the sensitive word "甲乙丙丁" in the audio to be audited, and the second duration can determine the duration of the sensitive word "甲乙丙丁" in the audio to be audited. Through the first duration and the second duration, the auditor can conveniently know the position of the sensitive word "甲乙丙丁" in the audio to be audited.

[0164] Step S750: The server determines whether the audio to be audited corresponding to the target text segment includes multiple audio segments.

[0165] If so, execute Step S760: The server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be audited. 。

[0166] Specifically, if the audio to be audited corresponding to the target text segment includes multiple audio segments, it means that the audio to be audited corresponding to the sensitive word is divided into multiple audio segments; for the convenience of subsequent description, the audio segment corresponding to the target text segment of the target pinyin group is marked as the target audio segment. And obtain the duration between the start time of the target audio segment and the start time of the corresponding audio to be audited. 。

[0167] Step S770: The server calculates the first duration according to the following formula:

[0168] ,

[0169] In the formula, is the first duration; is the total number of characters in the target text segment of the target pinyin group.

[0170] Specifically, the meaning of the above calculation formula is that the position of the text phrase to which the target pinyin group belongs in the corresponding text segment is also the position of the target pinyin group in the audio to be audited. Therefore, the ratio of the first character of the text phrase to which the target pinyin group belongs to the total number of characters in the target text segment of the target pinyin group is equal to the ratio of the appearance time of the first character of the text phrase to which the target pinyin group belongs to the total duration of the target audio segment.

[0171] For example, the serial number of "A" in the sensitive word "A, B, C, D" in the target text segment is 90, and the total number of characters in the target text segment of the target pinyin group is 360; the duration of the target audio segment is (60 seconds), then the estimated appearance time of "A" in the target audio segment is 15 seconds; and since the target audio segment is a part of the audio to be audited, (if the target audio segment in this embodiment is the second audio segment, then is 54 seconds), and then subtract the second redundancy duration, the final first duration calculated is 67 seconds, that is, the estimated appearance time of "A" in the sensitive word "A, B, C, D" in the audio to be audited is the 67th second. In addition, according to the foregoing formula, the estimated duration of the sensitive word "A, B, C, D" in the audio to be audited can be calculated as 8 seconds.

[0172] If not, execute step S780: The server calculates the first duration according to the following formula:

[0173] ,

[0174] Specifically, if not, it means that there is only one audio segment in the audio to be audited (that is, the audio to be audited itself), so on the basis of the foregoing formula, the first duration no longer needs to add ; for example: the serial number of "A" in the sensitive word "A, B, C, D" in the target text segment is 20, and the total number of characters in the target text segment of the target pinyin group is 60; the duration of the target audio segment (that is, the audio to be audited) is (60 seconds), and then subtract the second redundancy duration, the final first duration calculated is 18 seconds, that is, the estimated appearance time of "A" in the sensitive word "A, B, C, D" in the audio to be audited is the 18th second. In addition, according to the foregoing formula, the estimated duration of the sensitive word "A, B, C, D" in the audio to be audited can be calculated as 8 seconds.

[0175] Step S790: The server uses the first duration and the second duration as the position information for expressing the appearance position of the text phrase to which the target pinyin group belongs in the audio to be audited.

[0176] In the eighth embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the seventh embodiment, after step S760, the following steps are further included:

[0177] Step S810: The server calculates the average reading speed of the target audio segment of the target pinyin group based on the total number of characters in the target text segment of the target pinyin group and the segmentation duration

[0178] ,

[0179] In the formula, The average reading speed of the target audio segment of the target pinyin group.

[0180] Step S820: The server determines whether the following condition is satisfied: the ratio of the average reading speed of the target audio segment of the target pinyin group to the normal reading speed is less than a preset lower limit (eg, 0.5).

[0181] Specifically, if the ratio of the average reading speed of the target audio segment of the target pinyin group to the normal reading speed is less than the preset lower limit, it means that the average reading speed of the target audio segment is significantly lower than the normal reading speed (under the premise that the reading is continuous, it means that there is a blank period of at least half the length in the target audio segment, that is, a period without any voice, which may be a period of pure music); that is to say, the actual voice in the target audio segment is not completely distributed in the entire target audio segment, but is concentrated in a certain period of the target audio segment; therefore, the position of the text phrase to which the target pinyin group belongs in the corresponding text segment is not the position of the target pinyin group in the audio to be reviewed, so the first duration cannot be calculated based on the formula of the seventh embodiment.

[0182] If so, step S830 is executed: the server divides the target audio segment of the target pinyin group from the middle time to obtain a first audio segment in the front and a second audio segment in the back.

[0183] Step S840: The server performs speech recognition on the first audio segment to obtain a corresponding first text segment, and performs speech recognition on the second audio segment to obtain a corresponding second text segment.

[0184] Step S850: The server calculates the total number of characters in the first text segment based on the total number of characters in the first text segment. and the total number of characters in the second text segment Calculate the first duration.

[0185] Specifically, the target audio segment is divided from the middle moment to obtain a first audio segment in front and a second audio segment in the back, based on the total number of characters in the first text segment. and the total number of characters in the second text segment It can reflect the distribution of actual speech in the target audio segment, thereby calculating the first duration.

[0186] In a ninth embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the eighth embodiment, step S850 includes the following steps:

[0187] Step S910: If the total number of characters in the first text segment is 0, the server calculates the first duration according to the following formula:

[0188] .

[0189] Specifically, if the total number of characters in the first text segment is 0, indicating that the actual speech is all concentrated in the second text segment. Therefore, the ratio of the first character of the text phrase belonging to the target pinyin group to the total number of characters in the target text segment of the target pinyin group (that is, the total number of characters in the second text segment) is equal to the ratio of the appearance time of the first character of the text phrase belonging to the target pinyin group to half of the total duration of the target audio segment. The above formula can be obtained based on the calculation formula of the first duration in the seventh embodiment.

[0190] Step S920: If the total number of characters in the second text segment is 0, the server calculates the first duration according to the following formula:

[0191] .

[0192] Specifically, if the total number of characters in the second text segment is 0, indicating that the actual speech is all concentrated in the first text segment. Therefore, the ratio of the first character of the text phrase to which the target pinyin group belongs to and the total number of characters in the target text segment of the target pinyin group (that is, the total number of characters in the first text segment) is equal to the ratio of the appearance time of the first character of the text phrase to which the target pinyin group belongs to half of the total duration of the target audio segment. The above formula can be obtained based on the calculation formula for the first duration in the seventh embodiment.

[0193] Step S930: If the total number of characters in the first text segment and the total number of characters in the second text segment are both greater than 0, the server calculates the first duration according to the following formula:

[0194] .

[0195] Specifically, if the total number of characters in the first text segment and the total number of characters in the second text segment are all greater than 0, indicating that the text is distributed in both the first and second text segments, that is, the actual speech is distributed in both the first and second halves of the target audio segment; assuming that the interval between the start time of the actual speech and the middle time of the target audio segment is , and there is , the interval between the end time of the actual speech and the middle time of the target audio segment is ; The position of the text phrase to which the target pinyin group belongs in the corresponding text segment is also the position of the target pinyin group in the actual speech, and the duration of the actual speech is , then the interval duration between the first character of the text phrase to which the target pinyin group belongs and the start time of the actual speech can be derived as follows: , and the interval duration between the start time of the actual speech and the start time of the target audio segment is . Based on the calculation formula of the seventh embodiment, the formula for calculating the first duration in this embodiment can be derived as the above formula.

[0196] For example: If the total number of characters in the first text segment is 10, and the total number of characters in the second text segment is 20, the arrangement serial number of 'A' in the sensitive word 'A, B, C, D' in the target text segment is 5, and the total number of characters in the target text segment of the target pinyin group is 30; the duration of the target audio segment is (60 seconds), then the estimated appearance time of 'A' in the target audio segment is 25 seconds; and since the target audio segment is a part of the audio to be audited, (if the target audio segment in this embodiment is the second audio segment, then is 54 seconds) needs to be added, and then the final first duration calculated after subtracting the second redundancy duration is 77 seconds, that is, the estimated appearance time of 'A' in the sensitive word 'A, B, C, D' in the audio to be audited is the 77th second.

[0197] The present invention also provides an intelligent audio audit system based on speech recognition. This system applies the intelligent audio audit method based on speech recognition; the system includes a server and an audit terminal that are communicatively connected to each other; the server stores multiple sensitive word sets, and each sensitive word set includes multiple different sensitive words, and each sensitive word includes at least 2 characters.

[0198] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0199] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are only illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the purpose of the present invention and the scope protected by the claims. All these belong to the protection scope of the present invention.

Claims

1. An intelligent audio audit method based on speech recognition, characterized in that: Applicable to an intelligent audio audit system based on speech recognition; the system comprises a server and an audit terminal that are communicatively connected to each other; The server stores a sensitive word set, the sensitive word set includes a plurality of different sensitive words, and each sensitive word includes at least 2 characters; the method includes: The server determines a sensitive pinyin group and a similar pinyin group corresponding to the sensitive word set, wherein the sensitive pinyin group is a pinyin group of each sensitive word in the sensitive word set, and the similar pinyin group is a pinyin group with a pronunciation similar to the sensitive pinyin group; After the audit terminal obtains the audio to be audited, it sends the audio to be audited to the server; The server performs speech recognition on the received audio to be reviewed to obtain the corresponding text segment; The server splits and reorganizes the text segment based on the sensitive pinyin group to obtain a text phrase, and marks the pinyin of the text phrase as a text pinyin group; The server determines whether there is a text pinyin group that is consistent with the sensitive pinyin group or the similar pinyin group; If so, the server marks the text pinyin group that is consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, generates location information for describing the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed, sends the target pinyin group and the corresponding location information to the review terminal, and marks the audio to be reviewed as reviewed audio; If not, the server generates a review pass message indicating that the audio to be reviewed does not contain sensitive words, sends the review pass message to the review terminal, and marks the audio to be reviewed as reviewed audio; The server performs speech recognition on the received audio to be reviewed to obtain a corresponding text segment, including: The server determines whether the received audio to be reviewed meets the segmentation condition: the duration of the audio to be reviewed is greater than the segmentation duration; If the requirements are met, the server divides the audio to be reviewed to obtain multiple audio segments; The server performs speech recognition on the audio segment to obtain multiple text segments; If not, the server performs speech recognition on the audio to be reviewed to obtain a text segment; The server splits and reorganizes the text segments based on the sensitive pinyin groups to obtain text phrases, and marks the pinyin of the text phrases as text pinyin groups, including: The server obtains the total number of characters S in the text segment; The server obtains the most sensitive words, and marks the most sensitive words as the longest word number C; The server sets N=1; The server sets i=1; The server combines the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a character phrase; The server sets i=i+1 and determines whether i+N≥S holds; If so, let N=N+1, and determine whether N≥C holds; If N ≥ C, obtain the pinyin of all the obtained text phrases and use them as text pinyin groups; If N≥C is not true, execute the server to set i=1 and the following steps; If not, executing the server to combine the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a character phrase, and the following steps; If the above conditions are met, the server segments the audio to be reviewed to obtain multiple audio segments, including: The server determines the overlap duration based on the longest word number C , where the overlap duration Less than split time ; The server determines the start and end times of each audio segment, including: The server sets j=1; The server uses the start time of the audio to be reviewed as the start time of the jth audio segment, and the interval between the start time of the jth audio segment and the start time of the jth audio segment is the split duration. The moment of is taken as the end moment of the jth audio segment; The server sets j=j+1; The server will be before the end time of the j-1th audio segment, and the interval between the end time of the j-1th audio segment and the end time of the j-1th audio segment will be the overlap time. The moment of is taken as the starting moment of the jth audio segment; The server determines whether the interval between the end time of the audio to be reviewed and the start time of the jth audio segment is less than or equal to the split time. ; If yes, the server uses the end time of the audio to be reviewed as the end time of the jth audio segment; If not, the server will be after the start time of the jth audio segment, and the interval between the start time of the jth audio segment is the split duration. As the end time of the j-th audio segment, and execute the server setting j=j+1, and the subsequent steps; The server divides the audio to be reviewed based on the start time and end time of each audio segment to obtain multiple audio segments; The server determines the overlap duration based on the longest word number C ,include: The server obtains the normal reading speed , and based on normal reading speed Calculate the overlap time with the longest word C : ， Where A is the first redundancy duration, which takes a positive value; The generating of the position information for expressing the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed includes: The server marks the text segment corresponding to the text phrase to which the target pinyin group belongs as the target text segment; The server obtains the sequence number of the first character in the character phrase to which the target pinyin group belongs in the target text segment , and the sequence number of the last character in the character phrase to which the target pinyin group belongs in the target text segment ; The server uses the duration between the estimated appearance time of the first character in the character phrase to which the target pinyin group belongs in the audio to be reviewed and the start time of the audio to be reviewed as the first duration; The server uses the estimated duration of the text phrase to which the target pinyin group belongs in the audio to be reviewed as the second duration, wherein the calculation formula for the second duration is: ， In the formula, is the second duration; B is the second redundant duration, which takes a positive value; is the unit reading speed; The server determines whether the audio to be reviewed corresponding to the target text segment includes multiple audio segments; If yes, the server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be reviewed. ; The server calculates the first duration according to the following formula: ， In the formula, For the first duration; The total number of characters in the target text segment of the target pinyin group; If not, the server calculates the first duration according to the following formula: ， The server uses the first duration and the second duration as position information for expressing the position of the text phrase belonging to the target pinyin group in the audio to be reviewed.

2. According to claim 1, the intelligent audio audit method based on speech recognition is characterized in that: Also includes: The server marks the moment when the audit terminal receives the audited audio as the first moment, marks the moment when the audit terminal receives the target pinyin group and the corresponding position information, or the audit pass information as the second moment, and uses the duration between the first moment and the second moment as the audit duration corresponding to the audited audio; The server establishes a review time estimation model; The server obtains historical review information corresponding to all reviewed audios within a preset time period in the past, wherein the historical review information includes the duration of the reviewed audios, the total number of words in all text segments corresponding to the reviewed audios, and the review duration corresponding to the reviewed audios; The server uses the duration of the audited audio in the historical audit information and the total number of words in all text segments corresponding to the audited audio as input parameters of the audit time estimation model, and uses the audit time corresponding to the audited audio in the historical audit information as the output parameter of the audit time estimation model for training, so as to obtain a fully trained audit time estimation model; The server obtains the duration of the audio to be reviewed and the total number of characters in all text segments corresponding to the audio to be reviewed, and inputs the trained review duration estimation model to obtain the output estimated review duration corresponding to the audio to be reviewed; The server sends the estimated review time corresponding to the audio to be reviewed to the review terminal.

3. According to claim 1, the intelligent audio audit method based on speech recognition is characterized in that: The server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be reviewed. , and later: The total number of characters in the target text segment based on the target pinyin group of the server and split duration Calculate the average reading speed of the target audio segment of the target pinyin group; ， In the formula, The average reading speed of the target audio segment for the target pinyin group; The server determines whether the following conditions are met: the ratio of the average reading speed of the target audio segment of the target pinyin group to the normal reading speed is less than a preset lower limit; If yes, the server divides the target audio segment of the target pinyin group from the middle time to obtain a first audio segment in front and a second audio segment in the back; The server performs speech recognition on the first audio segment to obtain a corresponding first text segment, and performs speech recognition on the second audio segment to obtain a corresponding second text segment; The server based on the total number of characters in the first text segment and the total number of characters in the second text segment Calculate the first duration.

4. According to claim 3, the intelligent audio audit method based on speech recognition is characterized in that: The server is based on the total number of characters in the first text segment and the total number of characters in the second text segment Calculate the first duration, including: If the total number of characters in the first text segment is 0, the server calculates the first duration according to the following formula: ， If the total number of characters in the second text segment is 0, the server calculates the first duration according to the following formula: ， If the total number of characters in the first text segment and the total number of characters in the second text segment are both greater than 0, the server calculates the first duration according to the following formula: 。 5. An intelligent audio audit system based on speech recognition, characterized in that: An intelligent audio review method based on speech recognition as described in any one of claims 1 to 4 is applied; the system includes a server and a review terminal that are communicatively connected to each other; the server stores multiple sensitive word sets, the sensitive word sets include multiple sensitive words that are different from each other, and each sensitive word includes at least 2 characters.

Images