Intelligent audio auditing method and system based on voice recognition

Abstract

The invention relates to the technical field of audio auditing, in particular to an intelligent audio auditing method and system based on voice recognition. A sensitive pinyin group and a similar pinyin group corresponding to a sensitive word set are firstly determined, and then voice recognition is performed to obtain a corresponding character field; splitting and recombining the character fields to obtain character phrases, and obtaining character pinyin groups of the character phrases; judging whether a character pinyin group consistent with the sensitive pinyin group or the similar pinyin group exists or not; if the sensitive pinyin group or the similar pinyin group exists, marking the character pinyin group consistent with the sensitive pinyin group or the similar pinyin group as a target pinyin group, and sending the target pinyin group and the corresponding position information to an auditing terminal for re-checking, so that the auditing accuracy is ensured; the auditor only needs to recheck the target pinyin group and the corresponding position information; the auditing efficiency is improved, and the auditing accuracy is higher by adopting a pinyin comparison mode.

Description

Technical Field

[0001] The present invention relates to the technical field of audio auditing, and in particular to an intelligent audio auditing method and system based on speech recognition. Background Art

[0002] With the continuous development of information technology, Internet audio live broadcast and on-demand services have attracted widespread attention for their novel forms and rich content. However, the audio information in audio services may be mixed with illegal content, so effective review and supervision of audio information is required.

[0003] In the prior art, the review and supervision of audio information is mainly manual review, that is, reviewers listen to the audio information. Although this method can ensure the accuracy of the review, it is very time-consuming and labor-intensive, and it is difficult to conduct efficient and accurate review of the massive amount of audio information on the Internet. Summary of the invention

[0004] The main purpose of the present invention is to provide an intelligent audio review method and system based on speech recognition, aiming to solve the problem that the existing audio information review scheme is time-consuming and labor-intensive, and it is difficult to conduct efficient and accurate review of the massive amount of audio information on the Internet.

[0005] The technical solution proposed by the present invention is: An intelligent audio audit method based on speech recognition is applied to an intelligent audio audit system based on speech recognition; the system comprises a server and an audit terminal which are connected to each other in communication; the server stores a sensitive word set, the sensitive word set comprises a plurality of sensitive words which are different from each other, and each sensitive word comprises at least 2 characters; the method comprises: The server determines a sensitive pinyin group and a similar pinyin group corresponding to the sensitive word set, wherein the sensitive pinyin group is a pinyin group of each sensitive word in the sensitive word set, and the similar pinyin group is a pinyin group with a pronunciation similar to the sensitive pinyin group; After the audit terminal obtains the audio to be audited, it sends the audio to be audited to the server; The server performs speech recognition on the received audio to be reviewed to obtain the corresponding text segment; The server splits and reorganizes the text segment based on the sensitive pinyin group to obtain a text phrase, and marks the pinyin of the text phrase as a text pinyin group; The server determines whether there is a text pinyin group that is consistent with the sensitive pinyin group or the similar pinyin group; If so, the server marks the text pinyin group that is consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, generates location information for describing the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed, sends the target pinyin group and the corresponding location information to the review terminal, and marks the audio to be reviewed as reviewed audio; If not, the server generates review pass information to indicate that the audio to be reviewed does not contain sensitive words, sends the review pass information to the review terminal, and marks the audio to be reviewed as reviewed audio.

[0006] Preferably, the server performs speech recognition on the received audio to be reviewed to obtain a corresponding text segment, including: The server determines whether the received audio to be reviewed meets the segmentation condition: the duration of the audio to be reviewed is greater than the segmentation duration; If the requirements are met, the server divides the audio to be reviewed to obtain multiple audio segments; The server performs speech recognition on the audio segment to obtain multiple text segments; If not satisfied, the server performs speech recognition on the audio to be reviewed to obtain a text segment.

[0007] Preferably, the server splits and reorganizes the text segments based on the sensitive pinyin groups to obtain text phrases, and marks the pinyin of the text phrases as text pinyin groups, including: The server obtains the total number of characters S in the text segment; The server obtains the most sensitive words, and marks the most sensitive words as the longest word number C; The server sets N=1; The server sets i=1; The server combines the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a character phrase; The server sets i=i+1 and determines whether i+N≥S holds; If so, let N=N+1, and determine whether N≥C holds; If N ≥ C, obtain the pinyin of all the obtained text phrases and use them as text pinyin groups; If N≥C is not true, execute the server to set i=1 and the following steps; If not, the server is executed to combine the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a character phrase, and the following steps.

[0008] Preferably, it also includes: The server marks the moment when the audit terminal receives the audited audio as the first moment, marks the moment when the audit terminal receives the target pinyin group and the corresponding position information, or the audit pass information as the second moment, and uses the duration between the first moment and the second moment as the audit duration corresponding to the audited audio; The server establishes a review time estimation model; The server obtains historical review information corresponding to all reviewed audios within a preset time period in the past, wherein the historical review information includes the duration of the reviewed audios, the total number of words in all text segments corresponding to the reviewed audios, and the review duration corresponding to the reviewed audios; The server uses the duration of the audited audio in the historical audit information and the total number of words in all text segments corresponding to the audited audio as input parameters of the audit time estimation model, and uses the audit time corresponding to the audited audio in the historical audit information as the output parameter of the audit time estimation model for training, so as to obtain a fully trained audit time estimation model; The server obtains the duration of the audio to be reviewed and the total number of characters in all text segments corresponding to the audio to be reviewed, and inputs the trained review duration estimation model to obtain the output estimated review duration corresponding to the audio to be reviewed; The server sends the estimated review time corresponding to the audio to be reviewed to the review terminal.

[0009] Preferably, if the above conditions are met, the server segments the audio to be audited to obtain multiple audio segments, including: The server determines the overlap duration based on the longest word number C , where the overlap duration Less than split time ; The server determines the start and end times of each audio segment, including: The server sets j=1; The server uses the start time of the audio to be reviewed as the start time of the jth audio segment, and the interval between the start time of the jth audio segment and the start time of the jth audio segment is the split duration. The moment of is taken as the end moment of the jth audio segment; The server sets j=j+1; The server will be before the end time of the j-1th audio segment, and the interval between the end time of the j-1th audio segment and the end time of the j-1th audio segment will be the overlap time. The moment of is taken as the starting moment of the jth audio segment; The server determines whether the interval between the end time of the audio to be reviewed and the start time of the jth audio segment is less than or equal to the split time. ; If yes, the server uses the end time of the audio to be reviewed as the end time of the jth audio segment; If not, the server will be after the start time of the jth audio segment, and the interval between the start time of the jth audio segment is the split duration. As the end time of the j-th audio segment, and execute the server setting j=j+1, and the subsequent steps; The server divides the audio to be reviewed based on the start time and the end time of each audio segment to obtain multiple audio segments.

[0010] Preferably, the server determines the overlap duration based on the longest word number C ,include: The server obtains the normal reading speed , and based on normal reading speed Calculate the overlap time with the longest word C : , In the formula, is the overlap duration; A is the first redundant duration, which takes a positive value.

[0011] Preferably, the generating of the position information for expressing the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed includes: The server marks the text segment corresponding to the text phrase to which the target pinyin group belongs as the target text segment; The server obtains the sequence number of the first character in the character phrase to which the target pinyin group belongs in the target text segment , and the sequence number of the last character in the character phrase to which the target pinyin group belongs in the target text segment ; The server uses the duration between the estimated appearance time of the first character in the character phrase to which the target pinyin group belongs in the audio to be reviewed and the start time of the audio to be reviewed as the first duration; The server uses the estimated duration of the text phrase to which the target pinyin group belongs in the audio to be reviewed as the second duration, wherein the calculation formula for the second duration is: , In the formula, is the second duration; B is the second redundant duration, which takes a positive value; is the unit reading speed; The server determines whether the audio to be reviewed corresponding to the target text segment includes multiple audio segments; If yes, the server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be reviewed. ; The server calculates the first duration according to the following formula: , In the formula, For the first duration; The total number of characters in the target text segment of the target pinyin group; If not, the server calculates the first duration according to the following formula: , The server uses the first duration and the second duration as position information for expressing the position of the text phrase belonging to the target pinyin group in the audio to be reviewed.

[0012] Preferably, the server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be reviewed. , and later: The total number of characters in the target text segment based on the target pinyin group of the server and split duration Calculate the average reading speed of the target audio segment of the target pinyin group; , In the formula, The average reading speed of the target audio segment for the target pinyin group; The server determines whether the following conditions are met: the ratio of the average reading speed of the target audio segment of the target pinyin group to the normal reading speed is less than a preset lower limit; If yes, the server divides the target audio segment of the target pinyin group from the middle time to obtain a first audio segment in front and a second audio segment in the back; The server performs speech recognition on the first audio segment to obtain a corresponding first text segment, and performs speech recognition on the second audio segment to obtain a corresponding second text segment; The server based on the total number of characters in the first text segment and the total number of characters in the second text segment Calculate the first duration.

[0013] Preferably, the server is based on the total number of characters in the first text segment. and the total number of characters in the second text segment Calculate the first duration, including: If the total number of characters in the first text segment is 0, the server calculates the first duration according to the following formula: , If the total number of characters in the second text segment is 0, the server calculates the first duration according to the following formula: , If the total number of characters in the first text segment and the total number of characters in the second text segment are both greater than 0, the server calculates the first duration according to the following formula: .

[0014] The present invention also proposes an intelligent audio audit system based on speech recognition, which applies an intelligent audio audit method based on speech recognition; the system includes a server and an audit terminal that are communicatively connected to each other; the server stores multiple sensitive word sets, the sensitive word sets include multiple sensitive words that are different from each other, and each sensitive word includes at least 2 characters.

[0015] Through the above technical solution, the following beneficial effects can be achieved: The intelligent audio audit method based on speech recognition proposed in the present invention can audit the audio for sensitive words more efficiently and accurately; first determine the sensitive pinyin group and similar pinyin group corresponding to the sensitive word set, and then perform speech recognition on the received audio to be audited to obtain the corresponding text segment; then split and reorganize the text segment to obtain the text phrase, and obtain the text pinyin group of the text phrase; then compare the text pinyin group with the sensitive pinyin group and the similar audio group to determine whether there is a text pinyin group consistent with the sensitive pinyin group or the similar pinyin group; if so, it means that there is a text phrase that is consistent or similar in pronunciation to the sensitive word, indicating that the sensitive word may appear in the audio to be audited, then directly mark the text pinyin group consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, and then send the target pinyin group and the corresponding position information to the audit terminal, so that the operator of the audit terminal can review the recognition result, and further ensure the audit accuracy; this method no longer requires the auditor to manually listen to all the voices of the audio to be audited, and only needs to review the target pinyin group and the corresponding position information; this scheme improves the audit efficiency, and the pinyin comparison method makes the audit accuracy higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of the process steps of the first embodiment of an intelligent audio review method based on speech recognition proposed by the present invention. DETAILED DESCRIPTION

[0018] It should be understood that the specific embodiments described herein are only for explaining the present invention and are not used to limit the present invention.

[0019] The present invention provides an intelligent audio review method and system based on speech recognition.

[0020] As shown in the attached Figure 1 In the first embodiment of an intelligent audio review method based on speech recognition proposed by the present invention, this method is applied to an intelligent audio review system based on speech recognition; the system includes a server and a review terminal (a terminal operated by a reviewer) that are communicatively connected to each other; the server stores a set of sensitive words, the set of sensitive words includes a plurality of different sensitive words, and each sensitive word includes at least 2 characters; this embodiment includes the following steps: Step S110: The server determines a sensitive pinyin group and a similar pinyin group corresponding to the set of sensitive words, where the sensitive pinyin group is the pinyin group of each sensitive word in the set of sensitive words, and the similar pinyin group is a pinyin group that is similar in pronunciation to the sensitive pinyin group.

[0021] Specifically, for example, the set of sensitive words includes the following sensitive words "Jia Bing" (not an actual sensitive word, only for example) and "Jia Yi Bing Ding"; then the corresponding sensitive pinyin groups are "jiabing" and "jiayibingding", and correspondingly, the similar pinyin groups of the sensitive pinyin group "jiabing" include but are not limited to "jiabin", "jiapin", and "gabin"; the similar pinyin groups of the sensitive pinyin group "jiayibingding" include but are not limited to "jiayibinding", "gayibingding", and "jiaxibingding".

[0022] Step S120: After the review terminal obtains the audio to be reviewed, it sends the audio to be reviewed to the server.

[0023] Step S130: The server performs speech recognition on the received audio to be reviewed to obtain a corresponding text segment.

[0024] Specifically, the text segment here is the corresponding text segment formed after the speech in the audio to be reviewed is converted.

[0025] Step S140: The server splits and reorganizes the text segment based on the sensitive pinyin group to obtain text phrases, and marks the pinyin of the text phrases as the text pinyin group.

[0026] Specifically, after splitting the text segment, all possible combinations of text phrases in the text segment can be obtained, and these text phrases are the phrases that need to be compared with the sensitive words.

[0027] Step S150: The server determines whether there is a text pinyin group that is the same as the sensitive pinyin group or a similar pinyin group.

[0028] Specifically, during the speech recognition process, there may be differences between the characters in the recognized text segment and the actual speech in the audio to be audited. For example, "门神" (door god) may be recognized as "闷声" (muffled sound). Therefore, directly comparing the word groups split from the text segment obtained by speech recognition with sensitive words will result in errors. In this embodiment, instead of directly comparing the word groups with sensitive words, the pinyin groups of the word groups are compared with the sensitive pinyin groups, which can more accurately identify sensitive words and thus improve the audit accuracy rate. In addition, comparing the pinyin groups of the word groups with similar pinyin groups can conduct a more comprehensive audit of the audio to be audited, avoiding missing sensitive words due to similar pronunciations, thereby ensuring the final audit effect.

[0029] If so, execute Step S160: The server marks the text pinyin group that is the same as the sensitive pinyin group or a similar pinyin group as the target pinyin group, generates position information for expressing the position of the word group to which the target pinyin group belongs in the audio to be audited, sends the target pinyin group and the corresponding position information to the audit terminal, and marks the audio to be audited as an audited audio.

[0030] Specifically, sending the target pinyin group and the corresponding position information to the audit terminal facilitates the operator of the audit terminal to review the recognition result and further ensures the audit accuracy rate.

[0031] If not, execute Step S170: The server generates an audit passing information for expressing that the audio to be audited does not contain sensitive words, sends the audit passing information to the audit terminal, and marks the audio to be audited as an audited audio.

[0032] The intelligent audio audit method based on speech recognition proposed in the present invention can audit the audio for sensitive words more efficiently and accurately; first determine the sensitive pinyin group and similar pinyin group corresponding to the sensitive word set, and then perform speech recognition on the received audio to be audited to obtain the corresponding text segment; then split and reorganize the text segment to obtain the text phrase, and obtain the text pinyin group of the text phrase; then compare the text pinyin group with the sensitive pinyin group and the similar audio group to determine whether there is a text pinyin group consistent with the sensitive pinyin group or the similar pinyin group; if so, it means that there is a text phrase that is consistent or similar in pronunciation to the sensitive word, indicating that the sensitive word may appear in the audio to be audited, then directly mark the text pinyin group consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, and then send the target pinyin group and the corresponding position information to the audit terminal, so that the operator of the audit terminal can review the recognition result, and further ensure the audit accuracy; this method no longer requires the auditor to manually listen to all the voices of the audio to be audited, and only needs to review the target pinyin group and the corresponding position information; this scheme improves the audit efficiency, and the pinyin comparison method makes the audit accuracy higher.

[0033] In a second embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the first embodiment, step S130 includes the following steps: Step S210: The server determines whether the received audio to be reviewed meets the segmentation condition: the duration of the audio to be reviewed is greater than the segmentation duration (e.g. 1 minute).

[0034] Step S220: If satisfied, the server divides the audio to be reviewed to obtain multiple audio segments.

[0035] Step S230: The server performs speech recognition on the audio segment to obtain multiple text segments.

[0036] Step S240: If not satisfied, the server performs speech recognition on the audio to be reviewed to obtain a text segment.

[0037] Specifically, the audio to be reviewed is segmented to obtain audio segments, so that the server can perform speech recognition on each audio segment at the same time to improve efficiency. In addition, the position of the target pinyin group can be directly locked in a separate audio segment (rather than the entire audio to be reviewed) in the future, so as to more accurately and quickly generate the position information used to express the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed.

[0038] In a third embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the first embodiment, step S140 includes the following steps: Step S310: The server obtains the total number of characters S in the text segment.

[0039] Specifically, for example, in this embodiment, the text segment is "I am Jia, Yi, Bing, Ding". Then the corresponding total number of characters S is 7.

[0040] Step S320: The server obtains the sensitive word with the most characters and marks the number of characters of the sensitive word with the most characters as the longest number of characters C.

[0041] Specifically, as in the above embodiment, the sensitive word with the most characters is "Jia, Yi, Bing, Ding", so the longest number of characters C is 4.

[0042] Step S330: The server sets N = 1.

[0043] Step S340: The server sets i = 1.

[0044] Step S350: The server combines the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a text phrase.

[0045] Step S360: The server sets i = i + 1 and determines whether i + N ≥ S holds.

[0046] If so, execute Step S370: Set N = N + 1 and determine whether N ≥ C holds.

[0047] Step S380: If N ≥ C holds, obtain the pinyin of all the obtained text phrases and use it as the text pinyin group.

[0048] Step S390: If N ≥ C does not hold, execute Step S340 and the subsequent steps.

[0049] If not, execute Step S350 and the subsequent steps.

[0050] Specifically, this embodiment gives the detailed steps of how to split the text segment to obtain multiple text phrases and obtain the pinyin of the text phrases; through the above steps, after splitting the text segment "I am Jia, Yi, Bing, Ding", the following text phrases can be obtained: "I am", "am Jia", "Jia, Yi", "Yi, Bing", "Bing, Ding", "Ding", "I am Jia", "am Jia, Yi", "Jia, Yi, Bing", "Yi, Bing, Ding", "Bing, Ding", "I am Jia, Yi", "am Jia, Yi, Bing", "Jia, Yi, Bing, Ding" and "Yi, Bing, Ding". This can exhaust all the text combinations in the text segment to obtain a more complete set of text phrases, thus ensuring the accuracy of subsequent reviews.

[0051] In the fourth embodiment of an intelligent audio review method based on speech recognition proposed by the present invention, based on the first embodiment, this embodiment further includes the following steps: Step S410: The server marks the moment when the audio to be reviewed sent by the review terminal is received as the first moment, marks the moment when the review terminal receives the target pinyin group and corresponding location information, or the review pass information as the second moment, and uses the duration between the first moment and the second moment as the review duration corresponding to the audio to be reviewed.

[0052] Step S420: The server establishes a review time estimation model.

[0053] Step S430: The server obtains the historical review information corresponding to all reviewed audios within a preset time period in the past (for example, 3 months), wherein the historical review information includes the duration of the reviewed audios, the total number of characters in all text segments corresponding to the reviewed audios, and the review duration corresponding to the reviewed audios.

[0054] Step S440: The server uses the duration of the reviewed audio in the historical review information and the total number of characters in all text segments corresponding to the reviewed audio as input parameters of the review time estimation model, and uses the review duration corresponding to the reviewed audio in the historical review information as the output parameter of the review time estimation model for training, so as to obtain a trained review time estimation model.

[0055] Step S450: The server obtains the duration of the audio to be reviewed and the total number of characters in all text segments corresponding to the audio to be reviewed, and inputs the trained review duration estimation model to obtain the output estimated review duration corresponding to the audio to be reviewed.

[0056] Step S460: The server sends the estimated review duration corresponding to the audio to be reviewed to the review terminal.

[0057] Specifically, the solution of this embodiment establishes a review time estimation model, through which the review time of the audio to be reviewed can be estimated, so that the reviewer can know the estimated review time of the audio to be reviewed in time, which increases practicality.

[0058] In a fifth embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the third embodiment, step S220 includes the following steps: Step S510: The server determines the overlap duration based on the longest word number C (e.g. 6 seconds), where the overlap duration Less than split time .

[0059] Specifically, the overlap time here That is, when splitting the audio to be reviewed, it is the duration of the overlapping period at the head and tail of two adjacent audio segments; for example, if the total duration of the audio to be reviewed is 180 minutes. Then the start time of the first audio segment is the start time of the audio to be reviewed, and the end time of the first audio segment is the 60th second of the audio to be reviewed; the start time of the second audio segment is the 54th second of the audio to be reviewed (the duration between the start time of the second audio segment and the end time of the first audio segment is the overlapping duration, which is 6 seconds), and the end time of the second audio segment is the 104th second of the audio to be reviewed.

[0060] The reason for setting the overlapping duration in this embodiment is as follows: If the audio to be reviewed is directly split, some text phrases will be lost; for example: if the text recognized by the voice at the last moment of the first audio segment is "A", and the text recognized by the voice at the earliest moment of the second audio segment is "BCD", since the first audio segment and the second audio segment are separately subjected to voice recognition, the coherent phrase "ABCD" cannot be recognized, but in fact, the phrase "ABCD" exists in the audio to be reviewed; by setting the overlapping duration, both the end of the first audio segment and the start of the second audio segment can recognize complete text phrases, avoiding the above situation of losing text phrases.

[0061] Step S520: The server determines the start time and end time of each audio segment, which specifically includes the following steps: Step S521: The server sets j = 1.

[0062] Step S522: The server takes the start time of the audio to be reviewed as the start time of the j-th audio segment, and takes the time that is after the start time of the j-th audio segment and the interval duration between it and the start time of the j-th audio segment is the splitting duration as the end time of the j-th audio segment.

[0063] Step S523: The server sets j = j + 1.

[0064] Step S524: The server takes the time that is before the end time of the (j - 1)-th audio segment and the interval duration between it and the end time of the (j - 1)-th audio segment is the overlapping duration as the start time of the j-th audio segment.

[0065] Step S525: The server determines whether the interval duration between the end time of the audio to be reviewed and the start time of the j-th audio segment is less than or equal to the splitting duration .

[0066] If so, execute Step S526: The server takes the end time of the audio to be reviewed as the end time of the j-th audio segment.

[0067] If not, execute step S527: the server will be after the start time of the jth audio segment, and the interval between the start time of the jth audio segment and the start time of the jth audio segment is the split time. The moment of is taken as the end moment of the j-th audio segment, and step S523 and subsequent steps are executed.

[0068] Specifically, through the above steps, after segmenting the audio to be reviewed with a total length of 180 minutes, the following audio segments can be obtained: The starting time of the first audio segment is the starting time of the audio to be reviewed, and the ending time of the first audio segment is the 60th second of the audio to be reviewed; the starting time of the second audio segment is the 54th second of the audio to be reviewed, and the ending time of the second audio segment is the 104th second of the audio to be reviewed; the starting time of the third audio segment is the 98th second of the audio to be reviewed, and the ending time of the third audio segment is the 158th second of the audio to be reviewed; the starting time of the fourth audio segment is the 152th second of the audio to be reviewed, and the ending time of the fourth audio segment is the 180th second of the audio to be reviewed.

[0069] Step S530: The server divides the audio to be reviewed based on the start time and the end time of each audio segment to obtain multiple audio segments.

[0070] Specifically, this embodiment provides a specific solution for how to segment the audio to be reviewed to obtain multiple audio segments.

[0071] In a sixth embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the fifth embodiment, step S510 includes the following steps: Step S610: The server obtains the normal reading speed (in words per second, e.g. 1 word per second) and based on normal reading speed Calculate the overlap time with the longest word C : , In the formula, is the overlap duration; A is the first redundant duration, which takes a positive value, such as 1 second.

[0072] Specifically, the overlap time here needs to take into account the maximum number of words C, and the overlap time is set to at least ensure that the text with the longest number of words can be read at a normal reading speed within the overlap time; The calculation formula is set as: .

[0073] In the seventh embodiment of the intelligent audio audit method based on speech recognition proposed by the present invention, based on the sixth embodiment, the step of generating position information for expressing the position of the text phrase to which the target pinyin group belongs in the audio to be audited in step S160 includes the following steps: Step S710: The server marks the text segment corresponding to the text phrase to which the target pinyin group belongs as the target text segment.

[0074] Step S720: The server obtains the sequence number of the first character in the character phrase to which the target pinyin group belongs in the target text segment. , and the sequence number of the last character in the character phrase to which the target pinyin group belongs in the target text segment .

[0075] Specifically, under normal circumstances, the speech in the audio to be reviewed is continuous, so the position of the text phrase to which the target pinyin group belongs in the corresponding text segment is also the position of the target pinyin group in the audio to be reviewed; therefore, the arrangement sequence number of the first character in the text phrase to which the target pinyin group belongs in the target text segment is , and the sequence number of the last character in the character phrase to which the target pinyin group belongs in the target text segment It can reflect the position of the text phrase belonging to the target pinyin group in the audio to be reviewed.

[0076] Step S730: The server uses the duration between the estimated appearance time of the first character in the text phrase to which the target pinyin group belongs in the audio to be reviewed and the start time of the audio to be reviewed as the first duration.

[0077] Step S740: The server uses the estimated duration of the text phrase to which the target pinyin group belongs in the audio to be reviewed as the second duration, wherein the calculation formula for the second duration is: , In the formula, is the second duration; B is the second redundant duration, which takes a positive value, for example, 2 seconds; The unit reading speed is 1 word per second, which is used to unify the units on both sides of the formula.

[0078] Specifically, by setting the second redundant time length, the estimated time period for the text phrases belonging to the target pinyin group to appear in the audio to be reviewed can be increased, thereby increasing the probability of the text phrases belonging to the target pinyin group appearing in the audio to be reviewed within the estimated time period, so as to facilitate reviewers to conduct review.

[0079] Specifically, the above calculation formula means: based on the number of sensitive words and the normal reading speed, the estimated duration of the sensitive words in the audio to be reviewed is determined.

[0080] For example, if the target pinyin group of the sensitive word "jiayibingding" that appears in a to-be-reviewed audio is "jiayibingding", that is, the target pinyin group is "jiayibingding", and the text phrase to which the target pinyin group belongs is "jiayibingding"; the duration between the estimated appearance time of the character "jia" in the sensitive word "jiayibingding" and the start time of the to-be-reviewed audio is the first duration, and the estimated duration of the sensitive word "jiayibingding" is the second duration; that is, the first duration can locate the starting position of the sensitive word "jiayibingding" in the to-be-reviewed audio, and the second duration can determine the duration of the sensitive word "jiayibingding" in the to-be-reviewed audio. Through the first duration and the second duration, the reviewer can conveniently know the position of the sensitive word "jiayibingding" in the to-be-reviewed audio.

[0081] Step S750: The server determines whether the to-be-reviewed audio corresponding to the target text segment includes multiple audio segments.

[0082] If so, execute Step S760: The server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding to-be-reviewed audio. 。

[0083] Specifically, if the to-be-reviewed audio corresponding to the target text segment includes multiple audio segments, it means that the to-be-reviewed audio corresponding to the sensitive word is divided into multiple audio segments; for the convenience of subsequent description, the audio segment corresponding to the target text segment of the target pinyin group is marked as the target audio segment. And obtain the duration between the start time of the target audio segment and the start time of the corresponding to-be-reviewed audio. 。

[0084] Step S770: The server calculates the first duration according to the following formula: , In the formula, is the first duration; is the total number of characters in the target text segment of the target pinyin group.

[0085] Specifically, the meaning of the above calculation formula is that the position of the text phrase to which the target pinyin group belongs in the corresponding text segment is also the position of the target pinyin group in the to-be-reviewed audio. Therefore, the ratio of the first character of the text phrase to which the target pinyin group belongs to the total number of characters in the target text segment of the target pinyin group is equal to the ratio of the appearance time of the first character of the text phrase to which the target pinyin group belongs to the total duration of the target audio segment.

[0086] For example, the serial number of "A" in the sensitive word "A, B, C, D" in the target text segment is 90, and the total number of characters in the target text segment of the target pinyin group is 360; the duration of the target audio segment is (60 seconds), then the estimated appearance time of "A" in the target audio segment is 15 seconds; and since the target audio segment is part of the audio to be audited, it is also necessary to add (if the target audio segment in this embodiment is the second audio segment, then is 54 seconds), and then subtract the second redundant duration, and the final first duration calculated is 67 seconds, that is, the estimated appearance time of "A" in the sensitive word "A, B, C, D" in the audio to be audited is the 67th second. In addition, according to the foregoing formula, the estimated duration of the sensitive word "A, B, C, D" in the audio to be audited can be calculated to be 8 seconds.

[0087] If not, execute step S780: The server calculates the first duration according to the following formula: , Specifically, if not, it means that there is only one audio segment in the audio to be audited (which is the audio to be audited itself). Therefore, on the basis of the foregoing formula, the first duration no longer needs to add ; for example: the serial number of "A" in the sensitive word "A, B, C, D" in the target text segment is 20, and the total number of characters in the target text segment of the target pinyin group is 60; the duration of the target audio segment (i.e., the audio to be audited) is (60 seconds), and then subtract the second redundant duration, and the final first duration calculated is 18 seconds, that is, the estimated appearance time of "A" in the sensitive word "A, B, C, D" in the audio to be audited is the 18th second. In addition, according to the foregoing formula, the estimated duration of the sensitive word "A, B, C, D" in the audio to be audited can be calculated to be 8 seconds.

[0088] Step S790: The server uses the first duration and the second duration as the position information for expressing the position of the text phrase to which the target pinyin group belongs in the audio to be audited.

[0089] In the eighth embodiment of an intelligent audio auditing method based on speech recognition proposed by the present invention, based on the seventh embodiment, after step S760, the following steps are further included: Step S810: The server calculates the average reading speed of the target audio segment of the target pinyin group based on the total number of characters in the target text segment of the target pinyin group and the segmentation duration , wherein, is the average reading speed of the target audio segment of the target pinyin group.

[0090] Step S820: The server determines whether the following condition is satisfied: the ratio of the average reading speed of the target audio segment of the target pinyin group to the normal reading speed is less than a preset lower limit (eg, 0.5).

[0091] Specifically, if the ratio of the average reading speed of the target audio segment of the target pinyin group to the normal reading speed is less than the preset lower limit, it means that the average reading speed of the target audio segment is significantly lower than the normal reading speed (under the premise that the reading is continuous, it means that there is a blank period of at least half the length in the target audio segment, that is, a period without any voice, which may be a period of pure music); that is to say, the actual voice in the target audio segment is not completely distributed in the entire target audio segment, but is concentrated in a certain period of the target audio segment; therefore, the position of the text phrase to which the target pinyin group belongs in the corresponding text segment is not the position of the target pinyin group in the audio to be reviewed, so the first duration cannot be calculated based on the formula of the seventh embodiment.

[0092] If so, step S830 is executed: the server divides the target audio segment of the target pinyin group from the middle time to obtain a first audio segment in the front and a second audio segment in the back.

[0093] Step S840: The server performs speech recognition on the first audio segment to obtain a corresponding first text segment, and performs speech recognition on the second audio segment to obtain a corresponding second text segment.

[0094] Step S850: The server calculates the total number of characters in the first text segment based on the total number of characters in the first text segment. and the total number of characters in the second text segment Calculate the first duration.

[0095] Specifically, the target audio segment is divided from the middle moment to obtain a first audio segment in front and a second audio segment in the back, based on the total number of characters in the first text segment. and the total number of characters in the second text segment It can reflect the distribution of actual speech in the target audio segment, thereby calculating the first duration.

[0096] In a ninth embodiment of an intelligent audio audit method based on speech recognition proposed by the present invention, based on the eighth embodiment, step S850 includes the following steps: Step S910: If the total number of characters in the first text segment is 0, the server calculates the first duration according to the following formula: .

[0097] Specifically, if the total number of characters in the first text segment is 0, indicating that the actual speech is all concentrated in the second text segment. Therefore, the ratio of the first character of the text phrase belonging to the target pinyin group to the total number of characters in the target text segment of the target pinyin group (that is, the total number of characters in the second text segment) is equal to the ratio of the appearance time of the first character of the text phrase belonging to the target pinyin group to half of the total duration of the target audio segment. The above formula can be obtained based on the calculation formula of the first duration in the seventh embodiment.

[0098] Step S920: If the total number of characters in the second text segment is 0, the server calculates the first duration according to the following formula: .

[0099] Specifically, if the total number of characters in the second text segment is 0, indicating that the actual speech is all concentrated in the first text segment. Therefore, the ratio of the first character of the text phrase to which the target pinyin group belongs to and the total number of characters in the target text segment of the target pinyin group (that is, the total number of characters in the first text segment) is equal to the ratio of the appearance time of the first character of the text phrase to which the target pinyin group belongs to half of the total duration of the target audio segment. The above formula can be obtained based on the calculation formula for the first duration in the seventh embodiment.

[0100] Step S930: If the total number of characters in the first text segment and the total number of characters in the second text segment are both greater than 0, the server calculates the first duration according to the following formula: .

[0101] Specifically, if the total number of characters in the first text segment and the total number of characters in the second text segment are all greater than 0, indicating that the text is distributed in both the first and second text segments, that is, the actual speech is distributed in both the first and second halves of the target audio segment; assuming that the interval between the start time of the actual speech and the middle time of the target audio segment is , and there is , the interval between the end time of the actual speech and the middle time of the target audio segment is ; The position of the text phrase to which the target pinyin group belongs in the corresponding text segment is also the position of the target pinyin group in the actual speech, and the duration of the actual speech is , then we can deduce that the interval between the first character of the text phrase belonging to the target pinyin group and the start time of the actual speech is: , and the interval between the start time of the actual speech and the start time of the target audio segment is , based on the calculation formula of the seventh embodiment, the formula for calculating the first duration in this embodiment can be derived as the above formula.

[0102] For example: If the total number of words in the first text segment is 10, and the total number of words in the second text segment is 20, the arrangement serial number of "A" in the sensitive word "A, B, C, D" in the target text segment is 5, and the total number of words in the target text segment of the target pinyin group is 30; the duration of the target audio segment is (60 seconds), then the estimated appearance time of "A" in the target audio segment is 25 seconds; and because the target audio segment is part of the audio to be audited, so it is also necessary to add (if the target audio segment in this embodiment is the second audio segment, then is 54 seconds), and then subtract the second redundant duration, and the finally calculated first duration is 77 seconds, that is, the estimated appearance time of "A" in the sensitive word "A, B, C, D" in the audio to be audited is the 77th second.

[0103] The present invention also proposes an intelligent audio audit system based on speech recognition. This system applies the intelligent audio audit method based on speech recognition; the system includes a server and an audit terminal that are communicatively connected to each other; the server stores multiple sensitive word sets, and each sensitive word set includes multiple different sensitive words, and each sensitive word includes at least 2 words.

[0104] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0105] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are only illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the purpose of the present invention and the scope protected by the claims. These all belong to the protection scope of the present invention.

Claims

1. An intelligent audio audit method based on speech recognition, characterized in that: Applicable to an intelligent audio audit system based on speech recognition; the system comprises a server and an audit terminal that are communicatively connected to each other; The server stores a sensitive word set, the sensitive word set includes a plurality of different sensitive words, and each sensitive word includes at least 2 characters; the method includes: The server determines a sensitive pinyin group and a similar pinyin group corresponding to the sensitive word set, wherein the sensitive pinyin group is a pinyin group of each sensitive word in the sensitive word set, and the similar pinyin group is a pinyin group with a pronunciation similar to the sensitive pinyin group; After the audit terminal obtains the audio to be audited, it sends the audio to be audited to the server; The server performs speech recognition on the received audio to be reviewed to obtain the corresponding text segment; The server splits and reorganizes the text segment based on the sensitive pinyin group to obtain a text phrase, and marks the pinyin of the text phrase as a text pinyin group; The server determines whether there is a text pinyin group that is consistent with the sensitive pinyin group or the similar pinyin group; If so, the server marks the text pinyin group that is consistent with the sensitive pinyin group or the similar pinyin group as the target pinyin group, generates location information for describing the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed, sends the target pinyin group and the corresponding location information to the review terminal, and marks the audio to be reviewed as reviewed audio; If not, the server generates review pass information to indicate that the audio to be reviewed does not contain sensitive words, sends the review pass information to the review terminal, and marks the audio to be reviewed as reviewed audio.

2. According to claim 1, the intelligent audio audit method based on speech recognition is characterized in that: The server performs speech recognition on the received audio to be reviewed to obtain a corresponding text segment, including: The server determines whether the received audio to be reviewed meets the segmentation condition: the duration of the audio to be reviewed is greater than the segmentation duration; If the requirements are met, the server divides the audio to be reviewed to obtain multiple audio segments; The server performs speech recognition on the audio segment to obtain multiple text segments; If not satisfied, the server performs speech recognition on the audio to be reviewed to obtain a text segment.

3. According to claim 2, an intelligent audio audit method based on speech recognition is characterized in that: The server splits and reorganizes the text segments based on the sensitive pinyin groups to obtain text phrases, and marks the pinyin of the text phrases as text pinyin groups, including: The server obtains the total number of characters S in the text segment; The server obtains the most sensitive words, and marks the most sensitive words as the longest word number C; The server sets N=1; The server sets i=1; The server combines the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a character phrase; The server sets i=i+1 and determines whether i+N≥S holds; If so, let N=N+1, and determine whether N≥C holds; If N ≥ C, obtain the pinyin of all the obtained text phrases and use them as text pinyin groups; If N≥C is not true, execute the server to set i=1 and the following steps; If not, the server is executed to combine the i-th character in the text segment and the N adjacent characters after the i-th character to obtain a character phrase, and the following steps.

4. According to claim 1, the intelligent audio audit method based on speech recognition is characterized in that: Also includes: The server marks the moment when the audit terminal receives the audited audio as the first moment, marks the moment when the audit terminal receives the target pinyin group and the corresponding position information, or the audit pass information as the second moment, and uses the duration between the first moment and the second moment as the audit duration corresponding to the audited audio; The server establishes a review time estimation model; The server obtains historical review information corresponding to all reviewed audios within a preset time period in the past, wherein the historical review information includes the duration of the reviewed audios, the total number of words in all text segments corresponding to the reviewed audios, and the review duration corresponding to the reviewed audios; The server uses the duration of the audited audio in the historical audit information and the total number of words in all text segments corresponding to the audited audio as input parameters of the audit time estimation model, and uses the audit time corresponding to the audited audio in the historical audit information as the output parameter of the audit time estimation model for training, so as to obtain a fully trained audit time estimation model; The server obtains the duration of the audio to be reviewed and the total number of characters in all text segments corresponding to the audio to be reviewed, and inputs the trained review duration estimation model to obtain the output estimated review duration corresponding to the audio to be reviewed; The server sends the estimated review time corresponding to the audio to be reviewed to the review terminal.

5. The intelligent audio audit method based on speech recognition according to claim 3 is characterized in that: If the above conditions are met, the server segments the audio to be reviewed to obtain multiple audio segments, including: The server determines the overlap duration based on the longest word number C , where the overlap duration Less than split time ; The server determines the start and end times of each audio segment, including: The server sets j=1; The server uses the start time of the audio to be reviewed as the start time of the jth audio segment, and the interval between the start time of the jth audio segment and the start time of the jth audio segment is the split duration. The moment of is taken as the end moment of the jth audio segment; The server sets j=j+1; The server will be before the end time of the j-1th audio segment, and the interval between the end time of the j-1th audio segment and the end time of the j-1th audio segment will be the overlap time. The moment of is taken as the starting moment of the jth audio segment; The server determines whether the interval between the end time of the audio to be reviewed and the start time of the jth audio segment is less than or equal to the split time. ; If yes, the server uses the end time of the audio to be reviewed as the end time of the jth audio segment; If not, the server will be after the start time of the jth audio segment, and the interval between the start time of the jth audio segment is the split duration. As the end time of the j-th audio segment, and execute the server setting j=j+1, and the subsequent steps; The server divides the audio to be reviewed based on the start time and the end time of each audio segment to obtain multiple audio segments.

6. The intelligent audio audit method based on speech recognition according to claim 5 is characterized in that: The server determines the overlap duration based on the longest word number C ,include: The server obtains the normal reading speed , and based on normal reading speed Calculate the overlap time with the longest word C : ， Where A is the first redundancy duration, which takes a positive value.

7. The intelligent audio audit method based on speech recognition according to claim 6 is characterized in that: The generating of the position information for expressing the position of the text phrase to which the target pinyin group belongs in the audio to be reviewed includes: The server marks the text segment corresponding to the text phrase to which the target pinyin group belongs as the target text segment; The server obtains the sequence number of the first character in the character phrase to which the target pinyin group belongs in the target text segment , and the sequence number of the last character in the character phrase to which the target pinyin group belongs in the target text segment ; The server uses the duration between the estimated appearance time of the first character in the character phrase to which the target pinyin group belongs in the audio to be reviewed and the start time of the audio to be reviewed as the first duration; The server uses the estimated duration of the text phrase to which the target pinyin group belongs in the audio to be reviewed as the second duration, wherein the calculation formula for the second duration is: ， In the formula, is the second duration; B is the second redundant duration, which takes a positive value; is the unit reading speed; The server determines whether the to-be-audited audio corresponding to the target text segment includes multiple audio segments; If yes, the server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be reviewed. ; The server calculates the first duration according to the following formula: ， In the formula, For the first duration; The total number of characters in the target text segment of the target pinyin group; If not, the server calculates the first duration according to the following formula: ， The server uses the first duration and the second duration as position information for expressing the position of the text phrase belonging to the target pinyin group in the audio to be reviewed.

8. The intelligent audio audit method based on speech recognition according to claim 7 is characterized in that: The server marks the audio segment corresponding to the target text segment of the target pinyin group as the target audio segment, and obtains the duration between the start time of the target audio segment and the start time of the corresponding audio to be reviewed. , and later: The total number of characters in the target text segment based on the target pinyin group of the server and split duration Calculate the average reading speed of the target audio segment of the target pinyin group; ， In the formula, The average reading speed of the target audio segment for the target pinyin group; The server determines whether the following conditions are met: the ratio of the average reading speed of the target audio segment of the target pinyin group to the normal reading speed is less than a preset lower limit; If yes, the server divides the target audio segment of the target pinyin group from the middle time to obtain a first audio segment in front and a second audio segment in the back; The server performs speech recognition on the first audio segment to obtain a corresponding first text segment, and performs speech recognition on the second audio segment to obtain a corresponding second text segment; The server based on the total number of characters in the first text segment and the total number of characters in the second text segment Calculate the first duration.

9. The intelligent audio audit method based on speech recognition according to claim 8 is characterized in that: The server is based on the total number of characters in the first text segment and the total number of characters in the second text segment Calculate the first duration, including: If the total number of characters in the first text segment is 0, the server calculates the first duration according to the following formula: ， If the total number of characters in the second text segment is 0, the server calculates the first duration according to the following formula: ， If the total number of characters in the first text segment and the total number of characters in the second text segment are both greater than 0, the server calculates the first duration according to the following formula: 。 10. An intelligent audio audit system based on speech recognition, characterized in that: An intelligent audio audit method based on speech recognition as described in any one of claims 1 to 9 is applied; the system includes a server and an audit terminal that are communicatively connected to each other; the server stores multiple sensitive word sets, the sensitive word sets include multiple sensitive words that are different from each other, and each sensitive word includes at least 2 characters.

Images