A video speech recognition method and system for rule-breaking short videos

Abstract

The application relates to the technical field of voice data processing, in particular to a video voice recognition method and system for illegal short videos, which comprises the following steps: obtaining a forbidden word library and a video to be detected, and converting Chinese characters in the forbidden word library and the video to be detected into pinyin strings without tones; the step of comparing each Chinese character to be detected with each forbidden word in the forbidden word library comprises the following steps: determining forbidden word similarity weights of each Chinese character to be detected; analyzing the closeness of the serial numbers of the most similar Chinese characters of each Chinese character to be detected and adjacent Chinese characters in the forbidden words based on the pinyin strings, combining the forbidden word similarity weights, and determining forbidden word matching values of each Chinese character to be detected; determining whether each Chinese character to be detected is a forbidden Chinese character based on the distribution of the forbidden word matching values of each Chinese character to be detected and adjacent Chinese characters; and regarding the video to be detected containing the forbidden Chinese character as an illegal video. The application aims to improve the detection accuracy and efficiency of illegal short videos.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice data processing, in particular to a video voice recognition method and system for illegal short videos. BACKGROUND

[0002] Short video illegal detection is a demand that is increasingly highlighted with the rapid development of the short video industry. The existing short video illegal detection technology usually extracts the audio track of the video content, identifies the Chinese characters in the short video by voice recognition, and completes the simple matching of Chinese characters and prohibited words.

[0003] However, when detecting illegal short videos, there may be homophonic words of prohibited words in the short video. The homophonic word problem of prohibited words in the short video refers to the fact that some users intentionally use words that are similar in pronunciation but different in spelling to express the same or similar meaning in order to evade the detection system. At the same time, due to the influence of factors such as pronunciation, accent, background noise, etc., the recognition result may deviate from the original voice. Therefore, the existing technology cannot effectively deal with the above problems, resulting in low detection accuracy of illegal short videos. SUMMARY

[0004] In view of the above, it is necessary to provide a video voice recognition method and system for illegal short videos, which improves the detection accuracy and efficiency of illegal short videos compared with traditional illegal short video detection methods:

[0005] In a first aspect, the embodiments of the present application provide a video voice recognition method for illegal short videos, which comprises the following steps:

[0006] Obtain the prohibited word library and the Chinese characters in the video to be detected, and convert the prohibited word library and the Chinese characters in the video to be detected into pinyin strings without tone;

[0007] Compare each to-be-detected Chinese character with each prohibited word in the prohibited word library to determine whether each to-be-detected Chinese character is a prohibited Chinese character, and the specific steps are as follows:

[0008] (1) Based on the pinyin strings of the to-be-detected Chinese characters and the adjacent Chinese characters, the structural similarity between the pinyin strings of the to-be-detected Chinese characters and the pinyin strings of the prohibited words, and the similarity of the occurrence of the same characters, the structural similarity and the character similarity of each to-be-detected Chinese character are obtained, and then the prohibited word similarity weight of each to-be-detected Chinese character is determined;

[0009] (2) Based on the pinyin string analysis, the closeness of the sequence number of the most similar Chinese character of each to-be-detected Chinese character and the adjacent Chinese characters in the prohibited word is determined, and the prohibited word matching value of each to-be-detected Chinese character is determined in combination with the prohibited word similarity weight;

[0010] (3) determining whether each to-be-detected Chinese character is a forbidden Chinese character based on a distribution of the forbidden word matching value of each to-be-detected Chinese character and a neighboring Chinese character;

[0011] The to-be-detected video in which the forbidden Chinese character appears is regarded as a rule violation video.

[0012] In one embodiment, the structure similarity determination process is as follows:

[0013] The pinyin number sequence of the forbidden word is obtained based on the pinyin string of the forbidden word.

[0014] For each to-be-detected Chinese character, the pinyin number sequence of the to-be-detected vocabulary composed of the to-be-detected Chinese character and a neighboring Chinese character is obtained according to the obtaining method of the pinyin number sequence of the forbidden word, and the pinyin number sequence of the to-be-detected vocabulary is slidingly intercepted into a plurality of intercepted sequences equal in length to the pinyin number sequence of the forbidden word.

[0015] The difference in fractal dimension between the pinyin number sequence of the forbidden word and each intercepted sequence is calculated respectively and recorded as a fractal difference.

[0016] The maximum value of all the fractal differences of the to-be-detected Chinese character is taken as the structure similarity of the to-be-detected Chinese character.

[0017] In one embodiment, the obtaining method of the pinyin number sequence of the forbidden word is as follows: all the letters in the pinyin string of the forbidden word are numbered in the order of the 26-letter alphabet and arranged in the order of Chinese characters to form the pinyin number sequence of the forbidden word.

[0018] In one embodiment, the character similarity determination process is as follows:

[0019] The number and occurrence frequency and average occurrence interval of each number in each intercepted sequence are taken as the local feature value of each number in each intercepted sequence, and the local feature value of each number in the pinyin number sequence of the forbidden word is obtained according to the obtaining method of the local feature value of each number in each intercepted sequence.

[0020] The average value of the difference in local feature value of all the same numbers between the pinyin number sequence of the forbidden word and each intercepted sequence is calculated, and the maximum value of all the average values of the to-be-detected Chinese character is taken as the character similarity of the to-be-detected Chinese character.

[0021] In one embodiment, the forbidden word similarity weight calculation method is as follows:

[0022] The structure similarity and the character similarity of each to-be-detected Chinese character are fused, and the inverse of the sum of the predetermined value greater than 0 is taken as the normalization value of the forbidden word similarity weight of each to-be-detected Chinese character.

[0023] In one embodiment, the determination process of the forbidden word matching value is as follows:

[0024] All the letters in the pinyin string of each Chinese character are numbered in the order of the 26-letter alphabet and arranged in the order of pronunciation to form the pinyin number sequence of each Chinese character;

[0025] The sum value of the difference values of all the same position elements between the pinyin number sequence of the to-be-detected Chinese character and the pinyin number sequence of any Chinese character in the forbidden word is calculated, the serial number of the forbidden word Chinese character corresponding to the minimum value among all the sum values of the to-be-detected Chinese character is counted, and is recorded as a first serial number;

[0026] According to the first serial number, a second serial number and a third serial number are obtained for the adjacent Chinese characters on the left and right sides of the to-be-detected Chinese character, and the difference between the average of the second serial number and the third serial number and the first serial number is calculated;

[0027] Based on the minimum value and the difference, the local consistency value between the to-be-detected Chinese character and the forbidden word is determined.

[0028] The fusion result of the local consistency value and the forbidden word similarity weight is taken as the forbidden word matching value of the to-be-detected Chinese character.

[0029] In one embodiment, the calculation method of the local consistency value is as follows: the inverse of the fusion result of the minimum value and the difference is taken as the index of an exponential function with a natural constant as the base, and the calculation result of the exponential function is taken as the local consistency value between the to-be-detected Chinese character and the forbidden word.

[0030] In one embodiment, the process of determining whether each to-be-detected Chinese character is a forbidden Chinese character is as follows:

[0031] The length of the forbidden word is recorded as the control length, and for each to-be-detected Chinese character, the to-be-detected Chinese character and the adjacent Chinese characters form the control length of the word with a length of the control length, which is recorded as the control word.

[0032] The average value of the forbidden word matching value of each Chinese character in each control word is calculated respectively.

[0033] Each to-be-detected Chinese character is compared with all the forbidden words in the forbidden word library to obtain all the average values of each to-be-detected Chinese character, and whether each to-be-detected Chinese character is a forbidden Chinese character is determined based on all the average values of each to-be-detected Chinese character.

[0034] In one embodiment, the determination method of the forbidden Chinese character is as follows: when the number of average values of any to-be-detected Chinese character that is greater than the preset matching threshold is not 0, the any to-be-detected Chinese character is determined to be a forbidden Chinese character, otherwise, it is a non-forbidden Chinese character.

[0035] In a second aspect, the embodiments of the present application further provide a video and voice recognition system for short videos in violation, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the video and voice recognition method for short videos in violation according to any one of the above embodiments when executing the computer program.

[0036] The present application has at least the following beneficial effects:

[0037] The present application uses the difference in fractal dimension and the difference in local eigenvalue to reflect the similarity between the pinyin string of the detected word and the forbidden word in structure and character level, thereby improving the accuracy of detecting homophonic words and biased voice recognition results.

[0038] Further, the local consistency of the detected Chinese character and adjacent Chinese characters with the forbidden word is analyzed, the closeness of the serial number of the most similar Chinese character of the detected Chinese character and adjacent Chinese characters in the forbidden word is calculated, that is, the probability that the detected Chinese character and adjacent Chinese characters form a forbidden word, and the overall similarity is combined, thereby further improving the accuracy of detecting whether the detected Chinese character can form a forbidden word with adjacent Chinese characters.

[0039] Further, in order to quickly and accurately identify the violation content from a large number of videos, the forbidden judgment is performed on each detected Chinese character, and the detected video in which the forbidden Chinese character appears is regarded as a violation video, thereby realizing efficient screening of short video content and improving the accuracy and efficiency of violation short video content detection. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0041] Figure 1 A step flowchart of a video and voice recognition method for short videos in violation provided by an embodiment of the present application;

[0042] Figure 2 A schematic diagram of the determination process of the forbidden word matching value;

[0043] Figure 3 A schematic diagram of the determination process of whether the detected Chinese character is a forbidden Chinese character. DETAILED DESCRIPTION

[0044] In the description of the embodiments of the present application, the words "exemplary", "or", "for example", etc. are used to mean as an example, instance, or illustration. Any embodiment or design presented as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or design solutions. Rather, use of the words "exemplary", "or", "for example" is intended to present related concepts in a concrete manner.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. It is to be understood that the use of the singular herein includes the plural unless specifically stated otherwise.

[0046] In addition, it should be pointed out that the terms "first", "second" in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0047] The specific scheme of a video voice recognition method and system for illegal short videos provided by the present application will be specifically described below in combination with the drawings.

[0048] Please refer to Figure 1 which shows a step flowchart of a video voice recognition method for illegal short videos provided by an embodiment of the present application, which includes the following steps:

[0049] Step 1, obtain the forbidden word library and the Chinese characters in the video to be detected, and convert the forbidden word library and the Chinese characters in the video to be detected into pinyin strings without tone.

[0050] Obtain the forbidden word library from an open source website, and convert all Chinese characters in the forbidden word library into pinyin strings without tone using the pypinyin library.

[0051] Collect all the videos to be detected, and uniformly convert the videos to be detected into MP4 format for batch processing. Use audio editing software or programming library, such as FFmpeg, Libav, etc., to extract the audio track from the video to be detected, and convert all the extracted audios into MP3 format. Convert all stereo audios into monaural format for subsequent analysis.

[0052] Pretreat all audios, including audio denoising and de-echo operation, then convert all the pretreated audios into Chinese characters using the Whisper model, and finally convert each Chinese character obtained by conversion into a pinyin string without tone using the pypinyin library.

[0053] In this embodiment, the minimum mean square adaptive filter is used for audio de-noising and de-echo operation. On the basis of realizing the audio de-noising and de-echo operation, the implementer can use other existing technologies as the implementation manner, and the present application does not make special limitation.

[0054] Step 2, compare each to-be-detected Chinese character with each forbidden word in the forbidden word library respectively, and determine whether each to-be-detected Chinese character is a forbidden Chinese character.

[0055] Due to the influence of factors such as pronunciation, accent, background noise and the like, the speech recognition technology may be affected, resulting in deviation between the recognition result and the original speech. In addition, there are a large number of homophonic different words in Chinese language, and simple matching is easy to misjudge or miss. Therefore, by extracting the features of the forbidden words and the words appearing in the short video, the deep attributes of the words such as syllable composition can be captured, so as to perform more accurate matching at the semantic level.

[0056] Taking any one forbidden word in the forbidden word library as an example, the specific steps of determining whether each to-be-detected Chinese character is a forbidden Chinese character are as follows:

[0057] Step 2.1, based on the similarity in structure between the pinyin string of each to-be-detected Chinese character and the pinyin string of the forbidden word, and the similarity of the occurrence of the same characters, the structural similarity and the character similarity of each to-be-detected Chinese character are obtained respectively, and then the forbidden word similarity weight of each to-be-detected Chinese character is determined.

[0058] Taking each to-be-detected Chinese character as the center, a window with a size of 2n-1 is constructed, where n is the number of Chinese characters in the forbidden word. The pinyin of each Chinese character in the window is composed of initial and final, so that the pinyin has structural diversity. All the Chinese characters in the window form a to-be-detected word, and all the letters in the pinyin string of the to-be-detected word are numbered in the order of 26 letter table and arranged in the order of Chinese characters to form the pinyin number sequence of the to-be-detected word. At the same time, all the letters in the pinyin string of the forbidden word are numbered in the order of 26 letter table and arranged in the order of Chinese characters to form the pinyin number sequence of the forbidden word. For example, if the to-be-detected word is “forbidden word”, its pinyin is “weijinci”, and the converted pinyin number sequence is “23 5 9 10 9 14 3 9”.

[0059] It should be understood that when the window exceeds the boundary of the audio converted into Chinese characters, the window is truncated.

[0060] For each Chinese character to be detected, in order to keep the pinyin number sequence of the word to be detected consistent with the length of the pinyin number sequence of the forbidden word, the pinyin number sequence of the word to be detected is cut into multiple cut sequences with the same length as the pinyin number sequence of the forbidden word, the difference in fractal dimension between the pinyin number sequence of the forbidden word and each cut sequence is calculated respectively and recorded as a fractal difference. The calculation of the fractal dimension is a known technology, which will not be described herein. The maximum value of all the fractal differences of the Chinese character to be detected is taken as the structural similarity of the Chinese character to be detected.

[0061] In order to reduce the misjudgment caused by factors such as pronunciation, accent or background noise, the number and its occurrence frequency and average occurrence interval of each number in the pinyin number sequence of the forbidden word and each cut sequence are extracted. The number and its occurrence frequency and average occurrence interval of each number in each cut sequence are taken as the local feature value of each number in each cut sequence; the local feature value of each number in the pinyin number sequence of the forbidden word is obtained by using the obtaining method of the local feature value of each number in each cut sequence. For example, in the pinyin number sequence "23 5 9 10 9 14 39" of the forbidden word, the occurrence frequency of number 1 is 0 and the average occurrence interval is 0, so the local feature value of number 1 is "100"; the occurrence frequency of number 9 is 3 and the average occurrence interval is 1.5, so the local feature value of number 9 is 931.5. The difference in local feature value of each same number between the pinyin number sequence of the forbidden word and each cut sequence is calculated respectively, and the average of the difference in local feature value of all same numbers between the pinyin number sequence of the forbidden word and each cut sequence is calculated. The maximum value of all the averages of the Chinese character to be detected is taken as the character similarity of the Chinese character to be detected.

[0062] Further, the structural similarity and the character similarity of the Chinese character to be detected are fused, and the fusion result is taken as the forbidden word similarity weight of the Chinese character to be detected, which is the reciprocal of the sum of the preset value greater than 0. The preset value greater than 0 is used to avoid the denominator being 0.

[0063] In this embodiment, the difference between the fractal dimensions and the difference between the local feature values are both the absolute value of the difference, as other embodiments, on the basis of being able to measure the difference between the fractal dimensions and the difference between the local feature values, the implementer can use other calculation methods, such as ratio relationship, and the present application does not make special limitation.

[0064] It should be understood that fusion means combining multiple independent variables together in a way to enhance the overall effect, such as addition relationship, multiplication relationship, etc., and the implementer can limit it according to the actual situation.

[0065] In this embodiment, the product of the structural similarity and character similarity of the Chinese character to be detected is calculated. The normalized value of the sum of the product and a preset value greater than 0 is used as the prohibited word similarity weight of the Chinese character to be detected. The calculation formula is as follows: In the formula, A represents the similarity weight of the Chinese characters to be detected for prohibited words, norm() represents the normalization operation, B represents the structural similarity of the Chinese characters to be detected, and C represents the character similarity of the Chinese characters to be detected. This indicates a preset value greater than 0, the purpose of which is to avoid a denominator of 0. The value is preset by a person, and the implementer can set it himself. In this embodiment... The value is 0.01.

[0066] In this embodiment, the Min-Max normalization method is used for normalization. The Min-Max normalization method is a well-known technique and will not be described in detail here.

[0067] It should be noted that: fractal dimension measures the structural complexity of the pinyin string; fractal difference reflects the structural similarity between the truncated sequence of the pinyin number sequence of the word to be tested and the pinyin number sequence of the prohibited word; the larger the fractal difference, the smaller the similarity between the two in terms of pinyin structure, and the smaller the similarity weight of the prohibited word; the frequency of use and distribution pattern of the numbers are reflected by all numbers and their occurrence frequency and average occurrence interval, thus characterizing the similarity at the character level between the truncated sequence of the pinyin number sequence of the word to be tested and the pinyin number sequence of the prohibited word; the larger the mean, the smaller the character similarity between the truncated sequence and the pinyin number sequence of the prohibited word, and the smaller the similarity weight of the prohibited word; the similarity weight of the prohibited word reflects the overall similarity between the pinyin number sequence of the word to be tested and the pinyin number sequence of the prohibited word; the closer the value of the similarity weight of the prohibited word is to 1, the higher the similarity, and vice versa.

[0068] Step 2.2: Based on the analysis of the pinyin string, analyze the similarity between the sequence number of the most similar Chinese characters in the prohibited words for each Chinese character to be detected and its adjacent Chinese characters, and combine the prohibited word similarity weight to determine the prohibited word matching value for each Chinese character to be detected.

[0069] Number all letters in the pinyin string of each Chinese character according to the order of the 26 letters of the alphabet, and arrange them according to the order of pronunciation to form the pinyin number sequence of each Chinese character;

[0070] The sum of the difference values of all identical position elements between the pinyin number sequence of the to-be-detected Chinese character and the pinyin number sequence of any Chinese character in the forbidden word is calculated, and the minimum value of all the sum values of the to-be-detected Chinese character is recorded as the first sequence number, wherein if the length of the pinyin number sequence of the to-be-detected Chinese character and the length of the pinyin number sequence of any Chinese character in the forbidden word are not equal, 0 is added to the tail of the pinyin number sequence with shorter length. For example, when the forbidden word is “not allowed”, the sequence numbers of the Chinese characters “not”, “allowed” and “allowed” in the forbidden word are 1, 2 and 3 respectively.

[0071] The second sequence number and the third sequence number are obtained by using the same method as the first sequence number for the Chinese characters adjacent to the left and right sides of the to-be-detected Chinese character.

[0072] The difference between the average of the second sequence number and the third sequence number and the first sequence number is recorded as the forbidden sequence number difference.

[0073] The inverse of the fusion result of the minimum value and the forbidden sequence number difference is taken as the index of the exponential function with the natural constant as the base number, and the calculation result of the exponential function is taken as the local consistency value between the to-be-detected Chinese character and the forbidden word.

[0074] The fusion result of the local consistency value and the forbidden word similarity weight is taken as the forbidden word matching value of the to-be-detected Chinese character.

[0075] In this embodiment, the difference value between elements is the absolute value of the difference.

[0076] In this embodiment, the difference value is the absolute value of the difference, and the calculation formula of the forbidden sequence number difference is , wherein y represents the forbidden sequence number difference, respectively represent the first sequence number, the second sequence number and the third sequence number; as another embodiment, on the basis of being able to distinguish between and z, the implementer can use other calculation methods, such as ratio relationship, and the present application does not make special limitation.

[0077] In this embodiment, the inverse of the cumulative sum of the minimum value and the forbidden sequence number difference is taken as the index of the exponential function with the natural constant as the base number, and the calculation result of the exponential function is taken as the local consistency value between the to-be-detected Chinese character and the forbidden word.

[0078] In this embodiment, the product of the local consistency value and the forbidden word similarity weight is taken as the forbidden word matching value of the to-be-detected Chinese character.

[0079] It should be noted that: and value quantifies the difference between the two pinyin number sequence, and the smaller the value, the higher the matching degree of the to-be-detected Chinese character and the certain Chinese character in the prohibited word; the prohibited sequence difference reflects the proximity of the sequence numbers of the to-be-detected Chinese character and the adjacent Chinese character to the most similar Chinese character in the prohibited word, the smaller the prohibited sequence difference, the greater the probability that the to-be-detected Chinese character and the adjacent Chinese character form a prohibited word; the local consistency value reflects the matching degree of the to-be-detected Chinese character and the adjacent Chinese character and the Chinese character in the prohibited word, the greater the local consistency value, the higher the matching degree of the to-be-detected Chinese character and the adjacent Chinese character and the prohibited word, and the greater the prohibited word matching value; the prohibited word similarity weight reflects the overall similarity between the pinyin number sequence of the to-be-detected word and the pinyin number sequence of the prohibited word, the greater the prohibited word similarity weight, the greater the prohibited word matching value. The determination process of the prohibited word matching value is shown in Figure 2

[0080] Step 2.3, based on the distribution of the prohibited word matching value of each to-be-detected Chinese character and the adjacent Chinese character, determine whether each to-be-detected Chinese character is a prohibited Chinese character.

[0081] If the length of the prohibited word is n, for each to-be-detected Chinese character, the to-be-detected Chinese character and the adjacent Chinese character form n words with a length of n, denoted as control words, wherein the to-be-detected Chinese character is located at the end of the control word, the beginning of the control word, and the middle of the control word.

[0082] The average value of the prohibited word matching value of all Chinese characters in each control word is calculated respectively.

[0083] Each to-be-detected Chinese character is compared with all prohibited words in the prohibited word library to obtain all the average values of each to-be-detected Chinese character.

[0084] When the number of the average value of any to-be-detected Chinese character greater than the preset matching threshold is not 0, the pronunciation and structure of the any to-be-detected Chinese character and the prohibited word are similar, and it is determined that the any to-be-detected Chinese character is a prohibited Chinese character, otherwise, it is a non-prohibited Chinese character. The determination process of whether the to-be-detected Chinese character is a prohibited Chinese character is shown in Figure 3

[0085] In this embodiment, the value of the preset matching threshold is 0.8, and the value of the preset matching threshold is preset by human, which can be set by the implementer, and the present application does not make special restrictions.

[0086] All to-be-detected videos are screened, and the to-be-detected video in which the prohibited Chinese character appears is taken as a rule-violating video.

[0087] ​​Based on the same inventive concept as the above method, the embodiments of the present application also provide a video voice recognition system for illegal short videos, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the above methods for video voice recognition of illegal short videos when executing the computer program.

[0088] To sum up, the present application uses the difference in fractal dimension and the difference in local feature value to reflect the similarity between the pinyin string of the detected word and the forbidden word in structure and character level, thereby improving the accuracy of detecting homophonic words and voice recognition results with deviations;

[0089] Further, the local consistency degree of the detected Chinese character and adjacent Chinese characters with the forbidden word is analyzed, the closeness degree of the detected Chinese character and adjacent Chinese characters to the serial number of the most similar Chinese character in the illegal word is calculated, that is, the probability that the detected Chinese character and adjacent Chinese characters form a forbidden word, and the overall similarity degree is combined, thereby further improving the accuracy of detecting whether the detected Chinese character can form a forbidden word with adjacent Chinese characters;

[0090] Further, in view of the demand for quickly and accurately identifying illegal content from a large number of videos, the forbidden judgment is performed on each detected Chinese character, and the detected video in which the forbidden Chinese character appears is regarded as an illegal video, thereby realizing efficient screening of short video content and improving the accuracy and efficiency of illegal short video content detection.

[0091] The flowcharts and block diagrams in the drawings show the possible implementation architecture, function and operation of the system, method and computer program product according to the embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code containing one or more executable instructions for implementing a specified logic function. In some alternative implementations, the functions noted in the blocks can also occur in an order different from that noted in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. In the description corresponding to the flowcharts and block diagrams in the drawings, the operations or steps corresponding to different blocks can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. Each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0092] It is apparent to a person skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, but can be implemented in other concrete forms without departing from the essential characteristics of the present application. Therefore, the above-described embodiments of the present application should be considered in all respects as illustrative and not restrictive.

Claims

1. A method for video speech recognition of a short video in violation, characterized in that, The method comprises the following steps: obtaining a forbidden word library and Chinese characters in a video to be detected, and converting the forbidden word library and the Chinese characters in the video to be detected into pinyin strings without tones; comparing each Chinese character to be detected with each forbidden word in the forbidden word library to determine whether each Chinese character to be detected is a forbidden Chinese character, and the specific steps are as follows: (1) based on the pinyin string of the word composed of each Chinese character to be detected and adjacent Chinese characters, the structural similarity between the pinyin string of the word and the pinyin string of the forbidden word, and the similarity of the occurrence of the same characters, the structural similarity of each Chinese character to be detected and the character similarity are obtained, and then the forbidden word similarity weight of each Chinese character to be detected is determined; (2) based on the pinyin string, the closeness of the serial number of the most similar Chinese character of each Chinese character to be detected and adjacent Chinese characters in the forbidden word is analyzed, and the forbidden word matching value of each Chinese character to be detected is determined in combination with the forbidden word similarity weight; (3) based on the distribution of the forbidden word matching value of each Chinese character to be detected and adjacent Chinese characters, it is determined whether each Chinese character to be detected is a forbidden Chinese character; the video to be detected in which the forbidden Chinese character appears is regarded as a rule-violating video; the determination process of the structural similarity is as follows: obtaining the pinyin number sequence of the forbidden word based on the pinyin string of the forbidden word; for each Chinese character to be detected, the pinyin number sequence of the word to be detected composed of the Chinese character to be detected and adjacent Chinese characters is obtained according to the obtaining method of the pinyin number sequence of the forbidden word, and the pinyin number sequence of the word to be detected is slid and cut into multiple cut sequences with the same length as the pinyin number sequence of the forbidden word; the difference in fractal dimension between the pinyin number sequence of the forbidden word and each cut sequence is calculated respectively, and is recorded as a fractal difference; the maximum value of all the fractal differences of the Chinese character to be detected is taken as the structural similarity of the Chinese character to be detected; the obtaining method of the pinyin number sequence of the forbidden word is as follows: all the letters in the pinyin string of the forbidden word are numbered in the order of the 26-letter table, and are arranged in the order of Chinese characters to form the pinyin number sequence of the forbidden word; the determination process of the forbidden word matching value is as follows: all the letters in the pinyin string of each Chinese character are numbered in the order of the 26-letter table, and are arranged in the order of pronunciation to form the pinyin number sequence of each Chinese character; the sum of the difference values of all the same position elements between the pinyin number sequence of the Chinese character to be detected and any Chinese character in the forbidden word is calculated, the serial number of the forbidden word Chinese character corresponding to the minimum value of all the sum values of the Chinese character to be detected in the forbidden word is counted, and is recorded as a first serial number; for the Chinese characters adjacent to the left and right sides of the Chinese character to be detected, the second serial number and the third serial number are obtained according to the obtaining method of the first serial number, and the difference between the mean value of the second serial number and the third serial number and the first serial number is calculated; based on the minimum value and the difference, the local consistency value between the Chinese character to be detected and the forbidden word is determined; the fusion result of the local consistency value and the forbidden word similarity weight is taken as the forbidden word matching value of the Chinese character to be detected.

2. The method for video speech recognition of a short video in violation according to claim 1, wherein, the determination process of the character similarity is as follows: The local characteristic values of each number in each intercepted sequence are composed of the number and the occurrence frequency and the average interval of occurrence of each number in each intercepted sequence; the local characteristic values of each number in the pinyin number sequence of the forbidden word are obtained according to the method for obtaining the local characteristic values of each number in each intercepted sequence; The average of the differences of the local characteristic values of all the same numbers between the pinyin number sequence of the forbidden word and each intercepted sequence is calculated; the maximum value of all the averages of the to-be-detected Chinese character is taken as the character similarity of the to-be-detected Chinese character.

3. The method for video speech recognition of a short video in violation as claimed in claim 1, wherein, The calculation method of the forbidden word similarity weight is: The structure similarity and the character similarity of each to-be-detected Chinese character are fused, and the reciprocal of the sum of the normalization value of the fusion result and a preset value greater than 0 is taken as the forbidden word similarity weight of each to-be-detected Chinese character.

4. The method for video speech recognition of a short video in violation according to claim 1, wherein, The calculation method of the local consistency value is: the reciprocal of the fusion result of the minimum value and the difference amount is taken as the index of an exponential function with a natural constant as the base number, and the calculation result of the exponential function is taken as the local consistency value between the to-be-detected Chinese character and the forbidden word.

5. The method for video speech recognition of a short video in violation as claimed in claim 1, wherein, The process of determining whether each to-be-detected Chinese character is a forbidden Chinese character is: The length of the forbidden word is recorded as the control length, and for each to-be-detected Chinese character, the to-be-detected Chinese character and the adjacent Chinese character form the control length number of words with a length of the control length, which is recorded as the control word; The average value of the forbidden word matching value of all Chinese characters in each control word is calculated respectively; Each to-be-detected Chinese character is compared with all forbidden words in the forbidden word library to obtain all the average values of each to-be-detected Chinese character; whether each to-be-detected Chinese character is a forbidden Chinese character is determined based on all the average values of each to-be-detected Chinese character.

6. The method for video speech recognition of a short video in violation as claimed in claim 5, wherein, The determination method of the forbidden Chinese character is: when the number of average values greater than the preset matching threshold of any to-be-detected Chinese character is not 0, the any to-be-detected Chinese character is determined as a forbidden Chinese character, otherwise, it is a non-forbidden Chinese character.

7. A video speech recognition system for short video violation, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, The processor executes the computer program to realize the steps of the video voice recognition method for a short video in violation of regulations according to any one of claims 1-6.

Images