Method and apparatus for processing video data

Abstract

Embodiments of the present disclosure provide a video data processing method and device, relating to the technical field of computer, which solves the problem of high error rate caused by current manual acceptance of video. The method comprises: obtaining video data to be put, wherein the video data comprises oral broadcast data and picture data; performing segmentation on the oral broadcast data through an asr interface and an open-source punctuation sentence segmentation algorithm to obtain segmented asr text; filtering invalid information in the picture data through an ocr interface and preset invalid information to obtain filtered ocr text; and obtaining a matching result of the video data according to the segmented asr text, the filtered ocr text and a preset word matching rule. The embodiments of the present disclosure are suitable for the acceptance process of brand parties for the video to be put.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to a method and apparatus for processing video data. Background Technology

[0002] To standardize the quality of videos across their internal brand campaigns, brands need to control the quality of videos distributed across different social media channels to ensure that the product's selling points are fully showcased. Video production typically involves the brand providing rules or templates, such as the required key information to be included in the video, the placement of different information at different times and locations within the video, and the minimum duration for which this information must be displayed.

[0003] After the rules or templates are established, the re-streamed videos need to be checked according to the rules or templates to see if each video meets the brand's requirements. However, currently, most of the time, manual inspection is used to check each video one by one. Since it is necessary to check both the visuals and the narration in the video, the error rate of manual inspection is relatively large. Summary of the Invention

[0004] The purpose of the embodiments disclosed herein is to provide a video data processing method and device that solves the problem of a relatively high error rate caused by manual video acceptance. By automatically performing mutual verification of spoken data and video data, invalid information in the video is removed, thereby improving the accuracy of the video acceptance process.

[0005] To achieve the above objectives, according to a first aspect of this disclosure, a method for processing video data is provided, comprising: acquiring video data to be played, the video data including spoken data and video data; segmenting the spoken data using an ASR interface and an open-source punctuation segmentation algorithm to obtain segmented ASR text; filtering invalid information in the video data using an OCR interface and preset invalid information to obtain filtered OCR text; and obtaining a matching result of the video data based on the segmented ASR text, the filtered OCR text, and preset word matching rules.

[0006] In some embodiments of this disclosure, the step of segmenting the spoken data using the ASR interface and an open-source punctuation segmentation algorithm to obtain segmented ASR text includes: obtaining a first set of segmentation points corresponding to the spoken data through the ASR interface; merging the segments in the first set of segmentation points, and segmenting the merged text using the open-source punctuation segmentation algorithm to obtain a second set of segmentation points; and finding the intersection of the first set of segmentation points and the second set of segmentation points to obtain the segmented ASR text.

[0007] In some embodiments of this disclosure, the step of filtering invalid information in the image data through an OCR interface and preset invalid information to obtain filtered OCR text includes: converting the image data into OCR text through an OCR interface; filtering invalid information in the OCR text based on text coordinate information in the OCR text and position and proportion information in the preset invalid information to obtain filtered OCR text.

[0008] In some embodiments of this disclosure, obtaining the matching result of the video data based on the segmented ASR text, the filtered OCR text, and preset word matching rules includes: using a trie matching algorithm to match the segmented ASR text and the filtered OCR text with the specified words in the preset word matching rules to obtain a first matching result; using a pinyin pronunciation matching algorithm to match the segmented ASR text with the specified words in the preset word matching rules for the same pronunciation to obtain a second matching result; and searching for words that meet the requirement in the preset word matching rules, where specified words in the spoken data and the video data must appear simultaneously, to obtain the matching result of the video data.

[0009] In some embodiments of this disclosure, the method further includes: extracting the image data frame by frame and obtaining the OCR text of all frames through an OCR interface; performing deduplication processing on the OCR text of all frames according to a preset text deduplication rule and the DBscan clustering algorithm to obtain the OCR text after the first cleaning; obtaining the ASR text corresponding to the broadcast data through an ASR interface; obtaining the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning using information entropy and a specified threshold; and filtering the ASR segmentation result and the OCR segmentation result using the TF-IDF algorithm and a specified part-of-speech tag to obtain a new ASR word package and a new OCR word package.

[0010] In some embodiments of this disclosure, the step of deduplicating the OCR text of all frames according to preset text deduplication rules and the DBscan clustering algorithm to obtain the OCR text after the first cleaning includes: marking the positions of OCR text that meet the conditions in a two-dimensional matrix according to preset overlap, longest common subsequence ratio and text similarity in the preset text deduplication rules to obtain OCR text elements in the two-dimensional matrix; and using the DBscan clustering algorithm to cluster the OCR text elements in the two-dimensional matrix to obtain the OCR text after the first cleaning.

[0011] In some embodiments of this disclosure, obtaining the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning using information entropy and a specified threshold includes: calculating the information entropy between every two adjacent characters in the ASR text and the OCR text after the first cleaning; filtering the information entropy using the specified threshold to obtain the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning.

[0012] In some embodiments of this disclosure, the step of using the TF-IDF algorithm and specified part-of-speech tags to filter the ASR and OCR word segmentation results to obtain new ASR and OCR word packages includes: using the TF-IDF algorithm to calculate the scores of words in the ASR and OCR word segmentation results, and sorting them according to the scores to obtain ASR and OCR word segments sorted from high to low importance; and using the specified part-of-speech tags to filter the ASR and OCR word segments to obtain the new ASR and OCR word packages.

[0013] In some embodiments of this disclosure, the method further includes: matching the segmented ASR text and the filtered OCR text with the ASR new word package and the OCR new word package respectively to obtain the new word matching frequency.

[0014] According to a second aspect of this disclosure, a video data processing device is provided, including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring projected video data, the video data including spoken data and video data; segmenting the spoken data using an ASR interface and an open-source punctuation segmentation algorithm to obtain segmented ASR text; filtering invalid information in the video data using an OCR interface and preset invalid information to obtain filtered OCR text; and obtaining a matching result for the video data based on the segmented ASR text, the filtered OCR text, and preset word matching rules.

[0015] According to a third aspect of this disclosure, a machine-readable storage medium is provided that stores instructions for causing a machine to perform the video data processing method described above.

[0016] Other features and advantages of the embodiments of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate embodiments of the present disclosure and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present disclosure, but do not constitute a limitation on the embodiments of the present disclosure. In the drawings:

[0018] Figure 1 This is a schematic flowchart of a video data processing method provided according to an embodiment of the present disclosure;

[0019] Figure 2 This is a schematic diagram of the extended packet matching process according to an embodiment of the present disclosure;

[0020] Figure 3 This is a schematic diagram of OCR text deduplication according to an embodiment of the present disclosure;

[0021] Figure 4 This is a schematic diagram of OCR text element clustering according to an embodiment of the present disclosure;

[0022] Figure 5 This is a schematic diagram illustrating the frequency of all new words appearing in different videos, based on embodiments of this disclosure;

[0023] Figure 6 This is a schematic block diagram of a video data processing apparatus provided according to embodiments of the present disclosure.

[0024] The elements in the attached diagram are schematic and not drawn to scale. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are also within the scope of protection of this disclosure.

[0026] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having meanings consistent with their meanings in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. Furthermore, terms such as “first” and “second” are used only to distinguish one element (or part of an element) from another element (or another part of an element).

[0027] Figure 1It is a schematic flowchart of a method for processing video data provided by an embodiment of the present disclosure. As Figure 1 shown, the method includes the following steps:

[0028] Step S110, obtain the video data to be launched, where the video data includes oral broadcast data and picture data;

[0029] Step S120, split the oral broadcast data through the asr interface and an open-source punctuation-based sentence splitting algorithm to obtain the split asr text;

[0030] Step S130, filter out the invalid information in the picture data through the ocr interface and preset invalid information to obtain the filtered ocr text;

[0031] Step S140, obtain the matching result of the video data according to the split asr text, the filtered ocr text, and a preset word matching rule.

[0032] For the video data to be launched, the brand party will propose production rules. For a certain product line, the necessary keywords and core keywords of the video data are given. Among them, the necessary keywords are the words that the video data must contain, and the core keywords are the words that the video data must contain at least a specified number of times. For example, it is required that the video must contain one necessary keyword and at least one core keyword, or it is required that the video must contain two necessary keywords and at least one core keyword, etc. For example, the necessary keyword information given by the brand party is "Dove" and "Amino Bubble Cleanser", and the core keyword information given is "Skin Conditioning", "Skin Protection", "Deep Cleansing", "Gentle Cleansing", "Non-irritating". Then, according to the requirements of the brand party, if it is detected that the video contains two necessary keywords and any one of the five core keywords, the video is qualified.

[0033] In the embodiment of the present disclosure, according to the word package provided by the brand party, for example, the necessary keyword is A, and the core keywords are B, C, and D, and it is required to contain at least one core keyword. Then the preset word matching rule is A + B, or A + C, or A + D. In addition, according to the production rules provided by the brand party, for the specified words that must appear in both the oral broadcast data and the picture data, the specified words are set to appear at the same time in the split asr text and the filtered ocr text in the preset word matching rule.

[0034] For step S120, the spoken data is generally segmented according to semantics, with the granularity needing to be set appropriately. For example, the desired segmentation of "I often use this amino acid foaming facial cleanser because I'm too tired from daily work" is "I often use this amino acid foaming facial cleanser because I'm too tired from daily work." However, the segmentation results obtained through the ASR interface in traditional processing methods are too fine-grained. For example, the output might segment the sentence into "daily, work, too tired, I often, use this, amino acid foaming, facial cleanser," but the client's requested keyword is "amino acid foaming facial cleanser," which cannot be matched. However, the actual creator expressed this information. If all the text is concatenated into a single sentence, such as "I often use this amino acid foaming facial cleanser because I'm too tired from daily work," the text becomes too long, making it impossible to obtain a meaningful segmentation result. Therefore, in this embodiment, the first type of sentence segmentation set corresponding to the spoken data is obtained through the ASR interface, that is, the punctuation mark position 1 output by the ASR interface is recorded as the first type of sentence segmentation set. Next, the segments in the first type of segmentation set are merged to obtain the complete text. This complete text is then segmented again using an open-source punctuation segmentation algorithm to obtain the second type of segmentation set. This involves segmenting the merged complete text using the open-source punctuation segmentation algorithm, and the punctuation mark position 2 is recorded as the second type of segmentation set. The intersection of the first type of segmentation set and the second type of segmentation set is then calculated to obtain the segmented ASR text corresponding to the spoken data.

[0035] Regarding step S130, for the image data, since there may be invalid information similar to watermarks in the image, and this information is mostly located at the edge of the video and has small font size, in this embodiment of the disclosure, the image data is first converted into OCR text through an OCR interface. Then, based on the text coordinate information in the OCR text and the position and proportion information of the preset invalid information, the invalid information in the OCR text is filtered to obtain the filtered OCR text. For example, the position and proportion information of the preset invalid information can be set according to the actual video data being projected. For example, when the video data is 400*600, the data within 5% of the edge of the image can be set as invalid information for filtering, etc., thereby obtaining the filtered OCR text.

[0036] For step S140, firstly, using a trie matching algorithm, the segmented ASR text and the filtered OCR text are matched against the specified words in the preset word matching rules to obtain a first matching result. For example, the brand's rules include the essential keyword A and the core keywords B, C, and D, requiring at least one core keyword. Then, the rule words in the preset word matching rules are: A+B, or A+C, or A+D. The segmented ASR text and the filtered OCR text are matched separately, and the resulting matching result is set as the first matching result.

[0037] Subsequently, since there are homophones in the spoken data, a pinyin pronunciation matching algorithm is used to match the segmented ASR text with the words specified in the preset word matching rules to obtain a second matching result.

[0038] Finally, based on the requirement that specified words in the spoken data and the video data must appear simultaneously in the preset word matching rules, words that meet the requirements are searched in the first matching result and the second matching result to obtain the matching result of the video data. That is, the time of appearance of the specified words is searched in the first matching result and the second matching result; if they are the same, the match is considered successful; if they are different, the match is considered unsuccessful. Ultimately, the obtained matching result of the video data includes the matching results of essential keywords, the matching results of core keywords, and the matching results of whether the specified words appear simultaneously. If all the above matching results meet the requirements, the video is considered qualified; if the requirements are not met, the qualification rate of the deployed video data is obtained based on the ratio of qualified videos to all videos.

[0039] Through the embodiments of this disclosure, a set of cleaning rules is proposed for the detection results of ASR and OCR text in video data. This removes the interference caused by irregular segmentation in the spoken data and the data noise caused by redundant and repetitive text in the video data. By cross-checking the spoken data and the video data, invalid information in the video data is removed, thereby improving the rationality of the pass / fail assessment of the video data.

[0040] In addition, considering that the word information in the video data of some products is decided on the spot by the brand personnel, the words may be too professional and may not necessarily match the promotional effect of the brand's real selling points, so this embodiment of the disclosure provides matching of extended word packages.

[0041] like Figure 2 As shown, after acquiring the video data to be deployed, the method further includes the following steps:

[0042] Step S210: Extract the image data frame by frame and obtain the OCR text of all frames through the OCR interface;

[0043] Step S220: According to the preset text deduplication rules and DBscan clustering algorithm, the OCR text of all frames is deduplicated to obtain the OCR text after the first cleaning.

[0044] Step S230: Obtain the ASR text corresponding to the broadcast data through the ASR interface;

[0045] Step S240: Using information entropy and a specified threshold, obtain the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning.

[0046] Step S250: Using the TF-IDF algorithm and specified part-of-speech tags, filter the ASR segmentation results and the OCR segmentation results to obtain new ASR word packages and new OCR word packages;

[0047] Step S260: Match the segmented ASR text and the filtered OCR text with the ASR new word package and the OCR new word package respectively to obtain the new word matching frequency.

[0048] In this case, considering the presence of invalid information such as scrolling text in the image data, for example, the scrolling text "No watermark during ad placement, please do not steal others' materials," when it appears, the OCR information for each frame is "No watermark during ad placement, please do not steal," "No watermark during ad placement, please do not steal," "No watermark during ad placement, please do not steal," "No watermark during ad placement, please do not steal," "No watermark during ad placement, please do not steal others' materials," "No watermark during ad placement, please do not steal others' materials," "No watermark during ad placement, please do not steal others' materials," "No watermark during ad placement, please do not steal others' materials," "No watermark during ad placement, please do not steal others' materials," etc. Similarly, when a piece of text information lasts for several seconds, for example, "Dove Amino Acid Foaming Cleanser" appears for 3 seconds, it will appear in every frame of the extracted 6 frames. This word will be counted 6 times in the subsequent TF-IDF algorithm, while the actual text should only be counted once. Therefore, in this embodiment, the OCR text is deduplicated.

[0049] First, the image data is extracted frame by frame. The OCR text for each frame is obtained through an OCR interface. Simultaneously, the coordinate position of each OCR text in each frame can be recorded, such as... Figure 3As shown, taking a 3-frame example, where the same pattern represents OCR text with the same content, frame 1 has three OCR texts, frame 2 has two OCR texts, and frame 3 has two OCR texts. The first text in each of the three frames has the same content, and the third text in frame 1 has the same content as the third text in frame 2. Furthermore, for ease of recording, the OCR files in all frames can be indexed. For example, frame 1 has three OCR texts, denoted as 1-1, 1-2, 1-3; frame 2 has 2-1, 2-2; and frame 3 has 3-1, 3-2. All text in all frames is flattened into a single dimension, constructing a two-dimensional matrix to identify whether the OCR text has been deduplicated. The preset text deduplication rules include preset overlap, longest common subsequence ratio, and text similarity. For example, the preset overlap is >0.8, the longest common subsequence ratio is >0.5, and the text similarity is >0.5. In this embodiment, the text similarity calculation is performed using the T5 model for Chinese text. Furthermore, the frame difference between the OCR text and the corresponding frame does not exceed three frames; for example, the content of frame 1 is connected to the content of frame 3 at most, thus avoiding missed or incorrect OCR text recognition. The positions of all OCR text meeting the above conditions are marked in a two-dimensional matrix, resulting in OCR text elements in the two-dimensional matrix, such as... Figure 4 As shown, the DBscan clustering algorithm is used to cluster the OCR text elements in the two-dimensional matrix, that is, clustering based on reachability. For example, horizontally, 1-1 and 2-1 can be grouped into one cluster, and vertically, 2-1 and 3-1 can be grouped into another cluster, i.e., 1-1, 2-1, and 3-1 are grouped into one cluster; horizontally, 1-3 and 2-2 can be grouped into another cluster. Thus, four clusters are ultimately obtained: cluster 1 contains 1-1, 2-1, and 3-1; cluster 2 contains only 1-2; cluster 3 contains 1-3 and 2-2; and cluster 4 contains only 3-2. Furthermore, since only the longest OCR text is retained in each cluster, the final clusters are: 3-1 in cluster 1, 1-2 in cluster 2, 2-2 in cluster 3, and 3-2 in cluster 4. Finally, duplicate OCR text is removed to obtain the retained OCR text in each frame: 1-2 in frame 1, 2-2 in frame 2, and 3-1 and 3-2 in frame 3, thus obtaining the OCR text after the first cleaning.

[0050] Then, the ASR text corresponding to the broadcast data is obtained through the ASR interface. Then, using information entropy and a specified threshold, the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the first cleaned OCR text are obtained. In this embodiment, the information entropy between every two adjacent characters in the ASR text and the first cleaned OCR text is calculated, and the information entropy that should connect between two adjacent characters is obtained using the following formula (1):

[0051]

[0052] Where a is the information entropy, p(i) represents the probability of word i appearing in the corpus, p(j) represents the probability of word j appearing in the corpus, and P(i,j) represents the probability of word i and word j appearing simultaneously in the corpus.

[0053] After calculating the information entropy between each pair of adjacent characters, the information entropy is filtered using the specified threshold to obtain the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning. For example, in this embodiment, the specified threshold is 3, meaning that if the information entropy is greater than 3, the two adjacent characters should be connected. For example, in the phrase "amino acid components," "amino" should be connected with "base," "base" should be connected with "acid," "acid" should not be connected with "form," and "form" should be connected with "fraction." Thus, the words "amino acid" and "component" are obtained in this text. That is to say, high-frequency words are obtained using step S240, but not all high-frequency words are what the brand wants. Therefore, cleaning is performed in step S250.

[0054] Using the TF-IDF algorithm, the scores of words in the ASR and OCR segmentation results are calculated, and the words are sorted according to their scores to obtain ASR and OCR segmentations sorted from high to low importance. Here, TF is the ratio of the number of times a selected word appears in the video data to the total number of words in the video data, and IDF represents the degree to which a selected word appears in different video data. The degree to which a selected word appears is obtained using the following formula (2):

[0055]

[0056] Where, count doc The count represents the total number of video data. doc |i∈doc represents the number of videos in which the selected word appears.

[0057] Then, the TF*IDF scores of all selected words are obtained, and they are sorted according to the scores to obtain ASR and OCR words in descending order of importance. The higher the score, the more important the word is, and the lower the score, the more frequently the word appears. It is not an important word for the video and therefore the less important the word is.

[0058] Then, using the specified part-of-speech tags, the ASR and OCR word segments are filtered to obtain the new ASR and OCR word packages. Specifically, by judging the part-of-speech tags, common nouns, adjectives, and verbs are retained, while pure English words and numbers are filtered out, completing two rounds of text cleaning. For example, for a specific product line, the new words obtained in this way might be "high-purity hyaluronic acid," "dry and sensitive skin savior," and "thick skin without peeling."

[0059] Next, the segmented ASR text and the filtered OCR text are matched against the ASR new word package and the OCR new word package, respectively, to obtain the new word matching frequency. Similarly, using a trie matching algorithm, the segmented ASR text and the filtered OCR text are matched against the ASR new word package and the OCR new word package, respectively, to obtain matching result one. Then, because homophones exist in the spoken data, a pinyin pronunciation matching algorithm is used to match the segmented ASR text with the ASR new word package for the same pronunciation, to obtain matching result two. Finally, the new word matching results are sorted from high to low frequency, and the frequency of all new words appearing in different videos is counted to construct an inverted index for matching new words and videos. Simultaneously, the frequency of all new words appearing in different videos is counted, such as... Figure 5 As shown, video 1 matches words a, b, and d; video 2 matches words b and e; video 3 matches words a, b, c, and e; and video 4 matches words a, b, and e. The inverted result of the new words shows that word a appears in video 1, video 3, and video 4, and so on.

[0060] In addition, to help brands create more effective videos, analysis results of video detection can be provided based on the aforementioned preset word matching rules and the matching results of ASR and OCR new word packages. For example, if the essential keyword "amino acid foaming facial cleanser" provided by the brand appears in only 5 out of 100 videos, the brand will be alerted to the anomaly of this essential keyword. The brand will be prompted to check if this is due to a specific reason (such as the vocabulary being too specialized, causing the production team to not display it as required, or the given vocabulary not being required for this product line). At the same time, the new word discovery will also count videos that appear in multiple videos but are not in the word package provided by the brand, such as "hyaluronic acid moisturizing toner," prompting the brand to modify its rules, such as changing the vocabulary in the original word package. Additionally, the data in the inverted index can be used to re-analyze the data in the brand's modified rules, quickly providing matching results and calculating the pass rate of the video data. For example, under the brand's rule that "all essential keywords must appear and at least one core keyword must appear," the intersection of the number of videos matching the keywords can be combined with the union of the matching results in the core keywords to quickly calculate the pass rate of the videos after the rule modification.

[0061] Through the embodiments disclosed herein, consistency detection can be performed on the video data being distributed. A complete closed-loop workflow is constructed, encompassing data input, data cleaning, data analysis, and rule feedback. Addressing brands' needs for information expansion, the detection of new keywords helps brands supplement and improve their product keyword packages. Through data analysis, a rule feedback and suggestion mechanism is implemented, going beyond simply detecting the raw data provided by brands; it also helps brands formulate more reasonable rules for brand consistency detection.

[0062] Figure 6 This is a schematic block diagram of a video data processing device provided in an embodiment of this disclosure. Figure 6 As shown, the video data processing device 600 may include a processor 610, a memory 620, and a program stored in the memory 620 and executable on the processor 610. When the processor 610 executes the program, it may perform the following steps: acquiring projected video data, which includes spoken data and video data; segmenting the spoken data using an ASR interface and an open-source punctuation segmentation algorithm to obtain segmented ASR text; filtering invalid information in the video data using an OCR interface and preset invalid information to obtain filtered OCR text; and obtaining a matching result for the video data based on the segmented ASR text, the filtered OCR text, and preset word matching rules.

[0063] In the embodiments of this disclosure, when the processor 610 executes the program, it can also perform the following: obtain a first type of sentence segmentation set corresponding to the broadcast data through the ASR interface; merge the sentences in the first type of sentence segmentation set, and segment the merged whole text using the open-source punctuation sentence segmentation algorithm to obtain a second type of sentence segmentation set; find the intersection of the first type of sentence segmentation set and the second type of sentence segmentation set to obtain the segmented ASR text.

[0064] In the embodiments of this disclosure, when the processor 610 executes the program, it can also perform the following: converting the screen data into OCR text through the OCR interface; filtering the invalid information in the OCR text according to the text coordinate information in the OCR text and the position information and proportion information in the preset invalid information to obtain the filtered OCR text.

[0065] In embodiments of this disclosure, the processor 610 may further perform the following when executing the program: using a trie matching algorithm to match the segmented ASR text and the filtered OCR text with the specified words in the preset word matching rules to obtain a first matching result; using a pinyin pronunciation matching algorithm to match the segmented ASR text with the specified words in the preset word matching rules for the same pronunciation to obtain a second matching result; and according to the requirement in the preset word matching rules that the specified words in the spoken data and the video data must appear simultaneously, searching for words that meet the requirements in the first matching result and the second matching result to obtain the matching result of the video data.

[0066] In embodiments of this disclosure, the processor 610 may also perform the following when executing the program: after acquiring the video data to be played, extracting the video data frame by frame and obtaining the OCR text of all frames through the OCR interface; performing deduplication processing on the OCR text of all frames according to a preset text deduplication rule and the DBscan clustering algorithm to obtain the OCR text after the first cleaning; obtaining the ASR text corresponding to the spoken data through the ASR interface; obtaining the ASR word segmentation result corresponding to the ASR text and the OCR word segmentation result corresponding to the OCR text after the first cleaning using information entropy and a specified threshold; filtering the ASR word segmentation result and the OCR word segmentation result using the TF-IDF algorithm and a specified part-of-speech tag to obtain a new ASR word package and a new OCR word package.

[0067] In the embodiments of this disclosure, when the processor 610 executes the program, it can also perform the following: according to the preset overlap, longest common subsequence ratio and text similarity in the preset text deduplication rules, mark the positions of OCR texts that meet the conditions in a two-dimensional matrix to obtain OCR text elements in the two-dimensional matrix; and use the DBscan clustering algorithm to cluster the OCR text elements in the two-dimensional matrix to obtain the OCR text after the first cleaning.

[0068] In embodiments of this disclosure, the processor 610 may also perform the following when executing a program: calculating the information entropy between every two adjacent characters in the ASR text and the OCR text after the first cleaning; filtering the information entropy using the specified threshold to obtain the ASR word segmentation result corresponding to the ASR text and the OCR word segmentation result corresponding to the OCR text after the first cleaning.

[0069] In embodiments of this disclosure, when the processor 610 executes the program, it may also perform the following: using the TF-IDF algorithm to calculate the scores of words in the ASR segmentation results and the OCR segmentation results, and sorting them according to the scores to obtain ASR and OCR segmentations sorted from high to low importance; using the specified part-of-speech tag to filter the ASR and OCR segmentations to obtain the new ASR word package and the new OCR word package.

[0070] In embodiments of this disclosure, the processor 610 may also perform the following when executing a program: matching the segmented ASR text and the filtered OCR text with the ASR new word package and the OCR new word package respectively to obtain the new word matching frequency.

[0071] In embodiments of this disclosure, processor 610 may be, for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a processor based on a multi-core processor architecture, etc. Memory 620 may be any type of memory implemented using data storage technologies, including but not limited to random access memory, read-only memory, semiconductor-based memory, flash memory, disk storage, etc.

[0072] In other embodiments of this disclosure, a machine-readable storage medium is also provided, on which instructions are stored for causing a machine to perform the actions described above. Figures 1 to 2 The steps of the method shown.

[0073] In summary, the embodiments of this disclosure propose a set of cleaning rules for the detection results of ASR and OCR text in video data. This removes interference caused by irregular segmentation in spoken data and data noise caused by redundant and repetitive text in video data. Through mutual verification between spoken and video data, invalid information in the video data is removed, thereby improving the rationality of the pass / fail assessment of video data. Furthermore, consistency detection can be performed on the deployed video data, constructing a complete closed-loop workflow from data input, data cleaning, and data analysis to feedback rules. Addressing the brand's need for information expansion, the detection of new words helps the brand supplement and improve its product keyword package. Through data analysis, a feedback and suggestion mechanism for rules is implemented, going beyond simply detecting the raw data provided by the brand; it also helps the brand formulate more reasonable rules for brand consistency detection.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses and methods according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0075] Unless otherwise expressly indicated by the context, the singular form of words used herein and in the appended claims includes the plural form, and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Similarly, the terms “comprising” and “including” shall be interpreted as including rather than exclusively. Likewise, the terms “including” and “or” shall be interpreted as including unless such interpretation is expressly prohibited herein. Where the term “example” is used herein, particularly when it follows a set of terms, the “example” is merely exemplary and illustrative and should not be considered exclusive or extensive.

[0076] Further aspects and scope of adaptation become apparent from the description provided herein. It should be understood that various aspects of this application may be implemented individually or in combination with one or more other aspects. It should also be understood that the descriptions and specific embodiments herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0077] Several embodiments of this disclosure have been described in detail above. However, it is obvious that those skilled in the art can make various modifications and variations to the embodiments of this disclosure without departing from the spirit and scope of this disclosure. The scope of protection of this disclosure is defined by the appended claims.

Claims

1. A method for processing video data, characterized in that, The method includes: Acquire the video data to be delivered, including spoken data and video data; The spoken data is segmented using the ASR interface and an open-source punctuation mark segmentation algorithm to obtain the segmented ASR text. The invalid information in the image data is filtered out using the OCR interface and preset invalid information to obtain the filtered OCR text. Based on the segmented ASR text, the filtered OCR text, and the preset word matching rules, the matching result of the video data is obtained. The method further includes: The image data is extracted frame by frame, and the OCR text of all frames is obtained through the OCR interface; Based on the preset text deduplication rules and the DBscan clustering algorithm, the OCR text of all frames is deduplicated to obtain the OCR text after the first cleaning. The ASR text corresponding to the broadcast data is obtained through the ASR interface; Using information entropy and a specified threshold, the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning are obtained; Using the TF-IDF algorithm and specified part-of-speech tags, the ASR and OCR segmentation results are filtered to obtain new ASR and OCR word packages. The step of using information entropy and a specified threshold to obtain the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning includes: Calculate the information entropy between every two adjacent characters in the ASR text and the OCR text after the first cleaning, respectively; The information entropy is filtered using the specified threshold to obtain the ASR word segmentation result corresponding to the ASR text and the OCR word segmentation result corresponding to the OCR text after the first cleaning. The step of using the TF-IDF algorithm and specified part-of-speech tags to filter the ASR and OCR word segmentation results to obtain new ASR and OCR word packages includes: Using the TF-IDF algorithm, the scores of words in the ASR segmentation results and the OCR segmentation results are calculated, and the words are sorted according to the scores to obtain ASR and OCR segmentation results sorted from high to low importance. Using the specified part-of-speech tag, the ASR segmentation and the OCR segmentation are filtered to obtain the new ASR word package and the new OCR word package.

2. The video data processing method according to claim 1, characterized in that, The method of segmenting the broadcast data using the ASR interface and an open-source punctuation segmentation algorithm to obtain the segmented ASR text includes: The first set of sentence segments corresponding to the spoken data is obtained through the ASR interface; The sentences in the first type of sentence segmentation set are merged, and the merged whole text is segmented by the open-source punctuation mark sentence segmentation algorithm to obtain the second type of sentence segmentation set; The intersection of the first type of sentence segmentation set and the second type of sentence segmentation set is used to obtain the segmented ASR text.

3. The video data processing method according to claim 1, characterized in that, The step of filtering invalid information from the image data using the OCR interface and preset invalid information to obtain filtered OCR text includes: The image data is converted into OCR text via an OCR interface; Based on the text coordinate information in the OCR text and the position and proportion information of the preset invalid information, the invalid information in the OCR text is filtered to obtain the filtered OCR text.

4. The video data processing method according to claim 1, characterized in that, The process of obtaining the matching result of the video data based on the segmented ASR text, the filtered OCR text, and the preset word matching rules includes: Using a trie matching algorithm, the segmented ASR text and the filtered OCR text are matched with the specified words in the preset word matching rules to obtain the first matching result; Using a pinyin pronunciation matching algorithm, the segmented ASR text is matched with words in the preset word matching rules for the same pronunciation to obtain a second matching result; According to the requirement that specified words in the spoken data and the video data must appear simultaneously in the preset word matching rules, words that meet the requirements are searched in the first matching result and the second matching result to obtain the matching result of the video data.

5. The video data processing method according to claim 1, characterized in that, The OCR text of all frames is deduplicated according to preset text deduplication rules and the DBscan clustering algorithm, resulting in the OCR text after the first cleaning, which includes: Based on the preset overlap, longest common subsequence ratio and text similarity in the preset text deduplication rules, the positions of OCR texts that meet the conditions are marked in a two-dimensional matrix to obtain OCR text elements in the two-dimensional matrix; The DBscan clustering algorithm is used to cluster the OCR text elements in the two-dimensional matrix to obtain the OCR text after the first cleaning.

6. The video data processing method according to claim 1, characterized in that, The method further includes: The segmented ASR text and the filtered OCR text are matched with the ASR new word package and the OCR new word package respectively to obtain the new word matching frequency.

7. A video data processing device, characterized in that, The device includes: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the processor performs the following steps when executing the program: Acquire the video data to be delivered, including spoken data and video data; The spoken data is converted into segmented ASR text using the ASR interface and an open-source punctuation mark segmentation algorithm. The image data is converted into filtered OCR text using the OCR interface and preset invalid information. Based on the segmented ASR text, the filtered OCR text, and the preset word matching rules, the matching result of the video data is obtained. The processor also performs the following steps when executing the program: The image data is extracted frame by frame, and the OCR text of all frames is obtained through the OCR interface; Based on the preset text deduplication rules and the DBscan clustering algorithm, the OCR text of all frames is deduplicated to obtain the OCR text after the first cleaning. The ASR text corresponding to the broadcast data is obtained through the ASR interface; Using information entropy and a specified threshold, the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning are obtained; Using the TF-IDF algorithm and specified part-of-speech tags, the ASR and OCR segmentation results are filtered to obtain new ASR and OCR word packages. The step of using information entropy and a specified threshold to obtain the ASR segmentation result corresponding to the ASR text and the OCR segmentation result corresponding to the OCR text after the first cleaning includes: Calculate the information entropy between every two adjacent characters in the ASR text and the OCR text after the first cleaning, respectively; The information entropy is filtered using the specified threshold to obtain the ASR word segmentation result corresponding to the ASR text and the OCR word segmentation result corresponding to the OCR text after the first cleaning. The step of using the TF-IDF algorithm and specified part-of-speech tags to filter the ASR and OCR word segmentation results to obtain new ASR and OCR word packages includes: Using the TF-IDF algorithm, the scores of words in the ASR segmentation results and the OCR segmentation results are calculated, and the words are sorted according to the scores to obtain ASR and OCR segmentation results sorted from high to low importance. Using the specified part-of-speech tag, the ASR segmentation and the OCR segmentation are filtered to obtain the new ASR word package and the new OCR word package.

Images