Video content recognition method, device, equipment and storage medium

Abstract

The present disclosure provides a video content recognition method, apparatus, device, and storage medium. The method includes: in response to a received video to be reviewed, respectively obtaining the audio text data and video text data of the video to be reviewed; based on a preset teaching terminology and subject keyword library, determining the audio matching score group corresponding to each subject of the audio text data; based on a preset subject knowledge point text library, determining the video matching score group corresponding to each subject of the video text data; based on the audio matching score group and the video matching score group, determining the subject classification corresponding to the video to be reviewed. The present disclosure analyzes the audio and video of the video to be reviewed and verified separately, and matches them by keywords and text respectively, effectively ensuring the accuracy of the identified subject and improving the reliability and usability of the activity test results.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video content recognition method, apparatus, device, and storage medium. Background Art

[0002] With the development of internet applications, the number of online video users has steadily increased, and more and more people are using educational videos uploaded to online platforms for training and knowledge acquisition. Users can choose videos related to specific subjects or categories based on their needs. Uploaders can also upload educational videos covering different subjects. Online platforms are required to automatically identify the type of videos uploaded by uploaders to determine whether the videos contain relevant educational content and identify the specific subject.

[0003] In the existing technology, apart from manually setting labels, there is a lack of effective methods to identify the subject classification of uploaded videos, resulting in low recognition accuracy and inconvenience in use. Summary of the Invention

[0004] The present disclosure provides a video content recognition method, apparatus, device, and storage medium to improve the reliability and usability of activity test results.

[0005] In a first aspect, the present disclosure provides a video content recognition method, the video content recognition method comprising:

[0006] In response to the received video to be reviewed, respectively obtaining audio text data and video text data of the video to be reviewed;

[0007] Based on the preset teaching terms and subject keyword library, determine the audio matching score group corresponding to each subject of the audio text data, and the audio matching score group is used to represent the matching score between the audio text data and each subject;

[0008] Based on a preset subject knowledge point text library, determine the video matching score group corresponding to each subject of the video text data, and the video matching score group is used to represent the matching score between the video text data and each subject;

[0009] Based on the audio matching score group and the video matching score group, determine the subject classification corresponding to the video to be reviewed.

[0010] Optionally, based on a preset teaching terminology and subject keyword library, the audio matching score group corresponding to each subject of the audio text data is determined, including: matching the keywords in the audio text data with the teaching terminology and subject keyword library to determine the number of keywords for each subject contained in the audio text data; based on the number of keywords for each subject and the total number of keywords in the audio text data, determining the audio matching score corresponding to each subject in the audio matching score group.

[0011] Optionally, determining the number of keywords for each subject contained in the audio text data includes: when the keywords in the same audio text data match the keywords of at least two subjects, determining that the number of keywords for at least two subjects is increased by one respectively; accordingly, determining the audio matching score corresponding to each subject in the audio matching score group based on the number of keywords for each subject and the total number of keywords in the audio text data, including: when the sum of the number of keywords for each subject is greater than the total number of keywords in the audio text data, determining the audio matching score corresponding to each subject in the audio matching score group based on the number of keywords for each subject and the sum of the number of keywords for each subject.

[0012] Optionally, based on a preset subject knowledge point text library, determine the video matching score groups corresponding to each subject of the video text data, including: based on a sentence segmentation algorithm, segmenting the video text data into knowledge point sentences; based on a text matching algorithm, determining the subject to which the text in the subject knowledge point text library with the highest similarity to the knowledge point sentences belongs; determining the number of texts with the highest similarity for each subject contained in the video text data; based on the number of texts with the highest similarity for each subject and the number of knowledge point sentences, determine the video matching score corresponding to each subject in the video matching score group.

[0013] Optionally, the audio text data and video text data of the video to be reviewed are obtained separately, including: based on the speech recognition algorithm, obtaining the text content corresponding to the audio of the video to be reviewed as audio text data; based on the image text recognition algorithm, obtaining the text content contained in the image of the video to be reviewed as video text data.

[0014] Optionally, based on an image text recognition algorithm, the text content contained in the image of the video to be reviewed is obtained as video text data, including: obtaining the image content of each video frame of the video to be reviewed; determining, through an image comparison algorithm, that the video frame whose image content difference reaches a set threshold is a key frame; based on the image text recognition algorithm, obtaining the text content contained in the key frame as video text data.

[0015] Optionally, the preset teaching terms and subject keyword library is obtained by: acquiring standard teaching terms and subject term data, determining the keywords of the standard teaching terms and subject terms; and establishing the teaching terms and subject keyword library based on the AC dictionary tree algorithm and keywords.

[0016] Optionally, based on the audio matching score group and the video matching score group, determine the subject classification corresponding to the video to be reviewed, including: determining the matching score of each subject based on the audio matching score group and the video matching score group; determining the subject with the highest matching score as the subject classification of the video to be reviewed.

[0017] In a second aspect, the present disclosure provides a video content recognition device, the video content recognition device comprising:

[0018] An acquisition module, configured to respectively acquire audio text data and video text data of the video to be reviewed in response to the received video to be reviewed;

[0019] A first determination module is used to determine an audio matching score group corresponding to each subject of the audio text data based on a preset teaching terminology and subject keyword library, where the audio matching score group is used to represent a matching score between the audio text data and each subject;

[0020] The second determination module is used to determine the video matching score group corresponding to each subject of the video text data based on the preset subject knowledge point text library, and the video matching score group is used to represent the matching score between the video text data and each subject;

[0021] The processing module is used to determine the subject classification corresponding to the video to be reviewed based on the audio matching score group and the video matching score group.

[0022] Optionally, the first determination module is specifically used to match the keywords in the audio text data with the teaching terms and subject keyword library to determine the number of keywords for each subject contained in the audio text data; based on the number of keywords for each subject and the total number of keywords in the audio text data, determine the audio matching score corresponding to each subject in the audio matching score group.

[0023] Optionally, the first determination module is specifically used to, when the keywords in the same audio text data match the keywords of at least two subjects, determine that the number of keywords of at least two subjects is increased by one respectively; accordingly, based on the number of keywords of each subject and the total number of keywords in the audio text data, determine the audio matching score corresponding to each subject in the audio matching score group, including: when the sum of the number of keywords of each subject is greater than the total number of keywords in the audio text data, based on the number of keywords of each subject and the sum of the number of keywords of each subject, determine the audio matching score corresponding to each subject in the audio matching score group.

[0024] Optionally, the second determination module is specifically used to segment the video text data into knowledge point sentences based on a sentence segmentation algorithm; determine the subject to which the text with the highest similarity to the knowledge point sentence in the subject knowledge point text library belongs based on a text matching algorithm; determine the number of texts with the highest similarity for each subject contained in the video text data; and determine the video matching score corresponding to each subject in the video matching score group based on the number of texts with the highest similarity and the number of knowledge point sentences for each subject.

[0025] Optionally, the acquisition module is specifically used to obtain the text content corresponding to the audio of the video to be reviewed as audio text data based on a speech recognition algorithm; and to obtain the text content contained in the image of the video to be reviewed as video text data based on an image text recognition algorithm.

[0026] Optionally, the acquisition module is specifically used to obtain the image content of each video frame of the video to be reviewed; through the image comparison algorithm, determine the video frame whose image content difference reaches a set threshold as the key frame; based on the image text recognition algorithm, obtain the text content contained in the key frame as video text data.

[0027] Optionally, the first determination module is specifically used to obtain a preset teaching terminology and subject keyword library in the following manner: obtaining standard teaching terminology and subject terminology data, determining keywords of standard teaching terminology and subject terminology; and establishing a teaching terminology and subject keyword library based on the AC dictionary tree algorithm and keywords.

[0028] Optionally, the processing module is specifically used to determine the matching score of each subject based on the audio matching score group and the video matching score group; and determine the subject with the highest matching score as the subject classification of the video to be reviewed.

[0029] In a third aspect, the present disclosure further provides an electronic device, the electronic device comprising:

[0030] at least one processor;

[0031] and a memory communicatively coupled to the at least one processor;

[0032] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the electronic device to perform the video content recognition method corresponding to any embodiment of the first aspect of the present disclosure.

[0033] In a fourth aspect, the present disclosure further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement any video content recognition method as described in the first aspect of the present disclosure.

[0034] The video content recognition method, apparatus, device and storage medium provided by the present disclosure respectively obtain the audio text data and video text data of the video to be reviewed in response to the received video to be reviewed; then, based on the preset teaching terms and subject keyword library, determine the audio matching score group corresponding to each subject of the audio text data, and based on the preset subject knowledge point text library, determine the video matching score group corresponding to each subject of the video text data, and then determine the subject classification corresponding to the video to be reviewed based on the audio matching score group and the video matching score group. By analyzing the audio and video of the video to be reviewed and re-examined separately, and matching them by keywords and text respectively, the accuracy of the identified subject is effectively guaranteed. By combining the analysis results obtained from the audio text data and the video text data, the accuracy and reliability of the automatic identification of the subject corresponding to the video to be reviewed is guaranteed to the maximum extent, the recognition efficiency is improved, and the use is convenient. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0036] Figure 1 A diagram of an application scenario of the video content recognition method provided by an embodiment of the present disclosure;

[0037] Figure 2 A flowchart of a video content recognition method provided by one embodiment of the present disclosure;

[0038] Figure 3 A flowchart of a video content recognition method provided by another embodiment of the present disclosure;

[0039] Figure 4 A schematic structural diagram of a video content recognition device provided in yet another embodiment of the present disclosure;

[0040] Figure 5 A schematic structural diagram of an electronic device provided in yet another embodiment of the present disclosure.

[0041] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0042] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0043] The following detailed description of the technical solution of the present disclosure and how the technical solution of the present disclosure solves the above-mentioned technical problems is provided with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present disclosure will be described below in conjunction with the accompanying drawings.

[0044] With the development of Internet technology, the scale of online video users in my country has steadily increased, and subject education and training through online videos has been increasingly valued and applied. In the existing technology, production users can upload subject education and training videos to online platforms, and learners can choose educational videos of corresponding subjects according to the subjects they need to learn. Therefore, it is extremely important for online platforms to classify and identify a large number of online videos or live broadcast content by subject, and set corresponding tags to facilitate the screening of learners. However, there is no targeted method in the existing technology to automatically identify the subject to which the videos uploaded to the online platform belong. It can only be done by manually adding tags or simply classifying them through equipment, which has low recognition accuracy and is inconvenient to use.

[0045] In order to solve the above problems, the embodiments of the present disclosure provide a video content recognition method, which matches the audio and video content of the uploaded video to be reviewed with the keyword and text database of the subject and knowledge point respectively, and combines the matching results to determine the corresponding subject of the video to be reviewed. In this way, the accuracy of determining the subject to which the video to be reviewed belongs can be guaranteed to the greatest extent.

[0046] The following explains the application scenarios of the embodiments of the present disclosure:

[0047] Figure 1 This is an application scenario diagram of the video content recognition method provided by the embodiment of the present disclosure. Figure 1 As shown, the online platform server 100 receives the video to be reviewed uploaded by the producer user 110, adds the corresponding subject label after review, and after completing the video content recognition process, provides it to the learner user 120 for learning.

[0048] It should be noted that Figure 1In the scenario shown, only one producer user, online platform server and learner user is used as an example for illustration, but the present disclosure is not limited to this. That is, the number of producer users, online platform servers and learner users can be arbitrary.

[0049] The video content recognition method provided by the present disclosure is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0050] Figure 2 This is a flow chart of a video content recognition method provided by one embodiment of the present disclosure. Figure 2 As shown, the following steps are included:

[0051] Step S201: In response to the received video to be reviewed, respectively obtain the audio text data and video text data of the video to be reviewed.

[0052] Specifically, the execution subject of this method may be a server or a user terminal with data processing capabilities. In this embodiment, the description is made with the server as the execution subject.

[0053] When the server receives a video for review uploaded by a user, it will first process the video for review to extract the audio data and video data composed of each frame of images, and then obtain the text-related content contained in the audio data (i.e., audio text data) and the text-related content contained in the video data (i.e., video text data).

[0054] Furthermore, extracting audio text data from audio data can be achieved by using existing speech recognition algorithms; extracting video text data from video data can be achieved by using existing image and video algorithms to extract text from video frames containing text to obtain video text data.

[0055] Step S202: Based on the preset teaching terms and subject keyword library, determine the audio matching score group corresponding to each subject of the audio text data.

[0056] Among them, the audio matching score array is used to represent the matching scores between audio text data and various subjects.

[0057] Specifically, in subject training, in addition to different keywords for different subjects (such as "mechanics", "function", etc.), there are also common teaching terms for each subject, such as "course review" and "this question". Based on the teaching terms, the subdivision labels of the subject can be determined, such as whether it belongs to the "mathematics" subject in the elementary school stage or the "advanced function" subject in the university stage, because there will be obvious differences in the specific teaching terms between the two.

[0058] Therefore, when the audio text data is videotaped, the subject keywords and teaching term keywords contained therein are also analyzed for matching, which can maximize the accuracy of the video to be reviewed. The sub-discipline to which the video belongs is not just a simple identification of the major subject categories, thereby effectively ensuring the accuracy of the recognition.

[0059] Furthermore, the recognition results will be saved as multiple score groups composed of different subjects and teaching terms, and the corresponding scores need to be determined in turn.

[0060] For example, the recognition result of the audio text data includes: "Teacher and students, in this lesson, chemistry, gold, gold, repeated addition experiment results, physics, physics, adding experiment solution, gold solution, experiment results, gold solution, chemistry, copper-water, primary battery, sulfur, sulfuric acid solution, copper, copper, gold, hydrogen, copper, gold, hydrogen, sulfur, hydrogen sulfate, negative electrode, copper, positive electrode, description, sulfur, sulfuric acid, primary battery, dissolution, copper, zinc, sulfur, oxidation, chemistry, primary battery, copper, electron, zinc, hydrino, ion, ion, electron, hydrogen, copper, electrode, electrode reaction, electron, hydrino, ion, ion, copper, hydrino, ion, electron, hydrogen, electron, through, zinc, ion, cation, ion, positive electrode, sulfur, sulfuric acid, anion, ion, negative electrode, primary battery, primary battery, primary battery, gold, gold, primary battery, electrolysis, electrolyte, primary battery, electrode, electrolysis, electrolyte solution, primary battery, zinc, sulfur, sulfuric acid, oxygen, students, oxidation, sulfur, sulfuric acid, primary battery, primary battery, oxygen, oxidation",

[0061] Among them, there are 163 keywords, 4 teaching term keywords, and 159 subject keywords (143 in chemistry, 9 in physics, 5 in Chinese, and 2 in mathematics). From this, we can specifically calculate the matching score corresponding to each subject.

[0062] In some embodiments, the matching score of each subject is determined according to a set algorithm based on the number of keywords of the subject contained in the audio text data and the total number of keywords, such as calculating a ratio, percentage, or modulus.

[0063] In some embodiments, when the keywords included in the speech text data do not include keywords for a specific subject, the audio matching score for the specific subject is not calculated, thereby reducing the amount of computation and improving processing efficiency.

[0064] Step S203: Based on a preset subject knowledge point text library, determine the video matching score group corresponding to each subject of the video text data.

[0065] Among them, the video matching score group is used to represent the matching scores between video text data and various subjects.

[0066] Specifically, since the video text data usually contained in the video is usually specific knowledge point sentences rather than single keywords, and if the subject keywords are simply used to identify the keywords in the video text data, the degree of repetition with the audio text data is high. Even if the recognition results in the video text data are combined with the recognition results in the audio text data, the recognition accuracy cannot be significantly improved; and by matching the video text data based on the knowledge point text library, the matching database is different, and the recognition results will also be different from the subject keyword library. Combining the recognition results based on the knowledge point text library with the recognition results based on the subject keyword and teaching term keyword library can effectively improve the recognition accuracy.

[0067] Specifically, the result of identification through the subject knowledge point text library is also a specific matching score including multiple subjects, and the matching score of each subject is determined according to the set algorithm based on the number of knowledge points of the subject contained in the video text data and the total number of knowledge points.

[0068] Furthermore, knowledge points matching the video text data are identified and a single sentence matching method is adopted, such as "The law of lever requires a fulcrum" is a single sentence, rather than splitting "The law of lever" and "fulcrum" into two sentences; and the splitting of the video text data into single sentences can be achieved based on the set text processing rules, such as splitting sentences containing line breaks into different single sentences.

[0069] Step S204: Determine the subject classification corresponding to the video to be reviewed based on the audio matching score group and the video matching score group.

[0070] Specifically, based on the sum of the matching scores of each subject in the audio matching score group and the matching scores of each subject in the video matching score group, the subject with the closest matching score is determined, which is the subject classification corresponding to the video to be reviewed.

[0071] By combining the subject matching scores of the audio matching score group and the video matching score group, it is possible to effectively avoid a situation where the subject finally selected does not match the actual corresponding subject when the subject matching scores in one of the matching score groups are close.

[0072] For example, the subject matching scores determined by the audio matching score group are high school mathematics (0.31), high school physics (0.308), and teaching language (0.582), respectively. However, the subject matching scores determined by the video matching score group are high school physics (0.448), junior high school physics (0.452), and high school mathematics (0.1). If the audio matching score group is used alone, the subject corresponding to the video to be reviewed will be considered high school mathematics. If the video matching score group is used alone, the subject corresponding to the video to be reviewed will be considered junior high school physics. However, since there is a large amount of overlap in knowledge points between these subjects, only after combining the audio matching score group and the video matching score group can the subject classification of the video to be reviewed be determined to be high school physics.

[0073] The video content recognition method provided by the embodiment of the present disclosure obtains the audio text data and video text data of the video to be reviewed respectively in response to the received video to be reviewed; then, based on the preset teaching terms and subject keyword library, determines the audio matching score group corresponding to each subject of the audio text data, and based on the preset subject knowledge point text library, determines the video matching score group corresponding to each subject of the video text data, and then determines the subject classification corresponding to the video to be reviewed based on the audio matching score group and the video matching score group. By analyzing the audio and video of the video to be reviewed and re-examined respectively, and matching them by keywords and text respectively, the accuracy of the identified subject is effectively guaranteed. By combining the analysis results obtained from the audio text data and the video text data, the accuracy and reliability of the automatic identification of the subject corresponding to the video to be reviewed is guaranteed to the maximum extent, the recognition efficiency is improved, and the use is convenient.

[0074] Figure 3 The flowchart of a video content recognition method provided by the present disclosure is as follows. Figure 3 As shown, the video content recognition method provided by this embodiment includes the following steps:

[0075] Step S301: In response to the received video to be reviewed, based on a speech recognition algorithm, the text content corresponding to the audio of the video to be reviewed is obtained as audio text data.

[0076] In some embodiments, after receiving the video to be reviewed, the online platform server will automatically extract the audio data therein and save it in the cache, and then extract the audio text data therein through the speech recognition algorithm, and then save the audio text data in the cache for further processing; it can also directly extract the audio text data in the video data to be reviewed through the speech recognition algorithm, and then directly save the audio text data in the cache.

[0077] The speech recognition algorithm may adopt an ASR (Automatic Speech Recognition) algorithm for recognition.

[0078] Step S302: Based on an image text recognition algorithm, the text content contained in the image of the video to be reviewed is obtained as video text data.

[0079] In some embodiments, after receiving the video to be reviewed, the online platform server will extract the video data that does not contain audio from the video to be reviewed and save it to the cache, so as to extract the video text data therein through the image and text recognition algorithm; it can also directly extract the video text data from the video to be reviewed through the image and text recognition algorithm and save it to the cache.

[0080] The image text recognition algorithm may use an OCR (Optical Character Recognition) algorithm for recognition.

[0081] Specifically, with image recognition as the technical background, OCR technology can be used to achieve relatively accurate recognition of commonly used Chinese characters while ensuring a certain level of clarity. Moreover, in videos of subject education and training, subject knowledge points are generally presented in PPT handouts, so the knowledge point retrieval method is used for the text of image optical symbol recognition, which can determine the information contained in the video text data and the corresponding subject.

[0082] Step S303: Match the keywords in the audio text data with the teaching terms and keywords in the subject keyword library to determine the number of keywords for each subject contained in the audio text data.

[0083] Specifically, since the effect and accuracy of speech recognition at this stage are affected by various factors, especially the noisy background noise in most videos, the long text after recognition may be semantically incoherent due to errors in recognizing similar pronunciations. Therefore, the keyword matching method is used for the audio text data of speech recognition, and it is matched one by one with the keywords in the teaching terms and subject keyword library.

[0084] When the same keywords as those in the audio text data exist in the teaching terms and subject keyword database, it is considered a successful match, and the number of keywords for the corresponding subject or teaching terms is confirmed to increase by one.

[0085] By matching each keyword in the audio text data one by one, the number of keywords of each subject contained in the audio text data can be determined.

[0086] Furthermore, the preset teaching terms and subject keyword library is obtained by: obtaining standard teaching terms and subject terminology data, determining the keywords of standard teaching terms and subject terminology; and establishing the teaching terms and subject keyword library based on the AC dictionary tree algorithm and keywords.

[0087] By directly accessing existing standard teaching and subject terminology data, the cost of keyword library training is reduced and usage efficiency is improved. The AC dictionary tree algorithm effectively ensures keyword search speed and eliminates the need for further word segmentation, avoiding the problem of reduced matching accuracy due to inaccurate word segmentation.

[0088] For example, the keywords for teaching terms include: "classmates, teacher, this lesson", etc., the keywords for mathematics terms include "algebraic equations, geometry, function, matrix", etc., and the keywords for English terms include "voice, auxiliary verb, passive voice", etc.

[0089] In some embodiments, a library of teaching terms and subject keywords can be created using an inverted index algorithm, and knowledge points related to each subject can be added to the keyword library. The knowledge points in the keyword library can be iterated in actual use, with updated data directly added to the keyword library without the need for training.

[0090] Furthermore, when the keywords in the same audio text data match the keywords of at least two subjects, the number of keywords of at least two subjects is increased by one respectively; accordingly, when determining the audio matching score corresponding to each subject in the audio matching score group, when the sum of the number of keywords of each subject is greater than the total number of keywords in the audio text data, the audio matching score corresponding to each subject in the audio matching score group is determined based on the number of keywords of each subject and the sum of the number of keywords of each subject.

[0091] For example, if the keywords in the audio text data include "Spring and Autumn Period, Warring States Period, Qin and Han Dynasty", and "Spring and Autumn Period, Warring States Period" and "Qin and Han Dynasty" are both Chinese and historical terms, the number of keywords for Chinese and history subjects will be increased by two. Correspondingly, if the total number of keywords in the audio text data is 100, and the number of keywords for each subject is 50 for Chinese, 50 for history, and 20 for teaching terms, then the calculation will be based on the total number of keywords being 120, rather than the total number of keywords being 100.

[0092] In some embodiments, the audio matching score of the teaching term keywords is not considered in the calculation of the total number of subject keywords. As long as the audio text data contains the teaching term keywords, the audio matching score of the teaching term keywords is recorded as 1, otherwise it is 0. At this time, the teaching term keywords are used to determine whether the video to be reviewed is a subject education video.

[0093] Step S304: Based on the number of keywords for each subject and the total number of keywords in the audio text data, determine the audio matching score corresponding to each subject in the audio matching score group.

[0094] Specifically, the audio matching score may be determined based on the ratio of the number of keywords in each subject to the total number of keywords, or may be calculated based on a percentage.

[0095] For example, a certain audio text data contains 163 keywords, including 4 teaching terms and 159 subject keywords, including 143 in chemistry, 9 in physics, 5 in Chinese, and 2 in mathematics. The subject matching score is calculated by the ratio of the subject keywords to the total number of subject keywords. For example, if chemistry is 143 / 159 = 0.9, the matching scores for each subject are 0.9 for chemistry, 0.057 for physics, 0.03 for Chinese, and 0.013 for mathematics. The matching score for the teaching terms is 1 (i.e., at least one teaching term is present in the audio text data), and the corresponding audio matching score group is then determined.

[0096] In some embodiments, the audio matching scores for subjects in the audio matching score group whose audio matching scores are less than or equal to a set value can be set to 0. This can reduce the computational complexity. For example, in the above example, if the audio matching scores for subjects less than or equal to 0.3 are set to 0, the final audio matching score group will be: Teaching Language 1, Chemistry 0.9, Physics 0.057.

[0097] Step S305: Segment the video text data into knowledge point sentences based on a sentence segmentation algorithm.

[0098] Specifically, by segmenting video text data into knowledge point sentences rather than keywords, we can effectively avoid the difficulty of identifying subjects with ambiguous keywords. For example, keywords in history and politics are more common in daily conversations. Compared with subjects such as mathematics, physics, and chemistry, they lack highly representative vocabulary. Long text descriptions are needed to distinguish between teaching content and general historical explanations.

[0099] The sentence segmentation algorithm can be based on line breaks or sentence-ending punctuation, or can be trained using machine learning or neural networks. For example, if a text is presented in multiple paragraphs in a video frame, it can be segmented into different sentences based on the paragraphs or the sentence-ending punctuation within the paragraphs.

[0100] Step S306: Based on the text matching algorithm, determine the subject to which the text with the highest similarity to the knowledge point sentence in the subject knowledge point text library belongs.

[0101] Specifically, by setting up a subject knowledge point text library, it helps to avoid the large amount of duplication of keywords in similar subjects, such as history and Chinese, mathematics and physics.

[0102] Furthermore, the similarity between the sentences in the subject knowledge point text library and the knowledge point sentences can be determined based on the overlap calculated for each keyword in the knowledge point sentences. For example, the sentence in the subject knowledge point text library is "the sum of the squares of the two right-angled sides of a right triangle is equal to the square of the hypotenuse", and the knowledge point sentence in the video text data is "the sum of the squares of the two right-angled sides of a right triangle is equal to the square of the third side", where the total number of keywords is 7 (the content separated by spaces in the sentence is each keyword), of which 6 are the same as the keywords and sentences in the knowledge point text library, and only "the third side" and "the hypotenuse" are different, so the similarity can be obtained as 6 / 7=85.7%.

[0103] Step S307: Determine the number of texts with the highest similarity for each subject contained in the video text data.

[0104] Specifically, since the knowledge points of different subjects are different, each knowledge point sentence generally has only one text with the highest matching similarity in the subject knowledge point text library, and thus the subject with the highest similarity can be determined.

[0105] In some embodiments, the similarity includes a set threshold. When a knowledge point sentence does not find a sentence in the subject knowledge point text library with a similarity greater than or equal to the set threshold, it can be considered that the knowledge point sentence is not a teaching and training-related sentence, and the corresponding subject of the sentence is not calculated, such as "This period's homework is as follows", "Thank you", and other sentences.

[0106] Step S308: Based on the number of texts with the highest similarity and the number of knowledge point sentences for each subject, determine the video matching score corresponding to each subject in the video matching score group.

[0107] Specifically, the video matching score corresponding to each subject may be determined based on the ratio of the number of texts with the highest similarity to the number of knowledge point sentences in each subject.

[0108] For example, in a certain video text data, the video matching scores corresponding to each knowledge point sentence are 0.2 for high school chemistry, 0.1 for junior high school chemistry, and 0.05 for mathematics (excluding subjects with scores less than 0.05).

[0109] In some embodiments, the video matching score array may also include sentences of teaching terms as matching objects. In this case, the subject knowledge point text library also contains sentences of teaching terms, and the video matching score of the teaching terms is also calculated as 1 if the teaching terms are included and 0 if the teaching terms are not included.

[0110] Step S309: Determine the matching score of each subject based on the audio matching score group and the video matching score group.

[0111] The final matching score for each subject can be obtained by summing up the scores of each subject in the audio matching score array and the video matching score array in sequence.

[0112] In some embodiments, when the matching scores for each subject are high, but the score for teaching terms is 0, the video under review should be judged as a non-disciplinary education and training video. For example, if the final subject matching scores are: teaching terms 0, chemistry 1.18, physics 0.02, and mathematics 0.05, although the chemistry matching score confirms that the subject should be a chemistry education and training video, since the teaching terms are 0, it should still be considered a non-disciplinary education and training video. In this case, the video under review may be a short video of a certain subject, which may have a high keyword score but does not contain teaching terms.

[0113] Step S310: Determine the subject with the highest matching score as the subject classification of the video to be reviewed.

[0114] Among them, the higher the comprehensive evaluation matching score, the higher the probability that it contains educational videos of the target subject classification. Therefore, the subject with the highest matching score can generally be defaulted to the subject classification of the video to be reviewed.

[0115] The video content recognition method provided by the embodiment of the present disclosure obtains the text content corresponding to the audio and image of the video to be reviewed, and substitutes the audio text data and the video text data into the teaching terminology and subject keyword library, and the subject knowledge point text library respectively to determine the matching score of each subject, and then determines the subject classification of the video to be reviewed based on the subject with the highest matching score. In this way, it can effectively ensure that the identified subjects are based on different databases, thereby ensuring the accuracy and reliability of the recognition. At the same time, it can also eliminate non-subject education and training videos, thereby ensuring the automatic identification of the specific subject classification and type of the video to be reviewed.

[0116] Figure 4 This is a schematic diagram of the structure of a video content recognition device provided by the present disclosure. Figure 4 As shown, the video content recognition device 400 includes: an acquisition module 410, a first determination module 420, a second determination module 430 and a processing module 440.

[0117] An acquisition module 410 is configured to acquire audio text data and video text data of the video to be reviewed in response to the received video to be reviewed;

[0118] A first determination module 420 is configured to determine an audio matching score group corresponding to each subject of the audio text data based on a preset teaching terminology and subject keyword library, wherein the audio matching score group is used to represent a matching score between the audio text data and each subject;

[0119] The second determination module 430 is used to determine the video matching score group corresponding to each subject of the video text data based on the preset subject knowledge point text library, and the video matching score group is used to represent the matching score between the video text data and each subject;

[0120] The processing module 440 is used to determine the subject classification corresponding to the video to be reviewed based on the audio matching score group and the video matching score group.

[0121] Optionally, the first determination module 420 is specifically used to match the keywords in the audio text data with the teaching terms and the keywords in the subject keyword library to determine the number of keywords for each subject contained in the audio text data; based on the number of keywords for each subject and the total number of keywords in the audio text data, determine the audio matching score corresponding to each subject in the audio matching score group.

[0122] Optionally, the first determination module 420 is specifically used to, when the keywords in the same audio text data match the keywords of at least two subjects, determine that the number of keywords of at least two subjects is increased by one respectively; accordingly, based on the number of keywords of each subject and the total number of keywords in the audio text data, determine the audio matching score corresponding to each subject in the audio matching score group, including: when the sum of the number of keywords of each subject is greater than the total number of keywords in the audio text data, based on the number of keywords of each subject and the sum of the number of keywords of each subject, determine the audio matching score corresponding to each subject in the audio matching score group.

[0123] Optionally, the second determination module 430 is specifically used to segment the video text data into knowledge point sentences based on a sentence segmentation algorithm; determine the subject to which the text in the subject knowledge point text library has the highest similarity with the knowledge point sentences based on a text matching algorithm; determine the number of texts with the highest similarity for each subject contained in the video text data; and determine the video matching score corresponding to each subject in the video matching score group based on the number of texts with the highest similarity and the number of knowledge point sentences for each subject.

[0124] Optionally, the acquisition module 410 is specifically used to obtain the text content corresponding to the audio of the video to be reviewed as audio text data based on a speech recognition algorithm; and to obtain the text content contained in the image of the video to be reviewed as video text data based on an image text recognition algorithm.

[0125] Optionally, the acquisition module 410 is specifically used to obtain the image content of each video frame of the video to be reviewed; through the image comparison algorithm, determine the video frame whose image content difference reaches a set threshold as a key frame; based on the image text recognition algorithm, obtain the text content contained in the key frame as video text data.

[0126] Optionally, the first determination module 420 is specifically used to obtain a preset teaching terminology and subject keyword library in the following manner: obtaining standard teaching terminology and subject terminology data, determining the keywords of standard teaching terminology and subject terminology; and establishing a teaching terminology and subject keyword library based on the AC dictionary tree algorithm and keywords.

[0127] Optionally, the processing module 440 is specifically configured to determine the matching score of each subject based on the audio matching score group and the video matching score group; and determine the subject with the highest matching score as the subject classification of the video to be reviewed.

[0128] In this embodiment, the video content recognition device can analyze the audio and video of the video to be reviewed separately through the combination of various modules, and match them through keywords and text respectively, effectively ensuring the accuracy of the identified subjects. By combining the analysis results obtained from the audio text data and the video text data, the accuracy and reliability of the automatic identification of the corresponding subjects of the video to be reviewed can be maximized, the recognition efficiency can be improved, and it is convenient to use.

[0129] Figure 5 A schematic diagram of the structure of an electronic device provided by the present disclosure, such as Figure 5 As shown, the electronic device 500 includes a memory 510 and a processor 520 .

[0130] The memory 510 stores a computer program that can be executed by at least one processor 520. The computer program is executed by at least one processor 520 to enable the electronic device to implement the video content recognition method provided in any of the above embodiments.

[0131] The memory 510 and the processor 520 may be connected via a bus 530 .

[0132] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the method embodiments, which will not be repeated here.

[0133] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the following Figures 2 to 3 The video content recognition method of any corresponding embodiment.

[0134] Among them, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0135] One embodiment of the present disclosure provides a computer program product, which includes computer-executable instructions. When the computer-executable instructions are executed by a processor, the computer program product is used to implement the following Figures 2 to 3 The video content recognition method of any corresponding embodiment.

[0136] In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0137] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0138] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video content recognition method, characterized in that: include: In response to the received video to be reviewed, respectively obtaining audio text data and video text data of the video to be reviewed; Based on a preset teaching terminology and subject keyword library, determining an audio matching score group corresponding to each subject of the audio text data, wherein the audio matching score group is used to represent a matching score between the audio text data and each subject; the preset teaching terminology and subject keyword library is constructed based on standard teaching terminology and subject terminology data through an AC dictionary tree algorithm; Based on a preset subject knowledge point text library, determining a video matching score group corresponding to each subject of the video text data, wherein the video matching score group is used to represent a matching score between the video text data and each subject; the subject knowledge point text library includes preset sentences; Determine the subject classification corresponding to the video to be reviewed based on the audio matching score group and the video matching score group; The step of determining the video matching score groups corresponding to the video text data for each subject based on a preset subject knowledge point text library includes: Based on a sentence segmentation algorithm, the video text data is segmented into knowledge point sentences; Based on a text matching algorithm, determining the subject to which the text with the highest similarity to the knowledge point sentence in the subject knowledge point text library belongs; Determining the number of the texts with the highest similarity for each subject included in the video text data; Based on the number of the texts with the highest similarity and the number of the knowledge point sentences of each subject, the video matching score corresponding to each subject in the video matching score group is determined.

2. The video content recognition method according to claim 1, characterized in that The step of determining the audio matching score group corresponding to each subject of the audio text data based on the preset teaching terms and subject keyword library includes: Matching the keywords in the audio text data with the keywords in the teaching terms and subject keyword library to determine the number of keywords for each subject contained in the audio text data; Based on the number of keywords of each subject and the total number of keywords in the audio text data, the audio matching score corresponding to each subject in the audio matching score group is determined.

3. The video content recognition method according to claim 2, characterized in that: Determining the number of keywords for each subject contained in the audio text data includes: When a keyword in the same audio text data matches a keyword of at least two subjects, it is determined that the number of keywords of the at least two subjects is increased by one respectively; Accordingly, based on the number of keywords of each subject and the total number of keywords in the audio text data, determining the audio matching score corresponding to each subject in the audio matching score group includes: When the sum of the number of keywords for each subject is greater than the total number of keywords in the audio text data, the audio matching score corresponding to each subject in the audio matching score group is determined based on the number of keywords for each subject and the sum of the number of keywords for each subject.

4. The video content recognition method according to any one of claims 1 to 3, characterized in that: The step of respectively obtaining the audio text data and the video text data of the video to be reviewed includes: Based on the speech recognition algorithm, the text content corresponding to the audio of the video to be reviewed is obtained as audio text data; Based on an image text recognition algorithm, the text content contained in the image of the video to be reviewed is obtained as video text data.

5. The video content recognition method according to claim 4, characterized in that: The method of obtaining the text content contained in the image of the video to be reviewed as video text data based on the image text recognition algorithm includes: Obtaining the image content of each video frame of the video to be reviewed; Through the image comparison algorithm, the video frames whose image content differences reach the set threshold are determined as key frames; Based on an image text recognition algorithm, the text content contained in the key frame is obtained as video text data.

6. The video content recognition method according to any one of claims 1 to 3, characterized in that: The preset teaching terms and subject keyword library is obtained in the following manner: Acquiring standard teaching terms and subject terminology data, and determining keywords of the standard teaching terms and subject terminology; Based on the AC dictionary tree algorithm and the keywords, a teaching terminology and subject keyword library is established.

7. The video content recognition method according to any one of claims 1 to 3, characterized in that: Determining the subject classification corresponding to the video to be reviewed based on the audio matching score group and the video matching score group includes: Determine a matching score for each subject based on the audio matching score group and the video matching score group; The subject with the highest matching score is determined as the subject classification of the video to be reviewed.

8. A video content recognition device, characterized in that: include: An acquisition module, configured to respectively acquire audio text data and video text data of a video to be reviewed in response to a received video to be reviewed; A first determination module is configured to determine an audio matching score group corresponding to each subject of the audio text data based on a preset teaching terminology and subject keyword library, wherein the audio matching score group is used to represent a matching score between the audio text data and each subject; the preset teaching terminology and subject keyword library is constructed based on standard teaching terminology and subject terminology data using an AC dictionary tree algorithm; A second determination module is configured to determine, based on a preset subject knowledge point text library, a video matching score group corresponding to each subject of the video text data, wherein the video matching score group is used to represent a matching score between the video text data and each subject; the subject knowledge point text library includes preset sentences; A processing module, configured to determine a subject classification corresponding to the video to be reviewed based on the audio matching score group and the video matching score group; The second determination module is specifically configured to segment the video text data into knowledge point sentences based on a sentence segmentation algorithm; and determine the subject to which the text having the highest similarity to the knowledge point sentence in the subject knowledge point text library belongs based on a text matching algorithm; Determine the number of the texts with the highest similarity for each subject contained in the video text data; and determine the video matching score corresponding to each subject in the video matching score group based on the number of the texts with the highest similarity for each subject and the number of the knowledge point sentences.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the video content recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the video content recognition method according to any one of claims 1 to 7 when executed by a processor.

Images