Video label extraction method based on knowledge graph

Abstract

The application is a video label extraction method based on a knowledge graph, comprising: 1. According to professional knowledge, basic knowledge packages are sorted out, and word groups in the packages are sorted into excel according to hierarchical relationships and imported into a knowledge graph; 2. According to the characteristics of the feature word groups of the basic knowledge packages, application knowledge packages are sorted out, and the knowledge is sorted into excel and imported into the knowledge graph; 3. Deep learning speech recognition technology is used to recognize video caption information, and a document is formed in combination with a title and preprocessed; 4. Natural language processing keyword matching technology is used to find end node keywords and their basic knowledge package categories in the document according to the basic knowledge package categories; 5. The application knowledge package label is searched in reverse using the keywords, and the output label is customized and fine-tuned. The application has the advantages that: the knowledge graph data is more abundant; the work complexity is simplified, and the label work has better inheritance and systematicness; a big data analysis platform is provided for the development of industries and enterprises, data is more comprehensive, and relationships are clearer.

Description

TECHNICAL FIELD

[0001] The application relates to a video label extraction method based on a knowledge graph, and belongs to the technical field of deep learning, machine learning and knowledge graphs. BACKGROUND

[0002] On the basis of information, the connection between entities can be established to form "knowledge", and the knowledge graph is composed of a plurality of pieces of knowledge, and each piece of knowledge is represented as an spo triple. The knowledge graph is proposed by Google in 2012, and the purpose is to optimize search results and enable computers to understand the internal correlation of human information.

[0003] After years of development, the knowledge graph has mature and landed applications in many industries of artificial intelligence. According to the coverage of the knowledge graph, it is mainly divided into general knowledge graph and industry knowledge graph.

[0004] Due to the diversified and changeable application labels of the video media industry, it is not feasible to use a complete deep learning model for prediction. The keyword matching algorithm is a keyword search method with higher efficiency than the regular search method, and such a method can be applied to video label extraction in combination with the knowledge graph. For a video, the computational complexity and device requirements for directly processing the video or picture itself to label are relatively high, and therefore, the audio content of the video can be texted by using the relatively mature speech recognition technology, and the text is processed, so that the economic benefit is much higher, and the efficiency is also much higher.

[0005] For a new media enterprise, a large amount of video media materials and the like are produced every year, and a large amount of information of news and entertainment media is stored. Searching based on the materials is a difficult and arduous task, and integrating and mining the information is even more difficult. To solve these problems, the data and information are first reasonably and effectively arranged, and the effective arrangement needs to rely on an effective and sustainable label system, and the introduction of AI technology and the knowledge graph can solve the problem. SUMMARY

[0006] The application provides a video label extraction method based on a knowledge graph, and aims to fill the above-mentioned blank in the prior art and realize label of a video based on deep learning natural language processing technology and knowledge graph technology in the field of video media.

[0007] The technical solution of the application is a video label extraction method based on a knowledge graph, which is based on deep learning natural language processing technology and knowledge graph technology, applied to the field of video media, and includes the following steps.

[0008] The first step is to arrange a number of basic knowledge packages according to professional knowledge, and arrange the word groups in these basic knowledge packages according to the hierarchical relationship to excel and import into the knowledge graph. The basic knowledge package refers to all hierarchical categories and corresponding entity keywords of basic knowledge in a specific field;

[0009] The second step is to arrange the application knowledge package according to the combination characteristics of the characteristic words of the basic knowledge package obtained in the first step, arrange these knowledge to excel and import into the knowledge graph. The application knowledge package refers to the classification labels and corresponding feature keywords under certain specific rules set by video classification or application;

[0010] The third step is to use deep learning speech recognition technology to recognize the subtitle information of the video, combine the title to form a document and perform preprocessing;

[0011] The fourth step is to use the keyword matching technology in natural language processing to find the keywords of the end nodes in the basic knowledge package and their basic knowledge package categories in the document obtained in the third step according to the categories of the basic knowledge package obtained in the first step;

[0012] The fifth step is to use the keywords found in the fourth step to search the application knowledge package label in reverse, and finally perform customized fine-tuning to output the label.

[0013] Preferably, in the first step, the load csv command of Cypher is used to import into the neo4j knowledge graph, and the set of end nodes of all basic knowledge packages is baselist, which contains all keywords.

[0014] Preferably, in the second step, the set of end nodes of all application knowledge packages is applylist, which contains the list of corresponding basic knowledge packages and the set of keywords of all application knowledge packages.

[0015] Preferably, in the third step, the speech recognition technology uses deep learning based on attention mechanism and Transformer new neural network structure speech recognition technology to recognize the audio of the video into text, and processes the head and tail, and at the same time, the speech recognition text is preprocessed for numbers, web symbols and special symbols, and is combined with the title to form a to-be-tested document.

[0016] Preferably, in the fourth step, the open source keyword matching technology flashtext is used to find the basic knowledge package name and keyword set corresponding to the document according to baselist.

[0017] Preferably, in the fifth step, the application knowledge package label is searched. Assuming that the search result set of the document is set={A:[x1,x2,x3];B:[y1,y2,y3];C:[z1,z2,z3]}, the basic knowledge package contained in the document is list=[A,B,C]. First, through the reverse query of list and applylist, the element of applylist is found to be the list partlist of the subset of list. Then, it is verified whether the specific keyword set of the basic knowledge package in each element in partlist overlaps with the corresponding keyword set in set. If there is overlap, the application knowledge package label chain corresponding to the element in partlist is reversely queried and output.

[0018] Advantages of the present invention: 1) Compared with general knowledge graphs, the knowledge graph constructed by the present invention, which includes basic knowledge packages and application knowledge packages, has richer data and is more in line with the business needs of new media enterprises;

[0019] 2) The combined application of knowledge graphs, speech recognition, and NLP technologies to video label extraction greatly simplifies the complexity of the work, making labeling more integrated and systematic.

[0020] 3) Constructing a media knowledge graph can serve as basic data and provide a big data analysis platform for industry and enterprise development, with more comprehensive data and clearer relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the basic knowledge package graph structure in the video tag extraction method based on knowledge graph of the present invention.

[0022] Figure 2 It is a schematic diagram of the knowledge package graph structure applied in the video tag extraction method based on knowledge graph of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below with reference to examples and specific implementation methods.

[0024] The video tag extraction method based on knowledge graph is based on deep learning natural language processing technology and knowledge graph technology, and is applied to the video media field, including:

[0025] The first step is to organize several basic knowledge packages such as places, political figures, food, countries, etc. according to professional knowledge, and organize the phrases in these basic knowledge packages into Excel according to the hierarchical relationship and import them into the knowledge graph. Basic knowledge packages ( Figure 1 ) refers to all hierarchical categories of basic knowledge in a specific field and their corresponding entity keywords;

[0026] Second, according to the combination of the characteristic words of the basic knowledge package, the application knowledge package is sorted out, such as the feature of the "star" sub-label under the "entertainment circle" label is the place basic knowledge package plus star basic knowledge package information, and the knowledge is sorted out into excel and imported into the knowledge graph, and the application knowledge package ( Figure 2 ) refers to the classification label and its corresponding feature keywords under certain specific rules set by video classification or application;

[0027] Third, the speech recognition technology of deep learning is used to recognize the subtitle information of the video, and the document is formed by combining the title and preprocessed;

[0028] Fourth, the keyword matching technology in natural language processing is used to find the keywords of the end nodes in the basic knowledge package and their basic knowledge package categories in the above documents according to the categories of the basic knowledge package;

[0029] Fifth, the keywords found in the fourth step are used to search the application knowledge package label in reverse, and finally customized fine-tuning is performed to output the label.

[0030] Specifically,

[0031] In the first step, the sorting of the basic knowledge package is mainly like: China - Yunnan - Nujiang - Lushui County, Fugong, Gongshan, Lanping, Nujiang, such hierarchical knowledge sorting, and using the load csv command of Cypher to import into the neo4j knowledge graph; Set the end node set of all basic knowledge packages as baselist, which contains all the keywords.

[0032] In the second step, the label system of the application knowledge package can be like: China - Entertainment Culture - Entertainment Circle - Star - Place Name (Beijing, Shanghai, etc.) + Star Name, such structure is imported into the knowledge graph; Set the end node set of all application knowledge packages as applylist, which contains the basic knowledge package list and its keyword set corresponding to the application knowledge package.

[0033] In the third step, the speech recognition technology can use the speech recognition technology based on attention mechanism and Transformer neural network structure of deep learning to recognize the audio of the video into text, which can be processed by cutting off the head and tail; At the same time, the text recognized by speech recognition is preprocessed for numbers, web symbols and special symbols, and is combined with the title into a document to be tested.

[0034] In the fourth step, some open source keyword matching technologies such as flashtext can be used to find the basic knowledge package name and keyword set corresponding to the document according to baselist.

[0035] In the fifth step, the application knowledge package label is searched. Assuming that the search result set of the document is set={A:[x1,x2,x3];B:[y1,y2,y3];C:[z1,z2,z3]}, the basic knowledge package contained in the document is list=[A,B,C]. First, the list and the reverse pair of applylist are used to find the list partlist whose elements are the subset of list; then, it is verified whether the specific keyword set of the basic knowledge package in each element in partlist coincides with the corresponding keyword set in set. If there is a coincidence, the application knowledge package label chain corresponding to the element in partlist is reversely queried and output.

[0036] The above merely describes the preferred embodiments of the present application, and it should be noted that, for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application.

Claims

1. A method for video tag extraction based on a knowledge graph, characterized in that The method is based on deep learning natural language processing technology and knowledge graph technology, applied to the field of video media, comprising: First, according to professional knowledge, a number of basic knowledge packages are sorted out, and the word groups in these basic knowledge packages are sorted into excel according to the hierarchical relationship and imported into the knowledge graph. The basic knowledge package refers to all hierarchical categories and corresponding entity keywords of the basic knowledge of a specific field; Second, according to the combination characteristics of the characteristic words of the basic knowledge package obtained in the first step, the application knowledge package is sorted out, and the knowledge is sorted into excel and imported into the knowledge graph. The application knowledge package refers to the classification label and corresponding feature keyword under some specific rules set by video classification or application; Third, the speech recognition technology of deep learning is used to recognize the subtitle information of the video, and the title is combined to form a document and preprocessed; Fourth, the keyword matching technology in natural language processing is used to find the keywords of the end nodes in the basic knowledge package and their basic knowledge package categories in the document obtained in the third step according to the categories of the basic knowledge package obtained in the first step; Fifth, using the keywords found in the fourth step, the application knowledge package label is searched in reverse, and finally customized fine-tuning is performed to output the label; In the first step, the load csv command of Cypher is used to import into the neo4j knowledge graph, and the set of all end nodes of the basic knowledge package is baselist, which contains all the keywords; In the second step, the set of all end nodes of the application knowledge package is applylist, which contains the list of all application knowledge package corresponding basic knowledge packages and their keyword sets; In the third step, the speech recognition technology adopts the speech recognition technology based on attention mechanism and Transformer new neural network structure of deep learning, which recognizes the audio of the video into text, cuts off the head and tail for processing, and at the same time, the text of speech recognition is preprocessed for numbers, web symbols and special symbols, and is combined with the title into a document to be tested; In the fourth step, the open source keyword matching technology flashtext is used to find the basic knowledge package name and keyword set corresponding to the document according to baselist; In the fifth step, the application knowledge package label is searched. Assuming that the search result set of the document is set={A:[x1,x2,x3];B:[y1,y2,y3];C:[z1,z2,z3]}, A:[x1,x2,x3], B:[y1,y2,y3], C:[z1,z2,z3] are keyword sets, A, B, C are the numbers of the keyword sets, x1, x2, x3, y1, y2, y, z1, z2, z3 are keywords in the keyword sets, and the basic knowledge package contained in the document is list=[A, B, C]. First, the list and the applylist are reversely matched to find the list partlist whose elements are subsets of the applylist. Then, it is verified whether the specific keyword set of the basic knowledge package in each element in the partlist coincides with the corresponding keyword set in the set. If there is a coincidence, the application knowledge package label chain corresponding to the element in the partlist is reversely queried and output.

Images