Video recognition method, device, equipment and storage medium

Abstract

The present application discloses a video recognition method, apparatus, device and storage medium. The video recognition method includes: obtaining text information related to a video; obtaining semantic features of words in the text information, and obtaining a first feature, wherein the first feature includes a preset illegal word feature, or includes a preset illegal word feature and a part-of-speech feature of the words in the text information; calculating an attention feature representing the degree of illegality of the text information through the first feature and the semantic feature; and determining a recognition result of the video based on the attention feature, which belongs to the field of video review technology.

Description

Technical Field

[0001] The present application belongs to the field of video review, and specifically relates to a video recognition method, device, equipment and storage medium. Background Art

[0002] In recent years, Internet content platforms that focus on user-generated content (UGC) video models have developed rapidly. The UGC model allows every Internet user to become a content creator. On the one hand, it can help content platforms obtain a large amount of content at a lower cost. On the other hand, due to the massive amount of data and the loose uploading method, the quality of content varies. For example, some users will post some illegal content, such as pornography and vulgarity. These contents may not only damage the interests of the platform itself and affect the user experience, but may also have a bad impact on society, so it is necessary to identify the video.

[0003] Existing methods for identifying illegal content in videos have low recognition accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video recognition method, device, equipment and storage medium, which can solve the problem of low recognition accuracy of the existing methods for identifying illegal content in videos.

[0005] In a first aspect, an embodiment of the present application provides a video recognition method, comprising:

[0006] Get text information related to the video;

[0007] Acquire semantic features of words in the text information, and acquire a first feature, the first feature including a preset illegal word feature, or including the preset illegal word feature and a part-of-speech feature of the words in the text information;

[0008] An attention feature representing the degree of violation of text information is calculated by using the first feature and the semantic feature;

[0009] Based on the attention features, the recognition result of the video is determined.

[0010] In a second aspect, an embodiment of the present application provides a video recognition device, including:

[0011] An acquisition module, used to acquire text information related to the video;

[0012] The acquisition module is further used to acquire semantic features of words in the text information and acquire a first feature, wherein the first feature includes a preset illegal word feature, or includes a preset illegal word feature and a part-of-speech feature of the words in the text information;

[0013] A calculation module, used for calculating an attention feature representing the degree of violation of the text information through the first feature and the semantic feature;

[0014] The determination module is used to determine the recognition result of the video according to the attention feature.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0018] In an embodiment of the present application, text information related to a video is used as video identification data to identify whether the video is in violation of regulations, and an attention mechanism is introduced. The attention feature that characterizes the degree of violation of the text information is calculated through the attention mechanism, and then the recognition result of whether the video is in violation of regulations is determined based on the attention feature. This can identify implicit violation information that is not displayed in the form of an image in the video, thereby improving the recognition accuracy of the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of a video recognition method provided by an embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of a process for determining a video recognition result provided by an embodiment of the present application;

[0021] Figure 3 It is a flowchart of using three types of attention features to determine video recognition results provided by an embodiment of the present application;

[0022] Figure 4 is a structural schematic diagram of a video recognition device provided in an embodiment of the present application;

[0023] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0024] Figure 6 It is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] The existing methods for identifying illegal content in videos usually extract image frames from the video to identify the illegal element images therein. Image recognition can identify illegal information that is directly displayed in the form of images, but it is difficult for image recognition to identify some illegal information that is displayed implicitly in forms other than images, resulting in low recognition accuracy of the existing methods for identifying illegal content in videos.

[0028] The video recognition method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0029] Figure 1 FIG. 1 is a flow chart of a video recognition method provided by an embodiment of the present application. Figure 1 As shown, the method may include the following steps:

[0030] S110, obtaining text information related to the video.

[0031] The text information related to the video is generally used to explain the content displayed in the video, and is highly related to whether the video violates the regulations. When obtaining the text information related to the video, a text extraction algorithm can be applied to obtain the text information. The text extraction algorithm can extract the text information in the video image frame. For example, the text extraction algorithm can be selected as optical character recognition (OCR).

[0032] S120, obtaining semantic features of words in the text information, and obtaining a first feature, where the first feature includes a preset illegal word feature, or includes the preset illegal word feature and a part-of-speech feature of the words in the text information.

[0033] Among them, the semantic features of words in text information can be extracted by a semantic feature extraction algorithm, and the semantic feature extraction algorithm can extract semantic information in text information. For example, the semantic feature extraction algorithm can be selected as a BERT (Pre-training of Deep Bidirectional Transformers for Language Understanding) model. The preset illegal word features can be obtained by pre-recording illegal words and extracting illegal word features. The part-of-speech features can be obtained by a part-of-speech tagging (Part-of-Speech tagging or POS tagging) algorithm, and the part-of-speech tagging algorithm can identify the part-of-speech features of words in text information. The backbone network of the BERT model uses self-attention to calculate the relationship between texts, and uses a large amount of text corpus to perform self-supervisory training on it, so that the encoded text has common semantic information, obtains the feature encoding of the entire sentence, and the features of each word.

[0034] S130, calculating an attention feature representing the degree of violation of the text information by using the first feature and the semantic feature.

[0035] Among them, the attention feature between the first feature and the semantic feature can be calculated through the attention mechanism, and the core idea of ​​the attention mechanism is to find the correlation between them based on the input data. The attention feature that represents the degree of violation of the text information can be calculated through the attention mechanism.

[0036] S140, determining a recognition result of the video according to the attention feature.

[0037] Among them, the attention feature can reflect the degree of violation of the text information related to the video. According to the attention feature, the recognition result of the video can be determined.

[0038] In an embodiment of the present application, text information related to a video is used as video identification data to identify whether the video is in violation of regulations, and an attention mechanism is introduced. The attention feature that characterizes the degree of violation of the text information is calculated through the attention mechanism, and then the recognition result of whether the video is in violation of regulations is determined based on the attention feature. This can identify implicit violation information that is not displayed in the form of images in the video, thereby improving the recognition accuracy of the video.

[0039] In one embodiment, when the first feature includes a preset offending word feature, S130 may include: calculating a first attention feature between the semantic feature and the preset offending word feature.

[0040] S140 may include: determining a recognition result of the video according to the first attention feature.

[0041] The core idea of ​​the attention mechanism is to find the correlation between input data. The attention mechanism can calculate the first attention feature between the semantic feature and the preset illegal word feature. The first attention feature can reflect the degree of illegality of the text information related to the video. According to the first attention feature, the recognition result of the video can be determined.

[0042] In the embodiment of the present application, in order to better improve the accuracy of video recognition, considering that illegal events or some newly created illegal network words are often difficult to be understood by machine algorithms due to the obscure nature of their information. The embodiment of the present application incorporates the preset illegal word features into the calculation of the attention features, and the semantic features and the preset illegal word features are integrated through the attention mechanism, which can effectively provide additional prior knowledge to improve the accuracy of identifying whether the video is illegal during attention calculation.

[0043] In one embodiment, when the first feature includes a preset illegal word feature and a part-of-speech feature, S130 may include:

[0044] S1301, calculating a first attention feature between a semantic feature and a preset illegal word feature, and calculating a second attention feature between the semantic feature and a part-of-speech feature.

[0045] Among them, the first attention feature and the second attention feature can be calculated through the attention mechanism.

[0046] S1302, concatenate the first attention feature and the second attention feature to obtain a concatenated attention feature.

[0047] S140 may include: determining a recognition result of the video according to the spliced ​​attention feature.

[0048] The first attention feature can represent the correlation between the semantic feature and the preset illegal word feature; the second attention feature can represent the correlation between the semantic feature and the part-of-speech feature. Figure 2 As shown, the first attention feature and the second attention feature can be spliced ​​together through a feature fusion algorithm to obtain a spliced ​​feature. The feature fusion algorithm can fuse features. The feature fusion algorithm can be selected as a feature fusion algorithm based on deep learning theory. Then, the recognition result of the video is determined based on the spliced ​​attention feature.

[0049] In the embodiment of the present application, in order to better improve the accuracy of video recognition, considering the violation events or some newly created illegal network words, it is often difficult for the machine algorithm to understand due to the obscure attributes of their information or the newly created words. In addition, the semantic features of violations are often not displayed in the form of adverbs, such as "ah", "oh", nor in the form of quantifiers, such as "one", "many", and the expression of illegal information is often expressed as nouns and adjectives. The embodiment of the present application integrates the part-of-speech features and the preset illegal word features into the calculation of the attention features, which can effectively provide additional prior knowledge to improve the accuracy of identifying whether the video is illegal during the attention calculation.

[0050] In one embodiment, before S1302: concatenating the first attention feature and the second attention feature to obtain a concatenated attention feature, the method may further include:

[0051] A third attention feature is calculated between semantic features corresponding to the first word and the second word in the text information.

[0052] S1302 may include: splicing the first attention feature, the second attention feature, and the third attention feature to obtain a spliced ​​attention feature.

[0053] Among them, considering the semantic and grammatical connections between different words in text sentences, when calculating the splicing attention features, the self-attention feature can also be introduced to calculate the third attention feature between the semantic features corresponding to the first word and the second word in the text information, which improves the video accuracy. The third attention feature can be calculated through the attention mechanism.

[0054] In the embodiment of the present application, in order to better improve the accuracy of video recognition, considering the violation events or some newly created illegal network words, it is often difficult for the machine algorithm to understand due to the obscure attributes of their information or the newly created words. In addition, the semantic features of violations are often not displayed in the form of adverbs, such as "ah", "oh", nor in the form of quantifiers, such as "one", "many", and the expression form of illegal information is often expressed as nouns and adjectives. In addition, the context of the text information can better determine whether there is illegal information in the text information. The embodiment of the present application integrates the self-attention features of the part-of-speech features, the preset illegal word features, and the semantic features of the text into the calculation of the attention features, fully considering the semantics and grammatical connections between different words between text sentences, and can effectively provide additional prior knowledge to improve the accuracy of identifying whether the video is illegal during attention calculation.

[0055] To help understand, combine Figure 3 A specific embodiment of using the first attention feature, the second attention feature, and the third attention feature to determine the recognition result of the video is described. Figure 3As shown:

[0056] Considering that the semantic feature of the violation is likely to be the content of the preset violation word feature, it should be paid attention to. The first attention feature A between the semantic feature and the preset violation word feature is calculated. 1 .

[0057]

[0058] Among them, h t Represents any semantic feature, p i Represents any preset illegal word feature, N is the number of semantic features, and T represents matrix transposition. An illegal word library with preset illegal word features can be generated in advance and mapped into a Onehot vector. It is used for feature fusion calculation. In order to better identify some illegal events that occur in current events, illegal related words can be divided into several categories. Taking pornography as an example, the illegal related words can be divided into: sexual words, network words, network events, and physiological related words, as shown in Table 1.

[0059] Table 1

[0060]

[0061]

[0062] Consider the distribution relationship between the semantic features and part-of-speech features of violations, that is, the prior knowledge that violations are mostly distributed in adjectives and nouns. These prior distributions are captured through attention calculation, and the second attention feature A between the semantic features and part-of-speech features is calculated through the attention mechanism. 2 .

[0063]

[0064] Among them, j i Represents any part-of-speech feature. In order to better extract the semantic features of violations,

[0065] Incorporating part-of-speech features, in general, illegal words are adjectives and nouns, and there are rarely illegal quantifiers and pronouns. Incorporating part-of-speech tags can make recognition more accurate. The part-of-speech features can be shown in Table 2.

[0066] Table 2

[0067] Symbolic representation Part of Speech Part of Speech Vector No noun [1,0,0,0,0] A adjective [0,1,0,0,0] V verb [0,0,1,0,0] Ad adverb [0,0,0,1,0] W Punctuation [0,0,0,0,1]

[0068] Considering the correlation between the two words in the context, the third attention feature A between the semantic features corresponding to the first word and the second word in the text information is calculated. 3 .

[0069]

[0070] Among them, h t Represents any semantic feature, h i Represents any other semantic feature.

[0071] Then calculate the second splicing feature A final =concat([A 1 ; A 2 ; A 3 ]). And through the activation function sigmoid (*), the vector dimension of the second concatenated feature is reduced to obtain one-dimensional specific data, specifically:

[0072] For each semantic feature, an activation function value s can be calculated t ,s t =

[0073] sigmoid(A final ), the activation function value is compared with the preset violation threshold, and when it meets the preset violation threshold, the word corresponding to the semantic feature is classified as an illegal keyword.

[0074] For all s t , according to the SUM function through the preset weight k t After weighted summation, we get the overall activation function value l t , l t =sum(k t ×s t ).

[0075] Then, l t After concatenating and fusing with all the semantic features cls, it is reactivated using the activation function to obtain the final violation parameter low_score, low_score = Sigmod(W t ×

[0076] Concat([l t ;cls])),W t is a preset weight parameter. When the violation parameter meets the preset violation judgment condition, the video recognition result is determined. For example, if the violation parameter is greater than 0 and less than or equal to 0.3, the video recognition result is not a violation. If the violation parameter is greater than 0.3 and less than or equal to 0.4, the video recognition result is a suspected violation. If the violation parameter is greater than 0.4 and less than or equal to 1, the video recognition result is a violation.

[0077] In one embodiment, at S140: determining a recognition result of the video according to the attention feature, the method may further include:

[0078] Extracting image frames of the video; identifying illegal image information in the image frames to obtain image illegal identification results; S140: determining the identification results of the video according to the attention features, which may include:

[0079] S1401, determining a recognition result of the video according to the attention feature and the illegal image information existing in the image frame.

[0080] In one embodiment, S1401 may include:

[0081] A first violation judgment result of the video is determined according to the image violation recognition result.

[0082] Determine whether there is any illegal text information in the text information related to the video based on the attention features, and obtain a text violation recognition result; determine a second violation judgment result of the video based on the text violation recognition result; and determine that the recognition result of the video is a violation when at least one of the violation judgment result and the second violation judgment result is a video violation.

[0083] The illegal image information in the image frame can be identified by an image recognition algorithm, and the image recognition algorithm can identify the image information in the image frame. For example, the image recognition algorithm can be selected as a SURF algorithm.

[0084] In the embodiment of the present application, image recognition is introduced in conjunction with attention features to perform a two-stage judgment on the violation type of the video, and the two-stage judgment results are coordinated as the final result of video recognition, indicating the accuracy of whether the recognized video is in violation.

[0085] In one embodiment, the text information may include at least one of the following:

[0086] Video title text information, video image frame display text information, video comment text information.

[0087] In one embodiment, S140: determining the recognition result of the video according to the attention feature may include:

[0088] Calculate the activation function value of the attention feature through the activation function;

[0089] The recognition result of the video is determined based on the activation function value and semantic features.

[0090] In one embodiment, when the identification result of the video is identified as a violation, the method may further include:

[0091] Determine the target semantic feature corresponding to the target attention feature; the target attention feature is the attention feature whose activation function value meets the preset violation threshold.

[0092] Determine the position information of the illegal keywords in the video. The position information is used to assist the user in determining the video playback position corresponding to the illegal keywords when reviewing the video. The illegal keywords are words in the video-related text information corresponding to the target semantic features.

[0093] Among them, for each semantic feature corresponding to the attention feature, an activation function value s can be calculated t ,s t =sigmoid(A final ), the activation function value is compared with the preset violation threshold, and when it meets the preset violation threshold, the word corresponding to the semantic feature is classified as an illegal keyword.

[0094] In the embodiment of the present application, when a video is identified as suspected of violating regulations, location information of the violating keywords in the video can be provided to assist the user in reviewing the video, thereby improving the efficiency of user review.

[0095] In an embodiment of the present application, text information related to a video is used as video identification data to identify whether the video is in violation of regulations, and an attention mechanism is introduced. The attention feature that characterizes the degree of violation of the text information is calculated through the attention mechanism, and then the recognition result of whether the video is in violation of regulations is determined based on the attention feature. This can identify implicit violation information that is not displayed in the form of images in the video, thereby improving the recognition accuracy of the video.

[0096] It should be noted that the video recognition method provided in the embodiment of the present application can be executed by a video recognition device or a control module in the video recognition device for executing the video recognition method. In the embodiment of the present application, the video recognition device provided in the embodiment of the present application is described by taking the video recognition device executing the video recognition method as an example.

[0097] Figure 4 A schematic diagram of the structure of a video recognition device provided by an embodiment of the present application is shown. Figure 4 Each module in the device shown has the function of realizing Figure 1 The functions of each step in the process can achieve the corresponding technical effects. Figure 4 As shown, the device may include:

[0098] The acquisition module 410 is used to acquire text information related to the video.

[0099] The acquisition module 410 is further used to acquire semantic features of words in the text information and acquire a first feature, where the first feature includes a preset illegal word feature, or includes a preset illegal word feature and a part-of-speech feature of the words in the text information.

[0100] The calculation module 420 is used to calculate the attention feature representing the degree of violation of the text information through the first feature and the semantic feature.

[0101] The determination module 430 is used to determine the recognition result of the video according to the attention feature.

[0102] In an embodiment of the present application, text information related to a video is used as video identification data to identify whether the video is in violation of regulations, and an attention mechanism is introduced. The attention feature that characterizes the degree of violation of the text information is calculated through the attention mechanism, and then the recognition result of whether the video is in violation of regulations is determined based on the attention feature. This can identify implicit violation information that is not displayed in the form of an image in the video, thereby improving the recognition accuracy of the video.

[0103] In one embodiment, the calculation module 420 is configured to calculate a first attention feature between the semantic feature and the preset offending word feature when the first feature includes a preset offending word feature.

[0104] The determination module 430 is used to determine the recognition result of the video according to the first attention feature.

[0105] In one embodiment, the computing module 420 includes a computing unit 4201 and a splicing unit 4202 .

[0106] The calculation unit 4201 is used to calculate a first attention feature between the semantic feature and the preset illegal word feature, and calculate a second attention feature between the semantic feature and the part-of-speech feature when the first feature includes a preset illegal word feature and a part-of-speech feature.

[0107] The splicing unit 4202 is used to splice the first attention feature and the second attention feature to obtain a spliced ​​attention feature.

[0108] The determination module 430 is used to determine the recognition result of the video according to the spliced ​​attention feature.

[0109] In one embodiment, the calculation unit 4201 is also used to calculate a third attention feature between the semantic features corresponding to the first word and the second word in the text information before concatenating the first attention feature and the second attention feature to obtain the concatenated attention feature.

[0110] The splicing unit 4202 is specifically used to splice the first attention feature, the second attention feature, and the third attention feature to obtain a spliced ​​attention feature.

[0111] In one embodiment, the apparatus further includes an extraction module 440 and an identification module 450 .

[0112] The extraction module 440 is used to extract image frames of the video before determining the recognition result of the video based on the attention feature.

[0113] The identification module 450 is used to identify illegal image information in the image frame.

[0114] The determination module 430 is specifically used to determine the recognition result of the video according to the attention feature and the illegal image information existing in the image frame.

[0115] In one embodiment, the text information may include at least one of the following.

[0116] Video title text information, video image frame display text information, video comment text information.

[0117] In the embodiment of the present application, in order to better improve the accuracy of video recognition, considering the violation events or some newly created illegal network words, it is often difficult for the machine algorithm to understand due to the obscure attributes of their information or the newly created words. In addition, the semantic features of violations are often not displayed in the form of adverbs, such as "ah", "oh", nor in the form of quantifiers, such as "one", "many", and the expression form of illegal information is often expressed as nouns and adjectives. In addition, the context of the text information can better determine whether there is illegal information in the text information. The embodiment of the present application integrates the self-attention features of the part-of-speech features, the preset illegal word features, and the semantic features of the text into the calculation of the attention features, fully considering the semantics and grammatical connections between different words between text sentences, and can effectively provide additional prior knowledge to improve the accuracy of identifying whether the video is illegal during attention calculation.

[0118] The video recognition device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0119] The video recognition device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0120] The video recognition device provided in the embodiment of the present application can achieve Figures 1 to 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0121] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0122] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0123] The electronic device 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, and a processor 610.

[0124] Those skilled in the art will appreciate that the electronic device 600 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 610 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0125] It should be understood that in the embodiment of the present application, the input unit 604 may include a graphics processor (GPU) 6041 and a microphone 6042, and the graphics processor 6041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include but are not limited to a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 609 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 610 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface, and the application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 610.

[0126] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned video recognition method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0127] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned video recognition method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0129] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0130] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0132] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video recognition method, characterized in that: include: Get text information related to the video; Acquire semantic features of words in the text information, and acquire a first feature, where the first feature presets a feature of an illegal word and a part-of-speech feature of the words in the text information; Calculating a first attention feature between the semantic feature and the preset illegal word feature, and calculating a second attention feature between the semantic feature and the part-of-speech feature; Concatenating the first attention feature and the second attention feature to obtain a concatenated attention feature; Determine a recognition result of the video according to the spliced ​​attention feature.

2. The video recognition method according to claim 1, characterized in that: Before concatenating the first attention feature and the second attention feature to obtain a concatenated attention feature, the method further includes: Calculate a third attention feature between the semantic features respectively corresponding to the first word and the second word in the text information; The step of concatenating the first attention feature and the second attention feature to obtain a concatenated attention feature includes: The first attention feature, the second attention feature, and the third attention feature are concatenated to obtain the concatenated attention feature.

3. The video recognition method according to any one of claims 1 to 2, characterized in that: Before determining the recognition result of the video according to the spliced ​​attention feature, the method further includes: Extracting image frames of the video; Identifying illegal image information present in the image frame; The determining the recognition result of the video according to the spliced ​​attention feature includes: The recognition result of the video is determined according to the attention feature and the illegal image information existing in the image frame.

4. A video recognition device, characterized in that: include: An acquisition module, used to acquire text information related to the video; The acquisition module is further used to acquire semantic features of words in the text information and acquire a first feature, wherein the first feature includes a preset illegal word feature and a part-of-speech feature of the words in the text information; a calculation unit, configured to calculate a first attention feature between the semantic feature and the preset illegal word feature, and calculate a second attention feature between the semantic feature and the part-of-speech feature; a concatenation unit, configured to concatenate the first attention feature and the second attention feature to obtain a concatenated attention feature; A determination module is used to determine the recognition result of the video according to the spliced ​​attention feature.

5. The video recognition device according to claim 4, characterized in that: The calculation unit is further configured to calculate a third attention feature between the semantic features respectively corresponding to the first word and the second word in the text information before concatenating the first attention feature and the second attention feature to obtain the concatenated attention feature; The splicing unit is specifically used to splice the first attention feature, the second attention feature, and the third attention feature to obtain the spliced ​​attention feature.

6. The video recognition device according to any one of claims 4 to 5, characterized in that: The device also includes an extraction module and a recognition module; The extraction module is used to extract the image frame of the video before determining the recognition result of the video according to the spliced ​​attention feature; The identification module is used to identify illegal image information existing in the image frame; The determination module is specifically used to determine the recognition result of the video according to the attention feature and the illegal image information existing in the image frame.

7. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the video recognition method according to any one of claims 1 to 3.

8. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the video recognition method according to any one of claims 1 to 3 are implemented.

Images