A content recognition method and related apparatus

Abstract

Embodiments of the present application disclose a content recognition method and related device, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation, and auxiliary driving. The content to be recognized and content supplementary information of the content to be recognized are obtained, the content supplementary information includes object information corresponding to a publishing object publishing the content to be recognized and / or interaction information of the content to be recognized, the publishing object can reflect a publisher feature of publishing the content to be recognized, and the interaction information can reflect a view of a viewer of the content to be recognized on the content to be recognized. A joint feature of the content to be recognized determined according to the content to be recognized and the content supplementary information of the content to be recognized is more comprehensive and accurate in representing the content to be recognized, so that a category of the content to be recognized recognized according to the joint feature is more accurate. Therefore, the accuracy of the joint feature of the content to be recognized determined by multi-dimensional information is higher, and the precision of the category of the content to be recognized recognized is higher.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a content identification method and related devices. Background Art

[0002] In order to improve the user experience of the content platform, the content platform will display content by category. Taking videos as an example, videos are divided into reversal videos, funny videos, historical videos, etc.

[0003] Related technologies primarily rely on the content itself to identify whether it falls into a certain category. For example, to identify whether a video is a reverse video, the video is fed into a classification model. The classification model can identify the video's characteristics and determine whether it is reversed or non-reversed.

[0004] However, the recognition effect of this method is poor. Summary of the Invention

[0005] In order to solve the above technical problems, the present application provides a content recognition method and related devices for improving the recognition effect.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In one aspect, an embodiment of the present application provides a content identification method, the method comprising:

[0008] Acquire content to be identified and content supplementary information of the content to be identified, wherein the content supplementary information of the content to be identified includes at least one of object information corresponding to a publishing object that publishes the content to be identified and interaction information for the content to be identified;

[0009] determining a joint feature of the content to be identified based on the content to be identified and the content supplementary information of the content to be identified;

[0010] The category of the content to be identified is identified according to the joint features of the content to be identified.

[0011] On the other hand, an embodiment of the present application provides a content recognition device, the device comprising: an acquisition unit, a determination unit, and an identification unit;

[0012] The acquiring unit is configured to acquire content to be identified and content supplementary information of the content to be identified, wherein the content supplementary information of the content to be identified includes at least one of object information corresponding to a publishing object that published the content to be identified and interactive information for the content to be identified;

[0013] The determining unit is configured to determine a joint feature of the content to be identified based on the content to be identified and the content supplementary information of the content to be identified;

[0014] The identification unit is configured to identify the category of the content to be identified based on the joint features of the content to be identified.

[0015] In another aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory.

[0016] The memory is used to store a computer program and transmit the computer program to the processor;

[0017] The processor is configured to execute the method described above according to instructions in the computer program.

[0018] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the method described in the above aspects.

[0019] In another aspect, embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in the above aspects.

[0020] It can be seen from the above technical solution that the content to be identified and the content supplementary information of the content to be identified are obtained. The content supplementary information includes the object information corresponding to the publishing object of the content to be identified and / or the interactive information of the content to be identified, wherein the object information can reflect the characteristics of the publisher of the content to be identified, and the interactive information can reflect the views of the viewers of the content to be identified on the content to be identified, so that the content supplementary information can serve as a supplement to the content to be identified. Compared with the features determined only based on the content to be identified, the joint features of the content to be identified determined based on the content to be identified and the content supplementary information of the content to be identified are more comprehensive and accurate in describing the content to be identified, so that the category of the content to be identified identified based on the joint features is more accurate. Therefore, the accuracy of the joint features of the content to be identified determined by multi-dimensional information is higher, and the accuracy of the category of the content to be identified identified based on the more accurate joint features is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A schematic diagram of a content identification system provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of a content identification method provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of obtaining joint features provided in an embodiment of the present application;

[0025] Figure 4 A flowchart for determining the category of content to be identified provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of determining the similarity probability between contents provided in an embodiment of the present application;

[0027] Figure 6 Another flowchart for determining the category of content to be identified provided in an embodiment of the present application;

[0028] Figure 7 A flowchart of another method for determining the category of content to be identified provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of a content recognition device provided in an embodiment of the present application;

[0030] Figure 9 A schematic diagram of the structure of the server provided in the embodiment of the present application;

[0031] Figure 10 A schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application are described below with reference to the accompanying drawings.

[0033] In view of the fact that in the related art, the effect of identifying the category of content only through the content itself, that is, single-dimensional information, is poor, the embodiment of the present application provides a content identification method and related devices, which can more accurately determine the joint features of the content to be identified through multi-dimensional information, and thus the category of the content to be identified based on the more accurate joint features is more accurate.

[0034] The content recognition method provided in the embodiments of this application is realized based on artificial intelligence. Artificial intelligence (AI) is a theory, method, technology and application system for using a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

[0035] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0036] In the embodiments of this application, the artificial intelligence technologies mainly involved include the above-mentioned machine learning / deep learning, etc. Machine learning is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural network technology, etc.

[0037] In the content recognition method provided in the embodiments of this application, the artificial intelligence model used mainly involves the application of machine learning, which determines the joint features of the content to be recognized through machine learning, and further determines the category of the content to be recognized.

[0038] The method is applied to Figure 1The content recognition system shown in the figure includes a server and a terminal device, and the client is deployed on the terminal device, wherein the client can be run on the terminal device in the form of a browser, or can be run on the terminal device in the form of an independent application (APP), etc. The specific presentation form of the client is not limited here. The server involved in this application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can specifically be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application is not limited here. The number of servers and terminal devices is also not limited. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0039] Exemplarily, the server can obtain the content to be identified (e.g., video) and content supplementary information of the content to be identified from the content library. The content supplementary information includes at least one of the object information corresponding to the publishing object (e.g., account) that published the content to be identified (e.g., the category distribution of historical content published by the account, corresponding to the account information) and interactive information for the content to be identified (e.g., barrage, comments). Thus, the accuracy of the joint features of the content to be identified determined by the two-dimensional information of the content to be identified and the content supplementary information of the content to be identified is higher, and the accuracy of the category of the content to be identified identified based on the more accurate joint features is higher. Then, based on the category of the content to be identified, it is determined whether the content to be identified is to be pushed to the corresponding terminal device.

[0040] The content identification method provided in the embodiment of the present application can be executed by a server. However, in other embodiments of the present application, the terminal device can also have similar functions as the server to execute the content identification method provided in the embodiment of the present application, or the terminal device and the server can jointly execute the content identification method provided in the embodiment of the present application, and this embodiment does not limit this.

[0041] The content recognition method provided in this application can be applied to a content recognition device with data processing capabilities. Below, in conjunction with the accompanying drawings, a content recognition method provided in an embodiment of this application is introduced with a server as a content recognition device.

[0042] See also Figure 2 , Figure 2 This is a flow chart of a content identification method provided in an embodiment of the present application. Figure 2 As shown, the content identification method includes the following steps:

[0043] S201: Obtain the to-be-identified content and content supplementary information of the to-be-identified content.

[0044] The to-be-identified content is content that needs to be determined in a category, and the content is information such as a video, a picture, an article, and the like uploaded to a content platform for viewing by a content platform user. A content publisher generally needs to rely on an account of the content platform to upload the content to the content platform. For example, the content publisher publishes the to-be-identified content to the content platform through the account.

[0045] The supplementary information of the to-be-identified content includes one or more of object information corresponding to a publishing object that publishes the to-be-identified content and interaction information for the to-be-identified content. The publishing object refers to an object (such as an account or a client that publishes the to-be-identified content) on which the to-be-identified content is dependent. The object information is information for identifying the object. If the publishing object is an account, the account information can be a tag of the account, historical content published through the account, and the like. The account information can reflect the publisher characteristics of the to-be-identified content. For example, if an account often publishes funny videos, the probability that the to-be-identified content published by the account is a funny video is relatively large. The interaction information for the to-be-identified content is information generated by interaction between a user viewing the to-be-identified content and the to-be-identified content, such as a barrage, a comment, and the like. The interaction information can reflect the views of the viewer on the to-be-identified content.

[0046] As a possible implementation manner, the to-be-identified content and the content supplementary information of the to-be-identified content can be obtained from a content library corresponding to the content platform.

[0047] S202: Determine a joint feature of the to-be-identified content according to the to-be-identified content and the content supplementary information of the to-be-identified content.

[0048] The to-be-identified content can represent its own features, and the supplementary information of the to-be-identified content can reflect the publisher characteristics of the to-be-identified content or the views of the viewer on the to-be-identified content, which can be used as a supplement to the to-be-identified content. Compared with the features determined only according to the to-be-identified content, the joint features of the to-be-identified content determined according to the to-be-identified content and the content supplementary information of the to-be-identified content are more comprehensive and accurate in representing the to-be-identified content, so that the category of the to-be-identified content recognized according to the joint features is more accurate.

[0049] The joint feature is used to represent the characteristics of the to-be-identified content, and the category of the to-be-identified content can be recognized through the joint feature. Moreover, the joint feature is determined through multi-dimensional information, so that the accuracy of the category of the to-be-identified content recognized according to the joint feature with higher accuracy is higher.

[0050] S203: Recognize the category of the to-be-identified content according to the joint feature of the to-be-identified content.

[0051] It should be noted that the category of the content to be identified can be set according to the needs of the content platform, and can be a comedy category, a science category, a food category, etc.

[0052] The embodiments of the present application do not specifically limit the method of identifying the category of the content to be identified based on the joint features of the content to be identified. For example, identification is performed through a pre-trained model. For another example, a model can be trained to identify the category of the content to be identified based on the joint features of the content to be identified. A model can also be trained to identify whether the content to be identified is a category based on the joint features of the content to be identified, such as whether the content to be identified is a plot reversal video. Among them, a plot reversal video is a video with opposite plots at the beginning and end of the video. Reversal videos can enhance the expression effect of the video plot. By setting up reversal videos, more viewers can be attracted, the recognition accuracy of reversal videos can be improved, and a data basis can be provided for video platform recommendations, retrieval and distribution, etc.

[0053] It can be seen from the above technical solution that the content to be identified and the content supplementary information of the content to be identified are obtained. The content supplementary information includes the object information corresponding to the publishing object of the content to be identified and / or the interactive information of the content to be identified, wherein the object information can reflect the characteristics of the publisher of the content to be identified, and the interactive information can reflect the views of the viewers of the content to be identified on the content to be identified, so that the content supplementary information can serve as a supplement to the content to be identified. Compared with the features determined only based on the content to be identified, the joint features of the content to be identified determined based on the content to be identified and the content supplementary information of the content to be identified are more comprehensive and accurate in describing the content to be identified, so that the category of the content to be identified identified based on the joint features is more accurate. Therefore, the accuracy of the joint features of the content to be identified determined by multi-dimensional information is higher, and the accuracy of the category of the content to be identified identified based on the more accurate joint features is higher.

[0054] In addition, by automatically identifying the category of the content to be identified, not only can the cost of manual category labeling be reduced, but also the efficiency and effect of category identification can be improved, enriching each type of content on the content platform. Taking reversal videos as an example, the content library corresponding to reversal videos in the content platform is enriched, providing users who like to watch reversal videos with richer data resources, and improving the user experience of content platform recommendations, retrieval products, etc.

[0055] As a possible implementation method, the embodiment of the present application provides an S202, that is, a specific implementation method of determining the joint features of the content to be identified based on the content to be identified and the content supplementary information of the content to be identified, see S2021-S2023 for details.

[0056] S2021: Determine a first feature based on the content to be identified.

[0057] If the content to be identified is a video, the video includes each frame of the image that constitutes the video, the audio corresponding to each frame of the image, and the text corresponding to each frame of the image (such as titles, subtitles, narration, etc.). In order to deeply understand the video content, three models can be pre-trained to identify the image, audio, and text of the video respectively. Specifically, the content to be identified is input into the first model to obtain the image sub-features of the content to be identified; the content to be identified is input into the second model to obtain the audio sub-features of the content to be identified; the content to be identified is input into the third model to obtain the text sub-features of the content to be identified. Among them, the first model is used to extract the characteristics of each frame of the image in the video, the second model is used to extract the characteristics of the audio in the video, and the third model is used to extract the characteristics of the text in the video. The first feature is determined based on the image sub-features, audio sub-features, and text sub-features.

[0058] The embodiment of the present application does not specifically limit the first model, the second model and the third model. Figure 3 See Figure 3 , this figure is a schematic diagram of obtaining joint features provided in an embodiment of the present application.

[0059] Among them, the first model is an efficient network (EfficientNet) model and a self-attention mechanism (self-attention) model. It extracts the image key frame sequence from the content to be identified, inputs the image key frame sequence into the EfficientNet model, and obtains the picture frame representation of the content to be identified. Then, the self-attention model is used to construct a temporal fusion of multiple picture frames of the content to be identified to obtain image sub-features, that is, the image representation of the content to be identified.

[0060] The second model is a Visual Geometry Group Network (VGG) model (VGGish model for short) and a self-attention model based on TensorFlow (an open source software library). It extracts the audio frame sequence from the content to be recognized, inputs the audio frame sequence into the EfficientNet model, and obtains the audio frame representation of the content to be recognized. Then, the self-attention model is used to construct a temporal fusion of multiple audio frames of the content to be recognized to obtain audio sub-features, that is, the audio representation of the content to be recognized.

[0061] The third model is the Bidirectional Encoder Representations from Transformers (BERT) model. It extracts text from the content to be recognized, such as titles, subtitles, and dialogue. Through tokenization, the extracted text is concatenated and fed into the BERT model to generate text sub-features, which represent the text of the content to be recognized.

[0062] It should be noted that the dialogue in the video can be obtained through Automatic Speech Recognition (ASR) technology.

[0063] It should be noted that the first feature can be obtained by concatenating the image sub-features, the audio sub-features and the text sub-features.

[0064] Therefore, when the content to be identified is a video, by understanding the video's characteristics from the three dimensions of image, audio, and text, the resulting first feature is a multi-dimensional representation of the content to be identified, obtained by splicing the three dimensional representations. This can more accurately describe the characteristics of the content to be identified, thereby improving the accuracy of subsequent content classifications to be identified. In addition, through extensive experiments, it was found that the first features obtained using the EfficientNet model, the self-attention model, the VGGish model, and the BERT model performed the best. These models are all general models, simple to train, and can accelerate development.

[0065] S2022: Determine a second feature based on the supplementary content information of the content to be identified.

[0066] The content supplementary information of the content to be identified includes at least one of object information and interaction information, which will be described below respectively.

[0067] If the information to be identified includes interactive information, the BERT model can be used to extract textual information from the interactive information as a second feature. For example, the BERT model can extract comments and comments about the content to be identified, so that the obtained second feature can determine the category of the content to be identified based on the viewer's thoughts.

[0068] If the information to be identified includes object information, account information is used as an example for explanation. The category distribution of historical content published by the account can be determined based on the account information, and the second feature can be determined based on the category distribution. Among them, the category distribution is used to characterize the probability that historical content belongs to multiple categories. For example, in the historical content published by account A, funny videos account for 50%, popular science videos account for 30%, and other videos account for 20%. Therefore, through the publishing objects such as accounts, the category distribution of the historical content published by the publishing object is determined. Since the content published by the content publisher is stable, the object information can reflect the characteristics of the content to be identified from the side. Therefore, the category of the historical content published by the publishing object is used as the basis for identifying the content to be identified. For example, the category of the content to be identified is more likely to be the category with the largest proportion in the historical content, thereby improving the accuracy of subsequent identification of the category of the content to be identified.

[0069] Continue with Figure 3 For example, the object information of the content to be identified is determined based on the content to be identified, the category distribution of the historical content published by the publishing object is determined based on the object information, and the second feature is determined based on the fully connected network layer.

[0070] S2023: Determine a joint feature of the content to be identified based on the first feature and the second feature.

[0071] The embodiment of the present application does not specifically limit the method of obtaining the joint feature. For example, the joint feature of the content to be identified is obtained by splicing the first feature and the second feature.

[0072] Continue with Figure 3 For example, the joint features can be subsequently input into the fully connected network layer to obtain the category of the content to be identified. It should be noted that the fully connected network layer is used for classification and can be a pre-trained classification model, which is not specifically limited in this application.

[0073] Although historical content published by content publishers is stable, for example, historical content often covers the same field, content publishers are affected by factors such as traffic and may publish content related to hot events, resulting in jitter in the characteristics of their published historical content. Based on this, the embodiments of the present application propose a method for dynamically adjusting the degree of influence of each historical content on the content to be identified. The following example uses the content to be identified as a video as an example, see A1-A3.

[0074] A1: Determine the historical content published by the publishing object and the target category corresponding to the historical content based on the object information.

[0075] Continuing with the example of an account as the publishing target, the historical content published by the account is the content before the account published the content to be identified. When uploading content to the content platform, the content publisher selects a category for the content, or the content platform adds a corresponding category to the uploaded content. Therefore, the target category corresponding to the historical content can be obtained. The target category is one of the multiple categories included in the content platform, such as the comedy video category or the food video category.

[0076] A2: Determine the target probability of publishing the target category of the content based on the playback information and time information of the historical content.

[0077] The playback information of the historical content can reflect the importance of the historical content from the playback level, and the time information of the historical content can reflect the importance of the historical content from the time level.

[0078] If the historical content is a video, its playback information can be one or more combinations of video playback volume, video duration, and average video playback completion rate. The playback information of historical content can be used to determine the weight of a type of historical content in all categories of historical content. For example, a video with a large number of video playbacks, a long video duration, and a high average video playback completion rate may be a video that is popular with viewers and is also a content type that content publishers value more, and they will continue to create in this content type.

[0079] If the historical content is a video, its time information is the distance between the video release time and the present time. It should be noted that the shorter the distance from the present time, the later the video was released, that is, the shorter the time between it and the content to be identified. Since the creation of content publishers has a certain continuity, the content to be identified and the content with a close release time are more likely to belong to the same type.

[0080] A3: Determine the category distribution of historical content based on the target probability.

[0081] The category distribution of historical content can be expressed as du, where du can be an array of floating-point numbers with the number of elements being the number of categories. du[c] represents the probability that the historical content is of category c, and the sum of the elements of du is 1. If the category of the vth historical content published by the content publisher is c, then:

[0082] du[c] = (the number of video plays of the vth historical content published by the content publisher * the video duration of the vth historical content * the average video play completion rate of the vth historical content) / ln(e + the time from the release time of the vth historical content to the current time). Among them, ln(e + the time from the release time of the vth historical content to the current time) is the time decay coefficient. As a possible implementation method, the time from the release time of the video to the current time can be counted in weeks, to correspond to the situation that most content publishers create content in weeks and hot spots ferment in weeks, thereby improving the accuracy of the category distribution of historical content.

[0083] After all historical contents have been calculated as above, du is normalized, that is, the weight of each type of video over all types of videos, for example, represented by the array du[c]=du[c] / sum_idc(du[idc]).

[0084] Therefore, the degree of influence of each type of video on the content to be identified can be determined through the playback information of historical content, and the probability that the content to be identified belongs to the same type as the content to be identified can be determined through the time information of historical content. Therefore, the target probability of the publisher publishing the target category determined by the playback information and time information of historical content can be used to determine the probability of the publisher publishing each category, such as the probability of the account publishing funny videos, the probability of the account publishing popular science videos, etc., and thus the category distribution of historical content can be obtained. Therefore, by determining the category distribution of historical content, the categories of content frequently published by the publisher can be determined, avoiding the influence of content related to hot events, etc., thereby improving the recognition accuracy of the category of the content to be identified.

[0085] In order to further improve the accuracy of determining the category of the content to be identified, it is also possible to combine other dimensions to jointly identify the category of the content to be identified based on the joint feature dimension of the content to be identified. The embodiment of this application does not specifically limit the content of other dimensions. The following uses the joint features of the candidate content as other dimensions (see Figure 4 ), the interactive information of the content to be identified is used as another dimension (see Figure 6 ), and both of the above as other dimensions (see Figure 7 ), and determining the category of the content to be identified is used as an example to illustrate each.

[0086] See also Figure 4 , which is a flowchart of determining the category of content to be identified provided by an embodiment of the present application.

[0087] S401: Acquire multiple candidate contents of the content to be identified and content supplementary information of the multiple candidate contents.

[0088] Among them, the similarity between the candidate content and the content to be identified is greater than the first similarity threshold, or the similarity between the candidate content and the content to be identified is less than the second similarity threshold, and the first similarity threshold is greater than the second similarity threshold, for example, the first similarity threshold is 80%, and the second similarity threshold is 20%, that is, the candidate content is relatively similar or less similar to the content to be identified. The embodiment of the present application does not specifically limit the size of the first similarity threshold and the second similarity threshold, and those skilled in the art can set them according to actual needs. The content supplementary information of the candidate content includes one or more of the object information corresponding to the publishing object of the candidate content and the interactive information for the candidate content.

[0089] The embodiments of the present application do not specifically limit the method for obtaining candidate content. For example, candidate content can be determined by constructing an inverted index. Specifically, a search field is constructed using content supplementary information of the content to be identified, and an inverted index is constructed for the content in the content library of the content platform through methods such as Elasticsearch. The constructed inverted index is retrieved through the search field, and then the content and the similarity recall score between the content and the content to be identified are returned. The similarity recall score is used to indicate the degree of similarity, thereby determining candidate content through the similarity recall score.

[0090] Elasticsearch is a distributed, free, and open-source search and analytics engine suitable for all types of data, including text, numeric, geospatial, structured, and unstructured data. A reverse index is a way to retrieve candidate content. For example, a forward index indexes content by its label, while a reverse index indexes content by its label.

[0091] For example, if the content to be identified is a video, if you want to identify whether it is a reverse video, you can build a search field for the content to be identified through the title, subtitles, or dialogue of the content to be identified, or the category distribution of historical content posted by the account that posted the content to be identified. Use Elasticsearch to build an inverted index for reverse videos and non-reversed videos in the content platform, use the aforementioned search field to match in the inverted index, return the similarity recall score of each video in the index to be sorted, and use videos with a similarity recall score greater than a first similarity threshold and videos with a similarity recall score less than a second similarity threshold as candidate content for the candidate content to be identified.

[0092] S402: Determine joint features of the candidate content based on the candidate content and the content supplementary information of the candidate content.

[0093] The embodiment of the present application does not specifically limit the specific method of determining the joint features of the candidate content, and those skilled in the art can set it according to actual needs. For example, based on the candidate content and the content supplementary information of the candidate content, the joint features of the candidate content can be determined by referring to the aforementioned S202, that is, the specific implementation method of determining the joint features of the content to be identified based on the content to be identified and the content supplementary information of the content to be identified. Figure 5 , the candidate content and the content to be identified are taken as videos as an example for explanation.

[0094] See also Figure 5 , this figure is a schematic diagram of determining the similarity probability between contents provided in an embodiment of the present application.

[0095] Figure 5 The left side shows the joint features of the content to be identified based on the content to be identified and the content supplementary information of the content to be identified. For details, please refer to Figure 3 As shown, the joint features of the content to be identified are determined, which will not be repeated here.

[0096] Figure 5 The right side shows the joint features of the candidate content determined based on the candidate content and its supplementary content information. Specifically, the image keyframe sequence from the candidate content is extracted and fed into the EfficientNet model to construct a representation of the candidate content's image frames. A self-attention model then performs a temporal fusion of multiple image frames from the candidate content to obtain image sub-features, i.e., the image representation of the candidate content. The audio frame sequence from the candidate content is extracted and fed into the EfficientNet model to construct a representation of the candidate content's audio frames. A self-attention model then performs a temporal fusion of multiple audio frames from the candidate content to obtain audio sub-features, i.e., the audio representation of the candidate content. Text from the candidate content, such as titles, subtitles, and dialogue, is extracted and concatenated through tokenization and fed into the BERT model to obtain text sub-features, i.e., the text representation of the candidate content. The first feature of the candidate content is determined based on the image, audio, and text sub-features. The second feature of the candidate content is determined using a fully connected network layer based on the category distribution of the historical content of the publisher who published the candidate content. A joint feature of the candidate content is determined according to the first feature and the second feature of the candidate content.

[0097] S403: Determine the probability that the content to be identified belongs to the target category based on the joint features of the content to be identified and the joint features of the candidate content.

[0098] The embodiments of the present application do not specifically limit the method of determining the probability that the content to be identified belongs to the target category. Two methods are used as examples below for illustration.

[0099] Method 1: determine the comprehensive features of the content to be identified based on the joint features of the content to be identified and the joint features of the candidate content, and then determine the probability that the content to be identified belongs to the target category based on the comprehensive features.

[0100] Method 2: Determine a first probability that the content to be identified belongs to the target category based on the joint features of the content to be identified; determine a second probability that the content to be identified belongs to the target category based on the joint features of the content to be identified and the joint features of the candidate content; and determine the probability that the content to be identified belongs to the target category based on the first probability and the second probability. For example, the probability that the content to be identified belongs to the target category can be determined based on the first probability and the weight corresponding to the first probability, and the second probability and the weight corresponding to the second probability.

[0101] The embodiments of the present application do not specifically limit the method for determining the second probability. For example, the similarity probability of the content to be identified and the candidate content is determined based on the joint features of the content to be identified and the joint features of the candidate content; the second probability that the content to be identified belongs to the target category is determined based on the similarity probability, the playback information of the candidate content, and whether the candidate content belongs to the target category.

[0102] For example, see Figure 4 , concatenate the joint features of the content to be identified and the joint features of the candidate content, input the concatenated joint features of the content to be identified and the joint features of the candidate content into the fully connected network layer, and obtain the similarity probability between the content to be identified and the candidate content. The second probability is determined according to the following formula:

[0103] The second probability = sum_s(similar probability between the content to be identified and the candidate content * weight coefficient of the candidate content * whether the candidate content is the target category), that is, the second probability is obtained by summing the similarity probability between the content to be identified and each candidate content * weight coefficient of the candidate content * whether the candidate content is the target category.

[0104] Among them, the weight coefficient of the candidate content is reflected by the playback information of the candidate content. If the playback information of the candidate content includes the duration of the candidate content, the number of times the candidate content is played and the average playback completion of the candidate content, then the weight coefficient of the candidate content = (duration of the candidate content * number of times the candidate content is played * average playback completion of the candidate content) / sum of the duration of all candidate contents * sum of the number of times all candidate contents are played * sum of the average playback completion of all candidate contents.

[0105] Whether the candidate content belongs to the target category. If the candidate content belongs to the target category, it is 1; if the candidate content does not belong to the target category, it is 0. For example, if the content to be identified is a reverse video, if the candidate content belongs to the reverse video, it is 1; if the candidate content does not belong to the reverse video, it is 0.

[0106] The weight coefficient of the candidate content can be determined through the playback information of the candidate content, that is, the influence of a candidate video among all candidate videos can be determined through the playback information of the candidate content. For example, a video with a large number of video plays, a long video duration, and a high average video playback completion rate may be a video that is popular with viewers. It may be content that the content publisher pays more attention to, and the content publisher can create similar content for this content. However, since there is also content that is dissimilar to the content to be identified in the candidate content, the impact of the dissimilar content on the content to be identified can be reduced by the dimension of whether the candidate content is the target category, even if the dissimilar content has a higher weight. Therefore, the second probability that the content to be identified belongs to the target category is comprehensively determined by the similarity probability, the playback information of the candidate content, and whether the candidate content belongs to the target category. Not only can the characteristics of the content to be identified be further strengthened through comparative learning, but the weight of the candidate content can also be adjusted through the playback information, thereby improving the recognition accuracy of the category of the content to be identified.

[0107] S404: Determine the category of the content to be identified based on the probability that the content to be identified belongs to the target category.

[0108] The target category is one of multiple categories. The embodiment of the present application does not specifically limit the method of determining the category of the content to be identified.

[0109] For example, if a model is trained to identify the category of content to be identified based on its joint features, each of the multiple categories can be used as a target category, and the probability that the content to be identified belongs to each category can be determined, so that the category with the larger probability value can be used as the category of the content to be identified.

[0110] For example, if a model is trained to identify whether content to be identified belongs to a certain category based on its joint features, if the probability that the content to be identified belongs to the target category exceeds 50%, then the category of the content to be identified is the target category. For example, if the probability that the content to be identified belongs to the reverse video category exceeds 50%, then the category of the content to be identified is reverse video.

[0111] Therefore, by including candidate content that is relatively similar to the content to be identified and relatively dissimilar content, the characteristics of the content to be identified are further strengthened through comparative learning, and it is used as a dimension, and combined with the dimension of the content to be identified and the content supplementary information dimension of the content to be identified, the accuracy of the category of the content to be identified determined by multi-dimensional information is higher.

[0112] The following description will be made by taking the interactive information of the content to be identified as another dimension to determine the category of the content to be identified as an example.

[0113] join Figure 6 , this figure is another flowchart for determining the category of content to be identified provided by an embodiment of the present application.

[0114] S601: Acquire interactive information for content to be identified.

[0115] The interactive information of the content to be identified can be obtained from the interaction data between the viewers who view the content to be identified and the content to be identified, such as barrage, comments, etc., which are used to represent the viewers' views on the content to be identified. For example, comments can be used to determine whether the content to be identified is a reverse video.

[0116] S602: Determine a third probability that the content to be identified belongs to the target category according to the interaction information.

[0117] The embodiment of the present application does not specifically limit the method for determining the third probability. Figure 6 , input text such as barrage or comments into the BERT model, and use the BERT model to obtain the third probability that the content to be identified is a reverse video.

[0118] It should be noted that the BERT model is trained on a dataset of bullet comments and comments that have been labeled as reverse videos. After calculating the probability of each bullet comment and other text representing a reverse video, the multiple bullet comments, comments and other texts of the content to be identified are weighted to calculate the third probability that the content to be identified belongs to the target category.

[0119] For example, the probability that the content to be identified is a reversal video = sum (number of interactions for each barrage, comment, etc.) / number of interactions for all interactions of the content to be identified * probability that the barrage, comment, etc. is a reversal video calculated by the BERT model.

[0120] S603: Determine a first probability that the content to be identified belongs to the target category according to the joint features of the content to be identified.

[0121] S604: Determine the probability that the content to be identified belongs to the target category based on the first probability and the third probability.

[0122] For example, the probability that the content to be identified belongs to the target category may be determined based on the first probability and the weight corresponding to the first probability, and the third probability and the weight corresponding to the third probability.

[0123] S605: Determine the category of the content to be identified based on the probability that the content to be identified belongs to the target category.

[0124] Therefore, by taking the interactive information as a dimension and combining it with the dimension of the content to be identified and the dimension of the content supplementary information of the content to be identified, the accuracy of the category of the content to be identified determined by the multi-dimensional information is higher.

[0125] The following takes the joint feature of the candidate content and the interaction information of the content to be identified as an example to illustrate the determination of the category of the content to be identified.

[0126] participate Figure 7 The figure is another flow chart for determining the category of the content to be identified according to an embodiment of the present application.

[0127] S701: Obtain a plurality of candidate contents of the content to be identified and content supplementary information of the plurality of candidate contents.

[0128] Wherein, the similarity between the candidate content and the content to be identified is greater than a first similarity threshold or the similarity between the candidate content and the content to be identified is less than a second similarity threshold.

[0129] S702: Determine the joint feature of the candidate content according to the candidate content and the content supplementary information of the candidate content.

[0130] S703: Obtain the interaction information of the content to be identified.

[0131] S704: Determine the third probability that the content to be identified belongs to the target category according to the interaction information.

[0132] It should be noted that the present embodiment does not specifically limit the execution order of S703-S704, that is, S703 is not limited to be executed only after S702, as long as S703-S704 are completed before S707.

[0133] S705: Determine the first probability that the content to be identified belongs to the target category according to the joint feature of the content to be identified.

[0134] S706: Determine the second probability that the content to be identified belongs to the target category according to the joint feature of the content to be identified and the joint feature of the candidate content.

[0135] S707: Determine the probability that the content to be identified belongs to the target category according to the first probability, the second probability and the third probability.

[0136] For example, the probability that the content to be identified belongs to the reverse category video = first weight * first probability + second weight * second probability + third weight * third probability.

[0137] The present embodiment does not specifically limit the size of the first weight, the second weight and the third weight, and as a possible implementation manner, the first weight, the second weight and the third weight are all floating point numbers between 0-1, and the sum is 1, for example, the first weight is 0.5, the second weight is 0.3, and the third weight is 0.2.

[0138] Therefore, by combining multiple dimensions such as the content dimension to be identified, the content supplementary information dimension of the content to be identified, the joint feature dimension of the candidate content and the interactive information dimension, the category of the content to be identified is determined, thereby improving the accuracy of the category of the content to be identified.

[0139] In response to the content identification method provided in the above embodiment, an embodiment of the present application further provides a content identification device.

[0140] See also Figure 8 , which is a schematic diagram of a content recognition device provided by an embodiment of the present application. As shown in FIG8 , the content recognition device 800 includes: an acquisition unit 801, a determination unit 802, and an identification unit 803;

[0141] The acquiring unit 801 is configured to acquire content to be identified and content supplementary information of the content to be identified, wherein the content supplementary information of the content to be identified includes at least one of object information corresponding to a publishing object that published the content to be identified and interaction information for the content to be identified;

[0142] The determining unit 802 is configured to determine a joint feature of the content to be identified based on the content to be identified and the content supplementary information of the content to be identified;

[0143] The identification unit 803 is configured to identify the category of the content to be identified based on the joint features of the content to be identified.

[0144] It can be seen from the above technical solution that the content to be identified and the content supplementary information of the content to be identified are obtained. The content supplementary information includes the object information corresponding to the publishing object of the content to be identified and / or the interactive information of the content to be identified, wherein the object information can reflect the characteristics of the publisher of the content to be identified, and the interactive information can reflect the views of the viewers of the content to be identified on the content to be identified, so that the content supplementary information can serve as a supplement to the content to be identified. Compared with the features determined only based on the content to be identified, the joint features of the content to be identified determined based on the content to be identified and the content supplementary information of the content to be identified are more comprehensive and accurate in describing the content to be identified, so that the category of the content to be identified identified based on the joint features is more accurate. Therefore, the accuracy of the joint features of the content to be identified determined by multi-dimensional information is higher, and the accuracy of the category of the content to be identified identified based on the more accurate joint features is higher.

[0145] As a possible implementation, the determining unit 802 includes a first feature determining unit, a second feature determining unit, and a joint feature determining unit;

[0146] The first feature determining unit is configured to determine a first feature according to the content to be identified;

[0147] The second feature determining unit is configured to determine a second feature based on the supplementary content information of the content to be identified;

[0148] The joint feature determination unit is configured to determine the joint feature of the content to be identified based on the first feature and the second feature.

[0149] As a possible implementation manner, if the content supplementary information of the content to be identified includes object information corresponding to a publishing object that published the content to be identified, the second feature determination unit is specifically configured to:

[0150] determining, based on the object information, a category distribution of historical content published by the publishing object, wherein the category distribution is used to represent the probability that the historical content belongs to a plurality of categories;

[0151] The second feature is determined according to the category distribution.

[0152] As a possible implementation, if the content to be identified is a video, the second feature determination unit is specifically configured to:

[0153] determining, according to the object information, the historical content published by the publishing object and a target category corresponding to the historical content, the target category being one of the multiple categories;

[0154] Determining a target probability of the publishing object publishing the target category according to the playback information of the historical content and the time information of the historical content;

[0155] The category distribution of the historical content is determined according to the target probability.

[0156] As a possible implementation manner, the playback information includes at least one of the video playback volume, video duration, and average video playback completion rate, and the time information includes the time from the video release time to the present.

[0157] As a possible implementation, if the content to be identified is a video, the first feature determination unit is specifically configured to:

[0158] According to the content to be identified, obtaining image sub-features of the content to be identified by using a first model;

[0159] According to the content to be identified, obtaining audio sub-features of the content to be identified by using a second model;

[0160] According to the content to be identified, obtaining text sub-features of the content to be identified through a third model;

[0161] The first feature is determined according to the image sub-feature, the audio sub-feature, and the text sub-feature.

[0162] As a possible implementation manner, the apparatus further includes a supplement determination unit, configured to:

[0163] Acquire multiple candidate contents of the content to be identified and multiple content supplementary information of the candidate contents, wherein the similarity between the candidate contents and the content to be identified is greater than a first similarity threshold or the similarity between the candidate contents and the content to be identified is less than a second similarity threshold;

[0164] determining a joint feature of the candidate content based on the candidate content and the content supplementary information of the candidate content;

[0165] The identification unit 803 is specifically configured to:

[0166] Determining, based on the joint features of the content to be identified and the joint features of the candidate content, a probability that the content to be identified belongs to a target category;

[0167] The category of the content to be identified is determined according to the probability that the content to be identified belongs to the target category.

[0168] As a possible implementation, the identification unit 803 is specifically configured to:

[0169] determining a first probability that the content to be identified belongs to the target category based on the joint features of the content to be identified;

[0170] determining a second probability that the content to be identified belongs to the target category based on the joint features of the content to be identified and the joint features of the candidate content;

[0171] The probability that the to-be-identified content belongs to the target category is determined according to the first probability and the second probability.

[0172] As a possible implementation, the identification unit 803 is specifically configured to:

[0173] determining, based on the joint features of the content to be identified and the joint features of the candidate content, a similarity probability between the content to be identified and the candidate content;

[0174] A second probability that the to-be-identified content belongs to the target category is determined based on the similarity probability, the playback information of the candidate content, and whether the candidate content belongs to the target category.

[0175] As a possible implementation, the apparatus further includes an interaction unit, configured to:

[0176] Acquiring interactive information for the content to be identified;

[0177] determine a third probability that the to-be-recognized content belongs to a target category according to the interaction information;

[0178] The identifying the category of the to-be-recognized content according to the joint feature of the to-be-recognized content comprises:

[0179] determine a first probability that the to-be-recognized content belongs to the target category according to the joint feature of the to-be-recognized content;

[0180] determine a probability that the to-be-recognized content belongs to the target category according to the first probability and the third probability.

[0181] As a possible implementation manner, the apparatus further comprises a supplementary determining unit and an interaction unit;

[0182] The supplementary determining unit is configured to:

[0183] obtain a plurality of candidate contents of the to-be-recognized content and content supplementary information of the plurality of candidate contents, the candidate contents having a similarity greater than a first similarity threshold with the to-be-recognized content or the candidate contents having a similarity less than a second similarity threshold with the to-be-recognized content;

[0184] determine a joint feature of the candidate contents according to the candidate contents and the content supplementary information of the candidate contents;

[0185] The interaction unit is configured to:

[0186] obtain interaction information for the to-be-recognized content;

[0187] determine a third probability that the to-be-recognized content belongs to a target category according to the interaction information;

[0188] The identifying unit 803 is specifically configured to:

[0189] determine a first probability that the to-be-recognized content belongs to the target category according to the joint feature of the to-be-recognized content;

[0190] determine a second probability that the to-be-recognized content belongs to the target category according to the joint feature of the to-be-recognized content and the joint feature of the candidate contents;

[0191] determine a probability that the to-be-recognized content belongs to the target category according to the first probability, the second probability and the third probability.

[0192] The present application also provides a computer device, which is the computer device described above. The computer device can be a server or a terminal device. The content recognition device described above can be built into the server or the terminal device. The computer device provided by the present application embodiment will be introduced from the perspective of hardware entity. Figure 9 The following is a schematic diagram of the server structure. Figure 10 Shown is a schematic diagram of the structure of the terminal equipment.

[0193] See also Figure 9 , this figure is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1400 may have relatively large differences due to different configurations or performances, and may include one or more processors 1422, such as central processing units (CPUs), a memory 1432, and one or more application programs 1442 or a storage medium 1430 for data 1444 (for example, one or more massive storage devices). Among them, the memory 1432 and the storage medium 1430 may be temporary storage or permanent storage. The program stored in the storage medium 1430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the processor 1422 may be configured to communicate with the storage medium 1430 to execute a series of instruction operations in the storage medium 1430 on the server 1400.

[0194] The server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1458, and / or one or more operating systems 1441, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0195] The steps performed by the server in the above embodiment can be based on the Figure 9 The server structure shown.

[0196] The CPU 1422 is configured to execute the following steps:

[0197] Acquire content to be identified and content supplementary information of the content to be identified, wherein the content supplementary information of the content to be identified includes at least one of object information corresponding to a publishing object that publishes the content to be identified and interaction information for the content to be identified;

[0198] According to the to-be-identified content and the content supplement information of the to-be-identified content, a joint feature of the to-be-identified content is determined;

[0199] According to the joint feature of the to-be-identified content, a category of the to-be-identified content is identified.

[0200] Optionally, the CPU 1422 can also execute the method steps of any specific implementation manner of the content identification method in the embodiments of the present application.

[0201] Referring to Figure 10 , the figure is a structural schematic diagram of a terminal device provided by an embodiment of the present application. Figure 10 shows a block diagram of part of the structure of a smart phone related to the terminal device provided by the embodiment of the present application, which includes a radio frequency (RF) circuit 1510, a memory 1520, an input unit 1530, a display unit 1540, a sensor 1550, an audio circuit 1560, a wireless fidelity (WiFi) module 1570, a processor 1580, and a power supply 1590, and the like. Those skilled in the art can understand that the structure of the smart phone shown in the figure does not constitute a limitation on the smart phone, and the smart phone can include more or fewer components than those shown in the figure, or combine certain components, or different arrangement of components. Figure 10 The structure of the smart phone shown in the figure does not constitute a limitation on the smart phone, and the smart phone can include more or fewer components than those shown in the figure, or combine certain components, or different arrangement of components.

[0202] The following will be described in combination with Figure 10 The various constituent components of the smart phone will be specifically introduced:

[0203] The RF circuit 1510 can be used for receiving and sending signals in the process of information or call, in particular, receiving the downlink information of the base station and processing by the processor 1580; in addition, sending the uplink data to the base station.

[0204] The memory 1520 can be used for storing software programs and modules, and the processor 1580 processes various function applications and data processing of the smart phone by running the software programs and modules stored in the memory 1520.

[0205] The input unit 1530 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the smartphone. Specifically, the input unit 1530 may include a touch panel 1531 and other input devices 1532. The touch panel 1531, also known as a touch screen, can collect user touch operations on or near it and drive the corresponding connection device according to a pre-set program. In addition to the touch panel 1531, the input unit 1530 may also include other input devices 1532. Specifically, other input devices 1532 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0206] The display unit 1540 can be used to display information input by the user or information provided to the user, as well as various menus of the smartphone. The display unit 1540 may include a display panel 1541. Optionally, the display panel 1541 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0207] The smartphone may also include at least one sensor 1550, such as a light sensor, a motion sensor, or other sensors. Other sensors that may be included in the smartphone, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., are not described in detail here.

[0208] Audio circuit 1560, speaker 1561, and microphone 1562 provide an audio interface between the user and the smartphone. Audio circuit 1560 converts received audio data into electrical signals and transmits them to speaker 1561, where they are converted into sound signals for output. Microphone 1562, on the other hand, converts collected sound signals into electrical signals, which are then received by audio circuit 1560 and converted into audio data. The audio data is then processed by processor 1580 and transmitted to, for example, another smartphone via RF circuit 1510, or stored in memory 1520 for further processing.

[0209] Processor 1580 is the control center of the smartphone. It connects all parts of the smartphone using various interfaces and lines. It executes or runs software programs and / or modules stored in memory 1520 and accesses data stored in memory 1520 to perform various functions of the smartphone and process data. Optionally, processor 1580 may include one or more processing units.

[0210] The smartphone also includes a power supply 1590 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1580 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0211] Although not shown, the smartphone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0212] In the embodiment of the present application, the memory 1520 included in the smart phone can store program codes and transmit the program codes to the processor.

[0213] The processor 1580 included in the smart phone can execute the content identification method provided in the above embodiment according to the instructions in the program code.

[0214] An embodiment of the present application further provides a computer-readable storage medium for storing a computer program, wherein the computer program is used to execute the content identification method provided in the above embodiment.

[0215] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the content identification method provided in various optional implementations of the above aspects.

[0216] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the above-mentioned storage medium can be at least one of the following media: read-only memory (English: Read-Only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0217] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0218] The above is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A content identification method, characterized in that: The method comprises: Obtaining content to be identified and supplementary content information of the content to be identified, wherein the content to be identified includes one or more of a video, a picture, and an article, and the supplementary content information of the content to be identified includes at least one of object information corresponding to a publishing object that published the content to be identified and interactive information for the content to be identified; determining a joint feature of the content to be identified based on the content to be identified and the content supplementary information of the content to be identified; Identifying the category of the content to be identified based on the joint features of the content to be identified; The determining, based on the content to be identified and the content supplementary information of the content to be identified, a joint feature of the content to be identified includes: determining a first feature according to the content to be identified; determining a second feature based on the supplementary information of the content to be identified; determining a joint feature of the content to be identified based on the first feature and the second feature; If the content supplementary information of the content to be identified includes object information corresponding to a publishing object that published the content to be identified, determining the second feature based on the content supplementary information of the content to be identified includes: determining, based on the object information, a category distribution of historical content published by the publishing object, wherein the category distribution is used to represent the probability that the historical content belongs to a plurality of categories; The second feature is determined according to the category distribution.

2. The method according to claim 1, characterized in that If the to-be-identified content is a video, determining the category distribution of the historical content published by the publishing object according to the object information includes: determining, according to the object information, the historical content published by the publishing object and a target category corresponding to the historical content, the target category being one of the multiple categories; Determining a target probability of the publishing object publishing the target category according to the playback information of the historical content and the time information of the historical content; The category distribution of the historical content is determined according to the target probability.

3. The method according to claim 2, characterized in that The playback information includes at least one of the video playback volume, video duration, and average video playback completion rate, and the time information includes the time from the video release time to the present.

4. The method according to claim 1, wherein If the content to be identified is a video, determining the first feature according to the content to be identified includes: According to the content to be identified, obtaining image sub-features of the content to be identified by using a first model; According to the content to be identified, obtaining audio sub-features of the content to be identified by using a second model; According to the content to be identified, obtaining text sub-features of the content to be identified through a third model; The first feature is determined according to the image sub-feature, the audio sub-feature, and the text sub-feature.

5. The method according to claim 1, characterized in that The method further comprises: Acquire multiple candidate contents of the content to be identified and content supplementary information of the multiple candidate contents, wherein the similarity between the candidate contents and the content to be identified is greater than a first similarity threshold or the similarity between the candidate contents and the content to be identified is less than a second similarity threshold, and the first similarity threshold is greater than the second similarity threshold; determining a joint feature of the candidate content based on the candidate content and the content supplementary information of the candidate content; The identifying the category of the content to be identified according to the joint features of the content to be identified includes: Determining, based on the joint features of the content to be identified and the joint features of the candidate content, a probability that the content to be identified belongs to a target category; The category of the content to be identified is determined according to the probability that the content to be identified belongs to the target category.

6. The method according to claim 5, characterized in that The determining, based on the joint features of the content to be identified and the joint features of the candidate content, the probability that the content to be identified belongs to the target category includes: determining a first probability that the content to be identified belongs to the target category based on the joint features of the content to be identified; determining a second probability that the content to be identified belongs to the target category based on the joint features of the content to be identified and the joint features of the candidate content; The probability that the to-be-identified content belongs to the target category is determined according to the first probability and the second probability.

7. The method according to claim 6, characterized in that The determining, based on the joint features of the content to be identified and the joint features of the candidate content, a second probability that the content to be identified belongs to the target category includes: determining, based on the joint features of the content to be identified and the joint features of the candidate content, a similarity probability between the content to be identified and the candidate content; A second probability that the to-be-identified content belongs to the target category is determined based on the similarity probability, the playback information of the candidate content, and whether the candidate content belongs to the target category.

8. The method according to claim 1, characterized in that The method further comprises: Acquiring interactive information for the content to be identified; determining a third probability that the content to be identified belongs to a target category according to the interaction information; The identifying the category of the content to be identified according to the joint features of the content to be identified includes: determining a first probability that the content to be identified belongs to the target category based on the joint features of the content to be identified; The probability that the to-be-identified content belongs to the target category is determined according to the first probability and the third probability.

9. The method according to claim 1, characterized in that The method further comprises: Acquire multiple candidate contents of the content to be identified and multiple content supplementary information of the candidate contents, wherein the similarity between the candidate contents and the content to be identified is greater than a first similarity threshold or the similarity between the candidate contents and the content to be identified is less than a second similarity threshold; determining a joint feature of the candidate content based on the candidate content and the content supplementary information of the candidate content; Acquiring interactive information for the content to be identified; determining a third probability that the content to be identified belongs to a target category according to the interaction information; The identifying the category of the content to be identified according to the joint features of the content to be identified includes: Determining, based on the joint features of the content to be identified and the joint features of the candidate content, a probability that the content to be identified belongs to a target category; determining a first probability that the content to be identified belongs to the target category based on the joint features of the content to be identified; determining a second probability that the content to be identified belongs to the target category based on the joint features of the content to be identified and the joint features of the candidate content; The probability that the to-be-identified content belongs to the target category is determined according to the first probability, the second probability, and the third probability.

10. A content recognition device, characterized in that: The device includes: an acquisition unit, a determination unit and an identification unit; The acquisition unit is configured to acquire content to be identified and content supplementary information of the content to be identified, wherein the content to be identified includes one or more of a video, a picture, and an article, and the content supplementary information of the content to be identified includes at least one of object information corresponding to a publishing object that published the content to be identified and interactive information for the content to be identified; The determining unit is configured to determine a joint feature of the content to be identified based on the content to be identified and the content supplementary information of the content to be identified; The identification unit is configured to identify the category of the content to be identified based on the joint features of the content to be identified; The determining unit includes a first feature determining unit, a second feature determining unit and a joint feature determining unit; The first feature determining unit is configured to determine a first feature according to the content to be identified; The second feature determining unit is configured to determine a second feature based on the supplementary content information of the content to be identified; The joint feature determination unit is configured to determine the joint feature of the content to be identified based on the first feature and the second feature; If the content supplementary information of the content to be identified includes object information corresponding to a publishing object that published the content to be identified, the second feature determination unit is specifically configured to: determining, based on the object information, a category distribution of historical content published by the publishing object, wherein the category distribution is used to represent the probability that the historical content belongs to a plurality of categories; The second feature is determined according to the category distribution.

11. The device according to claim 10, characterized in that If the content to be identified is a video, the second feature determination unit is specifically configured to: determining, according to the object information, the historical content published by the publishing object and a target category corresponding to the historical content, the target category being one of the multiple categories; Determining a target probability of the publishing object publishing the target category according to the playback information of the historical content and the time information of the historical content; The category distribution of the historical content is determined according to the target probability.

12. The device according to claim 11, characterized in that The playback information includes at least one of the video playback volume, video duration, and average video playback completion rate, and the time information includes the time from the video release time to the present.

13. The device according to claim 10, characterized in that If the content to be identified is a video, the first feature determination unit is specifically configured to: According to the content to be identified, obtaining image sub-features of the content to be identified by using a first model; According to the content to be identified, obtaining audio sub-features of the content to be identified by using a second model; According to the content to be identified, obtaining text sub-features of the content to be identified through a third model; The first feature is determined according to the image sub-feature, the audio sub-feature, and the text sub-feature.

14. The device according to claim 10, characterized in that The apparatus further includes a supplement determination unit, configured to: Acquire multiple candidate contents of the content to be identified and multiple content supplementary information of the candidate contents, wherein the similarity between the candidate contents and the content to be identified is greater than a first similarity threshold or the similarity between the candidate contents and the content to be identified is less than a second similarity threshold; determining a joint feature of the candidate content based on the candidate content and the content supplementary information of the candidate content; The identification unit is specifically used to: Determining, based on the joint features of the content to be identified and the joint features of the candidate content, a probability that the content to be identified belongs to a target category; The category of the content to be identified is determined according to the probability that the content to be identified belongs to the target category.

15. The device according to claim 14, characterized in that The identification unit is specifically used to: determining a first probability that the content to be identified belongs to the target category based on the joint features of the content to be identified; determining a second probability that the content to be identified belongs to the target category based on the joint features of the content to be identified and the joint features of the candidate content; The probability that the to-be-identified content belongs to the target category is determined according to the first probability and the second probability.

16. The device according to claim 15, characterized in that The identification unit is specifically used to: determining, based on the joint features of the content to be identified and the joint features of the candidate content, a similarity probability between the content to be identified and the candidate content; A second probability that the to-be-identified content belongs to the target category is determined based on the similarity probability, the playback information of the candidate content, and whether the candidate content belongs to the target category.

17. The device according to claim 10, characterized in that The device further comprises an interaction unit, configured to: Acquiring interactive information for the content to be identified; determining a third probability that the content to be identified belongs to a target category according to the interaction information; The identifying the category of the content to be identified according to the joint features of the content to be identified includes: determining a first probability that the content to be identified belongs to the target category based on the joint features of the content to be identified; The probability that the to-be-identified content belongs to the target category is determined according to the first probability and the third probability.

18. The device according to claim 10, characterized in that The apparatus further comprises a supplement determination unit and an interaction unit; The supplement determination unit is configured to: Acquire multiple candidate contents of the content to be identified and multiple content supplementary information of the candidate contents, wherein the similarity between the candidate contents and the content to be identified is greater than a first similarity threshold or the similarity between the candidate contents and the content to be identified is less than a second similarity threshold; determining a joint feature of the candidate content based on the candidate content and the content supplementary information of the candidate content; The interaction unit is configured to: Acquiring interactive information for the content to be identified; determining a third probability that the content to be identified belongs to a target category according to the interaction information; The identification unit is specifically used to: determining a first probability that the content to be identified belongs to the target category based on the joint features of the content to be identified; determining a second probability that the content to be identified belongs to the target category based on the joint features of the content to be identified and the joint features of the candidate content; The probability that the to-be-identified content belongs to the target category is determined according to the first probability, the second probability, and the third probability.

19. A computer device, characterized in that: The computer device includes a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the method according to any one of claims 1 to 9 according to instructions in the computer program.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, characterized in that When the method is executed on a computer device, the computer device is enabled to execute the method according to any one of claims 1 to 9.

Images