A content recognition method, related apparatus, device, and storage medium

Abstract

This application discloses a content recognition method, which can be used as a supporting technology for WeChat Official Accounts. The method includes obtaining a first feature vector corresponding to the content to be recognized; recalling M content samples that meet similarity conditions from the Q content samples based on the first feature vector and the second feature vector of each of the Q content samples; statistically analyzing the content samples for each category identifier based on the category identifier of each of the M content samples to obtain the number of content samples corresponding to each category identifier; and obtaining the target category label corresponding to the content to be recognized based on the number of content samples corresponding to each category identifier. This application also provides related devices, equipment, and storage media. This method can directly cluster the obtained content to be recognized, thereby better adapting to streaming content scenarios and improving the real-time performance of content recognition.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, related apparatus, device, and storage medium for content recognition. Background Technology

[0002] In the daily operation of WeChat official accounts, due to the relatively high barrier to entry for content creation, some entities engage in unauthorized reposting, plagiarism, machine-generated content, and copy-pasting to mass-distribute text and image content in a matrix-like manner. They acquire a large amount of traffic in a low-cost and inexpensive way, and then monetize it through advertising and traffic redirection to reap substantial profits.

[0003] Matrix-style mass messaging can cause significant damage to the content ecosystem of WeChat official accounts. Currently, clustering algorithms are commonly used to identify content that falls into this category. Specifically, when a certain amount of content accumulates, it is clustered, and the clustering results are used to determine whether the content belongs to matrix-style mass messaging.

[0004] The inventors discovered that existing solutions have at least the following problems: While the methods described above can save on manual review costs to some extent, they require waiting for a certain amount of content to accumulate before clustering can be performed. Furthermore, they also need to consider merging or re-clustering the results of two consecutive clustering sessions. Therefore, the real-time performance of content recognition is poor. Summary of the Invention

[0005] This application provides a method, related apparatus, device, and storage medium for content recognition. This application can directly cluster the acquired content to be recognized, thereby better adapting to streaming content scenarios and improving the real-time performance of content recognition.

[0006] In view of this, this application provides a content recognition method, comprising:

[0007] Obtain the first feature vector corresponding to the content to be identified;

[0008] Based on the first feature vector and the second feature vector of each of the Q content samples, M content samples that meet the similarity condition are recalled from the Q content samples. Each content sample corresponds to a category identifier. M and Q are both integers greater than or equal to 1, and M is less than or equal to Q.

[0009] Based on the category identifier of each content sample in the M content samples, the number of content samples corresponding to each category identifier is obtained by counting the content samples for each category identifier;

[0010] Based on the number of content samples corresponding to each category identifier, obtain the target category label corresponding to the content to be identified.

[0011] This application also provides a content recognition device, comprising:

[0012] The acquisition module is used to acquire the first feature vector corresponding to the content to be identified;

[0013] The acquisition module is also used to recall M content samples that meet the similarity condition from the Q content samples based on the first feature vector and the second feature vector of each content sample in the Q content samples, wherein each content sample corresponds to a category identifier, M and Q are both integers greater than or equal to 1, and M is less than or equal to Q.

[0014] The determination module is used to count the number of content samples corresponding to each category identifier based on the category identifier of each content sample in the M content samples;

[0015] The acquisition module is also used to obtain the target category label corresponding to the content to be identified based on the number of content samples corresponding to each category identifier.

[0016] In one possible design, in another implementation of another aspect of the embodiments of this application, the content recognition device further includes a processing module;

[0017] The processing module is also used to perform pairwise similarity calculations on the content obtained within a preset time period before obtaining the first feature vector corresponding to the content to be identified;

[0018] The determination module is also used to treat content with a similarity greater than or equal to a similarity threshold as the same clustering result;

[0019] The acquisition module is also used to acquire the content to be identified based on the same clustering result.

[0020] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0021] The processing module is also used to perform pairwise similarity calculations on the content obtained within the preset time period if the number of contents obtained within the preset time period reaches the number threshold before obtaining the first feature vector corresponding to the content to be identified. The number threshold is an integer greater than 1.

[0022] The determination module is also used to treat content with a similarity greater than or equal to a similarity threshold as the same clustering result;

[0023] The acquisition module is also used to acquire the content to be identified based on the same clustering result.

[0024] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0025] The processing module is also used to generate a new category identifier for the content to be identified if there is no content sample after obtaining the first feature vector corresponding to the content to be identified.

[0026] The processing module is further configured to store the identifier of the content to be identified and the first feature vector into the first database according to the mapping relationship between the first feature vector and the content to be identified, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified.

[0027] The processing module is also used to store the identifier of the content to be identified and the new category identifier in the second database according to the mapping relationship between the new category identifier and the content to be identified.

[0028] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0029] The acquisition module is specifically used to retrieve the top T content samples with the highest similarity from the Q content samples by calling the first database based on the first feature vector and the second feature vector of each content sample in the Q content samples, where T is an integer greater than or equal to 1;

[0030] From the first T content samples, obtain M content samples that meet the similarity criteria.

[0031] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0032] The acquisition module is also used to obtain a recall number setting request before recalling the top T content samples with the highest similarity from the Q content samples by calling the first database based on the first feature vector and the second feature vector of each content sample in the Q content samples, wherein the recall number setting request carries the value of T;

[0033] The acquisition module is specifically used to execute the step of retrieving the top T content samples with the highest similarity from the Q content samples by calling the first database if the value of T is less than or equal to the value of Q.

[0034] The acquisition module is also used to recall Q content samples by calling the first database if the value of T is greater than the value of Q.

[0035] The acquisition module is also used to acquire M content samples that meet the similarity criteria from Q content samples.

[0036] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0037] The acquisition module is specifically used to determine the similarity between the content to be identified and the content sample for each of the first T content samples, based on the first feature vector and the second feature vector of the content sample.

[0038] If there is at least one content sample among the first T content samples with a similarity greater than or equal to the similarity threshold, then at least one content sample is taken as M content samples.

[0039] or,

[0040] The acquisition module is specifically used to determine the similarity distance between the content to be identified and the content sample for each of the first T content samples, based on the first feature vector and the second feature vector of the content sample.

[0041] If there is at least one content sample among the first T content samples whose similarity distance is less than or equal to the similarity distance threshold, then at least one content sample is taken as M content samples.

[0042] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0043] The processing module is also used to generate a new category identifier for the content to be identified if there are no content samples that meet the similarity conditions after obtaining the first feature vector corresponding to the content to be identified.

[0044] The processing module is further configured to store the identifier of the content to be identified and the first feature vector into the first database according to the mapping relationship between the first feature vector and the content to be identified, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified.

[0045] The processing module is also used to store the identifier of the content to be identified and the new category identifier in the second database according to the mapping relationship between the new category identifier and the content to be identified.

[0046] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0047] The determination module is specifically used to associate content samples with the same category identifier with the same category identifier based on the category identifier of each content sample in M ​​content samples, so as to obtain the number of content samples corresponding to each category identifier;

[0048] The acquisition module is specifically used to determine the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier.

[0049] Based on the target category identifier, determine the target category label corresponding to the content to be identified.

[0050] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0051] The processing module is also used to determine the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier, and then store the identifier of the content to be identified and the first feature vector in the first database according to the mapping relationship between the first feature vector and the content to be identified. The identifier of the content to be identified is used to uniquely identify the content to be identified.

[0052] The processing module is also used to store the identifier of the content to be identified and the target category identifier in the second database according to the mapping relationship between the target category identifier and the content to be identified.

[0053] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0054] The acquisition module is specifically used to determine the number of content samples associated with each category identifier in at least one category identifier;

[0055] The category identifier corresponding to the maximum number of content samples is used as the target category identifier for the content to be identified.

[0056] Based on the target category identifier, determine the target category label corresponding to the content to be identified.

[0057] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0058] The acquisition module is specifically used to determine the number of content samples associated with each category identifier in at least one category identifier;

[0059] If at least two category identifiers are associated with the same number of content samples, then the average ranking corresponding to each category identifier is calculated based on the content samples associated with each of the at least two category identifiers and the recall similarity ranking of the content samples.

[0060] Use the category identifier corresponding to the highest average ranking as the target category identifier for the content to be identified.

[0061] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0062] The acquisition module is specifically used to determine the number of content samples associated with each category identifier in at least one category identifier;

[0063] If at least two category identifiers are associated with the same number of content samples, then calculate the average feature vector corresponding to each category identifier based on the content samples associated with each of the at least two category identifiers.

[0064] The target category identifier for the content to be identified is determined based on the similarity between the average feature vector corresponding to each category identifier and the first feature vector.

[0065] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0066] The acquisition module is specifically used to respond to the labeling instructions for the target category identifier and determine the target category label corresponding to the content to be identified;

[0067] The acquisition module is also used to take the content samples associated with the target category identifier and the content to be identified as the training sample set, wherein the training sample set is labeled with the target category label;

[0068] The acquisition module is also used to acquire the content features and account information of each training sample in the training sample set;

[0069] The acquisition module is also used to acquire clustering similarity features of the set of samples to be trained;

[0070] The acquisition module is also used to obtain the probability distribution of the first category through a category recognition model based on the clustering similarity features of the set of samples to be trained, the content features of each sample to be trained, and the account information.

[0071] The processing module is also used to update the model parameters of the category recognition model based on the target category label and the probability distribution of the first category.

[0072] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0073] The acquisition module is specifically used to take the content samples associated with the target category identifier and the content to be identified as a set of prediction samples;

[0074] Obtain the content features and account information of each prediction sample in the prediction sample set;

[0075] Obtain the cluster similarity features of the predicted sample set;

[0076] Based on the clustering similarity features of the predicted sample set, the content features of each predicted sample, and account information, the probability distribution of the second category is obtained through a category recognition model.

[0077] The target category label corresponding to the content to be identified is determined based on the probability distribution of the second category.

[0078] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0079] The processing module is also used to extract features from the content to be identified if the content to be identified is text content, and obtain a first feature vector.

[0080] or,

[0081] The processing module is also used to encode the content to be identified by an encoder to obtain an image feature vector if the content to be identified is an image.

[0082] The processing module is also used to perform a nonlinear transformation on the image feature vector through the projection head to obtain the first feature vector.

[0083] This application also provides a computer device, including: a memory, a processor, and a bus system;

[0084] The memory is used to store programs;

[0085] The processor is used to execute programs in memory, and the processor is used to execute the methods mentioned above according to the instructions in the program code;

[0086] Bus systems are used to connect memory and processor to enable communication between them.

[0087] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0088] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the above aspects.

[0089] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0090] This application provides a content recognition method. First, a first feature vector corresponding to the content to be recognized is obtained. Then, based on the first feature vector and the second feature vector of each of the Q content samples, M content samples that meet the similarity condition are recalled from the Q content samples. Based on this, the number of content samples corresponding to each category identifier is statistically analyzed according to the category identifier of each of the M content samples. Finally, the target category label of the content to be recognized is obtained by combining at least one category identifier. Through this method, when new content to be recognized is obtained, several content samples with a high degree of similarity to the content to be recognized are first recalled. Then, based on the known clustering categories of these content samples, the clustering category to which the content to be recognized can be classified is determined, thereby obtaining the target category label. It is evident that for the obtained content to be recognized, clustering can be performed using statistical category identifiers, which can better adapt to streaming content scenarios and improve the real-time performance of content recognition. Attached Figure Description

[0091] Figure 1 This is a schematic diagram of the physical architecture of the content recognition system in the embodiments of this application;

[0092] Figure 2 This is a schematic diagram of the logical architecture of the content recognition system in the embodiments of this application;

[0093] Figure 3 This is a flowchart illustrating the content recognition method in an embodiment of this application;

[0094] Figure 4 This is a schematic diagram of a settings interface of the content recognition system in an embodiment of this application;

[0095] Figure 5 This is a schematic diagram of an annotation interface of the content recognition system in this application embodiment;

[0096] Figure 6 This is a schematic diagram of the structure of the category recognition model in an embodiment of this application;

[0097] Figure 7 This is a schematic diagram of a result display interface of the content recognition system in an embodiment of this application;

[0098] Figure 8 This is a schematic diagram illustrating text content-based recognition in an embodiment of this application;

[0099] Figure 9 This is a schematic diagram illustrating image content-based recognition in an embodiment of this application;

[0100] Figure 10 This is a schematic diagram of a content recognition device in an embodiment of this application;

[0101] Figure 11 This is a schematic diagram of the structure of a terminal device in an embodiment of this application;

[0102] Figure 12 This is a schematic diagram of the server structure in an embodiment of this application. Detailed Implementation

[0103] This application provides a method, related apparatus, device, and storage medium for content recognition. This application can directly cluster the acquired content to be recognized, thereby better adapting to streaming content scenarios and improving the real-time performance of content recognition.

[0104] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0105] With the development of internet technology, the amount of data available for users to browse and search is constantly increasing. To better analyze and utilize this data, clustering algorithms can be used to compare data similarity and group data with high similarity into a single category. Clustering is an unsupervised learning method that aims to divide a dataset into multiple sub-data sets according to certain rules; each sub-data set is called a cluster. Clustering analysis has wide applications in many fields, including text analysis, image analysis, business applications, web search, recommendation systems, and biomedicine.

[0106] To enable better real-time data processing, this application provides a streaming clustering method applicable to the following scenarios. It should be noted that practical applications are not limited to these.

[0107] I. Detect matrix-style content;

[0108] Matrix content refers to entities (e.g., studios, individual businesses, or individuals) registering a large number of public accounts and then publishing content that requires minimal creation cost. This content is typically similar, template-based, plagiarized, or copied and pasted. By leveraging a large number of accounts with extremely low content creation costs, they conduct massive mass distribution. Articles from a group of accounts are highly similar and contain little usable information.

[0109] II. Deduplicate articles already read by users;

[0110] For trending topics, there may be a large amount of related reports and news content, and users may read many articles with similar content. Therefore, the recommendation system uses the content recognition method provided in this application to identify articles that users have already read, thereby pushing more articles with less content duplication to users.

[0111] The following section will introduce the content recognition system from the perspective of physical architecture.

[0112] The content recognition method proposed in this application can be applied to... Figure 1 Please refer to the content recognition system shown. Figure 1 , Figure 1 This is a schematic diagram of the physical architecture of the content recognition system in this application embodiment. As shown in the figure, the friend recommendation system for interactive information includes a server and terminal devices, with the client deployed on the terminal devices. The client can run on the terminal devices via a browser or as a standalone application (APP). The specific form of the client is not limited here. The server involved in this application can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal devices can be smartphones, tablets, laptops, PDAs, personal computers, smart TVs, smartwatches, in-vehicle devices, wearable devices, etc., but are not limited to these. The terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not limited here. The number of servers and terminal devices is also not limited. The solution provided in this application can be completed independently by the terminal device, independently by the server, or jointly by the terminal device and the server. This application does not make any specific limitations on this.

[0113] by Figure 1Taking the content recognition system shown as an example, authors can publish content (e.g., articles or images) through terminal devices. The server, based on the content acquired in real time, clusters the acquired content by calling the feature vectors of existing content in the database, and feeds the clustering results back to the backend. Backend personnel can review whether the content belongs to similar categories based on the clustering results, or automatically analyze the clustering results through a neural network to obtain feedback on whether the content belongs to similar categories. Similar content can then no longer be pushed to users.

[0114] The following section will introduce the content recognition system from the perspective of its logical architecture.

[0115] The content recognition method proposed in this application can be applied to... Figure 2 Please refer to the content recognition system shown. Figure 2 , Figure 2 This is a schematic diagram of the logical architecture of the content recognition system in this application embodiment. As shown in the figure, the content recognition system is mainly divided into a data layer, a clustering layer, and an indexing layer, specifically:

[0116] For example, for the data layer, in step S11, X new samples (e.g., articles) are obtained for mass distribution, where X is an integer greater than or equal to 1. In step S12, the feature vector of each new sample is calculated. Based on this, if the value of X is large (e.g., greater than 5000), then in step S13, these X new samples are pre-clustered. Thus, in step S14, Y cluster centers are obtained after clustering, typically Y being less than X.

[0117] For example, for the clustering layer, Y cluster centers are used as Y samples. For ease of explanation, the following description will use any one of the Y samples as an example. In step S21, several (e.g., 1k) sample samples that are most similar to the sample are retrieved from the first database. In step S22, the category records of several (e.g., 1k) sample samples are obtained by querying the second database. In step S23, the number of categories of T sample samples is counted by voting, and the category with the most categories is taken as the category of the sample.

[0118] For example, for the clustering layer, in step B31, the identifier of the sample and its corresponding feature vector are written to the first database, and in step B32, the identifier of the sample and its corresponding category identifier are written to the second database.

[0119] Based on the above introduction, the content recognition method in this application will be described below. Please refer to [link / reference]. Figure 3 One embodiment of the content recognition method in this application includes:

[0120] 110. Obtain the first feature vector corresponding to the content to be identified;

[0121] In one or more embodiments, the content recognition device acquires the content to be recognized and performs feature extraction processing on the content to be recognized to obtain a first feature vector.

[0122] It should be noted that the content recognition device can be deployed on a server, on a terminal device, or in a system consisting of a server and a terminal device; this application does not impose any limitations.

[0123] 120. Based on the first feature vector and the second feature vector of each of the Q content samples, recall M content samples that meet the similarity condition from the Q content samples, where each content sample corresponds to a category identifier, M and Q are both integers greater than or equal to 1, and M is less than or equal to Q.

[0124] In one or more embodiments, the content recognition device, based on a first feature vector, first recalls T content samples from a first database, and then selects M content samples from the T content samples that meet the similarity condition. The first database is a database for similarity vector retrieval and is capable of providing fuzzy matching functionality.

[0125] Specifically, the first database stores Q groups of content sample information. Each group of content sample information includes the identity document (ID) of the content sample and its corresponding second feature vector. For example, the ID of the content sample is "0", and its corresponding second feature vector is "[101,108,3523,656,…,7984]".

[0126] It should be noted that the first database includes, but is not limited to, the Facebook artificial intelligence similarity search (FAISS) database, and the hierarchical navigable small world (HNSW) network.

[0127] 130. Based on the category identifier of each content sample in M ​​content samples, count the content samples for each category identifier to obtain the number of content samples corresponding to each category identifier;

[0128] In one or more embodiments, the content recognition device retrieves the category identifier of each content sample from a second database based on the ID of each content sample. This allows for the counting of content samples under each category identifier, essentially performing a vote on each category identifier. The number of content samples corresponding to each category identifier is then obtained. The second database is either a relational database or a key-value database.

[0129] Specifically, the second database stores the IDs of content samples and their corresponding category identifiers. For example, the category identifier for a content sample with ID "0" is "A".

[0130] It should be noted that the second database includes, but is not limited to, relational database management systems (MySQL) and Structured Query Language Server (SQL Server) databases.

[0131] 140. Based on the number of content samples corresponding to each category identifier, obtain the target category label corresponding to the content to be identified.

[0132] In one or more embodiments, the content recognition device can select the category identifier with the largest number of content samples as the target category identifier based on the number of content samples corresponding to each category identifier. Based on this, the content to be identified and the content samples belonging to the same target category identifier are grouped into a cluster result, and this cluster result is labeled to obtain the target category label of the cluster result. Thus, the target category label of the content to be identified can be determined.

[0133] For example, the target category label can be represented as a label, such as "1" or "0", where a target category label of "1" indicates that the content belongs to a category with high similarity (e.g., matrix content), and a target category label of "0" indicates that the content belongs to a category with low similarity (e.g., not matrix content).

[0134] For example, a target category label can be represented as a probability value ranging from 0 to 1. The higher the probability value, the greater the similarity of the content; conversely, the lower the probability value, the less similar the content.

[0135] This application provides a content recognition method. In this method, when new content to be identified is obtained, several content samples with a high degree of similarity to the content to be identified are first recalled. Then, based on the known clustering categories of these content samples, the clustering category to which the content to be identified can be assigned is determined, thereby obtaining the target category label. It is evident that for the obtained content to be identified, clustering can be performed using statistical category labels, which better adapts to streaming content scenarios and improves the real-time performance of content recognition.

[0136] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided in this application, before obtaining the first feature vector corresponding to the content to be identified, the following may be included:

[0137] Perform pairwise similarity calculations on the content acquired within a preset time period;

[0138] Content with a similarity greater than or equal to the similarity threshold will be treated as the same cluster result;

[0139] The content to be identified is obtained based on the same clustering result.

[0140] In one or more embodiments, a time-based pre-clustering method is described. As can be seen from the foregoing embodiments, streaming clustering may encounter high concurrency issues. To address this, this application sets a preset time period, allowing the content acquired within that preset time period to be clustered first, i.e., pre-clustering. This yields Y clustering results.

[0141] Specifically, assuming a preset duration of 10 seconds, X new pieces of content are received within this preset duration. For each piece of content, a feature vector is calculated. For example, in one case, the pairwise similarity (e.g., cosine similarity) can be calculated, resulting in [X(X-1) / 2] similarities. Content with similarities greater than or equal to a similarity threshold is grouped into the same cluster, thus yielding Y cluster results. For example, in another case, the pairwise similarity distance (e.g., cosine distance) can be calculated, resulting in [X(X-1) / 2] similarity distances. Content with similarity distances less than or equal to a similarity distance threshold is grouped into the same cluster, thus yielding Y cluster results.

[0142] Based on this, for each of the Y clustering results, its corresponding cluster center is determined, and this cluster center is used as the content to be identified for that clustering result. Therefore, Y clustering results yield Y cluster centers, and each cluster center belongs to one content to be identified.

[0143] It is understandable that a cluster center is a component within the clustering result. The first component in the clustering result can be used as the cluster center, or a component can be randomly selected from the clustering result; there are no restrictions here.

[0144] It should be noted that if a cluster center belongs to the target category identifier, then other content in the clustering results to which that cluster center belongs will also be marked with the target category identifier.

[0145] Secondly, this application provides a time-based pre-clustering method. This method allows for the pre-clustering of content acquired over a period of time, resulting in multiple cluster centers. Each cluster center is treated as a new piece of content (i.e., content to be identified), thereby reducing the amount of content and enabling the handling of high-concurrency content requests.

[0146] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided in this application, before obtaining the first feature vector corresponding to the content to be identified, the following may be included:

[0147] If the number of contents obtained within the preset time period reaches the quantity threshold, then pairwise similarity calculation is performed on the contents obtained within the preset time period, where the quantity threshold is an integer greater than 1;

[0148] Content with a similarity greater than or equal to the similarity threshold will be treated as the same cluster result;

[0149] The content to be identified is obtained based on the same clustering result.

[0150] In one or more embodiments, a pre-clustering method based on quantity is introduced. As can be seen from the foregoing embodiments, high concurrency of content may occur in streaming clustering. To address this, this application sets a preset duration and a quantity threshold. If the number of contents acquired within the preset duration reaches the quantity threshold, the content acquired within the preset duration can be clustered first, i.e., pre-clustering can be performed. This yields Y clustering results.

[0151] Specifically, assume a preset duration of 10 seconds and a quantity threshold of 5000. If X new pieces of content are received within the preset duration (e.g., within 3 seconds), where X equals 5000, then the feature vector for each piece of content is calculated. For example, in one scenario, pairwise similarity (e.g., cosine similarity) can be calculated, and content with similarity greater than or equal to the similarity threshold is grouped into the same cluster, thus yielding Y cluster results. Alternatively, in another scenario, pairwise similarity distance (e.g., cosine distance) can be calculated, and content with similarity distance less than or equal to the similarity distance threshold is grouped into the same cluster, thus yielding Y cluster results.

[0152] Based on this, for each of the Y clustering results, its corresponding cluster center is determined, and this cluster center is used as the content to be identified for that clustering result. Therefore, Y clustering results yield Y cluster centers, and each cluster center belongs to one content to be identified.

[0153] It is understandable that a cluster center is a component within the clustering result. The first component in the clustering result can be used as the cluster center, or a component can be randomly selected from the clustering result; there are no restrictions here.

[0154] It should be noted that if a cluster center belongs to the target category identifier, then other content in the clustering results to which that cluster center belongs will also be marked with the target category identifier.

[0155] Secondly, this application provides a method for pre-clustering based on quantity. Using this method, a certain amount of content can be pre-clustered to obtain multiple cluster centers. Each cluster center is treated as a new piece of content (i.e., content to be identified), thereby reducing the number of contents and thus enabling the handling of high-concurrency content processing.

[0156] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment in which, after obtaining the first feature vector corresponding to the content to be identified, the following may be included:

[0157] If no content sample exists, a new category identifier for the content to be identified is generated.

[0158] Based on the mapping relationship between the first feature vector and the content to be identified, the identifier of the content to be identified and the first feature vector are stored in the first database, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified;

[0159] Based on the mapping relationship between the newly added category identifier and the content to be identified, the identifier of the content to be identified and the newly added category identifier are stored in the second database.

[0160] In one or more embodiments, a method for automatically classifying the first piece of content is described. As can be seen from the foregoing embodiments, for the first piece of content to be identified entering the system, since there are no other content samples available for clustering, this content to be identified can be used as the first content sample.

[0161] Specifically, for the first piece of content to be identified, it is assumed that the content to be identified belongs to a separate category, and a new category identifier is set for this category, thus obtaining the new category identifier. Based on this, the content to be identified can be used as a new content sample, and the first database and the second database are updated based on the ID of the content to be identified, the first feature vector of the content to be identified, and the new category identifier of the content to be identified.

[0162] Assume the ID of the content to be identified is "0", the first feature vector of the content to be identified is [-0.39, 0.75, 0.82, ..., -0.11], and the new category identifier of the content to be identified is "A". Based on this, for ease of explanation, please refer to Table 1, which is a schematic diagram of the first database storage relationship.

[0163] Table 1

[0164] Content Sample ID Feature vector 0 [-0.39,0.75,0.82,…,-0.11]

[0165] Therefore, the content to be identified is the newly added content sample in the first database, and the first feature vector of the content to be identified is the second feature vector of the content sample.

[0166] Please refer to Table 2, which is a schematic diagram of the second database storage relationship.

[0167] Table 2

[0168] Content Sample ID Category Identifier 0 A

[0169] Therefore, the content to be identified is the newly added content sample in the second database, and the newly added category identifier of the content to be identified is the category identifier of the content sample.

[0170] It should be noted that the information and headers in the above list are for illustrative purposes only. In actual applications, other headers may be added, which are not limited here.

[0171] Secondly, this application provides a method for automatically classifying the first piece of content. Using this method, the first piece of content to be identified entering the system is automatically classified into one category, thus achieving automatic content classification and facilitating subsequent processing.

[0172] Optionally, in the above Figure 3Based on the corresponding embodiments, this application provides another optional embodiment in which, according to the first feature vector and the second feature vector of each of the Q content samples, M content samples that meet the similarity condition are recalled from the Q content samples. Specifically, this may include:

[0173] Based on the first feature vector and the second feature vector of each of the Q content samples, the top T content samples with the highest similarity are recalled from the Q content samples by calling the first database, where T is an integer greater than or equal to 1.

[0174] From the first T content samples, obtain M content samples that meet the similarity criteria.

[0175] In one or more embodiments, a method for extracting M content samples from Q content samples is described. As can be seen from the foregoing embodiments, the first database can recall the top T content samples with the highest similarity to the content to be identified from the stored Q content samples. Then, it further obtains M content samples that meet the similarity criteria from these top T content samples.

[0176] Specifically, assuming T is 1000 and Q is 50 million, the first database recalls the top 1000 most similar content samples based on the first feature vector of the content to be identified and the second feature vector of each of the Q content samples. Since the top T content samples may not all be sufficiently similar to the content to be identified, they need to be truncated and filtered to remove those that do not meet the similarity criteria, resulting in the remaining M content samples for subsequent clustering. Here, M is an integer less than or equal to T.

[0177] Secondly, this application provides a method for extracting M content samples from Q content samples. By using this method, calling the first database can recall the T most similar content samples, thus making the clustering efficiency essentially independent of the number of existing categories, thereby improving real-time performance. Furthermore, selecting M content samples that meet the similarity criteria from the T content samples allows for control over the rationality of the content samples during clustering.

[0178] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment, which may further include the following step before recalling the top T most similar content samples from the Q content samples by calling the first database, according to the first feature vector and the second feature vector of each content sample in the Q content samples:

[0179] Obtain the recall setting request, wherein the recall setting request carries the value of T;

[0180] Based on the first feature vector and the second feature vector of each of the Q content samples, the top T content samples with the highest similarity are retrieved from the Q content samples by calling the first database. Specifically, this may include:

[0181] If the value of T is less than or equal to the value of Q, then the step of recalling the top T content samples with the highest similarity from the Q content samples by calling the first database is executed, based on the first feature vector and the second feature vector of each content sample in the Q content samples.

[0182] It may also include:

[0183] If the value of T is greater than the value of Q, then Q content samples are retrieved by calling the first database;

[0184] Obtain M content samples that meet the similarity criteria from Q content samples.

[0185] In one or more embodiments, a method for customizing the recall quantity is introduced. As can be seen from the foregoing embodiments, the value of T can be set according to the actual situation. If the recall quantity (i.e., the T value) is greater than the number of content samples already stored in the first database (i.e., the Q value), then all Q content samples are directly recalled. Based on this, M content samples that meet the similarity condition can be obtained from the Q content samples.

[0186] Specifically, for ease of understanding, please refer to Figure 4 , Figure 4 This is a schematic diagram of a settings interface for the content recognition system in this application embodiment. As shown in the figure, A1 indicates the settings control, A2 indicates the recall quantity input area, A3 indicates the preset duration input area, A4 indicates the quantity threshold input area, and A5 indicates the confirmation control. Assuming "1000" is entered in the recall quantity input area and the confirmation control is clicked, a recall quantity setting request is triggered. This recall quantity setting request carries the value of T (e.g., 1000).

[0187] For example, a preset duration, such as 10 seconds, can be entered in the preset duration input area. For example, a quantity threshold, such as 5000, can be entered in the quantity threshold input area.

[0188] It should be noted that, Figure 4 The interface elements and layout shown are for illustrative purposes only and should not be construed as limiting the scope of this application.

[0189] Furthermore, this application embodiment provides a method for customizing the recall quantity. Through this method, backend personnel can set the recall quantity according to actual needs, thereby improving operational flexibility.

[0190] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment in which M content samples that satisfy the similarity condition are obtained from the first T content samples, specifically including:

[0191] For each of the first T content samples, the similarity between the content to be identified and the content sample is determined based on the first feature vector and the second feature vector of the content sample.

[0192] If there is at least one content sample among the first T content samples with a similarity greater than or equal to the similarity threshold, then at least one content sample is taken as M content samples.

[0193] or,

[0194] From the first T content samples, obtain M content samples that satisfy the similarity condition, which can specifically include:

[0195] For each of the first T content samples, the similarity distance between the content to be identified and the content sample is determined based on the first feature vector and the second feature vector of the content sample.

[0196] If there is at least one content sample among the first T content samples whose similarity distance is less than or equal to the similarity distance threshold, then at least one content sample is taken as M content samples.

[0197] In one or more embodiments, a method for obtaining M content samples based on similarity or similarity distance is described. As can be seen from the foregoing embodiments, similarity or similarity distance can be used to measure the degree of similarity between the content to be identified and the content samples. For ease of understanding, cosine similarity and cosine distance will be used as examples below.

[0198] I. Cosine similarity is used;

[0199] Specifically, a cosine similarity can be calculated based on the first feature vector and the second feature vector of the content sample. The closer the cosine similarity is to 1, the more similar the content to be identified is to the content sample. Based on this, if the cosine similarity between the content to be identified and the content sample is greater than or equal to the similarity threshold, then the content sample can be used as a sample for clustering.

[0200] II. Use cosine distance;

[0201] Specifically, cosine distance and cosine similarity can be calculated based on the first feature vector and the second feature vector of the content sample. The cosine distance is calculated by subtracting the cosine similarity from 1. The closer the cosine distance is to 0, the more similar the content to be identified is to the content sample. Based on this, if the cosine distance between the content to be identified and the content sample is less than or equal to the similarity distance threshold, the content sample can be used as a sample for clustering.

[0202] It should be noted that in practical applications, Euclidean distance, Manhattan distance, or Hamming distance can also be used to calculate the similarity between the content to be identified and the content sample; no limitation is made here.

[0203] Furthermore, this application provides a method for obtaining M content samples based on similarity or similarity distance. This method allows for the selection of content samples sufficiently similar to the content to be identified for clustering, enabling control over the rationality of content samples during clustering and thus improving clustering performance.

[0204] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment in which, after obtaining the first feature vector corresponding to the content to be identified, the following may be included:

[0205] If no content sample meets the similarity criteria, a new category identifier is generated for the content to be identified.

[0206] Based on the mapping relationship between the first feature vector and the content to be identified, the identifier of the content to be identified and the first feature vector are stored in the first database, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified;

[0207] Based on the mapping relationship between the newly added category identifier and the content to be identified, the identifier of the content to be identified and the newly added category identifier are stored in the second database.

[0208] In one or more embodiments, a processing method is described when the content to be identified does not meet the similarity condition. As can be seen from the foregoing embodiments, for content to be identified that is not similar to the content sample, since there are no other content samples available for clustering, the content to be identified can be treated as a content sample under a new category.

[0209] Specifically, after classifying the content to be identified into a new category, a new category identifier needs to be set for that category, thus obtaining the new category identifier. Based on this, the content to be identified can be used as a new content sample, and the first and second databases are updated based on the ID of the content to be identified, the first feature vector of the content to be identified, and the new category identifier of the content to be identified.

[0210] Assume the ID of the content to be identified is "618", the first feature vector of the content to be identified is [-0.39, 0.75, 0.82, ..., -0.11], and the new category identifier of the content to be identified is "E". Based on this, for ease of explanation, please refer to Table 3, which is a schematic diagram of the first database storage relationship.

[0211] Table 3

[0212] Content Sample ID Feature vector 0 [-0.41,0.52,0.33,…,-0.20] … … 618 [-0.39,0.75,0.82,…,-0.11]

[0213] Therefore, the content to be identified is the newly added content sample in the first database, and the first feature vector of the content to be identified is the second feature vector of the content sample.

[0214] Please refer to Table 4, which is a schematic diagram of the second database storage relationship.

[0215] Table 4

[0216] Content Sample ID Category Identifier 0 A … … 618 E

[0217] Therefore, the content to be identified is the newly added content sample in the second database, and the newly added category identifier of the content to be identified is the category identifier of the content sample. The first and second databases can store the category identifiers of all content samples within the current lifecycle (e.g., the most recent three months).

[0218] It should be noted that the information and headers in the above list are for illustrative purposes only. In actual applications, other headers may be added, which are not limited here.

[0219] Secondly, this application provides a method for handling situations where the content to be identified does not meet the similarity criteria. Through this method, content that is dissimilar to the content sample is automatically classified into a new category, thus achieving automatic content categorization and facilitating subsequent processing.

[0220] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment in which, according to the category identifier of each content sample in the M content samples, the number of content samples corresponding to each category identifier is statistically analyzed to obtain the number of content samples corresponding to each category identifier. Specifically, this may include:

[0221] Based on the category identifier of each content sample in the M content samples, the content samples with the same category identifier are associated with the same category identifier, and the number of content samples corresponding to each category identifier is obtained.

[0222] Based on the number of content samples corresponding to each category identifier, obtain the target category label corresponding to the content to be identified, which may specifically include:

[0223] Based on the number of content samples corresponding to each category identifier, determine the target category identifier corresponding to the content to be identified;

[0224] Based on the target category identifier, determine the target category label corresponding to the content to be identified.

[0225] In one or more embodiments, a method for determining target category labels based on a voting mechanism is introduced. As can be seen from the foregoing embodiments, after selecting M content samples, the category identifier of each content sample in the M content samples can be obtained by querying a second database. Based on this, content samples with the same category identifier can be associated with the same category identifier, thereby obtaining the number of content samples corresponding to each category identifier.

[0226] Specifically, each of the M content samples represents a "vote," and the category identifier of each content sample serves as the "vote type." Therefore, based on the category identifiers of these M content samples, voting is conducted for each category identifier, and the number of content samples associated with each category identifier is counted. Based on the number of content samples corresponding to each category identifier, one category identifier can be selected as the target category identifier for the content to be identified.

[0227] For example, a target category identifier can be associated with a known target category label, thereby directly determining the target category label. For instance, if the target category identifier is "W", the corresponding target category label indicates that the content to be identified belongs to matrix content.

[0228] For example, a target category identifier can be associated with an unknown target category label, thereby the target category label can be determined by manual annotation or by model prediction.

[0229] Secondly, this application embodiment provides a method for determining target category labels based on a voting mechanism. Considering that adding the recall process to the first database might reduce accuracy, a voting mechanism is introduced here to improve accuracy to some extent, thus balancing the efficiency and accuracy of clustering.

[0230] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment, which, after determining the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier, may further include:

[0231] Based on the mapping relationship between the first feature vector and the content to be identified, the identifier of the content to be identified and the first feature vector are stored in the first database, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified;

[0232] Based on the mapping relationship between the target category identifier and the content to be identified, the identifier of the content to be identified and the target category identifier are stored in the second database.

[0233] In one or more embodiments, a method for adding content to be identified as a content sample to a database is described. As can be seen from the foregoing embodiments, after clustering the content to be identified, the content to be identified can be used as a content sample under a new category.

[0234] Specifically, the content to be identified can be used as a new content sample, and the first database and the second database can be updated based on the ID of the content to be identified, the first feature vector of the content to be identified, and the target category identifier of the content to be identified.

[0235] Assume the ID of the content to be identified is "618", the first feature vector of the content to be identified is [-0.39, 0.75, 0.82, ..., -0.11], and the target category identifier of the content to be identified is "A". Based on this, for ease of explanation, please refer to Table 5, which is a schematic diagram of the first database storage relationship.

[0236] Table 5

[0237] Content Sample ID Feature vector 0 [-0.41,0.52,0.33,…,-0.20] … … 618 [-0.39,0.75,0.82,…,-0.11]

[0238] Therefore, the content to be identified is the newly added content sample in the first database, and the first feature vector of the content to be identified is the second feature vector of the content sample.

[0239] Please refer to Table 6, which is a schematic diagram of the second database storage relationship.

[0240] Table 6

[0241] Content Sample ID Category Identifier 0 A … … 618 A

[0242] Therefore, the content to be identified is the newly added content sample in the second database, and the target category identifier of the content to be identified is the category identifier of the content sample.

[0243] It should be noted that the information and headers in the above list are for illustrative purposes only. In actual applications, other headers may be added, which are not limited here. The first and second databases can store category identifiers for all content samples within the current lifecycle (e.g., the last three months).

[0244] Furthermore, this application embodiment provides a method for adding the content to be identified as a content sample to the database. Using this method, the clustered content to be identified can be used as a new content sample to update the database. This allows for a continuous increase in the number of content samples, which is beneficial for improving the accuracy of clustering.

[0245] Optionally, in the above Figure 3Based on the corresponding embodiments, this application provides another optional embodiment in which the target category identifier corresponding to the content to be identified is determined according to the number of content samples corresponding to each category identifier. Specifically, this may include:

[0246] Determine the number of content samples associated with each category identifier in at least one category identifier;

[0247] The category identifier corresponding to the maximum number of content samples is used as the target category identifier for the content to be identified.

[0248] Based on the target category identifier, determine the target category label corresponding to the content to be identified.

[0249] In one or more embodiments, a method for determining target category labels based on the number of content samples is described. As can be seen from the foregoing embodiments, after selecting M content samples, the number of content samples corresponding to each category label can be obtained after "voting".

[0250] Specifically, assuming M is 500, based on the category identifiers of these 500 content samples, a vote is taken for each category identifier, and the number of content samples associated with each category identifier is counted. For easier understanding, please refer to Table 7, which illustrates the relationship between each category identifier and the number of content samples.

[0251] Table 7

[0252] Category Identifier Content sample size A 300 B 100 C 50 D 50

[0253] Therefore, taking 500 content samples as an example, 300 samples are categorized as "A", 100 as "B", 50 as "C", and 50 as "D". This is equivalent to casting 300 votes for category "A" for the content to be identified. Since the number of votes for category "A" is far greater than the number of votes for other categories, the content to be identified can be classified as category "A" (i.e., the target category is now "A").

[0254] Furthermore, this application provides a method for determining target category labels based on the number of content samples. Using this method, the category identifier with the most votes can be used as the target category identifier for the content to be identified, thereby improving clustering accuracy to a certain extent and balancing clustering efficiency and accuracy.

[0255] Optionally, in the above Figure 3Based on the corresponding embodiments, this application provides another optional embodiment in which the target category identifier corresponding to the content to be identified is determined according to the number of content samples corresponding to each category identifier. Specifically, this may include:

[0256] Determine the number of content samples associated with each category identifier in at least one category identifier;

[0257] If at least two category identifiers are associated with the same number of content samples, then the average ranking corresponding to each category identifier is calculated based on the content samples associated with each of the at least two category identifiers and the recall similarity ranking of the content samples.

[0258] Use the category identifier corresponding to the highest average ranking as the target category identifier for the content to be identified.

[0259] In one or more embodiments, a method for determining target category identifiers based on average ranking is described. As can be seen from the foregoing embodiments, after selecting M content samples, the number of content samples corresponding to each category identifier can be obtained after "voting".

[0260] Specifically, assuming M is 500, based on the category identifiers of these 500 content samples, a vote is taken for each category identifier, and the number of content samples associated with each category identifier is counted. Taking 500 content samples as an example, 200 content samples are labeled "A", 200 are labeled "B", 75 are labeled "C", and 25 are labeled "D". This is equivalent to casting 200 votes for category "A" and 200 votes for category "B" for the content to be identified. In this case, category identifier "A" and category identifier "B" are tied.

[0261] Based on this, we can obtain the similarity ranking of each content sample among the M content samples in the T content samples. The higher the ranking of a content sample, the more similar it is to the content to be identified. For example, the content sample with the highest similarity ranking is most similar to the content to be identified. Suppose that the 200 content samples belonging to category "A" have an average ranking of 52 among the T content samples, and the 200 content samples belonging to category "B" have an average ranking of 78 among the T content samples. Then, the average ranking of 52 is higher than the average ranking of 78. Therefore, the content to be identified can be classified into category "A" (that is, the target category is now category "A").

[0262] Furthermore, this application provides a method for determining the target category identifier based on average ranking. Using this method, when the number of votes is the same, an average ranking strategy can be employed to select a category identifier as the target category identifier for the content to be identified. This improves clustering accuracy while balancing efficiency and precision. It also increases the feasibility and operability of the solution.

[0263] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment in which the target category identifier corresponding to the content to be identified is determined according to the number of content samples corresponding to each category identifier. Specifically, this may include:

[0264] Determine the number of content samples associated with each category identifier in at least one category identifier;

[0265] If at least two category identifiers are associated with the same number of content samples, then calculate the average feature vector corresponding to each category identifier based on the content samples associated with each of the at least two category identifiers.

[0266] The target category identifier for the content to be identified is determined based on the similarity between the average feature vector corresponding to each category identifier and the first feature vector.

[0267] In one or more embodiments, a method for determining target category identifiers based on average ranking is described. As can be seen from the foregoing embodiments, after selecting M content samples, the number of content samples corresponding to each category identifier can be obtained after "voting".

[0268] Specifically, assuming M is 500, based on the category identifiers of these 500 content samples, a vote is taken for each category identifier, and the number of content samples associated with each category identifier is counted. Taking 500 content samples as an example, 200 content samples are labeled "A", 200 are labeled "B", 75 are labeled "C", and 25 are labeled "D". This is equivalent to casting 200 votes for category "A" and 200 votes for category "B" for the content to be identified. In this case, category identifier "A" and category identifier "B" are tied.

[0269] Based on this, the second feature vector of each of the M content samples can be obtained from the first database. For the 200 content samples belonging to category "A", an average feature vector can be calculated, thus obtaining the average feature vector of category "A". Similarly, for the 200 content samples belonging to category "B", an average feature vector can be calculated, thus obtaining the average feature vector of category "B". Then, the similarity between the average feature vector of category "A" and the first feature vector, and the similarity between the average feature vector of category "B" and the first feature vector, are calculated respectively. For example, if the similarity between the average feature vector of category "A" and the first feature vector is greater, then the content to be identified can be classified as category "A" (i.e., the target category label is now category "A").

[0270] Furthermore, this application provides a method for determining the target category identifier based on the average ranking. Using this method, when the number of votes is the same, a vector average strategy can be employed to select one category identifier as the target category identifier for the content to be identified. This improves clustering accuracy while balancing efficiency and precision. It also increases the feasibility and operability of the solution.

[0271] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment in which the target category label corresponding to the content to be identified is determined according to the target category identifier, which may specifically include:

[0272] In response to the labeling instruction for the target category identifier, determine the target category label corresponding to the content to be identified;

[0273] It may also include:

[0274] The content samples associated with the target category identifier and the content to be identified are used as the training sample set, wherein the training sample set is labeled with the target category label;

[0275] Obtain the content features and account information of each training sample in the training sample set;

[0276] Obtain the cluster similarity features of the set of samples to be trained;

[0277] Based on the clustering similarity features of the training sample set, the content features of each training sample, and the account information, the probability distribution of the first category is obtained through a category recognition model.

[0278] The model parameters of the category recognition model are updated based on the target category label and the probability distribution of the first category.

[0279] In one or more embodiments, a method for training a category recognition model is described. As can be seen from the foregoing embodiments, for the clustering results corresponding to the target category identifier, backend personnel can conduct manual review in conjunction with the account's registration information to determine whether the account itself belongs to a low-cost content account operated in a matrix manner, and the content published by such accounts is all matrix content. Based on this, backend personnel can manually label the clustering results to obtain category labels, and use these category labels as the target category labels corresponding to the content to be identified.

[0280] Specifically, for ease of understanding, please refer to Figure 5 , Figure 5 This is a schematic diagram of an annotation interface for the content recognition system in this application embodiment. As shown in the figure, B1 indicates the annotation control, B2 indicates the category label input area, and B3 indicates the confirmation control. Assume the current clustering result's category identifier is "A", that is, the target category identifier is "A". The ID of the content to be identified is "77". Therefore, assuming "1" is entered in the category label input area and the confirmation control is clicked, a label annotation instruction for the target category identifier is triggered, where the label annotation instruction carries the target category label (e.g., "1").

[0281] It should be noted that, Figure 5 The interface elements and layout shown are for illustrative purposes only and should not be construed as limiting the scope of this application.

[0282] Based on this, the category recognition model can be trained using the labeled tags. For example, assume the training sample set includes content samples associated with the target category identifier and the content to be recognized. Each training sample in the training sample set is labeled with a target category tag (e.g., "1" or "0").

[0283] For example, it is necessary to obtain the content features and account information of each training sample in the training sample set. The content features are assumed to be text features, which include, but are not limited to, keyword features, fixed sentence structure features, word frequency features, word vectors (Word2Vec), and feature vectors output by the Bidirectional Encoder Representation from Transformers (BERT) model. The account information includes, but is not limited to, nickname, profile, account name, administrator, operator, penalty records, and mass messages.

[0284] For example, it is also necessary to obtain the clustering similarity features of the set of samples to be trained. These clustering similarity features include, but are not limited to, textual features of cluster ID, cluster size, and cluster centroids.

[0285] By utilizing the clustering similarity features of the training sample set, the content features of each training sample, and account information, a category recognition model outputs a first-class probability distribution. Based on this first-class probability distribution and the target class label, a gradient value can be calculated using the cross-entropy loss function. This gradient value is then used to update the model parameters of the category recognition model. The first-class probability distribution can be represented as a single probability value, or multiple probability values.

[0286] Taking the category recognition model as an example, which is a Deep Factorization Machine (DeepFM) model, please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of the category recognition model in an embodiment of this application. As shown in the figure, content features, account information, and related features from cluster similarity features (e.g., mass messaging features and account behavior features) can be used as dense features. Account information and related features from cluster similarity features (e.g., cluster ID and cluster size) can be used as discrete features. The dense features and discrete features are used as input to the category recognition model, and the category recognition model outputs a first category probability distribution. It is assumed that the first category probability distribution is a probability value, that is, the probability value represents the probability that the training sample set belongs to the matrix content.

[0287] It should be noted that this application uses the DeepFM model as the category recognition model. This is because the DeepFM model can effectively utilize both manually constructed business features and features learned by the model itself, placing basic features in the deep part and manually constructed business features in the factorization machines (FM) part, resulting in good performance. In practical applications, the extreme gradient boosting tree (XGBoost) model can also be used as the category recognition model. Alternatively, Automatic Feature Interaction (AutoInt) can be used as the category recognition model; no specific limitation is made here.

[0288] Furthermore, this application provides a method for training a category recognition model. By using existing sample data to periodically train the category recognition model, it is easier to predict clustering results subsequently, thus saving the cost of manual annotation and improving the efficiency of content recognition.

[0289] Optionally, in the above Figure 3 Based on the corresponding embodiments, this application provides another optional embodiment in which the target category label corresponding to the content to be identified is determined according to the target category identifier, which may specifically include:

[0290] The content samples associated with the target category identifier and the content to be identified are used as the prediction sample set;

[0291] Obtain the content features and account information of each prediction sample in the prediction sample set;

[0292] Obtain the cluster similarity features of the predicted sample set;

[0293] Based on the clustering similarity features of the predicted sample set, the content features of each predicted sample, and account information, the probability distribution of the second category is obtained through a category recognition model.

[0294] The target category label corresponding to the content to be identified is determined based on the probability distribution of the second category.

[0295] In one or more embodiments, a method for predicting target category identifiers using a category recognition model is described. As described in the foregoing embodiments, for the clustering results corresponding to the target category identifier, the clustering results can be used as a prediction sample set, wherein the prediction sample set includes content samples associated with the target category identifier and the content to be identified. Based on this, the content features and account information of each prediction sample in the prediction sample set, as well as the clustering similarity features of the prediction sample set, are obtained. Then, the clustering similarity features of the prediction sample set, the content features of each prediction sample, and the account information are used as input to the category recognition model, and the category recognition model outputs a second category probability distribution. For example, the second category probability distribution is a probability value; for instance, a probability value greater than or equal to 0.8 can be used as the category label "1", and a probability value less than 0.8 can be used as the category label "0". Assuming the second category probability distribution is 0.9, the target category label of the prediction sample set is "1", and the target category label corresponding to the content to be identified is also "1".

[0296] Specifically, for ease of understanding, please refer to Figure 7 , Figure 7 This is a schematic diagram of a result display interface of the content recognition system in this application embodiment. As shown in the figure, C1 is used to indicate the details viewing control. It can be seen that for a certain clustering result, its corresponding category identifier is "A", that is, the target category identifier is "A". Its corresponding category label is "1", that is, the target category label is "1", at this time, it is considered to belong to matrix content.

[0297] It should be noted that, Figure 7 The interface elements and layout shown are for illustrative purposes only and should not be construed as limiting the scope of this application.

[0298] Furthermore, this application provides a method for predicting target category identifiers using a category recognition model. This method saves the cost of manually labeling categories and improves the efficiency of content recognition.

[0299] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, it may also include:

[0300] If the content to be identified is text, then feature extraction is performed on the content to be identified to obtain the first feature vector;

[0301] or,

[0302] It may also include:

[0303] If the content to be identified is an image, then the encoder encodes the content to be identified to obtain an image feature vector;

[0304] The first feature vector is obtained by performing a nonlinear transformation on the image feature vector using a projection head.

[0305] In one or more embodiments, a processing method for different types of content to be identified is described. As can be seen from the foregoing embodiments, the content to be identified can be text content or image content, and different types of content are extracted using different methods to extract their corresponding feature vectors, which will be described separately below.

[0306] First, cluster the text content;

[0307] Specifically, if the content to be identified is text, a text feature extraction network can be used to extract the first feature vector. For example, the BERT model can be used to extract features and obtain the first feature vector. It can be understood that the first feature vector can be 768-dimensional, 1024-dimensional, or other dimensions; no limitation is made here.

[0308] For easier understanding, please refer to Figure 8 , Figure 8 This is a schematic diagram illustrating text content-based recognition in an embodiment of this application. As shown in the figure, the streaming text content is clustered by calling a first database and a second database, thereby obtaining a clustering result for each text content. Based on this, the features corresponding to each clustering result are used as input to a category recognition model, which outputs a category label for each text content.

[0309] 2. Cluster the image content;

[0310] Specifically, if the content to be identified is image content, an image feature extraction network can be used to extract the first feature vector. For example, a Simple Framework for Contrastive Learning of Visual Representations (SimCLR) can be used to extract features and obtain the first feature vector. SimCLR includes a base encoder and a projection head. The encoder encodes the content to be identified, extracting a representation vector to obtain an image feature vector. The encoder can use a Residual Neural Network-50 (ResNet-50). The image feature vector is input to the projection head, which performs a non-linear transformation on the image feature vector to obtain the first feature vector. The projection head can use a Multilayer Perceptron (MLP) and activation functions.

[0311] For easier understanding, please refer to Figure 9 , Figure 9 This is a schematic diagram illustrating image content-based recognition in an embodiment of this application. As shown in the figure, the streaming image content is clustered by calling a first database and a second database, thereby obtaining a clustering result for each image content. Based on this, the features corresponding to each clustering result are used as input to a category recognition model, which outputs a category label for each image content.

[0312] Secondly, this application provides a method for processing different types of content to be identified. Through this method, streaming clustering can be performed on text or image content, and their corresponding category labels can be identified, thus enabling the handling of a wider range of scenarios.

[0313] Traffic from low-cost content accounts accounts for approximately 40% of the platform's proactive recommendation business. During the gray-scale rollout of this solution, the approach provided in this application reduced the cost of matrix operations to below 5%, demonstrating significant effectiveness. This improved the quality and impact of the platform's content recommendation business, bringing more useful information and practical insights to end-users. By identifying low-cost content accounts, high-quality creators received increased traffic support, resulting in a 37% return rate, a 50% increase in readership, a significant rise in author activity, and a greater willingness to publish more high-quality content on the platform.

[0314] The content recognition device in this application is described in detail below. Please refer to [link / reference]. Figure 10 , Figure 10This is a schematic diagram of one embodiment of the content recognition device in this application. The content recognition device 20 includes:

[0315] The acquisition module 210 is used to acquire the first feature vector corresponding to the content to be identified;

[0316] The acquisition module 210 is further configured to recall M content samples that meet the similarity condition from the Q content samples based on the first feature vector and the second feature vector of each content sample in the Q content samples, wherein each content sample corresponds to a category identifier, M and Q are both integers greater than or equal to 1, and M is less than or equal to Q.

[0317] The determination module 220 is used to count the number of content samples corresponding to each category identifier based on the category identifier of each content sample in the M content samples.

[0318] The processing module 230 is also used to obtain the target category label corresponding to the content to be identified based on the number of content samples corresponding to each category identifier.

[0319] This application provides a content recognition device. Using this device, when new content to be recognized is obtained, several content samples with a high degree of similarity to the content to be recognized are first recalled. Then, based on the known clustering categories of these content samples, the clustering category to which the content to be recognized can be assigned is determined, thereby obtaining the target category label. It is evident that for the obtained content to be recognized, clustering can be performed using statistical category labels, which better adapts to streaming content scenarios and improves the real-time performance of content recognition.

[0320] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the content recognition device 20 provided in this application, the content recognition device 20 further includes a processing module 230;

[0321] The processing module 230 is also used to perform pairwise similarity calculation on the content obtained within a preset time period before obtaining the first feature vector corresponding to the content to be identified;

[0322] The determination module 220 is also used to treat content with a similarity greater than or equal to a similarity threshold as the same clustering result;

[0323] The acquisition module 210 is also used to acquire the content to be identified based on the same clustering result.

[0324] This application provides a content recognition device. Using this device, content acquired over a period of time can be pre-clustered to obtain multiple cluster centers. Each cluster center is then treated as a new piece of content (i.e., content to be recognized), thereby reducing the amount of content and enabling the handling of high-concurrency content processing.

[0325] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0326] The processing module 230 is also used to perform pairwise similarity calculation on the content obtained within the preset time period if the number of contents obtained within the preset time period reaches the number threshold before obtaining the first feature vector corresponding to the content to be identified. The number threshold is an integer greater than 1.

[0327] The determination module 220 is also used to treat content with a similarity greater than or equal to a similarity threshold as the same clustering result;

[0328] The acquisition module 210 is also used to acquire the content to be identified based on the same clustering result.

[0329] This application provides a content recognition device. Using this device, a certain amount of content can be pre-clustered to obtain multiple cluster centers. Each cluster center is then treated as a new piece of content (i.e., content to be recognized), thereby reducing the amount of content and enabling the handling of high-concurrency content processing.

[0330] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0331] The processing module 230 is also used to generate a new category identifier for the content to be identified if there is no content sample after obtaining the first feature vector corresponding to the content to be identified.

[0332] The processing module 230 is further configured to store the identifier of the content to be identified and the first feature vector into the first database according to the mapping relationship between the first feature vector and the content to be identified, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified.

[0333] The processing module 230 is also used to store the identifier of the content to be identified and the newly added category identifier in the second database according to the mapping relationship between the newly added category identifier and the content to be identified.

[0334] This application provides a content recognition device. Using this device, the first piece of content to be recognized entering the system is automatically categorized into one class, thus achieving automatic content classification and facilitating subsequent processing.

[0335] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0336] The acquisition module 210 is specifically used to retrieve the top T content samples with the highest similarity from the Q content samples by calling the first database based on the first feature vector and the second feature vector of each content sample in the Q content samples, where T is an integer greater than or equal to 1.

[0337] From the first T content samples, obtain M content samples that meet the similarity criteria.

[0338] This application provides a content recognition device. Using this device, the first database can be invoked to recall the T most similar content samples. This makes the clustering efficiency essentially independent of the number of existing categories, thereby improving real-time performance. Furthermore, by selecting M content samples from the T content samples that meet the similarity criteria, the rationality of the content samples during clustering can be controlled.

[0339] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0340] The acquisition module 210 is further configured to obtain a recall number setting request before recalling the top T content samples with the highest similarity from the Q content samples by calling the first database based on the first feature vector and the second feature vector of each content sample in the Q content samples, wherein the recall number setting request carries the value of T;

[0341] The acquisition module 210 is specifically used to execute the step of recalling the top T content samples with the highest similarity from the Q content samples by calling the first database if the value of T is less than or equal to the value of Q.

[0342] The acquisition module 210 is also used to recall Q content samples by calling the first database if the value of T is greater than the value of Q.

[0343] The acquisition module 210 is also used to acquire M content samples that meet the similarity conditions from Q content samples.

[0344] This application provides a content recognition device. Using this device, backend personnel can set the recall quantity according to actual needs, thereby improving operational flexibility.

[0345] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0346] The acquisition module 210 is specifically used to determine the similarity between the content to be identified and the content sample for each content sample in the first T content samples, based on the first feature vector and the second feature vector of the content sample.

[0347] If there is at least one content sample among the first T content samples with a similarity greater than or equal to the similarity threshold, then at least one content sample is taken as M content samples.

[0348] or,

[0349] The acquisition module 210 is specifically used to determine the similarity distance between the content to be identified and the content sample based on the first feature vector and the second feature vector of the content sample for each of the first T content samples.

[0350] If there is at least one content sample among the first T content samples whose similarity distance is less than or equal to the similarity distance threshold, then at least one content sample is taken as M content samples.

[0351] This application provides a content recognition device. Using this device, content samples sufficiently similar to the content to be recognized are selected for clustering. This allows control over the rationality of the content samples during clustering, thereby improving the clustering effect.

[0352] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0353] The processing module 230 is also used to generate a new category identifier for the content to be identified if there is no content sample that meets the similarity condition after obtaining the first feature vector corresponding to the content to be identified.

[0354] The processing module 230 is further configured to store the identifier of the content to be identified and the first feature vector into the first database according to the mapping relationship between the first feature vector and the content to be identified, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified.

[0355] The processing module 230 is also used to store the identifier of the content to be identified and the newly added category identifier in the second database according to the mapping relationship between the newly added category identifier and the content to be identified.

[0356] This application provides a content recognition device. Using this device, for content to be recognized that is dissimilar to a content sample, the content to be recognized is automatically classified into a new category, thereby achieving automatic content classification and facilitating subsequent processing.

[0357] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0358] The determination module 220 is specifically used to associate content samples with the same category identifier with the same category identifier based on the category identifier of each content sample in the M content samples, so as to obtain the number of content samples corresponding to each category identifier;

[0359] The acquisition module 210 is specifically used to determine the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier;

[0360] Based on the target category identifier, determine the target category label corresponding to the content to be identified.

[0361] In this embodiment, a content recognition device is provided. Using this device, considering that the recall process involving the first database might reduce accuracy, a voting mechanism is introduced to improve accuracy to some extent, thus balancing the efficiency and accuracy of clustering.

[0362] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0363] The processing module 230 is further configured to determine the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier, and then store the identifier of the content to be identified and the first feature vector in the first database according to the mapping relationship between the first feature vector and the content to be identified, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified.

[0364] The processing module 230 is also used to store the identifier of the content to be identified and the target category identifier in the second database according to the mapping relationship between the target category identifier and the content to be identified.

[0365] This application provides a content recognition device. Using this device, clustered content to be identified can be used as new content samples to update the database. This allows for a continuous increase in the number of content samples, which helps improve the accuracy of clustering.

[0366] Optionally, in the above Figure 10Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0367] The acquisition module 210 is specifically used to determine the number of content samples associated with each category identifier in at least one category identifier;

[0368] The category identifier corresponding to the maximum number of content samples is used as the target category identifier for the content to be identified.

[0369] Based on the target category identifier, determine the target category label corresponding to the content to be identified.

[0370] This application provides a content recognition device. Using this device, the category identifier with the most votes can be used as the target category identifier for the content to be identified, thereby improving clustering accuracy to a certain extent and balancing clustering efficiency and accuracy.

[0371] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0372] The acquisition module 210 is specifically used to determine the number of content samples associated with each category identifier in at least one category identifier;

[0373] If at least two category identifiers are associated with the same number of content samples, then the average ranking corresponding to each category identifier is calculated based on the content samples associated with each of the at least two category identifiers and the recall similarity ranking of the content samples.

[0374] Use the category identifier corresponding to the highest average ranking as the target category identifier for the content to be identified.

[0375] This application provides a content recognition device. Using this device, when the number of votes is the same, an average ranking strategy can be employed to select a category identifier as the target category identifier for the content to be identified. This improves clustering accuracy while balancing efficiency and precision. Furthermore, it increases the feasibility and operability of the solution.

[0376] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0377] The acquisition module 210 is specifically used to determine the number of content samples associated with each category identifier in at least one category identifier;

[0378] If at least two category identifiers are associated with the same number of content samples, then calculate the average feature vector corresponding to each category identifier based on the content samples associated with each of the at least two category identifiers.

[0379] The target category identifier for the content to be identified is determined based on the similarity between the average feature vector corresponding to each category identifier and the first feature vector.

[0380] This application provides a content recognition device. Using this device, when the number of votes is the same, a vector average strategy can be employed to select a category identifier as the target category identifier for the content to be recognized. This improves clustering accuracy while balancing efficiency and precision. Furthermore, it increases the feasibility and operability of the solution.

[0381] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0382] The acquisition module 210 is specifically used to respond to the label labeling instruction for the target category identifier and determine the target category label corresponding to the content to be identified;

[0383] The acquisition module 210 is also used to take the content sample associated with the target category identifier and the content to be identified as the training sample set, wherein the training sample set is labeled with the target category label;

[0384] The acquisition module 210 is also used to acquire the content features and account information of each training sample in the training sample set;

[0385] The acquisition module 210 is also used to acquire clustering similarity features of the set of samples to be trained;

[0386] The acquisition module 210 is also used to obtain the probability distribution of the first category through a category recognition model based on the clustering similarity features of the set of samples to be trained, the content features of each sample to be trained, and the account information.

[0387] The processing module 230 is also used to update the model parameters of the category recognition model based on the target category label and the probability distribution of the first category.

[0388] This application provides a content recognition device. Using this device, the category recognition model is periodically trained with existing sample data, facilitating subsequent prediction of clustering results, thereby saving the cost of manual annotation and improving the efficiency of content recognition.

[0389] Optionally, in the above Figure 10Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0390] The acquisition module 210 is specifically used to use the content samples associated with the target category identifier and the content to be identified as a prediction sample set;

[0391] Obtain the content features and account information of each prediction sample in the prediction sample set;

[0392] Obtain the cluster similarity features of the predicted sample set;

[0393] Based on the clustering similarity features of the predicted sample set, the content features of each predicted sample, and account information, the probability distribution of the second category is obtained through a category recognition model.

[0394] The target category label corresponding to the content to be identified is determined based on the probability distribution of the second category.

[0395] This application provides a content recognition device. Using this device saves the cost of manually labeling categories and improves the efficiency of content recognition.

[0396] Optionally, in the above Figure 10 Based on the corresponding embodiments, this application provides another embodiment of the content recognition device 20.

[0397] The processing module 230 is also used to extract features from the content to be identified if the content to be identified is text content, and obtain a first feature vector.

[0398] or,

[0399] The processing module 230 is also used to encode the content to be identified by an encoder to obtain an image feature vector if the content to be identified is image content;

[0400] The processing module 230 is also used to perform a nonlinear transformation on the image feature vector through the projection head to obtain the first feature vector.

[0401] This application provides a content recognition device. Using this device, text or image content can be streamed and clustered to identify its corresponding category tags, thus enabling the handling of a wider range of scenarios.

[0402] This application also provides another content recognition device that can be applied to terminal devices. For example... Figure 11As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablet computers, personal digital assistants (PDAs), point-of-sale terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0403] Figure 11 This diagram illustrates a partial structure of a mobile phone related to the terminal device provided in the embodiments of this application. (Reference) Figure 11 The mobile phone includes components such as a radio frequency (RF) circuit 310, a memory 320, an input unit 330, a display unit 340, a sensor 350, an audio circuit 360, a Wi-Fi module 370, a processor 380, and a power supply 390. Those skilled in the art will understand that... Figure 11 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0404] The following is combined with Figure 11 A detailed introduction to each component of a mobile phone:

[0405] RF circuit 310 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 380; additionally, it transmits uplink data to the base station. Typically, RF circuit 310 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 310 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0406] The memory 320 can be used to store software programs and modules. The processor 380 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0407] The input unit 330 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 330 may include a touch panel 331 and other input devices 332. The touch panel 331, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 331), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 331 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 380, and can also receive and execute commands sent by the processor 380. In addition, the touch panel 331 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 331, the input unit 330 may also include other input devices 332. Specifically, other input devices 332 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0408] The display unit 340 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 340 may include a display panel 341, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display. Furthermore, a touch panel 331 may cover the display panel 341. When the touch panel 331 detects a touch operation on or near it, it transmits the information to the processor 380 to determine the type of touch event. Subsequently, the processor 380 provides corresponding visual output on the display panel 341 based on the type of touch event. Although in Figure 11 In this embodiment, the touch panel 331 and the display panel 341 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 331 and the display panel 341 can be integrated to realize the input and output functions of the mobile phone.

[0409] The mobile phone may also include at least one sensor 350, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 341 according to the ambient light level, and the proximity sensor can turn off the display panel 341 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0410] Audio circuit 360, speaker 361, and microphone 362 provide an audio interface between the user and the mobile phone. Audio circuit 360 converts received audio data into electrical signals and transmits them to speaker 361, where speaker 361 converts them into sound signals for output. On the other hand, microphone 362 converts collected sound signals into electrical signals, which are received by audio circuit 360, converted into audio data, and then output to processor 380 for processing. The audio data is then transmitted via RF circuit 310 to, for example, another mobile phone, or output to memory 320 for further processing.

[0411] WiFi is a short-range wireless transmission technology. Through a WiFi module 370, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 11WiFi module 370 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0412] The processor 380 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 320, and by calling data stored in the memory 320. Optionally, the processor 380 may include one or more processing units; optionally, the processor 380 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 380.

[0413] The mobile phone also includes a power supply 390 (such as a battery) that supplies power to various components. Optionally, the power supply can be logically connected to the processor 380 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0414] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0415] The steps performed by the terminal device in the above embodiments can be based on this Figure 11 The terminal device structure is shown.

[0416] This application also provides another content recognition device that can be applied to a server. Figure 12 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 400 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 422 (e.g., one or more processors) and memory 432, and one or more storage media 430 (e.g., one or more mass storage devices) for storing application programs 442 or data 444. The memory 432 and storage media 430 can be temporary or persistent storage. The program stored in the storage media 430 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 422 may be configured to communicate with the storage media 430 and execute the series of instruction operations stored in the storage media 430 on the server 400.

[0417] Server 400 may also include one or more power supplies 426, one or more wired or wireless network interfaces 450, one or more input / output interfaces 458, and / or one or more operating systems 441, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0418] The steps performed by the server in the above embodiments can be based on this Figure 12 The server structure shown.

[0419] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0420] This application also provides a computer program product including a program, which, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0421] It is understood that in the specific implementation of this application, data such as account information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0422] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0423] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0424] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0425] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0426] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0427] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A content recognition method, characterized in that, include: Obtain the first feature vector corresponding to the content to be identified; Based on the first feature vector and the second feature vector of each of the Q content samples, M content samples that meet the similarity condition are recalled from the Q content samples, wherein each content sample corresponds to a category identifier, M and Q are both integers greater than or equal to 1, and M is less than or equal to Q; Based on the category identifier of each content sample in the M content samples, the number of content samples corresponding to each category identifier is obtained by counting the content samples for each category identifier; Based on the number of content samples corresponding to each category identifier, the target category identifier corresponding to the content to be identified is determined; Based on the target category identifier, determine the target category label corresponding to the content to be identified.

2. The method according to claim 1, characterized in that, Before obtaining the first feature vector corresponding to the content to be identified, the method further includes: Perform pairwise similarity calculations on the content acquired within a preset time period; Content with a similarity greater than or equal to the similarity threshold will be treated as the same cluster result; The content to be identified is obtained based on the same clustering result.

3. The method according to claim 1, characterized in that, Before obtaining the first feature vector corresponding to the content to be identified, the method further includes: If the number of contents obtained within a preset time period reaches a quantity threshold, then pairwise similarity calculation is performed on the contents obtained within the preset time period, wherein the quantity threshold is an integer greater than 1; Content with a similarity greater than or equal to the similarity threshold will be treated as the same cluster result; The content to be identified is obtained based on the same clustering result.

4. The method according to claim 1, characterized in that, After obtaining the first feature vector corresponding to the content to be identified, the method further includes: If no content sample exists, a new category identifier is generated for the content to be identified; Based on the mapping relationship between the first feature vector and the content to be identified, the identifier of the content to be identified and the first feature vector are stored in the first database, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified; Based on the mapping relationship between the newly added category identifier and the content to be identified, the identifier of the content to be identified and the newly added category identifier are stored in the second database.

5. The method according to claim 1, characterized in that, The step of recalling M content samples that meet the similarity condition from the Q content samples based on the first feature vector and the second feature vector of each content sample in the Q content samples includes: Based on the first feature vector and the second feature vector of each of the Q content samples, the top T content samples with the highest similarity are recalled from the Q content samples by calling the first database, where T is an integer greater than or equal to 1; From the first T content samples, obtain the M content samples that satisfy the similarity condition.

6. The method according to claim 5, characterized in that, Before retrieving the top T most similar content samples from the Q content samples by calling the first database based on the first feature vector and the second feature vector of each of the Q content samples, the method further includes: Obtain a recall setting request, wherein the recall setting request carries the value of T; The step of retrieving the top T content samples with the highest similarity from the Q content samples by calling the first database, based on the first feature vector and the second feature vector of each of the Q content samples, includes: If the value of T is less than or equal to the value of Q, then the step of recalling the top T content samples with the highest similarity from the Q content samples by calling the first database based on the first feature vector and the second feature vector of each content sample in the Q content samples. The method further includes: If the value of T is greater than the value of Q, then the Q content samples are recalled by calling the first database; From the Q content samples, obtain the M content samples that satisfy the similarity condition.

7. The method according to claim 5, characterized in that, The step of obtaining the M content samples that satisfy the similarity condition from the first T content samples includes: For each of the first T content samples, the similarity between the content to be identified and the content sample is determined based on the first feature vector and the second feature vector of the content sample. If there is at least one content sample among the first T content samples with a similarity greater than or equal to a similarity threshold, then the at least one content sample is taken as the M content samples. or, The step of obtaining the M content samples that satisfy the similarity condition from the first T content samples includes: For each of the first T content samples, the similarity distance between the content to be identified and the content sample is determined based on the first feature vector and the second feature vector of the content sample. If at least one content sample among the first T content samples has a similarity distance less than or equal to a similarity distance threshold, then that at least one content sample is taken as the M content samples.

8. The method according to claim 1, characterized in that, After obtaining the first feature vector corresponding to the content to be identified, the method further includes: If no content sample meets the similarity criteria, a new category identifier is generated for the content to be identified. Based on the mapping relationship between the first feature vector and the content to be identified, the identifier of the content to be identified and the first feature vector are stored in the first database, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified; Based on the mapping relationship between the newly added category identifier and the content to be identified, the identifier of the content to be identified and the newly added category identifier are stored in the second database.

9. The method according to claim 1, characterized in that, The step of counting the number of content samples corresponding to each category identifier based on the category identifier of each of the M content samples includes: Based on the category identifier of each of the M content samples, content samples with the same category identifier are associated with the same category identifier to obtain the number of content samples corresponding to each category identifier.

10. The method according to claim 9, characterized in that, After determining the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier, the method further includes: Based on the mapping relationship between the first feature vector and the content to be identified, the identifier of the content to be identified and the first feature vector are stored in the first database, wherein the identifier of the content to be identified is used to uniquely identify the content to be identified; Based on the mapping relationship between the target category identifier and the content to be identified, the identifier of the content to be identified and the target category identifier are stored in the second database.

11. The method according to claim 9, characterized in that, The step of determining the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier includes: Determine the number of content samples associated with each category identifier in at least one of the category identifiers; The category identifier corresponding to the maximum number of content samples is used as the target category identifier of the content to be identified.

12. The method according to claim 9, characterized in that, The step of determining the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier includes: Determine the number of content samples associated with each category identifier in at least one of the category identifiers; If at least two category identifiers are associated with the same number of content samples, then the average ranking corresponding to each category identifier is calculated based on the content samples associated with each of the at least two category identifiers and the recall similarity ranking of the content samples. The category identifier corresponding to the highest average ranking is used as the target category identifier for the content to be identified.

13. The method according to claim 9, characterized in that, The step of determining the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier includes: Determine the number of content samples associated with each category identifier in at least one of the category identifiers; If at least two category identifiers are associated with the same number of content samples, then the average feature vector corresponding to each category identifier is calculated based on the content samples associated with each of the at least two category identifiers. The target category identifier for the content to be identified is determined based on the similarity between the average feature vector corresponding to each category identifier and the first feature vector.

14. The method according to claim 9, characterized in that, The step of determining the target category label corresponding to the content to be identified based on the target category identifier includes: In response to a labeling instruction for the target category identifier, determine the target category label corresponding to the content to be identified; The method further includes: The content samples associated with the target category identifier and the content to be identified are used as a training sample set, wherein the training sample set is labeled with the target category label; Obtain the content features and account information of each training sample in the training sample set; Obtain the clustering similarity features of the set of samples to be trained; Based on the clustering similarity features of the set of samples to be trained, the content features of each sample to be trained, and the account information, the probability distribution of the first category is obtained through a category recognition model. The model parameters of the category recognition model are updated based on the target category label and the probability distribution of the first category.

15. The method according to claim 9, characterized in that, The step of determining the target category label corresponding to the content to be identified based on the target category identifier includes: The content samples associated with the target category identifier and the content to be identified are used as a prediction sample set; Obtain the content features and account information of each prediction sample in the prediction sample set; Obtain the clustering similarity features of the predicted sample set; Based on the clustering similarity features of the predicted sample set, the content features of each predicted sample, and the account information, the probability distribution of the second category is obtained through a category recognition model; The target category label corresponding to the content to be identified is determined based on the second category probability distribution.

16. The method according to any one of claims 1 to 15, characterized in that, The method further includes; If the content to be identified is text content, then feature extraction is performed on the content to be identified to obtain the first feature vector; or, The method further includes: If the content to be identified is image content, then the content to be identified is encoded by an encoder to obtain an image feature vector; The first feature vector is obtained by performing a nonlinear transformation on the image feature vector using a projection head.

17. A content recognition device, characterized in that, include: The acquisition module is used to acquire the first feature vector corresponding to the content to be identified; The acquisition module is further configured to recall M content samples that satisfy the similarity condition from the Q content samples based on the first feature vector and the second feature vector of each content sample in the Q content samples, wherein each content sample corresponds to a category identifier, M and Q are both integers greater than or equal to 1, and M is less than or equal to Q. The determination module is used to perform statistics on the content samples for each category identifier based on the category identifier of each content sample in the M content samples, and obtain the number of content samples corresponding to each category identifier; The acquisition module is further configured to determine the target category identifier corresponding to the content to be identified based on the number of content samples corresponding to each category identifier; and to determine the target category label corresponding to the content to be identified based on the target category identifier.

18. A computer device, characterized in that, include: Memory, processor, and bus system; The memory is used to store programs; The processor is configured to execute a program in the memory, and the processor is configured to execute the method of any one of claims 1 to 16 according to instructions in the program code; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

19. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1 to 16.

20. A computer program product, comprising a computer program and instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1 to 16.

Images