Method and apparatus for clustering videos

Abstract

Disclosed is a method and device for clustering videos, the method comprising: determining the title text of the video; determining the extended text for the title text based on the search click data associated with the video; determining at least two tag texts for clustering the video based on the title text and the extended text; clustering the at least two tag texts to obtain a clustering result for the video. By using the method for clustering videos provided by the present application, more texts for clustering videos are obtained based on the information in the stored search log, so that better clustering effects can be achieved based on richer semantic information. The clustering results of the video can provide users with more accurate recommendation information.

Description

Technical Field

[0001] The present application relates to the field of video search, and more specifically to a method and device for clustering videos. Background Art

[0002] With the rise of short video content, users can upload hundreds of thousands or even millions of short video content on the Internet every day. Short videos can be labeled and classification models can be trained according to a pre-established classification label system to achieve video classification. However, for hot videos (such as news videos), a large amount of video content may be uploaded in a short period of time. The pre-established classification label system is difficult to cover the information of emergencies, and there is no time to manually label the data. When performing unsupervised clustering on videos, due to the sparse semantic representation of video titles, the effect of clustering directly using video titles is not good. Summary of the invention

[0003] According to one aspect of the present application, a method for clustering videos is proposed, comprising: determining a title text of the video; determining an extended text for the title text based on search click data associated with the video; determining at least two tag texts for clustering the video based on the title text and the extended text; and clustering the at least two tag texts to obtain a clustering result for the video.

[0004] In some embodiments, the extended text includes at least one of an extended query string, an extended search text, and an extended title text, wherein the extended query string is used to search the video, the extended search text is obtained by performing a text search using the extended query string, and the extended title text is obtained by performing a text search using the title text.

[0005] In some embodiments, the extended text includes an extended query string, and based on search click data associated with the video, determining the extended text for the title text includes: determining at least one candidate query string based on the search click data, wherein the video is included in the search results of each candidate query string in the at least one candidate query string, and determining the extended query string using at least a portion of the at least one candidate query string.

[0006] In some embodiments, determining the extended query string using at least a portion of the at least one candidate query string includes: for each candidate query string in the at least one candidate query string, determining the click-through rate of the video in the search results of the candidate query string; and determining the extended query string using the candidate query string when the click-through rate of the video is greater than a first click-through rate threshold.

[0007] In some embodiments, the extended text includes an extended search text, and based on the search click data associated with the video, determining the extended text for the title text also includes: determining at least one candidate extended search text based on the text search results of the extended query string; and determining the extended search text using at least a portion of the at least one candidate extended search text.

[0008] In some embodiments, determining the extended search text using at least a portion of the at least one candidate extended search text includes: determining a click-through rate of each candidate extended search text in the at least one candidate extended search text, and determining the extended search text using the candidate extended search text if the click-through rate of the candidate extended search text is greater than a second click-through rate threshold.

[0009] In some embodiments, determining the extended search text using the candidate extended search text includes: determining the extended search text using at least one of a text title, a text abstract, and a text body of the candidate extended search text.

[0010] In some embodiments, the extended text includes an extended title text, and based on search click data associated with the video, determining the extended text for the title text includes: determining at least one candidate extended title text based on text search results of the title text of the video; and determining the extended title text using at least a portion of the at least one candidate extended title text.

[0011] In some embodiments, determining the extended title text using at least a portion of the at least one candidate extended title text includes: for each candidate extended title text in the at least one candidate extended title text, determining the degree of match between the candidate extended title text and the title text of the video, and when the degree of match is greater than a matching degree threshold, determining the extended title text using the candidate extended title text.

[0012] In some embodiments, clustering the at least two label texts to obtain a clustering result for the video includes: numerically processing the at least two label texts to determine at least two text vectors corresponding to the label texts respectively; clustering the at least two text vectors to obtain at least one category for the at least two label texts; and determining a clustering result for the video based on the at least one category.

[0013] In some embodiments, the at least two text vectors are clustered to obtain at least one category for the at least two label texts: the at least two text vectors are clustered based on the distance between the text vectors in the at least two text vectors to obtain the at least one category.

[0014] In some embodiments, clustering the at least two label texts to obtain a clustering result for the video includes: for each of the at least two label texts, segmenting the label text to obtain at least two label words in the label text; determining a first conditional probability that each label text belongs to a predetermined category; determining a second conditional probability that each category contains the label word; determining a probability that each label text belongs to a predetermined category based on the first conditional probability and the second conditional probability; clustering the label texts based on the probability that each label text belongs to the predetermined category to obtain a clustering result for the video.

[0015] In some embodiments, determining the probability that each label text belongs to a predetermined category based on the first conditional probability and the second conditional probability includes: determining the confidence of each label word based on the text type to which the label word belongs, the text type including at least one of title text, extended query string, extended search text and extended title text; adjusting the first conditional probability and the second conditional probability based on the confidence to obtain an adjusted first conditional probability and an adjusted second conditional probability; determining the probability that each label text belongs to a predetermined category based on the adjusted first conditional probability and the adjusted second conditional probability.

[0016] According to another aspect of the present application, a device for clustering videos is also provided, including: a title text determination unit, configured to determine the title text of the video; an extended text determination unit, configured to determine the extended text for the title text based on search click data associated with the video; a tag text determination unit, configured to determine at least two tag texts for clustering the video based on the title text and the extended text; and a clustering unit, configured to cluster the at least two tag texts to obtain a clustering result for the video.

[0017] In some embodiments, the extended text includes at least one of an extended query string, an extended search text, and an extended title text, wherein the extended query string is used to search the video, the extended search text is obtained by performing a text search using the extended query string, and the extended title text is obtained by performing a text search using the title text.

[0018] In some embodiments, the extended text includes an extended query string, and based on search click data associated with the video, determining the extended text for the title text includes: determining at least one candidate query string based on the search click data, wherein the video is included in the search results of each candidate query string in the at least one candidate query string, and determining the extended query string using at least a portion of the at least one candidate query string.

[0019] In some embodiments, determining the extended query string using at least a portion of the at least one candidate query string includes: for each candidate query string in the at least one candidate query string, determining the click-through rate of the video in the search results of the candidate query string; and determining the extended query string using the candidate query string when the click-through rate of the video is greater than a first click-through rate threshold.

[0020] In some embodiments, the extended text includes an extended search text, and based on the search click data associated with the video, determining the extended text for the title text also includes: determining at least one candidate extended search text based on the text search results of the extended query string; and determining the extended search text using at least a portion of the at least one candidate extended search text.

[0021] In some embodiments, determining the extended search text using at least a portion of the at least one candidate extended search text includes: determining a click-through rate of each candidate extended search text in the at least one candidate extended search text, and determining the extended search text using the candidate extended search text if the click-through rate of the candidate extended search text is greater than a second click-through rate threshold.

[0022] In some embodiments, determining the extended search text using the candidate extended search text includes: determining the extended search text using at least one of a text title, a text abstract, and a text body of the candidate extended search text.

[0023] In some embodiments, the extended text includes an extended title text, and based on search click data associated with the video, determining the extended text for the title text includes: determining at least one candidate extended title text based on text search results of the title text of the video; and determining the extended title text using at least a portion of the at least one candidate extended title text.

[0024] In some embodiments, determining the extended title text using at least a portion of the at least one candidate extended title text includes: for each candidate extended title text in the at least one candidate extended title text, determining the degree of match between the candidate extended title text and the title text of the video, and when the degree of match is greater than a matching degree threshold, determining the extended title text using the candidate extended title text.

[0025] In some embodiments, clustering the at least two label texts to obtain a clustering result for the video includes: numerically processing the at least two label texts to determine at least two text vectors corresponding to the label texts respectively; clustering the at least two text vectors to obtain at least one category for the at least two label texts; and determining a clustering result for the video based on the at least one category.

[0026] In some embodiments, the at least two text vectors are clustered to obtain at least one category for the at least two label texts: the at least two text vectors are clustered based on the distance between the text vectors in the at least two text vectors to obtain the at least one category.

[0027] In some embodiments, clustering the at least two label texts to obtain a clustering result for the video includes: for each of the at least two label texts, segmenting the label text to obtain at least two label words in the label text; determining a first conditional probability that each label text belongs to a predetermined category; determining a second conditional probability that each category contains the label word; determining a probability that each label text belongs to a predetermined category based on the first conditional probability and the second conditional probability; clustering the label texts based on the probability that each label text belongs to the predetermined category to obtain a clustering result for the video.

[0028] In some embodiments, determining the probability that each label text belongs to a predetermined category based on the first conditional probability and the second conditional probability includes: determining the confidence of each label word based on the text type to which the label word belongs, the text type including at least one of title text, extended query string, extended search text and extended title text; adjusting the first conditional probability and the second conditional probability based on the confidence to obtain an adjusted first conditional probability and an adjusted second conditional probability; determining the probability that each label text belongs to a predetermined category based on the adjusted first conditional probability and the adjusted second conditional probability.

[0029] According to another aspect of the present application, a device for clustering videos is also provided, the device comprising a memory and a processor, wherein the memory stores instructions, and when the instructions are executed by the processor, the processor executes the method for clustering videos as described above.

[0030] According to another aspect of the present application, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the processor executes the method for clustering videos as described above.

[0031] By using the method and device for clustering videos provided by the present application, the videos can be clustered using the title text of the videos and the extended text generated based on the search click data of the title text. The extended text generated based on the search click data of the title text includes content related to the video, so it can make up for the problem of poor clustering effect caused by the short title text of the original video, and achieve better clustering results. By obtaining more text for clustering videos based on the information in the stored search log, better clustering effects can be achieved based on richer semantic information. Furthermore, the clustering results of the videos can provide users with more accurate recommendation information. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The following drawings are not deliberately scaled according to the actual size, and the focus is on illustrating the main purpose of the present application.

[0033] Figure 1 An exemplary scene diagram of a system for clustering videos according to the present application is shown;

[0034] Figure 2 A schematic flow chart of a method for clustering videos according to the principles of the present application is shown;

[0035] Figure 3 The process of clustering label texts using the K-means clustering algorithm according to the present application is shown;

[0036] Figure 4 A schematic diagram showing a process of using an LDA clustering algorithm according to an embodiment of the present application is shown;

[0037] Figure 5 An exemplary process of determining an extended query string according to an embodiment of the present application is shown;

[0038] Figure 6 An exemplary process of determining an extended search text according to an embodiment of the present application is shown;

[0039] Figure 7 An exemplary process of determining an extended title text according to an embodiment of the present application is shown;

[0040] Figure 8 A schematic block diagram of an apparatus for clustering videos according to an embodiment of the present application is shown;

[0041] Figure 9A-9C An example of determining an extended text according to an embodiment of the present application is shown; and

[0042] Fig.10 The architecture of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0044] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0045] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0046] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0047] Figure 1 An exemplary scene diagram of a system for clustering videos according to the present application is shown. Figure 1 As shown, the system 100 may include a user terminal 110 , a network 120 , a server 130 , and a database 140 .

[0048] The user terminal 110 may be, for example, Figure 1 It is understood that, in fact, the user terminal can be any other type of electronic device capable of performing data processing, which can include but is not limited to desktop computers, laptops, tablet computers, smart phones, smart home devices, wearable devices, etc.

[0049] The user terminal provided by the present application can be used to receive the video to be clustered. For example, the user terminal can collect the video to be clustered through an image acquisition device (such as a camera, a video camera, etc.) set on the user terminal. For another example, the user terminal can also obtain the video to be clustered from an independently set image acquisition device (such as a camera, a video camera, a scanner, etc.). For another example, the user terminal can also receive the video to be clustered from a server or any other electronic device via a network.

[0050] The network 120 may be a single network or a combination of at least two different networks. For example, the network 120 may include, but is not limited to, a local area network, a wide area network, a public network, a private network, etc., or a combination of several of them.

[0051] The server 130 may be a single server or a server group, and each server in the group is connected via a wired or wireless network. A server group may be centralized, such as a data center, or distributed. The server 130 may be local or remote. The server 130 may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The user terminal 110 and the server 130 may be directly or indirectly connected via wired or wireless communications, and this application is not limited here. The user terminal may send the video to be clustered to the server 130 via the network 120, and the server 130 may execute the method for clustering the video provided in this application. In some implementations, the server 130 may perform clustering using an application built into the server. In other implementations, the server 130 may perform clustering by calling an application stored externally on the server.

[0052] The database 140 may generally refer to a device with a storage function. The database 140 is mainly used to store various data used, generated and output from the user terminal 110 and the server 130. For example, the database 140 may be used to store videos to be clustered and various search click data associated with the videos (e.g., search click logs). The database 140 may be local or remote. The database 140 may include various memories, such as random access memory (RAM), read only memory (ROM), etc. The storage devices mentioned above are just some examples, and the storage devices that can be used by the system are not limited thereto.

[0053] The database 140 may be connected or communicated with the server 130 or a part thereof via the network 120 , or directly connected or communicated with the server 130 , or a combination of the above two methods.

[0054] In some embodiments, the database 140 may be an independent device. In other embodiments, the database 140 may also be integrated in at least one of the user terminal 110 and the server 130. For example, the database 140 may be set on the user terminal 110 or on the server 130. For another example, the database 140 may also be distributed, with a portion of it being set on the user terminal 110 and another portion being set on the server 130.

[0055] In some methods, since the title text of a video may include text that can indicate the content of the video, clustering of the video can be achieved accordingly by clustering the title text of the video. However, since the length of the video title is short and the semantic representation is sparse, the effect of directly using a clustering model on the title text is not good.

[0056] The process of the method for clustering videos provided by the present application will be described in detail below.

[0057] In the technical solution provided in the present application, artificial intelligence methods can be used to process the title text of the video and the extended text determined based on the title text to achieve clustering of the video.

[0058] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0059] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0060] The method provided in this application relates to the technology of natural language processing in artificial intelligence, including text processing, semantic understanding, etc. The following will be specifically described in conjunction with the embodiments of this application.

[0061] In some embodiments, cloud technology can be used to implement the method for clustering videos provided in this application. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks in a wide area network or a local area network to achieve data calculation, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the application of cloud computing business model. It can form a resource pool, which is used on demand and is flexible and convenient. The background service of the technical network system requires a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. By integrating a large amount of resources and data in the network, the data required in the method for clustering videos to be implemented in this application can be obtained.

[0062] Figure 2 A schematic flow chart of a method for clustering videos according to the principles of the present application is shown.

[0063] In step S202, the title text of the video may be determined. The video in step S202 may include a video uploaded to the network platform within a predetermined time period. The predetermined time period mentioned here may be one hour, one day, one week, or any predefined time period. The title text of the video may include text that can indicate the content of the video.

[0064] In some embodiments, the title text may be text stored in association with the video. In some implementations, the title text may be text entered or selected by the user when uploading the video. In other implementations, the title text may be text generated by performing speech recognition on the audio in the video content.

[0065] In step S204, the extended text for the title text can be determined based on the search click data associated with the video. In the process of video clustering, the search click data associated with the video (such as search click log) can be obtained by using the content of big data. And the extended text for the title text can be determined by mining the massive information behind the big data.

[0066] Big data refers to a collection of data that cannot be captured, managed, and processed by conventional software tools within a certain time frame. It is a massive, high-growth, and diverse information asset that requires new processing models to have stronger decision-making power, insight discovery, and process optimization capabilities. With the advent of the cloud era, big data has also attracted more and more attention. Big data requires special technologies to effectively process large amounts of data within a tolerable time frame. Technologies applicable to big data include large-scale parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

[0067] In some embodiments, the search click data associated with the video may include the association relationship between the query string associated with the video and the search results. For example, when a user searches for a video in a video search engine, the association relationship between the query string input by the user and the video search results generated by the video search engine can be determined. For another example, when a user searches for a video in a text search engine, the association relationship between the query string input by the user and the video search results generated by the text search engine can be determined. The association relationship between the above query string and the search results can be stored in a database. In addition, the search click data associated with the video may also include click data of the search results, i.e., data indicating whether the user clicks on the search results. The click data may be used to indicate the association between the search results and the query string. If the click data indicates that the click rate of a certain search result is higher, it can be considered that the correlation between the search result and the query string is higher.

[0068] In some embodiments, the extended text may include at least one of an extended query string, an extended search text, and an extended title text, wherein the extended query string is used to search for videos, the extended search text is obtained by performing a text search using the extended query string, and the extended title text is obtained by performing a text search using the title text.

[0069] The following will combine Figure 5-Figure 7 The processes of determining the extended query string, the extended search text, and the extended title text are described separately and will not be repeated here.

[0070] In step S206, at least two label texts for clustering videos may be determined based on the title text determined in step S202 and the extended text determined in step S204.

[0071] As previously mentioned, the extended text determined in step S204 is the text associated with the video and the title text of the video. The extended text can indicate the content of the video at least to a certain extent. Therefore, the title text and the extended text can be used for the clustering process to be described next in the application at the same time to realize the clustering for the video.

[0072] In some embodiments, the title text and the extended text may be directly combined into the label text. In other embodiments, the title text and the extended text may be expanded using a knowledge graph. For example, the knowledge graph may be used to determine the contextual knowledge, synonym knowledge, near-synonymous knowledge, associative entity, and other knowledge graph extended texts of the words in the label text. Then, the title text, the extended text, and the knowledge graph extended text may be combined to form the label text.

[0073] In step S208, at least two label texts may be clustered to obtain a clustering result for the video. By applying a clustering algorithm to at least two label texts corresponding to each video to be clustered, a clustering result of the label text may be obtained. As described above, for each of the at least two videos to be clustered, the label text of the video may be determined based on steps S202 to S206. The label text of the video may indicate the content of the video. Therefore, the clustering result of the label text may be determined as the clustering result of the video.

[0074] The label text can be processed using a natural language processing method to convert the label text into a form that can be processed by a computer. For example, the label text can be converted using a trained text conversion model to obtain numerical data (such as a text vector) that can be processed by a computer. For another example, the label text can also be segmented using a trained word segmentation model. The above-mentioned natural language processing models such as text conversion models and word segmentation models can be obtained using any pre-established vocabulary. Different vocabulary libraries can be established for different application scenarios to obtain better results. For example, for the video clustering process of news, the label text can be processed using a natural language processing model trained based on a vocabulary library established based on news vocabulary. For another example, for the video clustering process of law, the label text can be processed using a natural language processing model trained based on a vocabulary library established based on legal vocabulary.

[0075] Any clustering algorithm can be used to process the label text to obtain the clustering result. Figure 3 , Figure 4 The principles of the present application are described by taking the K-means clustering algorithm and the LDA (Latent Dirichlet Allocation) clustering algorithm as examples. However, it is understandable that those skilled in the art can implement the method of the present application using any clustering algorithm without departing from the principles of the present application.

[0076] By using the method for clustering videos provided by the present application, the title text content of the video can be extended to obtain an extended text, wherein the extended text includes text indicating the video content. Then, clustering can be performed based on both the title text and the extended text to obtain a clustering result of the video. Since the extended text includes more text related to the video content, a better clustering effect can be obtained by using the method for clustering videos provided by the present application.

[0077] Figure 3 The process of clustering label text using the K-means clustering algorithm according to the present application is shown. Figure 3 The process shown in Figure 2 Step S208 in .

[0078] In step S302, at least two label texts corresponding to the videos to be clustered may be numerically processed to determine at least two text vectors corresponding to each label text. When using a computer to cluster the label texts, since the computer is often unable to directly process text data, when processing the label texts, the label texts need to be converted into numerical data first.

[0079] In some embodiments, each label text may be numerically processed using a trained doc2vec model. The doc2vec model may be used to generate a fixed-length feature representation from variable-length (i.e., indefinite-length) text (such as sentences, paragraphs, documents, etc.) as a text vector for the label text.

[0080] In other embodiments, for each label text in at least two label texts, the label text can be segmented to obtain at least two label words in the label text. Then, each label word can be numerically processed to obtain a word vector corresponding to each label word. For example, each label word can be numerically processed using a trained word2vec model. The word2vec model can be used to generate a fixed-length feature representation from a variable-length word as a text vector for a label word. At least two text vectors corresponding to the label text can be determined based on the word vectors of the label words in at least two label texts. For example, a text vector for the label text can be formed by combining the word vectors of the label words of the label text.

[0081] In step S304, at least two text vectors may be clustered to obtain at least one category for the corresponding at least two label texts.

[0082] In some embodiments, at least two text vectors may be clustered based on a distance between the text vectors in the at least two text vectors to obtain at least one category.

[0083] For example, in the K-means clustering algorithm, the objects to be clustered (text vectors in this application) can be pre-divided into K groups, where K is a preset positive integer. K objects can be randomly selected as initial cluster centers. Then, the distance between each object to be clustered and each cluster center can be calculated, and each object can be assigned to the cluster center with the nearest distance. For each cluster, each time an object is assigned, the cluster center of the cluster needs to be recalculated based on the distance between the objects included in the category. The above process will be repeated until the termination condition is met. In some implementations, the termination condition can be that no (or less than the minimum number) objects are reallocated to different clusters, or no (or less than the minimum number) cluster centers change, or the sum of squared errors of each cluster is locally minimized.

[0084] At least two text vectors may be clustered into K different categories using a K-means clustering algorithm, wherein each category includes at least one text vector. The category to which the text vector belongs may be determined as the category to which the label text belongs.

[0085] In step S306, a clustering result for the video may be determined based on the at least one category determined in step S304. As described above, each tag text indicates the video content in a corresponding video. Therefore, the at least one category determined for the tag text may be determined as the clustering result of the video.

[0086] Figure 4 A schematic diagram showing a process of using an LDA clustering algorithm according to an embodiment of the present application is shown.

[0087] In step S402, for each of the at least two label texts, the label text may be segmented to obtain at least two label words in the label text.

[0088] In step S404, the first conditional probability that each label text belongs to a predetermined k category (k is a positive integer) can be determined. Each predetermined category is composed of at least two category words corresponding to the category. In some embodiments, the probability that the label text belongs to the i-th category (i is a positive integer, and 1≤i≤k) can be determined using formula (1):

[0089] p ti =n ti / n (1)

[0090] where p ti Indicates the probability that the label text belongs to the i-th category, n ti It represents the number of category words belonging to the i-th category in the label text, and n represents the number of all words in the label text.

[0091] In step S406, for each label word determined in step S402, a second conditional probability that each category contains the label word can be determined. In some embodiments, the probability that the jth label word (j is a positive integer, where 1≤j≤Z, and Z is the total number of label words) belongs to the ith category can be determined using formula (2):

[0092] p wj =N wj / N (2)

[0093] where p wj represents the probability that the jth label word belongs to the i-th category, N wj represents the number of category words corresponding to the ith category for the j-th label word contained in different label texts, and N represents the total number of category words corresponding to the ith category.

[0094] In step S408 , the probability that each label text belongs to a predetermined category may be determined based on the first conditional probability and the second conditional probability.

[0095] Formulas (1) and (2) can be used to calculate the conditional probability p(w|d) that each label word belongs to the label text when it corresponds to a predetermined category, where p(w|d) = p ti *p wj .

[0096] In the LDA clustering process, we can first randomly assign p to all label texts and categories. ti and p wj Assign values, and then update the category to which each label word belongs by calculating the conditional probability p(w|d) of each label word. Continuously iterate and calculate the p of all label texts and categories. ti and p wj And update the conditional probability p(w|d) of each label word. When a predetermined number of iterations have passed or the result of the conditional probability p(w|d) meets the convergence condition, the converged conditional probability p(w|d) can be obtained and the probability of each label word belonging to each category can be calculated using the above method. The probability of each label text belonging to each category can also be calculated based on the principle of conditional probability.

[0097] In step S410, each label text may be clustered based on the probability that the label text belongs to a predetermined category to obtain a clustering result for the video.

[0098] In some embodiments, the probability that each label text belongs to the predetermined k categories can be used to assign the label text to the category corresponding to the highest probability, thereby determining the category to which each label text belongs and implementing clustering of the label text.

[0099] In some embodiments, the confidence of each label word can be determined based on the text type to which the label word belongs. The first conditional probability and the second conditional probability are adjusted based on the confidence to obtain the adjusted first conditional probability and the adjusted second conditional probability text type. Then, the probability that each label text belongs to a predetermined category can be determined based on the adjusted first conditional probability and the adjusted second conditional probability.

[0100] The text type may include at least one of the title text, the extended query string, the extended search text, and the extended title text. In some implementations, when the label words belong to different text types, different confidences may be set for the label words. For example, for the same word, when the word belongs to the title text, its confidence may be set to 1. When the word belongs to the extended query string, its confidence may be set to 0.8. When the word belongs to the extended search text, its confidence may be set to 0.7. When the word belongs to the extended title text, its confidence may be set to 0.9. It is understandable that those skilled in the art may arbitrarily set the confidences of words belonging to different text types according to actual conditions. The actual value of the confidence is not limited here. When a word belongs to at least two different text types at the same time, the word confidences corresponding to the different text types may be weighted averaged to obtain the confidence of the word.

[0101] The confidence of the tag words can be used to adjust the first conditional probability and the second conditional probability determined in step S404 and step S406. For example, the number of words n in formula (1) and formula (2) can be adjusted based on the confidence. ti and N wj For example, in determining the number of words n in formula (1) and formula (2), ti and N wj When the confidence level of a word m is A, its number can be calculated as A*M, where M is the number of statistical words n. ti or N wj The actual statistical frequency of word m is . In other words, the statistical frequency of words in the LDA clustering algorithm can be adjusted based on the confidence level.

[0102] The probability p(w|d) that each label text belongs to a predetermined category can be determined based on the adjusted first conditional probability and the adjusted second conditional probability, where p(w|d)=p' ti *p' wj , where p' ti represents the adjusted first conditional probability, p' wj represents the adjusted second conditional probability.

[0103] In step S410, the label texts may be clustered based on the probability that each label text belongs to a predetermined category to obtain a clustering result for the video. In some embodiments, when the probability that the label text belongs to a predetermined category is greater than a predefined probability threshold, it may be determined that the label text belongs to the predetermined category.

[0104] Figure 5 An exemplary process of determining an extended query string according to an embodiment of the present application is shown.

[0105] In step S502, at least one candidate query string may be determined based on search click data associated with the videos to be clustered.

[0106] In some embodiments, query strings whose search results include videos may be determined as candidate query strings.

[0107] In other embodiments, the query string including the click data of clicking on the video in the search results can be determined as the candidate query string. In other words, the video to be clustered is clicked under the candidate query string.

[0108] In step S504, an extended query string may be determined using at least a portion of at least one candidate query string.

[0109] In some embodiments, whether to use the candidate query string to determine the expanded query string may be determined based on the click-through rate of the video in the search results of the candidate query string.

[0110] In some implementations, for each candidate query string in at least one candidate query string, the click rate of the video in the search results of the candidate query string can be determined. In the case where the click rate of the video is greater than a first click rate threshold, the candidate query string can be used to determine the extended query string. The click rate of the video can be defined as the ratio of the number of times the video is clicked to the number of times the query string is queried, that is, the click rate of the video = the number of times the video is clicked / the number of times the query string is queried.

[0111] For example, the first click rate threshold may be 70%. It is understandable that those skilled in the art may arbitrarily set the value of the first click rate threshold according to actual conditions.

[0112] In some other embodiments, whether to use the candidate query string to determine the expanded query string may be determined based on the number of clicks on the video in the search results of the candidate query string.

[0113] In some implementations, for each candidate query string in at least one candidate query string, the number of clicks on the video in the search results of the candidate query string can be determined. In the case where the number of clicks on the video is greater than a click number threshold, the candidate query string can be used to determine an extended query string. For example, the click number threshold can be 200. It is understood that those skilled in the art can arbitrarily set the value of the click number threshold according to actual conditions.

[0114] In other embodiments, whether to use the candidate query string to determine the expanded query string may be determined based on the ranking of the click-through rate or the number of clicks of the video in the search results of the candidate query string.

[0115] In some implementations, for each candidate query string in at least one candidate query string, the ranking of the click-through rate or number of clicks of the video in the search results of the candidate query string can be determined. In the case where the ranking of the click-through rate or number of clicks of the video is greater than the ranking threshold, the candidate query string can be used to determine the extended query string. For example, the ranking threshold can be 20. In other words, the query string whose click-through rate or number of clicks of the video in the search results ranks in the top 20 can be determined as the extended query string of the video. It is understandable that those skilled in the art can arbitrarily set the numerical value of the ranking threshold according to actual conditions.

[0116] Figure 6 An exemplary process of determining an expanded search text according to an embodiment of the present application is shown.

[0117] In step S602, at least one candidate expanded search text may be determined based on the text search results of the expanded query string.

[0118] In some embodiments, the text search engine may be used to enter Figure 5 The process shown in FIG. 1 determines the expanded query string to obtain the text search result of the expanded query string. At least one text included in the text search result of the expanded query string can be determined as a candidate expanded search text.

[0119] In step S604, an extended search text may be determined using at least a portion of at least one candidate extended search text.

[0120] In some embodiments, whether to use the candidate expanded search text to determine the expanded search text may be determined based on the click rate of the candidate expanded search text in the text search results of the expanded query string.

[0121] In some implementations, for each candidate extended search text in at least one candidate extended search text, a click rate of the candidate extended search text may be determined. When the click rate of the candidate extended search text is greater than a second click rate threshold, the candidate extended search text may be used to determine the extended search text. The click rate of the candidate extended search text may be defined as the ratio of the number of times the candidate extended search text is clicked to the number of times the candidate query string is queried, that is, the click rate of the candidate extended search text = the number of times the candidate extended search text is clicked / the number of times the candidate query string is queried.

[0122] For example, the second click rate threshold may be 70%. It is understandable that those skilled in the art may arbitrarily set the value of the second click rate threshold according to actual conditions. The second click rate threshold and the first click rate threshold may be the same or different.

[0123] In some other embodiments, whether to use the candidate expanded search text to determine the expanded search text may be determined based on the number of clicks on the candidate expanded search text in the text search results of the expanded query string.

[0124] In some implementations, for each candidate extended search text in at least one candidate extended search text, the number of clicks on the candidate extended search text may be determined. In the case where the number of clicks on the candidate extended search text is greater than a click number threshold, the candidate extended search text may be determined as an extended query string. For example, the click number threshold may be 200. It is understood that those skilled in the art may arbitrarily set the value of the click number threshold according to actual conditions.

[0125] In some other embodiments, whether to use the candidate expanded search text to determine the expanded search text may be determined based on the ranking of the click rate or the number of clicks of the candidate expanded search text in the text search results of the expanded query string.

[0126] In some implementations, for each candidate extended search text in at least one candidate extended search text, the ranking of the click rate or number of clicks of the candidate extended search text can be determined. In the case where the ranking of the click rate or number of clicks of the candidate extended search text is greater than the ranking threshold, the candidate extended search text can be used to determine the extended search text. In other words, the query string whose ranking of the click rate or number of clicks of the candidate extended search text in the text search results based on the extended query string is in the top 20 can be determined as the extended query string of the video. It is understandable that those skilled in the art can arbitrarily set the numerical value of the ranking threshold according to actual conditions.

[0127] In some embodiments, determining the extended search text using the candidate extended search text includes: determining the extended search text using at least one of a text title, a text abstract, and a text body of the candidate extended search text.

[0128] Figure 7 An exemplary process of determining extended title text according to an embodiment of the present application is shown.

[0129] In step S702, at least one candidate extended title text may be determined based on the text search results of the title text of the video. In some embodiments, the title text of the video may be input into a text search engine to obtain a text search result of the title text of the video. At least one text included in the text search results of the title text may be determined as a candidate extended title text.

[0130] In step S704, the extended title text may be determined using at least a portion of at least one candidate extended title text.

[0131] In some embodiments, for each candidate extended title text in at least one candidate extended title text, the degree of match between the candidate extended title text and the title text of the video is determined. For example, the hash values ​​of the title text and the candidate extended title text can be calculated, and the degree of match between the candidate extended title text and the title text of the video can be determined based on the similarity between the hash values ​​of the title text and the candidate extended title text. In some implementations, when the degree of match is greater than a matching threshold, the extended title text is determined using the candidate extended title text. In other implementations, the candidate extended title texts can be sorted based on matching pairs, and the candidate extended title texts whose ranking is greater than a ranking threshold are determined as extended title texts.

[0132] Figure 8 FIG. 2 shows a schematic block diagram of a device for clustering videos according to an embodiment of the present application. Figure 8 As shown, the apparatus 800 may include a title text determination unit 810 , an extended text determination unit 820 , a tag text determination unit 830 , and a clustering unit 840 .

[0133] The title text determining unit 810 may be configured to determine the title text of the video. The video to be clustered may include at least two videos uploaded to the network platform within a predetermined time period. The predetermined time period mentioned here may be one hour, one day, one week, or any predefined time period. The title text of the video may include text that can indicate the content of the video.

[0134] In some embodiments, the title text may be text stored in association with the video. In some implementations, the title text may be text entered or selected by the user when uploading the video. In other implementations, the title text may be text generated by performing speech recognition on the audio in the video content.

[0135] The extended text determination unit 820 may be configured to determine an extended text for the title text based on search click data associated with the video.

[0136] In some embodiments, the search click data associated with the video may include the association relationship between the query string associated with the video and the search results. For example, when a user searches for a video in a video search engine, the association relationship between the query string input by the user and the video search results generated by the video search engine can be determined. For another example, when a user searches for a video in a text search engine, the association relationship between the query string input by the user and the video search results generated by the text search engine can be determined. The association relationship between the above query string and the search results can be stored in a database. In addition, the search click data associated with the video may also include click data of the search results, i.e., data indicating whether the user clicks on the search results. The click data may be used to indicate the association between the search results and the query string. If the click data indicates that the click rate of a certain search result is higher, it can be considered that the correlation between the search result and the query string is higher.

[0137] In some embodiments, the extended text may include at least one of an extended query string, an extended search text, and an extended title text, wherein the extended query string is used to search for videos, the extended search text is obtained by performing a text search using the extended query string, and the extended title text is obtained by performing a text search using the title text.

[0138] like Figure 8 As shown, the extended text determining unit 820 may include an extended query string determining unit 821 , an extended search text determining unit 822 , and an extended title text determining unit 823 .

[0139] The expanded query string determination unit 821 may be configured to determine an expanded query string based on search click data associated with the videos to be clustered.

[0140] The expanded query string determination unit 821 may be configured to determine at least one candidate query string based on search click data associated with the videos to be clustered.

[0141] In some embodiments, query strings whose search results include videos may be determined as candidate query strings.

[0142] In other embodiments, the query string including the click data of clicking on the video in the search results can be determined as the candidate query string. In other words, the video to be clustered is clicked under the candidate query string.

[0143] Further, the expanded query string determining unit 821 may be configured to determine an expanded query string using at least a portion of at least one candidate query string.

[0144] In some embodiments, whether to use the candidate query string to determine the expanded query string may be determined based on the click-through rate of the video in the search results of the candidate query string.

[0145] In some implementations, for each candidate query string in at least one candidate query string, the click rate of the video in the search results of the candidate query string can be determined. In the case where the click rate of the video is greater than a first click rate threshold, the candidate query string can be used to determine the extended query string. The click rate of the video can be defined as the ratio of the number of times the video is clicked to the number of times the query string is queried, that is, the click rate of the video = the number of times the video is clicked / the number of times the query string is queried.

[0146] For example, the first click rate threshold may be 70%. It is understandable that those skilled in the art may arbitrarily set the value of the first click rate threshold according to actual conditions.

[0147] In some other embodiments, whether to use the candidate query string to determine the expanded query string may be determined based on the number of clicks on the video in the search results of the candidate query string.

[0148] In some implementations, for each candidate query string in at least one candidate query string, the number of clicks on the video in the search results of the candidate query string can be determined. In the case where the number of clicks on the video is greater than a click number threshold, the candidate query string can be used to determine an extended query string. For example, the click number threshold can be 200. It is understood that those skilled in the art can arbitrarily set the value of the click number threshold according to actual conditions.

[0149] In other embodiments, whether to use the candidate query string to determine the expanded query string may be determined based on the ranking of the click-through rate or the number of clicks of the video in the search results of the candidate query string.

[0150] In some implementations, for each candidate query string in at least one candidate query string, the ranking of the click-through rate or number of clicks of the video in the search results of the candidate query string can be determined. In the case where the ranking of the click-through rate or number of clicks of the video is greater than the ranking threshold, the candidate query string can be used to determine the extended query string. For example, the ranking threshold can be 20. In other words, the query string whose click-through rate or number of clicks of the video in the search results ranks in the top 20 can be determined as the extended query string of the video. It is understandable that those skilled in the art can arbitrarily set the numerical value of the ranking threshold according to actual conditions.

[0151] The extended search text determination unit 822 may be configured to determine an extended search text for the title text based on search click data associated with the video.

[0152] The expanded search text determining unit 822 may be configured to determine at least one candidate expanded search text based on the text search result of the expanded query string.

[0153] In some embodiments, the text search engine may be used to enter Figure 5 The process shown in FIG. 1 determines the expanded query string to obtain the text search result of the expanded query string. At least one text included in the text search result of the expanded query string can be determined as a candidate expanded search text.

[0154] Further, the expanded search text determining unit 822 may be configured to determine an expanded search text using at least a portion of at least one candidate expanded search text.

[0155] In some embodiments, whether to use the candidate expanded search text to determine the expanded search text may be determined based on the click rate of the candidate expanded search text in the text search results of the expanded query string.

[0156] In some implementations, for each candidate extended search text in at least one candidate extended search text, a click rate of the candidate extended search text may be determined. When the click rate of the candidate extended search text is greater than a second click rate threshold, the candidate extended search text may be used to determine the extended search text. The click rate of the candidate extended search text may be defined as the ratio of the number of times the candidate extended search text is clicked to the number of times the candidate query string is queried, that is, the click rate of the candidate extended search text = the number of times the candidate extended search text is clicked / the number of times the candidate query string is queried.

[0157] For example, the second click rate threshold may be 70%. It is understandable that those skilled in the art may arbitrarily set the value of the second click rate threshold according to actual conditions. The second click rate threshold and the first click rate threshold may be the same or different.

[0158] In some other embodiments, whether to use the candidate expanded search text to determine the expanded search text may be determined based on the number of clicks on the candidate expanded search text in the text search results of the expanded query string.

[0159] In some implementations, for each candidate extended search text in at least one candidate extended search text, the number of clicks on the candidate extended search text may be determined. In the case where the number of clicks on the candidate extended search text is greater than a click number threshold, the candidate extended search text may be determined as an extended query string. For example, the click number threshold may be 200. It is understood that those skilled in the art may arbitrarily set the value of the click number threshold according to actual conditions.

[0160] In some other embodiments, whether to use the candidate expanded search text to determine the expanded search text may be determined based on the ranking of the click rate or the number of clicks of the candidate expanded search text in the text search results of the expanded query string.

[0161] In some implementations, for each candidate extended search text in at least one candidate extended search text, the ranking of the click rate or number of clicks of the candidate extended search text can be determined. In the case where the ranking of the click rate or number of clicks of the candidate extended search text is greater than the ranking threshold, the candidate extended search text can be used to determine the extended search text. In other words, the query string whose ranking of the click rate or number of clicks of the candidate extended search text in the text search results based on the extended query string is in the top 20 can be determined as the extended query string of the video. It is understandable that those skilled in the art can arbitrarily set the numerical value of the ranking threshold according to actual conditions.

[0162] In some embodiments, determining the extended search text using the candidate extended search text includes: determining the extended search text using at least one of a text title, a text abstract, and a text body of the candidate extended search text.

[0163] The extended title text determination unit 823 may be configured to determine the extended title text based on the search click data associated with the video to be clustered.

[0164] The extended title text determining unit 823 may be configured to determine at least one candidate extended title text based on the text search results of the title text of the video. In some embodiments, the title text of the video may be input into a text search engine to obtain the text search results of the title text of the video. At least one text included in the text search results of the title text may be determined as a candidate extended title text.

[0165] Furthermore, the extended title text determining unit 823 may also be configured to determine the extended title text using at least a portion of at least one candidate extended title text.

[0166] In some embodiments, for each candidate extended title text in at least one candidate extended title text, the degree of match between the candidate extended title text and the title text of the video is determined. For example, the hash values ​​of the title text and the candidate extended title text can be calculated, and the degree of match between the candidate extended title text and the title text of the video can be determined based on the similarity between the hash values ​​of the title text and the candidate extended title text. In some implementations, when the degree of match is greater than a matching threshold, the extended title text is determined using the candidate extended title text. In other implementations, the candidate extended title texts can be sorted based on matching pairs, and the candidate extended title texts whose ranking is greater than a ranking threshold are determined as extended title texts.

[0167] The label text determination unit 830 may be configured to determine at least two label texts for clustering videos based on the title text determined by the title text determination unit 810 and the extended text determined by the extended text determination unit 820 .

[0168] As previously mentioned, the extended text determined by the extended text determining unit 820 is a text associated with the title text of the video and the video. The extended text can indicate the content of the video at least to a certain extent. Therefore, the title text and the extended text can be used for the clustering process to be described next in the application at the same time to realize the clustering for the video.

[0169] In some embodiments, the title text and the extended text may be directly combined into the label text. In other embodiments, the title text and the extended text may be expanded using a knowledge graph. For example, the knowledge graph may be used to determine the contextual knowledge, synonym knowledge, near-synonymous knowledge, associative entity, and other knowledge graph extended texts of the words in the label text. Then, the title text, the extended text, and the knowledge graph extended text may be combined to form the label text.

[0170] The clustering unit 840 can be configured to cluster at least two label texts to obtain a clustering result for the video. By applying a clustering algorithm to at least two label texts corresponding to each video to be clustered, a clustering result of the label text can be obtained. The label text of the video can indicate the video content. Therefore, the clustering result of the label text can be determined as the clustering result of the video.

[0171] Any clustering algorithm can be used to process the tag text to obtain a clustering result. In some embodiments, the clustering unit 840 can be configured to perform the clustering algorithm in combination with the present application. Figure 3 or Figure 4 The process described in derives the clustering result of the video.

[0172] By using the method for clustering videos provided by the present application, the title text content of the video can be extended to obtain an extended text, wherein the extended text includes text indicating the video content. Then, clustering can be performed based on both the title text and the extended text to obtain a clustering result of the video. Since the extended text includes more text related to the video content, a better clustering effect can be obtained by using the method for clustering videos provided by the present application.

[0173] Figure 9A-9C An example of determining extended text according to an embodiment of the present application is shown.

[0174] like Fig. 9AAs shown, for "query string 1" 901, the search results 902 obtained by using the video search engine include "video 1" 903, "video 2" 904, and "video 3" 905. The click rate of each video can be calculated based on the number of times each video is clicked and the number of times query string 1 is queried. Taking the first click rate threshold of 70% as an example, when the click rate of "video 1" is 20%, the click rate of "video 2" is 40%, and the click rate of "video 3" is 80%, it can be considered that the correlation between "query string 1" and "video 3" is higher. Therefore, "query string 1" can be determined as an extended query string for "video 3".

[0175] like Fig. 9B As shown, you can use Fig. 9A The extended query string "query string 1" determined in the above example is searched for text to obtain the extended search text for "video 3". For example, a text search engine can be used to search for "query string 1" and obtain the text search results for "query string 1". Fig. 9B In the example shown, the text search results for "query string 1" include "text 1" 905, "text 2" 906, and "text 3" 907. Taking the second click rate threshold of 70% as an example, when the click rate of "text 1" is 75%, the click rate of "text 2" is 60%, and the click rate of "text 3" is 15%, it can be considered that "text 1" and "query string 1" have a higher correlation. Therefore, "text 1" can be determined as the extended search text for "video 3".

[0176] like Fig. 9C As shown, the title text "Title 3" of "Video 3" can be used to perform a text search to obtain the extended title text for "Video 3". Fig. 9C In the example shown, the text search results for "Title 3" include "Text 4", "Text 5", and "Text 6". "Text 4", "Text 5", "Text 6", and "Title 3" can be sorted based on their matching degree, and the top two "Text 6" and "Text 4" can be determined as the extended title texts of "Video 3".

[0177] In addition, the method or device according to the embodiment of the present application can also be used by Fig.10 The architecture of the computing device shown is implemented. Fig.10 The architecture of the computing device is shown. Fig.10As shown, the computing device 1000 may include a bus 1010, one or at least two CPUs 1020, a read-only memory (ROM) 1030, a random access memory (RAM) 1040, a communication port 1050 connected to a network, an input / output component 1060, a hard disk 1070, etc. The storage device in the computing device 1000, such as ROM 1030 or hard disk 1070, can store various data or files used for processing and / or communication of the target detection method provided in the present application and program instructions executed by the CPU. The computing device 1000 may also include a user interface 1080. Of course, Fig.10 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Fig.10 One or at least two components of the computing device shown.

[0178] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer-readable instructions are stored. When the instructions are executed by a computer, the above-mentioned method can be executed.

[0179] The program part of the technology can be considered as a "product" or "manufactured product" in the form of executable code and / or related data, which is participated in or realized by computer-readable media. Tangible and permanent storage media can include any memory or storage used by any computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives or any similar devices that can provide storage functions for software.

[0180] All or part of the software may sometimes communicate over a network, such as the Internet or other communications network. Such communications can load software from one computer device or processor to another. For example: from a server or host computer of a video target detection device to a hardware platform of a computer environment, or other computer environment that implements the system, or a system with similar functions related to providing information required for target detection. Therefore, another medium capable of transmitting software elements can also be used as a physical connection between local devices, such as light waves, radio waves, electromagnetic waves, etc., transmitted through cables, optical cables or air. Physical media used to carry carriers, such as cables, wireless connections or optical cables and the like, can also be considered as media that carry software. As used herein, unless limited to tangible "storage" media, other terms referring to computer or machine "readable media" refer to media involved in the process of the processor executing any instructions.

[0181] The present application uses specific words to describe the embodiments of the present application. For example, "first / second embodiment", "one embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0182] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0183] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0184] The above is an explanation of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined in the claims. It should be understood that the above is an explanation of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.

Claims

1. A method for clustering videos, comprising: determining a title text for the video; Determining extended text for the title text based on search click data associated with the video; The extended text includes an extended search text and an extended title text; the extended search text is determined based on a text search result obtained by performing a text search using an extended query string, the extended title text is determined based on a text search result obtained by performing a text search using the title text, and the extended query string is used to search for the video; Determining at least two tag texts for clustering the videos based on the title text and the extended text; Perform word segmentation on the at least two label texts to obtain at least two label words; For each label text, obtain a first number and a second number, wherein the first number is the number of label words of the category words belonging to the i-th category in the label text, and the second number is the number of all label words in the label text; based on the ratio of the first number to the second number, obtain a first conditional probability that the label text belongs to the i-th category, where i is a positive integer, and 1≤i≤k, and k is the total number of categories; For the j-th label word, obtain a third number and a fourth number, the third number being the number of category words belonging to the i-th category of the j-th label word contained in different label texts, and the fourth number being the total number of category words corresponding to the i-th category; Based on the ratio of the third number to the fourth number, obtaining a second conditional probability that the j-th label word belongs to the i-th category, where j is a positive integer, wherein 1≤j≤Z, and Z is the total number of label words; Determine the probability that each label text belongs to the i-th category based on the first conditional probability and the second conditional probability; The label texts are clustered based on the probability that each label text belongs to the i-th category to obtain a clustering result for the video.

2. The method of claim 1, wherein: The determining of the extended text for the title text based on the search click data associated with the video comprises: determining at least one candidate query string based on the search click data, wherein the video is included in the search results of each candidate query string in the at least one candidate query string, The expanded query string is determined using at least a portion of the at least one candidate query string.

3. The method of claim 2, wherein: Determining the expanded query string using at least a portion of the at least one candidate query string includes: For each candidate query string in the at least one candidate query string, determining a click rate of the video in the search results of the candidate query string; When the click rate of the video is greater than a first click rate threshold, the candidate query string is used to determine the extended query string.

4. The method of claim 1, wherein: The step of determining the extended text for the title text based on the search click data associated with the video further comprises: Determine at least one candidate expanded search text based on the text search results of the expanded query string; The expanded search text is determined using at least a portion of the at least one candidate expanded search text.

5. The method of claim 4, wherein: Determining the extended search text using at least a portion of the at least one candidate extended search text comprises: For each candidate expanded search text in the at least one candidate expanded search text, determining a click rate of the candidate expanded search text, In the case that the click rate of the candidate expanded search text is greater than a second click rate threshold, the expanded search text is determined by using the candidate expanded search text.

6. The method of claim 4, wherein: Determining the extended search text using the candidate extended search text includes: determining the extended search text using at least one of a text title, a text abstract, and a text body of the candidate extended search text.

7. The method of claim 1, wherein: The step of determining the extended text for the title text based on the search click data associated with the video comprises: determining at least one candidate extended title text based on the text search results of the title text of the video; The extended title text is determined using at least a portion of the at least one candidate extended title text.

8. The method of claim 7, wherein determining the extended title text using at least a portion of the at least one candidate extended title text comprises: For each candidate extended title text in the at least one candidate extended title text, Determine the degree of match between the candidate extended title text and the title text of the video, When the matching degree is greater than the matching degree threshold, the extended title text is determined using the candidate extended title text.

9. The method of claim 1, wherein: The determining the probability that each label text belongs to the i-th category based on the first conditional probability and the second conditional probability includes: Determining the confidence of each tag word based on the text type to which the tag word belongs, wherein the text type includes at least one of a title text, an extended query string, an extended search text, and an extended title text; Adjusting the first conditional probability and the second conditional probability based on the confidence level to obtain an adjusted first conditional probability and an adjusted second conditional probability; The probability that each label text belongs to the i-th category is determined based on the adjusted first conditional probability and the adjusted second conditional probability.

10. A device for clustering videos, comprising: a title text determination unit, configured to determine a title text of the video; an extended text determination unit, configured to determine extended text for the title text based on search click data associated with the video; the extended text includes an extended search text and an extended title text; the extended search text is determined based on a text search result obtained by performing a text search using an extended query string, the extended title text is determined based on a text search result obtained by performing a text search using the title text, and the extended query string is used to search for the video; a label text determination unit, configured to determine at least two label texts for clustering the videos based on the title text and the extended text; A clustering unit configured to perform word segmentation on the at least two label texts to obtain at least two label words; For each label text, a first number and a second number are obtained, wherein the first number is the number of label words belonging to the i-th category in the label text, and the second number is the number of all label words in the label text; based on the ratio of the first number to the second number, a first conditional probability that the label text belongs to the i-th category is obtained, where i is a positive integer, and 1≤i≤k, and k is the total number of categories; for the j-th label word, a third number and a fourth number are obtained, wherein the third number is the number of category words belonging to the i-th category of the j-th label word contained in different label texts, and the fourth number is the total number of all category words corresponding to the i-th category; Based on the ratio of the third number to the fourth number, obtaining a second conditional probability that the j-th label word belongs to the i-th category, where j is a positive integer, wherein 1≤j≤Z, and Z is the total number of label words; Determine the probability that each label text belongs to the i-th category based on the first conditional probability and the second conditional probability; The label texts are clustered based on the probability that each label text belongs to the i-th category to obtain a clustering result for the video.

11. A device for clustering videos, the device comprising a memory and a processor, wherein the memory stores instructions, and when the processor is used to execute the instructions, the processor executes the method for clustering videos as described in any one of claims 1-9.

12. A computer-readable storage medium having instructions stored thereon, wherein when the instructions are executed by a processor, the processor executes the method for clustering videos according to any one of claims 1 to 9.

Images