Processing method and device based on video stream clustering, equipment and medium

Abstract

The embodiment of the invention provides a processing method and device based on video stream clustering, equipment and a medium, and the method comprises the steps: obtaining a video stream, and carrying out the clustering of the video stream, and obtaining one or more clustering clusters; wherein different clusters correspond to different target attribute label information; and for the video streams in the one or more clustering clusters, generating mark information so as to retrieve video contents associated with the target attribute label information in the video streams according to the mark information. By means of the video content retrieval method and device, the video streams are clustered based on the clustering technology, the marks are added to the clustered video streams, then the video content needed by a user can be quickly retrieved according to the marks, and the video content retrieval efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a processing method, device, equipment and medium based on video stream clustering. Background Art

[0002] With the development of video technologies such as video conferencing, the number of video streams generated is increasing. In the case of a large number of video streams, users cannot perceive the video content in the video streams in a timely manner, and thus cannot quickly find the required video content from the large number of video streams. Summary of the invention

[0003] In view of the above problems, a processing method, apparatus, device and medium based on video stream clustering are proposed to overcome the above problems or at least partially solve the above problems, including:

[0004] A processing method based on video stream clustering, the method comprising:

[0005] Acquire a video stream, and cluster the video stream to obtain one or more clusters; wherein different clusters correspond to different target attribute label information;

[0006] For the video streams in the one or more clusters, tag information is generated to retrieve video content associated with the target attribute label information in the video stream according to the tag information.

[0007] Optionally, before clustering the video stream to obtain one or more clusters, the method further includes: determining reference attribute label information of the video stream;

[0008] The clustering of the video stream to obtain one or more clusters includes: clustering the video stream according to the reference attribute tag information to obtain one or more clusters.

[0009] Optionally, the determining the reference attribute tag information of the video stream includes:

[0010] Obtaining label description information collected for the video stream;

[0011] The video stream and the tag description information are input into a preset tag information model, and reference attribute tag information of the video stream input into the tag information model is obtained.

[0012] Optionally, obtaining the label description information collected for the video stream includes: extracting key information from the video stream and generating the label description information based on the key information; or obtaining user input information corresponding to the video stream and generating the label description information based on the user input information.

[0013] Optionally, after acquiring the label description information collected for the video stream, the method further includes:

[0014] The video stream and the tag description information are standardized.

[0015] Optionally, clustering the video stream according to the reference attribute tag information to obtain one or more clusters includes:

[0016] Extracting features from the video stream according to the reference attribute label information to obtain a feature vector;

[0017] The feature vectors are clustered to obtain one or more clusters.

[0018] Optionally, generating tag information for the video streams in the one or more clusters includes:

[0019] For the video streams in the one or more clusters, determining key frames or key time points;

[0020] According to the key frame or key time point, mark information is generated; wherein the mark information includes an association relationship between the key frame or time point and the target attribute label information.

[0021] A processing device based on video stream clustering, the device comprising:

[0022] A video stream clustering module, used to obtain a video stream and cluster the video stream to obtain one or more clusters; wherein different clusters correspond to different target attribute label information;

[0023] The video stream marking module is used to generate marking information for the video streams in the one or more clusters, so as to retrieve the video content associated with the target attribute label information in the video stream according to the marking information.

[0024] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method described above when executed by the processor.

[0025] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0026] The embodiments of the present invention have the following advantages:

[0027] In an embodiment of the present invention, a video stream is acquired and clustered to obtain one or more clustering clusters, where different clustering clusters correspond to different target attribute label information. Then, for the video streams in the one or more clustering clusters, tag information is generated to retrieve video content associated with the target attribute label information in the video stream according to the tag information. Clustering of video streams based on clustering technology and adding tags to the clustered video streams are achieved, so that the video content required by the user can be quickly retrieved according to the tags, thereby improving the efficiency of video content retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0029] Figure 1 is a flowchart of a processing method based on video stream clustering provided by some embodiments of the present invention;

[0030] Figure 2 is a flowchart of another processing method based on video stream clustering provided by some embodiments of the present invention;

[0031] Figure 3 is a flowchart of another processing method based on video stream clustering provided by some embodiments of the present invention;

[0032] Figure 4 is a flowchart of another processing method based on video stream clustering provided by some embodiments of the present invention;

[0033] Figure 5 It is a structural block diagram of another processing device based on video stream clustering provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0034] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Reference Figure 1 , shows a flowchart of a processing method based on video stream clustering provided by some embodiments of the present invention, which may specifically include the following steps:

[0036] Step 101: obtain a video stream, and cluster the video stream to obtain one or more clusters; wherein different clusters correspond to different target attribute label information.

[0037] In some examples, in a visual networking scenario, the video stream may be a video stream acquired based on the visual networking, such as a visual networking server may receive a real-time video stream and forward it to a computing server.

[0038] When obtaining batch video streams, the computing server can use clustering technology to cluster the video streams to obtain one or more clusters. Each cluster can correspond to one or more target attribute label information, and each video stream can belong to one of the clusters.

[0039] In some examples, the attribute tag information may be tag information corresponding to specific scenes, characters, events, and the like, and the specific attribute tag information is the part of the video content that the user is most concerned about.

[0040] Among them, clustering is a data processing method, which is widely used in image processing, data analysis and other fields. Its core idea is to use morphological operators to cluster and merge adjacent classification regions with similar characteristics. Specifically, clustering solves the common problem of lack of spatial continuity in classification images, such as the presence of spots or holes. These problems can be smoothed to a certain extent by low-pass filtering, but are often interfered by the encoding of adjacent categories.

[0041] In some embodiments of the present invention, before clustering the video stream to obtain one or more clusters, the process further includes: determining reference attribute label information of the video stream.

[0042] In some embodiments of the present invention, the determining the reference attribute tag information of the video stream includes:

[0043] Acquire label description information collected for the video stream; input the video stream and the label description information into a preset label information model, and acquire reference attribute label information of the video stream input by the label information model.

[0044] In practical applications, label description information collected for video streams can be obtained. The label description information can be a description of the video content in the video stream. Then the label description information and the video stream can be input into a label information model. The existing label information model can be used to perform preliminary analysis and classification of the video stream to generate predictive attribute labels, i.e., reference attribute label information.

[0045] In some embodiments of the present invention, obtaining label description information collected for the video stream includes: extracting key information from the video stream and generating label description information based on the key information; or obtaining user input information corresponding to the video stream and generating label description information based on the user input information.

[0046] In some embodiments, after obtaining the video stream, the computing server can collect attributes of the video stream, such as using video analysis technology (such as image recognition, voice recognition, etc.) to automatically extract key information from the video stream, such as people, scenes, objects, sounds, etc., and generate corresponding label description information, such as directly using the key information as label description information or optimizing the key information to obtain label description information.

[0047] In other embodiments, the user may manually input relevant information of the video before processing the video, such as descriptive text, keywords, and classification tags based on the video content, and then obtain the user input information corresponding to the video stream, and then generate label description information based on the user input information, such as directly using the user input information as the label description information or optimizing the user input information to obtain the label description information.

[0048] In some embodiments of the present invention, after acquiring the label description information collected for the video stream, the method further includes: performing standardization processing on the video stream and the label description information.

[0049] In practical applications, computing servers can perform data preprocessing on video streams, that is, standardization processing. In some examples, the collected reference attribute label information can be cleaned, deduplicated, and formatted to ensure the accuracy and consistency of the label information. In some examples, necessary encoding, decoding, transcoding and other operations can be performed on the video stream to facilitate subsequent clustering processing.

[0050] In some embodiments of the present invention, clustering the video stream to obtain one or more clusters includes: clustering the video stream according to the reference attribute tag information to obtain one or more clusters.

[0051] After the reference attribute tag information is determined, the reference attribute tag information can be used as a reference tag for the video content in the video stream. The reference attribute tag information is different according to the different video contents. When clustering the video stream, the video stream can be classified into a corresponding clustering cluster in combination with the reference attribute tag information.

[0052] In some examples, each video stream may belong to a cluster, and each cluster may correspond to one or more target attribute label information. The target attribute label information may be attribute label information selected from reference attribute label information of the video streams included in the cluster, or may be attribute label information obtained by aggregation.

[0053] In some embodiments of the present invention, clustering the video stream according to the reference attribute tag information to obtain one or more clusters includes:

[0054] According to the reference attribute label information, feature extraction is performed on the video stream to obtain a feature vector; and the feature vector is clustered to obtain one or more clusters.

[0055] In practical applications, a suitable clustering algorithm can be selected according to the characteristics of the video stream and the reference attribute label information. The computing server clusters the frames in the video stream based on the characteristics of the video frames (such as color, texture, motion, etc.) in order to identify different scenes or objects.

[0056] In some examples, the clustering algorithm may be a K-means clustering algorithm (k-means clustering algorithm).

[0057] Taking the K-means clustering algorithm as an example, the clustering process of the video stream is as follows:

[0058] 1. Data preparation

[0059] Feature extraction: Extract the features of each frame from the video stream. These features can be color histograms, texture features (such as gray-level co-occurrence matrix), motion features (such as optical flow), etc. In practical applications, the selection of features depends on the label content set in the early stage, that is, feature extraction is performed based on the reference attribute label information.

[0060] Feature vector construction: Represent the features of each frame as a feature vector. For example, if color histogram and texture features are used, each feature vector will contain a combination of these two parts of data.

[0061] 2. Clustering algorithm implementation

[0062] Choosing K value: Before starting clustering, you need to determine the number of clusters K to be formed. In some examples, the number of clusters K can be determined through domain knowledge, data visualization, or trial and error.

[0063] Initialize cluster centers: Randomly select K feature vectors as initial cluster centers. In some examples, K-means++ can be used to select the initial centers to reduce the number of iterations and improve algorithm efficiency.

[0064] Iterative optimization: For each feature vector, calculate its distance to each cluster center (commonly used Euclidean distance) and assign it to the nearest cluster center.

[0065] Update step: For each cluster, recalculate the cluster center (i.e., the mean of all feature vectors in the cluster).

[0066] Repeat the above two steps until the cluster center no longer changes significantly or the predetermined number of iterations is reached.

[0067] Result classification: To evaluate the quality of clustering results, indicators such as the Silhouette Coefficient can be used to measure the closeness and separation of clusters.

[0068] In practical applications, due to the large amount of video stream data to be processed, the computing server may need to process the data online or in batches, so multiple video streams will be randomly combined.

[0069] In addition, the computing server will also pre-process the noise and outliers that may be contained in the video stream (such as filtering and denoising). In some examples, if there is a lot of noise or outliers in the video stream, or the shape of the cluster is not spherical, you can choose to use DBSCAN (Density-Based Spatial Clustering of Applications with Noise), which does not require the number of clusters to be specified in advance and can identify clusters and noise points of any shape.

[0070] Step 102: Generate tag information for the video streams in the one or more clusters, so as to retrieve video content associated with the target attribute tag information in the video stream according to the tag information.

[0071] After clustering the video streams, each video stream has a cluster to which it belongs. Tags can be added to the video streams according to the cluster to which the video streams belong to generate tag information. The processed video streams and the corresponding tag information can then be output to the user or a subsequent processing system. The user can quickly retrieve key content segments in the video stream based on the tag information, thereby improving the utilization efficiency of the video content.

[0072] In related technologies, video content analysis and mining often rely on manual operations, which are inefficient and prone to errors. In the embodiments of the present invention, by collecting label description information of video streams or label description information input by users, and utilizing label information models, video streams can be automatically clustered according to their video content, thereby greatly improving the efficiency and accuracy of video management.

[0073] In some embodiments of the present invention, generating marking information for the video streams in the one or more clustering clusters includes: determining key frames or key time points for the video streams in the one or more clustering clusters; generating marking information based on the key frames or key time points; wherein the marking information includes the association between the key frames or time points and the target attribute label information.

[0074] In some examples, key frames associated with target attribute tag information may be marked in the video stream, or time points associated with target attribute tag information may be marked in the video stream.

[0075] In practical applications, the computing server can send the clustered video stream to the visual network server or IP network server or terminal, and the visual network server or IP network server or terminal marks the video stream. For example, for the video stream in each cluster, video analysis technology (such as frame recognition, timestamp extraction, etc.) is used to extract key frames or time points. The key frames or time points are associated with the target attribute label information in the video stream, representing the specific position of the target attribute label information in the video.

[0076] The visual network server or IP network server or terminal can mark and output the processed video stream in real time and process it in real time. In some examples, clustering processing and marking technology can be continuously applied during the real-time transmission or processing of the video stream to ensure that the system can receive new video streams, extract attribute label information, perform clustering processing and mark in real time.

[0077] In an embodiment of the present invention, a video stream is acquired and clustered to obtain one or more clustering clusters, where different clustering clusters correspond to different target attribute label information. Then, for the video streams in the one or more clustering clusters, tag information is generated to retrieve video content associated with the target attribute label information in the video stream according to the tag information. Clustering of video streams based on clustering technology and adding tags to the clustered video streams are achieved, so that the video content required by the user can be quickly retrieved according to the tags, thereby improving the efficiency of video content retrieval.

[0078] Reference Figure 2 , shows a flowchart of another processing method based on video stream clustering provided by some embodiments of the present invention, which may specifically include the following steps:

[0079] Step 201: Acquire a video stream and determine reference attribute tag information of the video stream.

[0080] In some examples, in a visual networking scenario, the video stream may be a video stream acquired based on the visual networking, such as a visual networking server may receive a real-time video stream and forward it to a computing server.

[0081] After obtaining the video stream, reference attribute tag information of the video stream may be determined. The reference attribute tag information may be used as a reference tag for the video content in the video stream. The reference attribute tag information may be different depending on the video content.

[0082] Step 202: cluster the video stream according to the reference attribute label information to obtain one or more clusters; wherein different clusters correspond to different target attribute label information.

[0083] In some examples, the attribute tag information may be tag information corresponding to specific scenes, characters, events, and the like, and the specific attribute tag information is the part of the video content that the user is most concerned about.

[0084] When clustering video streams, the video streams can be classified into corresponding clusters in combination with reference attribute label information. Specifically, when obtaining batch video streams, the computing power server can use clustering technology to cluster the video streams to obtain one or more clusters, each of which can correspond to one or more target attribute label information, and each video stream can belong to one of the clusters.

[0085] Among them, clustering is a data processing method, which is widely used in image processing, data analysis and other fields. Its core idea is to use morphological operators to cluster and merge adjacent classification regions with similar characteristics. Specifically, clustering solves the common problem of lack of spatial continuity in classification images, such as the presence of spots or holes. These problems can be smoothed to a certain extent by low-pass filtering, but are often interfered by the encoding of adjacent categories.

[0086] Step 203: Generate tag information for the video streams in the one or more clusters, so as to retrieve video content associated with the target attribute tag information in the video stream according to the tag information.

[0087] After clustering the video streams, each video stream has a cluster to which it belongs. Tags can be added to the video streams according to the cluster to which the video streams belong to generate tag information. The processed video streams and the corresponding tag information can then be output to the user or a subsequent processing system. The user can quickly retrieve key content segments in the video stream based on the tag information, thereby improving the utilization efficiency of the video content.

[0088] In related technologies, video content analysis and mining often rely on manual operations, which are inefficient and prone to errors. In the embodiments of the present invention, by collecting label description information of video streams or label description information input by users, and utilizing label information models, video streams can be automatically clustered according to their video content, thereby greatly improving the efficiency and accuracy of video management.

[0089] Reference Figure 3 , shows a flowchart of another processing method based on video stream clustering provided by some embodiments of the present invention, which may specifically include the following steps:

[0090] Step 301, obtain a video stream, extract key information from the video stream, and generate tag description information according to the key information; or obtain user input information corresponding to the video stream, and generate tag description information according to the user input information.

[0091] In some embodiments, after obtaining the video stream, the computing server can collect attributes of the video stream, such as using video analysis technology (such as image recognition, voice recognition, etc.) to automatically extract key information from the video stream, such as people, scenes, objects, sounds, etc., and generate corresponding label description information, such as directly using the key information as label description information or optimizing the key information to obtain label description information.

[0092] In other embodiments, the user may manually input relevant information of the video before processing the video, such as descriptive text, keywords, and classification tags based on the video content, and then obtain the user input information corresponding to the video stream, and then generate label description information based on the user input information, such as directly using the user input information as the label description information or optimizing the user input information to obtain the label description information.

[0093] Step 302: input the video stream and the tag description information into a preset tag information model, and obtain reference attribute tag information of the video stream input into the tag information model.

[0094] In practical applications, label description information collected for video streams can be obtained. The label description information can be a description of the video content in the video stream. Then the label description information and the video stream can be input into a label information model. The existing label information model can be used to perform preliminary analysis and classification of the video stream to generate predictive attribute labels, i.e., reference attribute label information.

[0095] Step 303: extract features from the video stream according to the reference attribute label information to obtain a feature vector, and cluster the feature vector to obtain one or more clusters; wherein different clusters correspond to different target attribute label information.

[0096] After the reference attribute tag information is determined, the reference attribute tag information can be used as a reference tag for the video content in the video stream. The reference attribute tag information is different according to the different video contents. When clustering the video stream, the video stream can be classified into a corresponding clustering cluster in combination with the reference attribute tag information.

[0097] In some examples, each video stream may belong to a cluster, and each cluster may correspond to one or more target attribute label information. The target attribute label information may be attribute label information selected from reference attribute label information of the video streams included in the cluster, or may be attribute label information obtained by aggregation.

[0098] In practical applications, a suitable clustering algorithm can be selected according to the characteristics of the video stream and the reference attribute label information. The computing server clusters the frames in the video stream based on the characteristics of the video frames (such as color, texture, motion, etc.) in order to identify different scenes or objects.

[0099] In some examples, the clustering algorithm may be a K-means clustering algorithm (k-means clustering algorithm).

[0100] Taking the K-means clustering algorithm as an example, the clustering process of the video stream is as follows:

[0101] 1. Data preparation

[0102] Feature extraction: Extract the features of each frame from the video stream. These features can be color histograms, texture features (such as gray-level co-occurrence matrix), motion features (such as optical flow), etc. In practical applications, the selection of features depends on the label content set in the early stage, that is, feature extraction is performed based on the reference attribute label information.

[0103] Feature vector construction: Represent the features of each frame as a feature vector. For example, if color histogram and texture features are used, each feature vector will contain a combination of these two parts of data.

[0104] 2. Clustering algorithm implementation

[0105] Choosing K value: Before starting clustering, you need to determine the number of clusters K to be formed. In some examples, the number of clusters K can be determined through domain knowledge, data visualization, or trial and error.

[0106] Initialize cluster centers: Randomly select K feature vectors as initial cluster centers. In some examples, K-means++ can be used to select the initial centers to reduce the number of iterations and improve algorithm efficiency.

[0107] Iterative optimization: For each feature vector, calculate its distance to each cluster center (commonly used Euclidean distance) and assign it to the nearest cluster center.

[0108] Update step: For each cluster, recalculate the cluster center (i.e., the mean of all feature vectors in the cluster).

[0109] Repeat the above two steps until the cluster center no longer changes significantly or the predetermined number of iterations is reached.

[0110] Result classification: To evaluate the quality of clustering results, indicators such as the Silhouette Coefficient can be used to measure the closeness and separation of clusters.

[0111] In practical applications, due to the large amount of video stream data to be processed, the computing server may need to process the data online or in batches, so multiple video streams will be randomly combined.

[0112] In addition, the computing server will also pre-process the noise and outliers that may be contained in the video stream (such as filtering and denoising). In some examples, if there is a lot of noise or outliers in the video stream, or the shape of the cluster is not spherical, you can choose to use DBSCAN (Density-Based Spatial Clustering of Applications with Noise), which does not require the number of clusters to be specified in advance and can identify clusters and noise points of any shape.

[0113] Step 304: Generate tag information for the video streams in the one or more clusters, so as to retrieve video content associated with the target attribute tag information in the video stream according to the tag information.

[0114] After clustering the video streams, each video stream has a cluster to which it belongs. Tags can be added to the video streams according to the cluster to which the video streams belong to generate tag information. The processed video streams and the corresponding tag information can then be output to the user or a subsequent processing system. The user can quickly retrieve key content segments in the video stream based on the tag information, thereby improving the utilization efficiency of the video content.

[0115] In related technologies, video content analysis and mining often rely on manual operations, which are inefficient and prone to errors. In the embodiments of the present invention, by collecting label description information of video streams or label description information input by users, and utilizing label information models, video streams can be automatically clustered according to their video content, thereby greatly improving the efficiency and accuracy of video management.

[0116] Reference Figure 4 , shows a flowchart of another processing method based on video stream clustering provided by some embodiments of the present invention, which may specifically include the following steps:

[0117] Step 401: obtain a video stream, and cluster the video stream to obtain one or more clusters; wherein different clusters correspond to different target attribute label information.

[0118] In some examples, in a visual networking scenario, the video stream may be a video stream acquired based on the visual networking, such as a visual networking server may receive a real-time video stream and forward it to a computing server.

[0119] When obtaining batch video streams, the computing server can use clustering technology to cluster the video streams to obtain one or more clusters. Each cluster can correspond to one or more target attribute label information, and each video stream can belong to one of the clusters.

[0120] In some examples, the attribute tag information may be tag information corresponding to specific scenes, characters, events, and the like, and the specific attribute tag information is the part of the video content that the user is most concerned about.

[0121] Among them, clustering is a data processing method, which is widely used in image processing, data analysis and other fields. Its core idea is to use morphological operators to cluster and merge adjacent classification regions with similar characteristics. Specifically, clustering solves the common problem of lack of spatial continuity in classification images, such as the presence of spots or holes. These problems can be smoothed to a certain extent by low-pass filtering, but are often interfered by the encoding of adjacent categories.

[0122] Step 402: determining key frames or key time points for the video streams in the one or more clusters.

[0123] In some examples, key frames associated with target attribute tag information may be marked in the video stream, or time points associated with target attribute tag information may be marked in the video stream.

[0124] Step 403, generating marking information according to the key frame or key time point, so as to retrieve the video content associated with the target attribute label information in the video stream according to the marking information; wherein the marking information includes the association relationship between the key frame or time point and the target attribute label information.

[0125] In practical applications, the computing server can send the clustered video stream to the visual network server or IP network server or terminal, and the visual network server or IP network server or terminal marks the video stream. For example, for the video stream in each cluster, video analysis technology (such as frame recognition, timestamp extraction, etc.) is used to extract key frames or time points. The key frames or time points are associated with the target attribute label information in the video stream, representing the specific position of the target attribute label information in the video.

[0126] The visual network server or IP network server or terminal can mark and output the processed video stream in real time and process it in real time. In some examples, clustering processing and marking technology can be continuously applied during the real-time transmission or processing of the video stream to ensure that the system can receive new video streams, extract attribute label information, perform clustering processing and mark in real time.

[0127] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0128] Reference Figure 5 , shows a schematic diagram of the structure of a processing device based on video stream clustering provided by some embodiments of the present invention, which may specifically include the following modules:

[0129] The video stream clustering module 501 is used to obtain a video stream and cluster the video stream to obtain one or more clusters; wherein different clusters correspond to different target attribute label information;

[0130] The video stream marking module 502 is configured to generate marking information for the video streams in the one or more clusters, so as to retrieve the video content associated with the target attribute tag information in the video stream according to the marking information.

[0131] In some embodiments of the present invention, before clustering the video stream to obtain one or more clusters, the method further includes: determining reference attribute label information of the video stream;

[0132] The clustering of the video stream to obtain one or more clusters includes: clustering the video stream according to the reference attribute tag information to obtain one or more clusters.

[0133] In some embodiments of the present invention, the determining the reference attribute tag information of the video stream includes:

[0134] Obtaining label description information collected for the video stream;

[0135] The video stream and the tag description information are input into a preset tag information model, and reference attribute tag information of the video stream input into the tag information model is obtained.

[0136] In some embodiments of the present invention, obtaining label description information collected for the video stream includes: extracting key information from the video stream and generating label description information based on the key information; or, obtaining user input information corresponding to the video stream, and generating label description information based on the user input information.

[0137] In some embodiments of the present invention, after acquiring the tag description information collected for the video stream, the method further includes:

[0138] The video stream and the tag description information are standardized.

[0139] In some embodiments of the present invention, clustering the video stream according to the reference attribute tag information to obtain one or more clusters includes:

[0140] Extracting features from the video stream according to the reference attribute label information to obtain a feature vector;

[0141] The feature vectors are clustered to obtain one or more clusters.

[0142] In some embodiments of the present invention, the step of generating tag information for the video streams in the one or more clusters includes:

[0143] For the video streams in the one or more clusters, determining key frames or key time points;

[0144] According to the key frame or key time point, mark information is generated; wherein the mark information includes an association relationship between the key frame or time point and the target attribute label information.

[0145] In an embodiment of the present invention, a video stream is acquired and clustered to obtain one or more clustering clusters, where different clustering clusters correspond to different target attribute label information. Then, for the video streams in the one or more clustering clusters, tag information is generated to retrieve video content associated with the target attribute label information in the video stream according to the tag information. Clustering of video streams based on clustering technology and adding tags to the clustered video streams are achieved, so that the video content required by the user can be quickly retrieved according to the tags, thereby improving the efficiency of video content retrieval.

[0146] Some embodiments of the present invention further provide an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the above method is implemented when the computer program is executed by the processor.

[0147] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0148] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above method when executed by a processor.

[0149] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0150] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0151] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0152] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0153] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0154] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0156] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0157] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0158] The above is a detailed introduction to the processing method, device, equipment and medium based on video stream clustering. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A processing method based on video stream clustering, characterized in that: The method comprises: Acquire a video stream, and cluster the video stream to obtain one or more clusters; wherein different clusters correspond to different target attribute label information; For the video streams in the one or more clusters, tag information is generated to retrieve video content associated with the target attribute label information in the video stream according to the tag information.

2. The method according to claim 1, characterized in that: Before clustering the video stream to obtain one or more clusters, the method further includes: determining reference attribute label information of the video stream; The clustering of the video stream to obtain one or more clusters includes: clustering the video stream according to the reference attribute tag information to obtain one or more clusters.

3. The method according to claim 2, characterized in that The determining of the reference attribute tag information of the video stream includes: Obtaining label description information collected for the video stream; The video stream and the tag description information are input into a preset tag information model, and reference attribute tag information of the video stream input into the tag information model is obtained.

4. The method according to claim 3, characterized in that The obtaining of the label description information collected for the video stream includes: extracting key information from the video stream and generating the label description information according to the key information; or obtaining user input information corresponding to the video stream and generating the label description information according to the user input information.

5. The method according to claim 3, characterized in that: After acquiring the label description information collected for the video stream, the method further includes: The video stream and the tag description information are standardized.

6. The method according to any one of claims 2 to 5, characterized in that: The clustering of the video stream according to the reference attribute tag information to obtain one or more clusters includes: Extracting features from the video stream according to the reference attribute label information to obtain a feature vector; The feature vectors are clustered to obtain one or more clusters.

7. The method according to claim 1, characterized in that The step of generating tag information for the video streams in the one or more clusters includes: For the video streams in the one or more clusters, determining key frames or key time points; According to the key frame or key time point, mark information is generated; wherein the mark information includes an association relationship between the key frame or time point and the target attribute label information.

8. A processing device based on video stream clustering, characterized in that: The device comprises: A video stream clustering module, used to obtain a video stream and cluster the video stream to obtain one or more clusters; wherein different clusters correspond to different target attribute label information; The video stream marking module is used to generate marking information for the video streams in the one or more clusters, so as to retrieve the video content associated with the target attribute label information in the video stream according to the marking information.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images