Hotspot video clip identification method and device, electronic equipment and medium

Abstract

The present application relates to artificial intelligence technology, disclose a kind of hot video segment identification method, comprising: using pre-trained graphic-text matching model, extract the video feature of each video segment and the text feature of each barrage text, in turn select a barrage text as a text to be matched, the initial video segment corresponding to the text to be matched and the video segment in the preset adjacent range are taken as the video segment set to be matched, the similarity between the text feature of the text to be matched and the video feature corresponding to the video segment set to be matched is calculated, the video segment that meets the preset similarity condition is selected as the matching video segment of the text to be matched, the barrage text after graphic-text matching is classified, according to the number of barrage text in each classification, the hot degree of video segment in the corresponding classification is calculated, and the hot video is determined according to the hot degree.The present application also proposes a kind of hot video segment identification device, equipment and medium.The present application can improve the accuracy of hot video segment identification in medical related video.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for identifying hotspot video clips. Background Technology

[0002] With the development of internet technology, in the medical field, traditional non-interactive video viewing functions are far from meeting user needs for promoting and popularizing medical and health information, medical science, and medical tips through recorded videos. To enhance user interaction, many medical video apps utilize bullet comments (or danmaku) to improve user experience and adjust their video delivery strategies based on user feedback, ensuring the quantity and quality of trending videos.

[0003] Traditional methods of extracting trending video clips rely heavily on manual labor, which introduces subjectivity and limitations. This means that video clips deemed valuable by a particular user may not be accepted by the general public, and could even convey the editor's negative emotions or unhealthy values, severely impacting the online environment of self-media platforms. It also easily leads to insufficient discovery of trending video clips.

[0004] Current technologies for automatically identifying trending videos using machines mainly include detection technologies based on the number and threshold of bullet comments in the video, and technologies based on the sentiment polarity of bullet comment text.

[0005] Detection methods based on the number of video bullet comments and the threshold size cannot meet the needs of users to select popular segments based on content and emotional preferences.

[0006] This technology, based on the sentiment polarity detection of bullet screen text, calculates the overall sentiment intensity of video segments by constructing a sentiment dictionary and examining the rate of change in sentiment intensity between adjacent video segments. It identifies trending video segments based on the characteristics of this sentiment change rate, thus meeting users' needs to search for trending segments using sentiment tendencies and keywords. However, this technology does not consider the latency issue of bullet screen text; that is, there is a time delay between the bullet screen comment text and the video time point or time period that resonates with the user. Relying solely on bullet screen text to identify trending video segments will introduce certain errors, and the accuracy of trending video segment identification needs further improvement. Summary of the Invention

[0007] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for identifying hotspot video clips, with the main objective of improving the accuracy of hotspot video clip identification.

[0008] To achieve the above objectives, the present invention provides a method for identifying hotspot video segments, comprising:

[0009] Obtain the initial video segment set and the corresponding initial bullet screen text set;

[0010] Using a pre-trained image-text matching model, extract the video features of each video segment in the initial video segment set and the text features of each bullet screen text in the initial bullet screen text set;

[0011] From the initial set of bullet screen texts, one bullet screen text is selected as the text to be matched, and the initial video segment corresponding to the text to be matched and the video segments within the preset adjacent range corresponding to the initial video segment are taken as the set of video segments to be matched.

[0012] Calculate the similarity between the text features of the text to be matched and the video features corresponding to the set of video segments to be matched. Select video segments that meet the preset similarity conditions from the set of video segments to be matched as the matching video segments of the text to be matched, and obtain the set of video bullet comments with image-text matching.

[0013] The bullet screen text in the video bullet screen set of the image-text matching is classified. The popularity degree of the video segment in the corresponding category is calculated according to the number of bullet screen texts in each category. The popular videos are determined according to the popularity degree.

[0014] Optionally, extracting video features from each video segment in the initial video segment set includes:

[0015] Each video segment in the initial video segment set is sequentially processed into frames to obtain a set of video frames corresponding to each video segment.

[0016] Extract the pixel values ​​of the RGB three channels of each video frame in the video frame set;

[0017] The extracted pixel values ​​are used to generate a pixel matrix for the corresponding video frame;

[0018] By fusing the pixel matrix corresponding to each video frame in the video frame set, the video features of the corresponding video segment are obtained.

[0019] Optionally, extracting the text features of each bullet screen text in the initial bullet screen text set includes:

[0020] Each of the bullet screen texts is segmented sequentially to obtain multiple text segments;

[0021] Select one text segment from the multiple text segmentations as the target segment, and count the number of times the target segment and its neighboring text segments co-occur within a preset neighborhood of the target segment.

[0022] Construct a co-occurrence matrix using the co-occurrence counts corresponding to each text segmentation;

[0023] The multiple text segments are converted into word vectors, and the word vectors are concatenated into a vector matrix;

[0024] The text vector matrix is ​​obtained by multiplying the co-occurrence matrix and the vector matrix.

[0025] Extract the text features of the bullet screen text from the text vector matrix.

[0026] Optionally, extracting the text features of the bullet screen text from the text vector matrix includes:

[0027] Select one text segment from the multiple text segmentations one by one as the target segmentation, and calculate the key value of the target segmentation based on the word vector of the target segmentation and the text vector matrix;

[0028] A predetermined number of text segments are selected as feature segments from the plurality of text segments according to the key values ​​in descending order;

[0029] The text features of the bullet screen text are obtained by concatenating the word vectors of the feature segmentation.

[0030] Optionally, classifying the text of the video bullet comments in the image-text matching set includes:

[0031] In the set of video bullet comments that match text and images, select one video segment as the target video segment in sequence;

[0032] Calculate the preset classification feature vector for each bullet screen text in the target video segment;

[0033] Cluster the bullet screen text in the target video segment based on the classification feature vector to obtain cluster centers;

[0034] Calculate the probability value between the classification feature vector of each cluster center and the preset classification label, and select the classification label corresponding to the probability value greater than the preset probability threshold as the classification of the target video segment.

[0035] Optionally, the step of calculating the popularity level of video segments in each category based on the number of bullet screen texts in each category, and determining popular videos based on the popularity level, includes:

[0036] Count the number of bullet screen texts for each video segment within the same category;

[0037] The number of statistics is used as the popularity level of the corresponding video segment, and all video segments in the same category are sorted according to the popularity level.

[0038] Select video segments within the preset sorting range as hot videos.

[0039] Optionally, obtaining the initial video segment set and the corresponding initial bullet screen text set includes:

[0040] Get long videos with bullet screen data;

[0041] The long video is divided into video segments of equal length according to a preset unit duration to obtain the initial video segment set;

[0042] Obtain the bullet screen data contained in each video segment of the initial video segment set, preprocess each bullet screen data, and obtain the initial bullet screen text set corresponding to the initial video segment set.

[0043] To address the above problems, the present invention also provides a hotspot video clip recognition device, the device comprising:

[0044] The image and text object acquisition module is used to acquire the initial video segment set and the initial bullet screen text set corresponding to the initial video segment set;

[0045] The image and text feature extraction module is used to extract the video features of each video segment in the initial video segment set and the text features of each bullet screen text in the initial bullet screen text set using a pre-trained image and text matching model.

[0046] The image-text alignment module is used to sequentially select one bullet screen text as the text to be matched from the initial bullet screen text set, and to take the initial video segment corresponding to the text to be matched and the video segments within a preset adjacent range corresponding to the initial video segment as the set of video segments to be matched; calculate the similarity between the text features of the text to be matched and the video features corresponding to the set of video segments to be matched, and select the video segments that meet the preset similarity conditions from the set of video segments to be matched as the matching video segments of the text to be matched, thereby obtaining the image-text matched video bullet screen set;

[0047] The image and text classification and hotspot statistics module is used to classify the bullet screen text in the video bullet screen collection of the image and text matching, calculate the hotspot degree of the video segment in the corresponding category based on the number of bullet screen texts in each category, and determine the hotspot video based on the hotspot degree.

[0048] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0049] Memory, storing at least one computer program; and

[0050] The processor executes the program stored in the memory to implement the hotspot video segment recognition method described above.

[0051] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the aforementioned hotspot video segment recognition method.

[0052] This invention utilizes a pre-trained image-text matching model to extract video features from each video segment in the initial video segment set and text features from each bullet screen text in the initial bullet screen text set. It calculates the similarity between the text features of the text to be matched and the corresponding video features in the video segment set, and selects video segments that meet preset similarity conditions as the matching video segments for the text to be matched. This achieves alignment between bullet screen text and video segments, improving the accuracy of subsequent classification of video segments and hotspot calculation using bullet screen text, thereby enhancing the accuracy of hotspot video segment identification. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating a hotspot video segment identification method according to an embodiment of the present invention.

[0054] Figure 2 This is a detailed implementation flowchart of one step in a hotspot video segment recognition method provided in an embodiment of the present invention;

[0055] Figure 3 This is a detailed implementation flowchart of another step in the hotspot video segment recognition method provided in an embodiment of the present invention;

[0056] Figure 4 This is a detailed implementation flowchart of another step in the hotspot video segment recognition method provided in an embodiment of the present invention;

[0057] Figure 5 This is a functional block diagram of a hotspot video clip recognition device provided in an embodiment of the present invention;

[0058] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the hotspot video segment recognition method according to an embodiment of the present invention.

[0059] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0060] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0061] This application provides a method for identifying trending video clips. The executing entity of the method includes, but is not limited to, at least one of electronic devices configured to execute the method provided in this application, such as a server or a terminal. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0062] Reference Figure 1 The diagram shown is a flowchart illustrating a method for identifying hotspot video clips according to an embodiment of the present invention. In this embodiment, the method for identifying hotspot video clips includes:

[0063] S1. Obtain the initial video segment set and the initial bullet screen text set corresponding to the initial video segment set;

[0064] In this embodiment of the invention, a medical guidance video is used as an example for illustration. The initial video segment set consists of multiple medical guidance videos of varying or specified lengths, such as videos on simple wound bandaging, home blood pressure measurement, and cervical spondylosis prevention exercises. The initial bullet screen text set refers to the collection of bullet screen text generated during the playback of each video segment in the initial video segment set.

[0065] In this embodiment of the invention, the initial video segment set and the initial bullet screen text set can be collected from a designated storage area. For example, the initial video segment set and the initial bullet screen text set can be collected from a database, blockchain, network cache, or other designated storage area using computer statements (Java statements, Python statements, etc.) with data scraping capabilities.

[0066] In detail, obtaining the initial video segment set and the initial bullet screen text set corresponding to the initial video segment set includes: obtaining a long video with bullet screen data; dividing the long video into video segments of equal length according to a preset unit duration to obtain the initial video segment set; obtaining the bullet screen data contained in each video segment in the initial video segment set, and preprocessing each bullet screen data to obtain the initial bullet screen text set corresponding to the initial video segment set.

[0067] For example, V1, V2, V3, V n These represent N long videos with accompanying bullet comments, each with a different duration. The preset unit length is L seconds. (i = 1, 2, ... N) represents the N video segments corresponding to the i-th long video.

[0068] In this embodiment of the invention, the preprocessing includes operations such as text deduplication, removal of useless characters, and filtering of emojis.

[0069] For example, suppose the i-th long video v i The j-th video segment (i = 1, 2, ..., N) There are 1, 2, ..., N bullet screen texts; j = 1, 2, ..., ni. These can be represented by... express.

[0070] It is understandable that, due to the time lag between the generation of bullet comment text and the playback time of the video, it is possible that a certain bullet comment text may appear out of place. In the video clip, the actual content expressed by the bullet comment was directed at... The initial set of bullet screen texts and the initial set of video segments are one or N video segments before each other. Therefore, there is a certain time delay between the initial set of bullet screen texts and the initial set of video segments, and they do not match perfectly.

[0071] S2. Using a pre-trained image-text matching model, extract the video features of each video segment in the initial video segment set and the text features of each bullet screen text in the initial bullet screen text set.

[0072] In this embodiment of the invention, the pre-trained image-text matching model is a neural network model based on an attention mechanism. Using a given training set of video segments and a training set of bullet screen text, the image-text matching model is trained so that it can find the optimal delay time parameter between the bullet screen text and the video segment.

[0073] It is understood that each video segment consists of a varying number of video frames. Each video frame can be regarded as a static image. Typically, an image consists of three channels: R, G, and B. Each channel can be regarded as a pixel matrix. The pixel matrix can be used to represent the features of the corresponding image, and then the video features of the corresponding video segment can be generated based on the features of the image.

[0074] For details, please refer to Figure 3 As shown, the extraction of video features from each video segment in the initial video segment set includes:

[0075] S21. Perform frame segmentation processing on each video segment in the initial video segment set in sequence to obtain a set of video frames corresponding to each video segment;

[0076] S22. Extract the pixel values ​​of the three RGB channels of each video frame in the video frame set;

[0077] S23. Use the extracted pixel values ​​to generate the pixel matrix of the corresponding video frame;

[0078] S24. Merge the pixel matrix corresponding to each video frame in the video frame set to obtain the video features of the corresponding video segment.

[0079] In one embodiment of the present invention, the pixel matrix of each video frame can be fused using the following formula:

[0080]

[0081] in, This represents the video feature corresponding to the j-th video segment in the i-th long video. They represent The pixel values ​​of the RGB channels in the x-th frame at second t, where W represents the parameters of the convolution transformation and b represents the corresponding bias term.

[0082] In this embodiment of the invention, since each bullet screen text is composed of natural language, directly analyzing the bullet screen text would consume a lot of computing resources, resulting in low analysis efficiency. Therefore, the bullet screen text can be converted into a text vector matrix, thereby converting the bullet screen text expressed in natural language into a numerical form.

[0083] In this embodiment of the invention, methods such as GloVe (Global Vectors for Word Representation) and Embedding Layer can be used to convert each bullet screen text into a text vector matrix. Furthermore, after converting the bullet screen text into a text vector matrix, feature extraction can be performed on the text vector matrix to obtain the text features of the bullet screen text. The text features include, but are not limited to, text scene, text theme, and text keywords.

[0084] In one embodiment of the present invention, see reference Figure 3 As shown, the extraction of text features for each bullet screen text in the initial bullet screen text set includes:

[0085] S21. Perform word segmentation on each of the bullet screen texts in sequence to obtain multiple text segments;

[0086] S22. Select one text segment from the plurality of text segments as the target segment, and count the number of times the target segment and its neighboring text segments co-occur within a preset neighborhood of the target segment.

[0087] S23. Construct a co-occurrence matrix using the co-occurrence counts corresponding to each text segmentation;

[0088] S24. Convert the multiple text segments into word vectors respectively, and concatenate the word vectors into a vector matrix;

[0089] S25. Perform a product operation using the co-occurrence matrix and the vector matrix to obtain the text vector matrix;

[0090] S26. Extract the text features of the bullet screen text from the text vector matrix.

[0091] In detail, the bullet screen text can be segmented using a preset standard dictionary to obtain multiple text segments, and the standard dictionary contains multiple standard segments.

[0092] For example, the bullet screen text can be searched in the standard dictionary according to different lengths. If the same standard word segmentation as the bullet screen text can be found, then the retrieved standard word segmentation can be determined as the text segmentation of the bullet screen text.

[0093] For example, a co-occurrence matrix can be constructed using the co-occurrence counts corresponding to each text segment, as shown below:

[0094]

[0095] Among them, X i,j The number of times keyword i co-occurs with adjacent word segment j in the bullet screen text.

[0096] Specifically, extracting the text features of the bullet screen text from the text vector matrix includes:

[0097] Select one text segment from the multiple text segmentations one by one as the target segmentation, and calculate the key value of the target segmentation based on the word vector of the target segmentation and the text vector matrix;

[0098] A predetermined number of text segments are selected as feature segments from the plurality of text segments according to the key values ​​in descending order;

[0099] The text features of the bullet screen text are obtained by concatenating the word vectors of the feature segmentation.

[0100] In detail, since the bullet screen text contains a large number of text segments, but not every text segment is a feature of the bullet screen text, it is necessary to filter the multiple text segments. In this embodiment of the invention, one text segment is selected as the target segment from the multiple text segments one by one. The key value of the target segment is calculated based on the word vector of the target segment and the text vector matrix, so as to filter out the feature segments that are representative of the bullet screen text based on the key value, so as to obtain the text features of the bullet screen text.

[0101] Specifically, the step of calculating the key value of the target word based on the word vector of the target word and the text vector matrix includes:

[0102] The key values ​​of the target word segment are calculated using the following key value algorithm:

[0103]

[0104] Where K is the key value, |W| is the text vector matrix, T is the matrix transpose symbol, and || is the modulo symbol. The word vectors for the target word segmentation.

[0105] In this embodiment of the invention, a preset number of text segments are selected as feature segments from the plurality of text segments in descending order of their key values.

[0106] For example, the multiple text segmentations include: text segment A, text segment B, and text segment C, wherein the key value of text segment A is 80, the key value of text segment B is 70, and the key value of text segment C is 30. If the preset number is 2, then text segment A and text segment B are selected as feature segments in descending order of the key values, and the word vectors of text segment A and text segment B are concatenated to obtain the text features of the bullet screen text.

[0107] S3. Select one bullet screen text from the initial bullet screen text set as the text to be matched, and take the initial video segment corresponding to the text to be matched and the video segments within the preset adjacent range corresponding to the initial video segment as the set of video segments to be matched.

[0108] Understandably, bullet screen comments typically contain a timestamp relative to the video playback time. This timestamp identifies exactly when a user posted the comment within a video segment. However, due to variations in user editing times and network speed, the bullet screen comment may not perfectly match the corresponding initial video segment. A single bullet screen comment could potentially correspond to a video segment preceding or following the initial video segment.

[0109] In this embodiment of the invention, the preset adjacent range can be set according to actual conditions. For example, the video segments within the preset adjacent range corresponding to the j-th initial video segment include the j-th (0 < ∈ ≤ j) video segment and the j+∈ video segment, where ∈ can be a natural number such as 1, 2, 3, etc. to The set of video segments to be matched consists of several video segments.

[0110] S4. Calculate the similarity between the text features of the text to be matched and the video features corresponding to the video segment set to be matched. From the video segment set to be matched, select the video segment that meets the preset similarity condition as the matching video segment of the text to be matched, and obtain the video barrage set of text-image matching.

[0111] In this embodiment of the invention, the similarity between the text features of the text to be matched and the video features of each video segment in the set of video segments to be matched can be calculated by using a preset activation function.

[0112] In this embodiment of the invention, the similarity can be calculated using the following activation function:

[0113]

[0114] in, This represents the t-th segment corresponding to the j-th video segment of the i-th long video. ij Textual features of bullet screen text This represents the range from the j-th (0 < 0 ≤ j) video segment to the j+th video segment corresponding to the i-th long video. There are a total of 2 ∈+1 video segments with video features.

[0115] In this embodiment of the invention, the preset similarity condition can be set according to the actual situation. For example, the preset similarity condition can be that when the similarity between the text features of a text to be matched and the video features of a video segment to be matched is greater than or equal to a preset similarity threshold, there is a matching relationship between the corresponding text to be matched and the video segment to be matched.

[0116] In this embodiment of the invention, in the video bullet screen set with image-text matching, the bullet screen text contained in each video segment achieves the effect of image-text alignment with the corresponding video segment.

[0117] S5. Classify the bullet screen text in the video bullet screen set of the image and text matching, calculate the hotness of the video segment in the corresponding category according to the number of bullet screen text in each category, and determine the hot video according to the hotness.

[0118] In this embodiment of the invention, the corresponding video segments can be classified by performing cluster analysis on the bullet screen text. The classification can be a sentiment polarity classification, including but not limited to positive, neutral, and negative. In practical applications, the classification can be set according to the needs of selecting trending videos.

[0119] For details, please refer to Figure 4 As shown, the classification of bullet screen text in the video bullet screen set of the image-text matching includes:

[0120] S51. In the set of video bullet comments for image and text matching, select one video segment as the target video segment in sequence;

[0121] S52. Calculate the preset classification feature vector of each bullet screen text in the target video segment;

[0122] S53. Cluster the bullet screen text in the target video segment according to the classification feature vector to obtain the cluster center;

[0123] S54. Calculate the probability value between the classification feature vector of each cluster center and the preset classification label, and select the classification label corresponding to the probability value greater than the preset probability threshold as the classification of the target video segment.

[0124] In this embodiment of the invention, methods such as GloVe (Global Vectors for Word Representation) and Embedding Layer can be used to calculate the classification feature vector of each bullet screen text.

[0125] In this embodiment of the invention, a preset activation function can be used to calculate the probability value between the classification feature vector of each cluster center and the preset classification label. The activation function includes, but is not limited to, the softmax activation function, the sigmoid activation function, and the ReLU activation function. The preset classification label includes, but is not limited to, positive, negative, and neutral.

[0126] In one embodiment of the present invention, the probability value can be calculated using the following activation function:

[0127]

[0128] Where p(a|x) is the probability that the cluster center's classification feature vector is between x and the classification label a, and w a Let T be the weight vector of category label a, T be the transpose operator, exp be the expected value operator, and A be the preset number of category labels.

[0129] Furthermore, the step of calculating the popularity level of video segments in each category based on the number of bullet screen texts, and determining popular videos based on the popularity level, includes:

[0130] Count the number of bullet comments for each video segment in the same category; use the counted number as the popularity level of the corresponding video segment, and sort all video segments in the same category according to the popularity level; select video segments within a preset sorting range as hot videos.

[0131] For example, the top 3 video segments are selected as trending videos.

[0132] In this embodiment of the invention, after each bullet screen text is matched and aligned with the corresponding video segment, the number of bullet screen texts in each video segment is not directly counted. Instead, the bullet screen texts in each video segment are first classified, which can achieve the effect of tagging each video segment. Then, the number of bullet screen texts in all video segments of the same category is counted, which can improve the accuracy of selecting popular videos.

[0133] This invention utilizes a pre-trained image-text matching model to extract video features from each video segment in the initial video segment set and text features from each bullet screen text in the initial bullet screen text set. It calculates the similarity between the text features of the text to be matched and the corresponding video features in the video segment set, and selects video segments that meet preset similarity conditions as the matching video segments for the text to be matched. This achieves alignment between bullet screen text and video segments, improving the accuracy of subsequent classification of video segments and hotspot calculation using bullet screen text, thereby enhancing the accuracy of hotspot video segment identification.

[0134] like Figure 5 The diagram shown is a functional block diagram of a hotspot video segment recognition device provided in an embodiment of the present invention.

[0135] The hotspot video clip recognition device 100 of this invention can be installed in an electronic device. Depending on its functions, the hotspot video clip recognition device 100 includes: an image / text object acquisition module 101, an image / text feature extraction module 102, an image / text alignment module 103, and an image / text classification and hotspot statistics module 104. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0136] In this embodiment, the functions of each module / unit are as follows:

[0137] The image and text object acquisition module 101 is used to acquire an initial video segment set and an initial bullet screen text set corresponding to the initial video segment set.

[0138] The image and text feature extraction module 102 is used to extract the video features of each video segment in the initial video segment set and the text features of each bullet screen text in the initial bullet screen text set using a pre-trained image and text matching model.

[0139] The image-text alignment module 103 is used to sequentially select one bullet screen text as the text to be matched from the initial bullet screen text set, and take the initial video segment corresponding to the text to be matched and the video segments within a preset adjacent range corresponding to the initial video segment as the set of video segments to be matched; calculate the similarity between the text features of the text to be matched and the video features corresponding to the set of video segments to be matched, and select the video segments that meet the preset similarity conditions from the set of video segments to be matched as the matching video segments of the text to be matched, so as to obtain the image-text matched video bullet screen set;

[0140] The image and text classification and hotspot statistics module 104 is used to classify the bullet screen text in the video bullet screen set of the image and text matching, calculate the hotspot degree of the video segment in the corresponding category according to the number of bullet screen texts in each category, and determine the hotspot video according to the hotspot degree.

[0141] In detail, each module in the hotspot video segment recognition device 100 described in this embodiment of the invention adopts the same characteristics as described above during use. Figures 1 to 4 The method used is the same as the hotspot video clip recognition method described above, and can produce the same technical effect, so it will not be repeated here.

[0142] like Figure 6 The diagram shown is a structural schematic of an electronic device for implementing a hotspot video segment recognition method according to an embodiment of the present invention.

[0143] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a hotspot video clip recognition program.

[0144] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 can include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a hotspot video clip recognition program, but also to temporarily store data that has been output or will be output.

[0145] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules (such as hotspot video clip recognition programs) stored in the memory 11, and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0146] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0147] Figure 6 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 6 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0148] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0149] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0150] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0151] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0152] The hotspot video segment recognition program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0153] Obtain the initial video segment set and the corresponding initial bullet screen text set;

[0154] Using a pre-trained image-text matching model, extract the video features of each video segment in the initial video segment set and the text features of each bullet screen text in the initial bullet screen text set;

[0155] From the initial set of bullet screen texts, one bullet screen text is selected as the text to be matched, and the initial video segment corresponding to the text to be matched and the video segments within the preset adjacent range corresponding to the initial video segment are taken as the set of video segments to be matched.

[0156] Calculate the similarity between the text features of the text to be matched and the video features corresponding to the set of video segments to be matched. Select video segments that meet the preset similarity conditions from the set of video segments to be matched as the matching video segments of the text to be matched, and obtain the set of video bullet comments with image-text matching.

[0157] The bullet screen text in the video bullet screen set of the image-text matching is classified. The popularity degree of the video segment in the corresponding category is calculated according to the number of bullet screen texts in each category. The popular videos are determined according to the popularity degree.

[0158] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0159] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0160] Obtain the initial video segment set and the corresponding initial bullet screen text set;

[0161] Using a pre-trained image-text matching model, extract the video features of each video segment in the initial video segment set and the text features of each bullet screen text in the initial bullet screen text set;

[0162] From the initial set of bullet screen texts, one bullet screen text is selected as the text to be matched, and the initial video segment corresponding to the text to be matched and the video segments within the preset adjacent range corresponding to the initial video segment are taken as the set of video segments to be matched.

[0163] Calculate the similarity between the text features of the text to be matched and the video features corresponding to the set of video segments to be matched. Select video segments that meet the preset similarity conditions from the set of video segments to be matched as the matching video segments of the text to be matched, and obtain the set of video bullet comments with image-text matching.

[0164] The bullet screen text in the video bullet screen set of the image-text matching is classified. The popularity degree of the video segment in the corresponding category is calculated according to the number of bullet screen texts in each category. The popular videos are determined according to the popularity degree.

[0165] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0166] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0167] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0168] The embodiments of this application can acquire and process relevant data based on holographic projection technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0169] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for identifying trending video clips, characterized in that, The method includes: Obtain the initial video segment set and the corresponding initial bullet screen text set; Using a pre-trained image-text matching model, extract the video features of each video segment in the initial video segment set and the text features of each bullet screen text in the initial bullet screen text set; From the initial set of bullet screen texts, one bullet screen text is selected as the text to be matched, and the initial video segment corresponding to the text to be matched and the video segments within the preset adjacent range corresponding to the initial video segment are taken as the set of video segments to be matched. Calculate the similarity between the text features of the text to be matched and the video features corresponding to the set of video segments to be matched. Select video segments that meet the preset similarity conditions from the set of video segments to be matched as the matching video segments of the text to be matched, and obtain the set of video bullet comments with image-text matching. In the set of video bullet comments for image-text matching, a video segment is selected sequentially as the target video segment. A preset classification feature vector is calculated for each bullet comment text in the target video segment. Based on the classification feature vector, the bullet comment texts in the target video segment are clustered to obtain cluster centers. The probability value between the classification feature vector of each cluster center and a preset classification label is calculated. The classification label corresponding to the probability value greater than a preset probability threshold is selected as the sentiment polarity classification of the target video segment. The number of bullet comment texts corresponding to each video segment in all video segments of the same category is counted. The counted number is used as the popularity level of the corresponding video segment. All video segments in the same category are sorted according to the popularity level. Video segments within a preset sorting range are selected as hot videos.

2. The hotspot video segment identification method as described in claim 1, characterized in that, The extraction of video features from each video segment in the initial video segment set includes: Each video segment in the initial video segment set is sequentially processed into frames to obtain a set of video frames corresponding to each video segment. Extract the pixel values ​​of the RGB three channels of each video frame in the video frame set; The extracted pixel values ​​are used to generate a pixel matrix for the corresponding video frame; By fusing the pixel matrix corresponding to each video frame in the video frame set, the video features of the corresponding video segment are obtained.

3. The hotspot video segment identification method as described in claim 1, characterized in that, The extraction of text features for each bullet screen text in the initial bullet screen text set includes: Each of the bullet screen texts is segmented sequentially to obtain multiple text segments; Select one text segment from the multiple text segmentations as the target segment, and count the number of times the target segment and its neighboring text segments co-occur within a preset neighborhood of the target segment. Construct a co-occurrence matrix using the co-occurrence counts corresponding to each text segmentation; The multiple text segments are converted into word vectors, and the word vectors are concatenated into a vector matrix; The text vector matrix is ​​obtained by multiplying the co-occurrence matrix and the vector matrix. Extract the text features of the bullet screen text from the text vector matrix.

4. The hotspot video segment identification method as described in claim 3, characterized in that, Extracting the text features of the bullet screen text from the text vector matrix includes: Select one text segment from the multiple text segmentations one by one as the target segmentation, and calculate the key value of the target segmentation based on the word vector of the target segmentation and the text vector matrix; A predetermined number of text segments are selected as feature segments from the plurality of text segments according to the key values ​​in descending order; The text features of the bullet screen text are obtained by concatenating the word vectors of the feature segmentation.

5. The hotspot video segment identification method as described in claim 1, characterized in that, The process of obtaining the initial video segment set and the corresponding initial bullet screen text set includes: Get long videos with bullet screen data; The long video is divided into video segments of equal length according to a preset unit duration to obtain the initial video segment set; Obtain the bullet screen data contained in each video segment of the initial video segment set, preprocess each bullet screen data, and obtain the initial bullet screen text set corresponding to the initial video segment set.

6. A hotspot video clip recognition device, characterized in that, The device includes: The image and text object acquisition module is used to acquire the initial video segment set and the initial bullet screen text set corresponding to the initial video segment set; The image and text feature extraction module is used to extract the video features of each video segment in the initial video segment set and the text features of each bullet screen text in the initial bullet screen text set using a pre-trained image and text matching model. The image-text alignment module is used to sequentially select one bullet screen text as the text to be matched from the initial bullet screen text set, and to take the initial video segment corresponding to the text to be matched and the video segments within a preset adjacent range corresponding to the initial video segment as the set of video segments to be matched; calculate the similarity between the text features of the text to be matched and the video features corresponding to the set of video segments to be matched, and select the video segments that meet the preset similarity conditions from the set of video segments to be matched as the matching video segments of the text to be matched, thereby obtaining the image-text matched video bullet screen set; The image and text classification and hotspot statistics module is used to sequentially select a video segment as the target video segment from the set of video bullet comments matched with images and text, calculate a preset classification feature vector for each bullet comment text in the target video segment, cluster the bullet comment text in the target video segment according to the classification feature vector to obtain cluster centers, calculate the probability value between the classification feature vector of each cluster center and the preset classification label, select the classification label corresponding to the probability value greater than the preset probability threshold as the sentiment polarity classification of the target video segment, count the number of bullet comment texts corresponding to each video segment in all video segments of the same category, use the counted number as the hotspot degree of the corresponding video segment, sort all video segments of the same category according to the hotspot degree, and select video segments within a preset sorting range as hot videos.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the hotspot video segment recognition method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the hotspot video segment identification method as described in any one of claims 1 to 5.

Images