Expert recommendation method and system based on video scenario analysis

Abstract

Disclosed is a method and system for recommending experts based on video scenario analysis. First, the pushed videos of the experts to be recommended are segmented to obtain a sequence of pushed video clips, then, the sequences of pushed video clips are respectively subjected to feature extraction to obtain a sequence of pushed video clip semantic feature maps, then, the sequences of pushed video clip semantic feature maps are respectively subjected to feature full perception analysis to obtain a sequence of pushed video clip full perception semantic feature vectors, then, the user input text retrieval content is semantically encoded to obtain a user input text retrieval semantic feature vector, then, the sequence of pushed video clip full perception semantic feature vectors and the user input text retrieval semantic feature vector are subjected to topic matching calculation to obtain a topic matching degree, and finally, based on the comparison between the topic matching degree and a predetermined threshold, it is determined whether to push the expert to be recommended. In this way, the accuracy and personalization of expert push can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent recommendation, and more specifically, to a method and system for expert recommendation based on video scenario analysis. Background Art

[0002] Video scenario analysis refers to the process of identifying and understanding the scenes, characters, actions, emotions and other elements in a video, and is an important part of video content understanding. Video scenario analysis can be used in a variety of application scenarios, such as video retrieval, video recommendation, video advertising, etc. In the field of video advertising, video scenario analysis can help users find videos of experts that match their interests and needs, thereby improving user satisfaction and retention.

[0003] However, current methods for recommending talent are mainly based on the user's historical behavior and social relationships, ignoring the semantic matching problem between the user's input sentence and the video of the talent to be recommended, resulting in inaccurate and in personalized recommendation results. In addition, there are some existing video matching methods based on user input keywords. This method can only perform semantic matching based on the user input keywords and the additional text of the video. Due to the diversity, complexity and dynamic nature of video data, it is not possible to perform talent recommendation based on video context analysis.

[0004] Therefore, an expert recommendation solution based on video scenario analysis is desired. Summary of the invention

[0005] In view of this, the present application proposes an expert recommendation method and system based on video scenario analysis, which can filter out expert videos related to the topic from a large number of expert videos according to the text retrieval content input by the user, and push the expert corresponding to the expert video to the user, thereby improving the accuracy and personalization of expert push.

[0006] According to one aspect of the present application, a method for expert recommendation based on video scenario analysis is provided, which includes:

[0007] Get the push videos of the talents to be recommended;

[0008] Performing video segmentation processing on the pushed video of the expert to be recommended to obtain a sequence of pushed video segments;

[0009] Performing feature extraction on the sequence of pushed video clips respectively by a video semantic feature extractor based on a deep neural network model to obtain a sequence of pushed video clip semantic feature maps;

[0010] Performing full-feature perception analysis on the sequences of the pushed video segment semantic feature maps respectively to obtain a sequence of pushed video segment full-feature perception semantic feature vectors;

[0011] Get the search content of the user input text;

[0012] Performing semantic encoding on the user input text retrieval content to obtain a user input text retrieval semantic feature vector;

[0013] Performing a topic matching calculation on the sequence of the fully perceived semantic feature vectors of the pushed video clip and the user input text retrieval semantic feature vector to obtain a topic matching degree; and

[0014] Based on the comparison between the subject matching degree and a predetermined threshold, it is determined whether to push the expert to be recommended.

[0015] According to another aspect of the present application, a talent recommendation system based on video scenario analysis is provided, which includes:

[0016] The video acquisition module is used to obtain the push videos of the talents to be recommended;

[0017] A video segmentation module, used for performing video segmentation processing on the pushed video of the expert to be recommended to obtain a sequence of pushed video clips;

[0018] A video feature extraction module, used for performing feature extraction on the sequence of pushed video clips respectively through a video semantic feature extractor based on a deep neural network model to obtain a sequence of pushed video clip semantic feature maps;

[0019] A feature full perception analysis module, used for performing feature full perception analysis on the sequence of the pushed video segment semantic feature graphs to obtain a sequence of pushed video segment full perception semantic feature vectors;

[0020] Input acquisition module, used to obtain the user input text retrieval content;

[0021] An input semantic encoding module is used to perform semantic encoding on the user input text retrieval content to obtain a user input text retrieval semantic feature vector;

[0022] a topic matching degree calculation module, used for performing topic matching degree calculation on the sequence of the fully perceived semantic feature vectors of the pushed video clips and the user input text retrieval semantic feature vector to obtain a topic matching degree; and

[0023] The push analysis module is used to determine whether to push the expert to be recommended based on the comparison between the subject matching degree and a predetermined threshold.

[0024] According to an embodiment of the present application, it first performs video segmentation processing on the pushed videos of the recommended expert to obtain a sequence of pushed video clips, then performs feature extraction on the sequence of pushed video clips to obtain a sequence of pushed video clip semantic feature maps, then performs feature full-perception analysis on the sequence of pushed video clip semantic feature maps to obtain a sequence of pushed video clip full-perception semantic feature vectors, then performs semantic encoding on the user input text retrieval content to obtain the user input text retrieval semantic feature vector, then performs topic matching calculation on the sequence of pushed video clip full-perception semantic feature vectors and the user input text retrieval semantic feature vector to obtain the topic matching degree, and finally, based on the comparison between the topic matching degree and a predetermined threshold, it is determined whether to push the recommended expert. In this way, the accuracy and personalization of expert push can be improved.

[0025] Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present application and, together with the description, serve to explain the principles of the present application.

[0027] Figure 1 A flowchart of an expert recommendation method based on video scenario analysis according to an embodiment of the present application is shown.

[0028] Figure 2 A schematic diagram of the architecture of an expert recommendation method based on video scenario analysis according to an embodiment of the present application is shown.

[0029] Figure 3 A block diagram of an expert recommendation system based on video scenario analysis according to an embodiment of the present application is shown.

[0030] Figure 4 An application scenario diagram of an expert recommendation method based on video scenario analysis according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0032] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0033] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0034] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0035] In response to the above technical problems, the technical concept of the present application is to obtain the pushed videos of the experts to be recommended and the text retrieval content input by the user, and introduce data processing and semantic understanding algorithms in the back end to perform semantic correlation analysis of the pushed videos and text retrieval content, so as to calculate the subject matching degree between the two. In this way, it is possible to screen out expert videos related to the topic from a large number of expert videos based on the text retrieval content input by the user, and push the expert corresponding to the expert video to the user, thereby improving the accuracy and personalization of expert push.

[0036] Figure 1 A flowchart of an expert recommendation method based on video scenario analysis according to an embodiment of the present application is shown. Figure 2 FIG. 1 is a schematic diagram showing the architecture of an expert recommendation method based on video scenario analysis according to an embodiment of the present application. Figure 1 and Figure 2As shown, according to the embodiment of the present application, the expert recommendation method based on video scenario analysis includes the following steps: S110, obtaining the pushed video of the expert to be recommended; S120, performing video segmentation processing on the pushed video of the expert to be recommended to obtain a sequence of pushed video clips; S130, performing feature extraction on the sequence of pushed video clips respectively through a video semantic feature extractor based on a deep neural network model to obtain a sequence of pushed video clip semantic feature maps; S140, performing feature full-perception analysis on the sequence of pushed video clip semantic feature maps respectively to obtain a sequence of pushed video clip full-perception semantic feature vectors; S150, obtaining user input text retrieval content; S160, semantically encoding the user input text retrieval content to obtain a user input text retrieval semantic feature vector; S170, performing topic matching calculation on the sequence of pushed video clip full-perception semantic feature vectors and the user input text retrieval semantic feature vector to obtain a topic matching degree; and, S180, determining whether to push the expert to be recommended based on a comparison between the topic matching degree and a predetermined threshold.

[0037] Specifically, in the technical solution of the present application, first, a push video of the expert to be recommended is obtained. Then, considering that the local video semantics in different time segments in the push video may be different, in order to be able to divide the push video into multiple meaningful segments so as to more accurately understand the video semantics of the push video, in the technical solution of the present application, the push video of the expert to be recommended is segmented to obtain a sequence of push video segments.

[0038] Then, the sequence of pushed video clips is subjected to feature mining in a video semantic feature extractor based on a three-dimensional convolutional neural network model to extract local semantic feature information of each video clip in the pushed video, thereby obtaining a sequence of semantic feature graphs of pushed video clips. In other words, the sequence of pushed video clips is processed in a video semantic feature extractor based on a three-dimensional convolutional neural network model to extract high-level semantic information from the video clips, such as the actions, objects, scenes, emotions, etc. of the expert in the video, so as to provide effective feature representation for video analysis and understanding.

[0039] Correspondingly, in step S130, the deep neural network model is a three-dimensional convolutional neural network model. It is worth mentioning that the three-dimensional convolutional neural network (3D Convolutional Neural Network, 3D CNN) is a deep neural network model for processing video and time series data. Unlike the traditional two-dimensional convolutional neural network (2D CNN), the three-dimensional convolutional neural network can perform convolution operations on the input data in the time dimension, thereby effectively capturing the spatiotemporal features in the video or time series data. Step S130 uses a video semantic feature extractor based on a deep neural network model, wherein the deep neural network model refers to a three-dimensional convolutional neural network. By dividing the video into a sequence of fragments and then extracting features for each fragment, a sequence of semantic feature maps of pushed video fragments is obtained. The purpose of this is to extract meaningful semantic information from the video for subsequent analysis and recommendation processes.

[0040] Furthermore, it is also considered that in each pushed video segment in the pushed video, the local video semantic features of the expert have feature association information based on the global feature of the segment, and due to the inherent limitations of convolution, it can only extract the local temporal semantic features of each pushed video segment. It should be understood that the feature full perception module based on the fully connected layer is a feature fusion method based on the fully connected layer, which can effectively utilize the correlation and complementarity between feature information at different levels, thereby enhancing the expression and distinction capabilities of the features. Based on this, in the technical solution of the present application, the sequence of semantic feature maps of the pushed video segments is further processed through the feature full perception module based on the fully connected layer to extract the fully perceived semantic feature information of each pushed video segment, thereby obtaining a sequence of fully perceived semantic feature vectors of the pushed video segments, so as to provide a unified feature representation for the overall semantic information of each video segment.

[0041] Correspondingly, in step S140, feature full-perception analysis is performed on the sequence of semantic feature maps of the pushed video clips to obtain a sequence of fully-perception semantic feature vectors of the pushed video clips, including: passing the sequence of semantic feature maps of the pushed video clips through a feature full-perception module based on a fully connected layer to obtain a sequence of fully-perception semantic feature vectors of the pushed video clips.

[0042] It is worth mentioning that the fully connected layer (Fully Connected Layer), also known as the densely connected layer, is a common layer type in deep neural networks. In the fully connected layer, each neuron is connected to all neurons in the previous layer, and each connection has a weight for calculating the linear combination between the input data and the weight, and then the result of the linear combination is nonlinearly transformed by the activation function to generate the final output. The fully connected layer plays an important role in deep neural networks. It can learn higher-level feature representations from the features of the previous layer to help the network perform more complex pattern recognition and decision-making. The fully connected layer can flexibly map the dimensions of the input data to the dimensions of the output data, so that the network can adapt to different tasks and data types. In an embodiment of the present application, the feature full perception module analyzes the sequence of the semantic feature map of the pushed video clip in a fully connected layer-based manner to obtain a sequence of fully perceived semantic feature vectors of the pushed video clip. This means that through the fully connected layer, the network can further process and abstract the semantic features of each pushed video clip to capture higher-level semantic information. The function of the fully connected layer is to linearly combine and nonlinearly transform the input semantic features to generate a more expressive feature vector, providing more accurate input for subsequent topic matching calculations and recommendation decisions.

[0043] In order to be able to match and retrieve the expert's push videos based on the user's input text, in the technical solution of the present application, it is also necessary to obtain the user input text retrieval content, and semantically encode the user input text retrieval content to obtain the user input text retrieval semantic feature vector. In particular, in a specific example of the present application, the user input text retrieval content can be "A family took the baby to the wild lake for camping and barbecue. The weather turned autumn. During the picnic, the baby was bitten by mosquitoes, and the mother used mosquito repellent." The semantic encoding process is to perform word segmentation on the user input text retrieval content and then process it through a retrieval content context encoder containing a word embedding layer to extract the global context-related feature information of each word in the user input text retrieval content, so as to facilitate the subsequent association and matching of the expert's push videos.

[0044] Correspondingly, in step S160, the user input text retrieval content is semantically encoded to obtain a user input text retrieval semantic feature vector, including: performing word segmentation processing on the user input text retrieval content to convert the user input text retrieval content into a word sequence composed of multiple words; using a word embedding layer of a retrieval content context encoder including a word embedding layer to map each word in the word sequence to a word vector to obtain a sequence of word vectors; using the retrieval content context encoder including the word embedding layer to perform global context semantic encoding on the sequence of word vectors to obtain multiple user input text retrieval feature vectors; and, cascading the multiple user input text retrieval feature vectors to obtain the user input text retrieval semantic feature vector.

[0045] It is worth mentioning that the word embedding layer is a commonly used technology in deep learning, which is used to convert discrete word representations into continuous real vector representations. It maps words to a low-dimensional vector space so that the semantic relationship between words can be reflected in the vector space. In text processing tasks, the word embedding layer is usually used as the first layer of a neural network to convert words in text data into word vectors. The advantage of doing this is that text data can be converted into continuous numerical representations, which facilitates further calculations and learning of neural networks. Specifically, in an embodiment of the present application, when semantic encoding is performed on the text retrieval content input by the user, the text needs to be segmented first, and the text is split into a sequence of multiple words. Then, a retrieval content context encoder containing a word embedding layer is used to map each word to a corresponding word vector to form a sequence of word vectors. The word embedding layer is obtained by using a pre-trained word vector model or automatically learning through the network, so that the semantic information of the word can be retained in the word vector. Through the processing of the word embedding layer, the text data can be represented as a sequence composed of word vectors. This representation method can better capture the semantic relationship and contextual information between words, and provide a richer information basis for subsequent contextual semantic encoding and feature extraction.

[0046] Next, in order to evaluate the semantic relevance between the text retrieval content input by the user and the video clip, so as to provide a more accurate basis for the recommendation of the video, in the technical solution of the present application, the subject matching degree between the sequence of the semantic feature vector of the text retrieval input by the user and the fully perceived semantic feature vector of the pushed video clip is further calculated. In particular, the subject matching degree is a matching method based on semantic similarity, which can use the correlation between the semantic feature vector of the text retrieval input by the user and the fully perceived semantic feature vector of the pushed video clip to calculate the semantic similarity between the two. If the subject matching degree is higher, it means that the semantic relevance between the text retrieval content input by the user and the video clip of the expert to be recommended is more relevant, so that the pushed video of the expert is more likely to meet the needs and interests of the user. Then, based on the comparison between the subject matching degree and the predetermined threshold, it is determined whether to push the expert to be recommended. In this way, according to the text retrieval content input by the user, the expert video related to the subject can be screened out from the massive expert videos, and the expert corresponding to the expert video can be pushed to the user, thereby improving the accuracy and personalization of expert push.

[0047] Correspondingly, in step S170, a topic matching degree is calculated for the sequence of the fully-perceived semantic feature vectors of the pushed video clip and the user-input text retrieval semantic feature vector to obtain a topic matching degree, including: calculating the topic matching degree between the sequence of the user-input text retrieval semantic feature vector and the fully-perceived semantic feature vector of the pushed video clip.

[0048] It is worth mentioning that the topic matching calculation is a method for measuring the topic relevance between the user input text and the pushed video clip, which is used to evaluate the degree of match between the user's search intent and the content of the pushed video to determine whether to recommend the video to the user. In the embodiment of the present application, the topic matching calculation involves the comparison between the user input text retrieval semantic feature vector and the sequence of the fully perceived semantic feature vector of the pushed video clip. Specifically, various similarity measurement methods can be used to calculate the topic matching, such as cosine similarity, Euclidean distance, Manhattan distance, etc. A common method for calculating the topic matching is to use cosine similarity. Cosine similarity measures the cosine value of the angle between two vectors, ranging from -1 to 1. If the two vectors are more similar, the cosine similarity is closer to 1; if the two vectors are less similar, the cosine similarity is closer to -1. Therefore, by calculating the cosine similarity between the user input text retrieval semantic feature vector and the sequence of the fully perceived semantic feature vector of the pushed video clip, the topic matching between them can be obtained. The purpose of the topic matching calculation is to help the system evaluate the correlation between the user's search intent and the content of the pushed video, so as to decide whether to recommend the corresponding expert of the video to the user. By calculating the topic matching degree, the matching degree between user needs and video content can be quantified, thus improving the accuracy of the recommendation system and user satisfaction.

[0049] In one example, calculating the topic matching degree between the user input text retrieval semantic feature vector and the sequence of the pushed video segment full perception semantic feature vector includes: calculating the topic matching degree between the user input text retrieval semantic feature vector and the sequence of the pushed video segment full perception semantic feature vector using the following matching degree formula; wherein the matching degree formula is:

[0050]

[0051] Where V is the fully perceived semantic feature vector of the pushed video clip, and A represents 1×N w The matrix, N w is equal to the scale of the fully perceived semantic feature vector of the pushed video clip, and B is 1×N h The matrix, N h is equal to the number of fully perceptual semantic feature vectors of pushed video segments in the sequence of fully perceptual semantic feature vectors of pushed video segments, σ ​​is the Sigmoid function, h i is the fully perceptual semantic feature vector of each pushed video clip in the sequence of the fully perceptual semantic feature vector of the pushed video clip, N represents the scale of the fully perceptual semantic feature vector of each pushed video clip in the sequence of the fully perceptual semantic feature vector of the pushed video clip, and S is the topic matching degree.

[0052] In particular, in the above technical scheme, the user input text retrieval semantic feature vector expresses the encoded text semantic features of the user input text retrieval content, and the sequence of the fully perceived semantic feature vectors of the pushed video clips expresses the image semantic features of the local time domain temporal sequence associations represented by each local time domain clip of the pushed video. Therefore, when calculating the topic matching degree between the user input text retrieval semantic feature vector and the sequence of the fully perceived semantic feature vectors of the pushed video clips, considering that the feature correspondence sparsity caused by the cross-modal difference between the user input text retrieval semantic feature vector and the sequence of the fully perceived semantic feature vectors of the pushed video clips may affect the accuracy of the calculated topic matching degree, it is expected to optimize the corresponding feature vector based on the significance and criticality of the feature expressions of the user input text retrieval semantic feature vector and the sequence of the fully perceived semantic feature vectors of the pushed video clips, thereby improving the accuracy of the topic matching degree.

[0053] Based on this, the present application performs correspondence correction on the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector with feature value as the granularity.

[0054] Correspondingly, a topic matching degree is calculated for the sequence of fully-perceived semantic feature vectors of the pushed video clips and the user-input text retrieval semantic feature vectors to obtain a topic matching degree, including: performing feature correction on the sequence of fully-perceived semantic feature vectors of the pushed video clips and the user-input text retrieval semantic feature vectors respectively to obtain a corrected sequence of fully-perceived semantic feature vectors of the pushed video clips and a corrected user-input text retrieval semantic feature vector; and, performing a topic matching degree calculation on the sequence of fully-perceived semantic feature vectors of the corrected pushed video clips and the corrected user-input text retrieval semantic feature vector to obtain the topic matching degree.

[0055] It should be understood that the topic matching calculation includes two key steps: feature correction and topic matching calculation. 1. Feature correction: a. Feature correction of the sequence of fully perceived semantic feature vectors of pushed video clips: By correcting the sequence of fully perceived semantic feature vectors of pushed video clips, the representation of video features can be optimized and the matching degree with the user input text can be improved. The specific correction method may involve weighting, scaling or other transformation operations of features to adapt to the matching requirements with the user input text. b. Feature correction of user input text retrieval semantic feature vectors: Correcting the user input text retrieval semantic feature vector can optimize the representation of text features and improve the matching degree with the pushed video. Similar to the correction of pushed video features, the correction of user input text may involve weighting, scaling or other transformation operations of features to adapt to the matching requirements with the pushed video. 2. Theme matching calculation: After feature correction, the sequence of the corrected fully perceived semantic feature vectors of the pushed video clips and the corrected semantic feature vectors of the user input text retrieval are used to calculate the theme matching. The specific calculation method can use cosine similarity or other similarity measurement methods to measure the similarity between the two vectors. The calculation result of the theme matching can be used as an indicator to evaluate the degree of match between the user's search intent and the pushed video content, and to decide whether to recommend the video to the user. Through feature correction and theme matching calculation, the semantic correlation between the user input text and the pushed video clip can be quantified, thereby improving the accuracy of the recommendation system and user satisfaction.

[0056] In one example, feature correction is performed on the sequence of fully-perceptual semantic feature vectors of the pushed video clip and the sequence of fully-perceptual semantic feature vectors of the user-input text retrieval semantic feature vectors to obtain a corrected sequence of fully-perceptual semantic feature vectors of the pushed video clip and a corrected user-input text retrieval semantic feature vector, including: performing correspondence correction on the sequence of the user-input text retrieval semantic feature vector and the sequence of fully-perceptual semantic feature vectors of the pushed video clip to obtain a correspondence correction feature vector; and, using the correspondence correction feature vector to correct the sequence of the user-input text retrieval semantic feature vector and the sequence of fully-perceptual semantic feature vectors of the pushed video clip to obtain a corrected sequence of fully-perceptual semantic feature vectors of the pushed video clip and the corrected user-input text retrieval semantic feature vector.

[0057] Among them, the correspondence correction is performed on the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector to obtain the correspondence correction feature vector, including: the correspondence correction is performed on the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector according to the following correction formula to obtain the correspondence correction feature vector; wherein the correction formula is:

[0058]

[0059] Among them, v 1i and v 2i are the feature values ​​of the cascade feature vector V2 obtained by sequentially cascading the user input text retrieval semantic feature vector V1 and the pushed video clip full perception semantic feature vector, and ||·||2 1 / 2 denote the square of the first norm of the feature vector and the square root of the second norm of the feature vector respectively, L denotes the length of the feature vector, and the user input text retrieval semantic feature vector V1 and the cascade feature vector V2 have the same length L, represents the positional addition of eigenvectors, represents the positional subtraction of feature vectors, α is the weight hyperparameter, exp represents the exponential function with the natural constant e as the base, and v ci is the eigenvalue of the correspondence-corrected eigenvector.

[0060] Here, the correspondence correction is based on the correspondence at the feature value granularity to divide the serialized correspondence representation of the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector into foreground manifold and background manifold based on the vector scale, so as to stack the dynamic feature value channelized association of the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector under the feature corresponding channel supermanifold aggregation mechanism, thereby marking the feature semantic information of the changes between the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector, and realizing the quasi-fully connected stacking fusion according to the variability of the semantic content between the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector in different computing dimensions. In this way, the feature value v is then ci The correspondence correction feature vector V c By respectively correcting the sequence of the semantic feature vector of the user input text retrieval and the fully perceived semantic feature vector of the pushed video clip and then calculating the subject matching degree, the calculation accuracy of the subject matching degree can be improved. In this way, expert push can be performed based on the subject matching degree of the text retrieval content input by the user and the expert pushed video. In this way, expert videos related to the subject can be screened out from a large number of expert videos based on the text retrieval content input by the user, and the expert corresponding to the expert video can be pushed to the user, thereby improving the accuracy and personalization of expert push.

[0061] In summary, the expert recommendation method based on video scenario analysis according to the embodiment of the present application can improve the accuracy and personalization of expert push.

[0062] Figure 3 FIG. 1 is a block diagram of an expert recommendation system 100 based on video scenario analysis according to an embodiment of the present application. Figure 3 As shown, according to the embodiment of the present application, the expert recommendation system 100 based on video scenario analysis includes: a video acquisition module 110, which is used to acquire the pushed video of the expert to be recommended; a video segmentation module 120, which is used to perform video segmentation processing on the pushed video of the expert to be recommended to obtain a sequence of pushed video clips; a video feature extraction module 130, which is used to extract features from the sequence of pushed video clips respectively through a video semantic feature extractor based on a deep neural network model to obtain a sequence of semantic feature maps of pushed video clips; a feature full perception analysis module 140, which is used to perform feature full perception on the sequence of semantic feature maps of pushed video clips respectively. Analysis is performed to obtain a sequence of fully perceived semantic feature vectors of the pushed video clips; an input acquisition module 150 is used to obtain user input text retrieval content; an input semantic encoding module 160 is used to semantically encode the user input text retrieval content to obtain a user input text retrieval semantic feature vector; a topic matching degree calculation module 170 is used to perform topic matching degree calculation on the sequence of fully perceived semantic feature vectors of the pushed video clips and the user input text retrieval semantic feature vector to obtain a topic matching degree; and a push analysis module 180 is used to determine whether to push the recommended expert based on a comparison between the topic matching degree and a predetermined threshold.

[0063] In one possible implementation, the deep neural network model is a three-dimensional convolutional neural network model.

[0064] Here, those skilled in the art can understand that the specific functions and operations of the various units and modules in the above-mentioned expert recommendation system 100 based on video scenario analysis have been described in detail above. Figure 1 to Figure 2 The description of the expert recommendation method based on video scenario analysis has been introduced in detail, and therefore, its repeated description will be omitted.

[0065] As described above, the expert recommendation system 100 based on video scenario analysis according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an expert recommendation algorithm based on video scenario analysis. In one possible implementation, the expert recommendation system 100 based on video scenario analysis according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the expert recommendation system 100 based on video scenario analysis can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the expert recommendation system 100 based on video scenario analysis can also be one of the many hardware modules of the wireless terminal.

[0066] Alternatively, in another example, the expert recommendation system 100 based on video scenario analysis and the wireless terminal may also be separate devices, and the expert recommendation system 100 based on video scenario analysis may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0067] Figure 4 FIG. 1 shows an application scenario diagram of an expert recommendation method based on video scenario analysis according to an embodiment of the present application. Figure 4 As shown, in this application scenario, first, the push video of the person to be recommended is obtained (for example, Figure 4 D1) and user input text retrieval content (for example, Figure 4 Then, the pushed video of the expert to be recommended and the text search content input by the user are input into a server (for example, Figure 4 In S) shown in , the server is able to use the expert recommendation algorithm based on video scenario analysis to process the pushed video of the expert to be recommended and the user input text retrieval content to determine whether to push the pushed video of the expert to be recommended.

[0068] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the function involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a dedicated hardware-based system that performs the function or action of the specification, or can be realized by a combination of special-purpose hardware and computer instructions.

[0069] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for recommending experts based on video scenario analysis, characterized in that: include: Get the push videos of the talents to be recommended; Performing video segmentation processing on the pushed video of the expert to be recommended to obtain a sequence of pushed video segments; Performing feature extraction on the sequence of pushed video clips respectively by a video semantic feature extractor based on a deep neural network model to obtain a sequence of pushed video clip semantic feature maps; Performing full-feature perception analysis on the sequences of the pushed video segment semantic feature maps respectively to obtain a sequence of pushed video segment full-feature perception semantic feature vectors; Get the search content of the user input text; Performing semantic encoding on the user input text retrieval content to obtain a user input text retrieval semantic feature vector; Performing a topic matching calculation on the sequence of the fully perceived semantic feature vectors of the pushed video clip and the user input text retrieval semantic feature vector to obtain a topic matching degree; as well as Based on the comparison between the subject matching degree and a predetermined threshold, determining whether to push the expert to be recommended; The process of calculating the topic matching degree of the sequence of the fully perceived semantic feature vectors of the pushed video clip and the user input text retrieval semantic feature vector to obtain the topic matching degree includes: Calculating the topic matching degree between the sequence of the user input text retrieval semantic feature vector and the pushed video segment full perception semantic feature vector; The step of calculating the topic matching degree between the user input text retrieval semantic feature vector and the sequence of the pushed video segment full perception semantic feature vector includes: The subject matching degree between the user input text retrieval semantic feature vector and the sequence of the pushed video segment full perception semantic feature vector is calculated by the following matching degree formula; The matching degree formula is: Where V is the fully perceived semantic feature vector of the pushed video clip, and A represents 1×N w The matrix, N w is equal to the scale of the fully perceived semantic feature vector of the pushed video clip, and B is 1×N h The matrix, N h is equal to the number of fully perceptual semantic feature vectors of pushed video segments in the sequence of fully perceptual semantic feature vectors of pushed video segments, σ ​​is the Sigmoid function, h i is each fully perceptual semantic feature vector of a pushed video segment in the sequence of fully perceptual semantic feature vectors of the pushed video segment, N represents the scale of each fully perceptual semantic feature vector of a pushed video segment in the sequence of fully perceptual semantic feature vectors of the pushed video segment, and S is the subject matching degree; Performing feature correction on the sequence of fully perceived semantic feature vectors of the pushed video clips and the user input text retrieval semantic feature vectors to obtain a sequence of fully perceived semantic feature vectors of the pushed video clips and a corrected semantic feature vector of the user input text retrieval; and Performing a topic matching calculation on the sequence of the corrected pushed video clip full perception semantic feature vector and the corrected user input text retrieval semantic feature vector to obtain the topic matching degree; The method of performing feature correction on the sequence of fully perceived semantic feature vectors of the pushed video clips and the semantic feature vectors of the user input text retrieval to obtain a sequence of fully perceived semantic feature vectors of the pushed video clips and a sequence of corrected semantic feature vectors of the user input text retrieval, respectively, comprises: Performing correspondence correction on the sequence of the user input text retrieval semantic feature vector and the pushed video segment full perception semantic feature vector to obtain a correspondence correction feature vector; and Using the correspondence correction feature vector, respectively, to correct the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector to obtain the sequence of the corrected pushed video clip full perception semantic feature vector and the corrected user input text retrieval semantic feature vector; Among them, the correspondence correction is performed on the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector to obtain the correspondence correction feature vector, including: the correspondence correction is performed on the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector according to the following correction formula to obtain the correspondence correction feature vector; wherein the correction formula is: Among them, v 1i and v 2i are the feature values ​​of the cascade feature vector V2 obtained by sequentially cascading the user input text retrieval semantic feature vector V1 and the pushed video clip full perception semantic feature vector, and ||·||2 1 / 2 denote the square of the first norm of the feature vector and the square root of the second norm of the feature vector respectively, L denotes the length of the feature vector, and the user input text retrieval semantic feature vector V1 and the cascade feature vector V2 have the same length L, represents the positional addition of eigenvectors, represents the positional subtraction of feature vectors, α is the weight hyperparameter, exp represents the exponential function with the natural constant e as the base, and v ci is the eigenvalue of the correspondence-corrected eigenvector.

2. The expert recommendation method based on video scenario analysis according to claim 1, characterized in that: The deep neural network model is a three-dimensional convolutional neural network model.

3. The expert recommendation method based on video scenario analysis according to claim 2 is characterized in that: Performing full-feature perception analysis on the sequence of the pushed video segment semantic feature graphs respectively to obtain a sequence of full-feature perception semantic feature vectors of the pushed video segment, including: The sequence of the pushed video segment semantic feature maps is passed through a feature full perception module based on a fully connected layer to obtain a sequence of the pushed video segment full perception semantic feature vectors.

4. The expert recommendation method based on video scenario analysis according to claim 3 is characterized in that: Semantically encoding the user input text search content to obtain a user input text search semantic feature vector, including: Performing word segmentation processing on the text search content input by the user to convert the text search content input by the user into a word sequence consisting of multiple words; Mapping each word in the word sequence to a word vector using a word embedding layer of a retrieval content context encoder including a word embedding layer to obtain a sequence of word vectors; Using the retrieval content context encoder including the word embedding layer to perform global context semantic encoding on the sequence of word vectors to obtain multiple user input text retrieval feature vectors; and The multiple user input text retrieval feature vectors are cascaded to obtain the user input text retrieval semantic feature vector.

5. A talent recommendation system based on video scenario analysis, characterized in that: include: The video acquisition module is used to obtain the push videos of the talents to be recommended; A video segmentation module, used for performing video segmentation processing on the pushed video of the expert to be recommended to obtain a sequence of pushed video clips; A video feature extraction module, used for performing feature extraction on the sequence of pushed video clips respectively through a video semantic feature extractor based on a deep neural network model to obtain a sequence of pushed video clip semantic feature maps; A feature full perception analysis module, used for performing feature full perception analysis on the sequence of the pushed video segment semantic feature graphs to obtain a sequence of pushed video segment full perception semantic feature vectors; Input acquisition module, used to obtain the user input text retrieval content; An input semantic encoding module is used to perform semantic encoding on the user input text retrieval content to obtain a user input text retrieval semantic feature vector; A topic matching degree calculation module, used for performing topic matching degree calculation on the sequence of the fully perceived semantic feature vectors of the pushed video clips and the user input text retrieval semantic feature vector to obtain a topic matching degree; as well as A push analysis module, used to determine whether to push the recommended person based on the comparison between the subject matching degree and a predetermined threshold; The process of calculating the topic matching degree of the sequence of the fully perceived semantic feature vectors of the pushed video clip and the user input text retrieval semantic feature vector to obtain the topic matching degree includes: Calculating the topic matching degree between the sequence of the user input text retrieval semantic feature vector and the pushed video segment full perception semantic feature vector; The step of calculating the topic matching degree between the user input text retrieval semantic feature vector and the sequence of the pushed video segment full perception semantic feature vector includes: The subject matching degree between the user input text retrieval semantic feature vector and the sequence of the pushed video segment full perception semantic feature vector is calculated by the following matching degree formula; The matching degree formula is: Where V is the fully perceived semantic feature vector of the pushed video clip, and A represents 1×N w The matrix, N w is equal to the scale of the fully perceived semantic feature vector of the pushed video clip, and B is 1×N h The matrix, N h is equal to the number of fully perceptual semantic feature vectors of pushed video segments in the sequence of fully perceptual semantic feature vectors of pushed video segments, σ ​​is the Sigmoid function, h i is each fully perceptual semantic feature vector of a pushed video segment in the sequence of fully perceptual semantic feature vectors of the pushed video segment, N represents the scale of each fully perceptual semantic feature vector of a pushed video segment in the sequence of fully perceptual semantic feature vectors of the pushed video segment, and S is the subject matching degree; Performing feature correction on the sequence of fully perceived semantic feature vectors of the pushed video clips and the user input text retrieval semantic feature vectors to obtain a sequence of fully perceived semantic feature vectors of the pushed video clips and a corrected semantic feature vector of the user input text retrieval; and Performing a topic matching calculation on the sequence of the corrected pushed video clip full perception semantic feature vector and the corrected user input text retrieval semantic feature vector to obtain the topic matching degree; The method of performing feature correction on the sequence of fully perceived semantic feature vectors of the pushed video clips and the semantic feature vectors of the user input text retrieval to obtain a sequence of fully perceived semantic feature vectors of the pushed video clips and a sequence of corrected semantic feature vectors of the user input text retrieval, respectively, comprises: Performing correspondence correction on the sequence of the user input text retrieval semantic feature vector and the pushed video segment full perception semantic feature vector to obtain a correspondence correction feature vector; and Using the correspondence correction feature vector, respectively, to correct the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector to obtain the sequence of the corrected pushed video clip full perception semantic feature vector and the corrected user input text retrieval semantic feature vector; Among them, the correspondence correction is performed on the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector to obtain the correspondence correction feature vector, including: the correspondence correction is performed on the sequence of the user input text retrieval semantic feature vector and the pushed video clip full perception semantic feature vector according to the following correction formula to obtain the correspondence correction feature vector; wherein the correction formula is: Among them, v 1i and v 2i are the feature values ​​of the cascade feature vector V2 obtained by sequentially cascading the user input text retrieval semantic feature vector V1 and the pushed video clip full perception semantic feature vector, and ||·||2 1 / 2 denote the square of the first norm of the feature vector and the square root of the second norm of the feature vector respectively, L denotes the length of the feature vector, and the user input text retrieval semantic feature vector V1 and the cascade feature vector V2 have the same length L, represents the positional addition of eigenvectors, represents the positional subtraction of feature vectors, α is the weight hyperparameter, exp represents the exponential function with the natural constant e as the base, and v ci is the eigenvalue of the correspondence-corrected eigenvector.

6. The expert recommendation system based on video scenario analysis according to claim 5, characterized in that: The deep neural network model is a three-dimensional convolutional neural network model.

Images