Multimedia search method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application disclose a multimedia search method and device, electronic equipment and storage medium, comprising: in response to a search operation of a target terminal, obtaining a preliminary multimedia, the search operation comprising a search word, and the preliminary multimedia being a search result corresponding to the search word; obtaining historical multimedia; based on a matching degree between the historical multimedia and the search word, screening out associated multimedia from the historical multimedia; performing first feature extraction processing on the associated multimedia to obtain associated features; performing second feature extraction processing on the preliminary multimedia to be processed to obtain to-be-processed features; performing correlation analysis processing on the search word, the associated features and the to-be-processed features to obtain an analysis result; and determining a display result of the preliminary multimedia to be processed based on the analysis result. The present application takes the interaction of the historical multimedia as an important basis when displaying the search result, so that the search result can better meet the personalized needs of the user, thereby improving the search efficiency.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically to a multimedia search method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of the internet, browsing multimedia information has become an important way for users to work, study, and entertain themselves. Users often need to search for more multimedia information while browsing. However, because search terms often have related meanings in multiple fields, users often need to further filter through the massive amount of search results to find content that interests them, thus reducing search efficiency. Summary of the Invention

[0003] This application provides a multimedia search method, apparatus, electronic device, and storage medium, which can improve the problem of low search efficiency in the prior art.

[0004] This application provides a multimedia search method that, in response to a search operation of a target terminal, firstly selects multimedia, wherein the search operation includes a search term, and the initially selected multimedia is the search result corresponding to the search term; secondly, it selects historical multimedia, wherein the historical multimedia is multimedia in which the target terminal has performed interactive behavior within a historical time period; thirdly, it filters related multimedia from the historical multimedia based on the matching degree between the historical multimedia and the search term; fourthly, it performs a first feature extraction process on the related multimedia to obtain related features; fifthly, it performs a second feature extraction process on the initially selected multimedia to obtain features to be processed; wherein the initially selected multimedia to be processed is any one of the initially selected multimedia; sixthly, it performs a correlation analysis process on the search term, the related features, and the features to be processed to obtain analysis results; and finally, based on the analysis results, it determines the display results of the initially selected multimedia to be processed.

[0005] This application provides a multimedia search device, the device comprising:

[0006] The initial multimedia selection unit is used to respond to a search operation of the target terminal and obtain initial multimedia, wherein the search operation includes a search term, and the initial multimedia is the search result corresponding to the search term;

[0007] A historical multimedia unit is used to acquire historical multimedia, wherein the historical multimedia is multimedia in which the target terminal has performed interactive behaviors within a historical time period;

[0008] The associated multimedia unit is used to filter associated multimedia from the historical multimedia based on the degree of matching between the historical multimedia and the search term;

[0009] The first extraction unit is used to perform a first feature extraction process on the associated multimedia to obtain associated features;

[0010] The second extraction unit is used to perform second feature extraction processing on the initial multimedia to be processed to obtain the features to be processed; wherein, the initial multimedia to be processed is any one of the initial multimedia;

[0011] The results display unit is used to perform relevance analysis on the search terms, the associated features, and the features to be processed to obtain analysis results; and based on the analysis results, to determine the display results of the preliminary multimedia to be processed.

[0012] In one embodiment, the associated features include an associated extraction sequence and associated multimedia features; correspondingly, the first extraction unit includes:

[0013] The associated text subunit is used to extract text from the associated multimedia to obtain the associated extraction result;

[0014] The sequential arrangement subunit is used to arrange multiple association extraction results in a set order to obtain an association extraction sequence;

[0015] The associated feature subunit is used to extract features from the associated extraction sequence to obtain associated multimedia features.

[0016] In one embodiment, the features to be processed include the extraction results to be processed and the initial selected features to be processed; correspondingly, the second extraction unit includes:

[0017] The text to be processed subunit is used to extract text from the initially selected multimedia to obtain the extraction result.

[0018] The feature subunit to be processed is used to extract features from the extraction result to obtain the initial selected features to be processed.

[0019] In one implementation, the associated multimedia unit includes:

[0020] The historical text subunit is used to perform text extraction processing on each of the historical multimedia files to obtain the corresponding historical multimedia extraction results.

[0021] The ratio calculation subunit is used to determine the hit ratio of the number of hits to the total number of historical multimedia extraction results for each historical multimedia, wherein the number of hits is the number of times the historical multimedia extraction results for the historical multimedia hit the search term.

[0022] The multimedia filtering subunit is used to filter out the associated multimedia from the historical multimedia based on the hit ratio and a preset ratio threshold.

[0023] In one implementation, the result display unit includes:

[0024] The relevance analysis subunit is used to perform relevance analysis on the search term, the associated features, and the features to be processed to obtain multiple relevance weight values, wherein each relevance weight value reflects the weight of the corresponding relevance coefficient;

[0025] The correlation score subunit is used to obtain a correlation score based on the multiple correlation weight values ​​and the correlation coefficient corresponding to each correlation weight value, wherein the correlation score is the analysis result.

[0026] In one implementation, the result display unit further includes:

[0027] The multimedia display subunit is used to display the initially selected multimedia in descending order of relevance scores.

[0028] In one implementation, the result display unit is specifically used to: perform correlation analysis on the search term, the associated features, and the features to be processed using a trained scoring model to obtain analysis results.

[0029] In one embodiment, the device further includes:

[0030] A training initial selection unit is used to obtain training initial selection multimedia, wherein the training initial selection multimedia is the search result corresponding to the training search term;

[0031] The training history unit is used to acquire training history multimedia, wherein the training history multimedia is multimedia in which the training terminal has performed interactive behaviors during the training history time period;

[0032] A matching degree unit is used to filter out training-related multimedia from the training history multimedia based on the matching degree between the training history multimedia and the training search term;

[0033] The first feature training unit is used to perform first feature extraction processing on the training associated multimedia to obtain training associated features;

[0034] The second feature training unit is used to perform second feature extraction processing on the initial multimedia to be processed for training, so as to obtain the training features; wherein, the initial multimedia to be processed for training is any one of the initial multimedia to be processed for training.

[0035] The training relevance analysis unit is used to perform relevance analysis on the training search terms, the training related features, and the training features to be processed using the scoring model to be trained, so as to obtain the training analysis results.

[0036] The loss function unit is used to construct a loss function based on the training analysis results, the initial selection of multimedia and labels;

[0037] The parameter update unit is used to update the parameters of the scoring model based on the loss function until convergence, thereby obtaining the trained scoring model.

[0038] In the multimedia search method provided in this application embodiment, in response to the search operation of the target terminal, search results corresponding to the search terms are obtained and denoted as preliminary selected multimedia. Multimedia in which the target terminal has performed interactive behavior within a historical time period are obtained and denoted as historical multimedia. Based on the matching degree between historical multimedia and search terms, historical multimedia is filtered to select related multimedia related to the search terms. Subsequently, feature extraction is performed on the related multimedia to obtain related features. Feature extraction is performed on any one of the preliminary selected multimedia (let's call it preliminary selected multimedia to be processed) to obtain features to be processed. Correlation analysis is performed on the search terms, related features, and features to be processed to obtain analysis results; and based on the analysis results, the display results of the preliminary selected multimedia to be processed are determined.

[0039] In this embodiment, when a user searches for multimedia information, the system first obtains the search results corresponding to the search term as preliminary search results. Then, it filters out related multimedia content that matches the search term from historical multimedia content. Subsequently, based on the related multimedia content and the search term, it performs a relevance analysis on each of the preliminary search results, obtaining the analysis results corresponding to each preliminary search result. Based on the analysis results, it determines the display method for multiple preliminary search results. This application uses the interaction data of historical multimedia content as an important basis when displaying search results, making the search results more suitable for users' personalized needs, thereby improving search efficiency. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1a This is a schematic diagram illustrating the application scenario of the multimedia search method provided in this application;

[0042] Figure 1bThis is a flowchart illustrating the multimedia search method provided in an embodiment of this application;

[0043] Figure 1c The specific process of correlation analysis in one particular implementation is illustrated;

[0044] Figure 2 This is a flowchart illustrating a multimedia search method provided in a specific embodiment of this application;

[0045] Figure 3 This is a schematic diagram of a multimedia search device provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] This application provides a multimedia search method, apparatus, electronic device, and storage medium.

[0049] Specifically, this multimedia search device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, desktop computer, etc. The server can be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0050] In some embodiments, the multimedia search device may also be integrated into multiple electronic devices. For example, the multimedia search device may be integrated into multiple servers, and the multimedia search method of this application may be implemented by multiple servers.

[0051] In some embodiments, the terminal can also be used as a server to implement some or all of the functions of a server.

[0052] Please see details Figure 1aThe method provided in this application embodiment may include: responding to a search operation of a target terminal, obtaining a preliminary selection of multimedia, wherein the search operation includes a search term, and the preliminary selection of multimedia is the search result corresponding to the search term; obtaining historical multimedia, wherein the historical multimedia is multimedia in which the target terminal has performed interactive behavior within a historical time period; filtering related multimedia from the historical multimedia based on the matching degree between the historical multimedia and the search term; performing a first feature extraction process on the related multimedia to obtain related features; performing a second feature extraction process on the preliminary selection of multimedia to be processed to obtain features to be processed; wherein the preliminary selection of multimedia to be processed is any one of the preliminary selection of multimedia; performing a correlation analysis process on the search term, related features, and features to be processed to obtain analysis results; and determining the display results of the preliminary selection of multimedia to be processed based on the analysis results.

[0053] In the above method, when a user searches for multimedia information, this application embodiment first obtains the search results corresponding to the search term as preliminary search results; then, it filters out related multimedia that matches the search term from historical multimedia; subsequently, based on the related multimedia and the search term, it performs a relevance analysis on each of the preliminary search results to obtain the analysis results corresponding to each preliminary search result, and determines the display method of multiple preliminary search results based on the analysis results. This application uses the interaction status of historical multimedia as an important basis when displaying search results, making the search results more able to meet the user's personalized needs, thereby improving search efficiency.

[0054] The multimedia search method provided in this application can be applied to applications with multimedia information browsing capabilities, such as news applications and short video applications. Specifically, it can be applied to applications that integrate recommendation and search systems, identifying potential user search needs and optimizing search result presentation by analyzing real-time multimedia information browsing behavior within the recommendation stream.

[0055] It is understood that in the embodiments of this application, data such as user information are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0056] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0057] This embodiment provides a multimedia search method. For example... Figure 1b As shown, the specific process of this method may include the following steps 110 to 160:

[0058] 110. In response to a search operation of the target terminal, obtain a preliminary selection of multimedia, wherein the search operation includes a search term, and the preliminary selection of multimedia is the search result corresponding to the search term.

[0059] Multimedia refers to information content that combines two or more media formats. Multimedia can include, but is not limited to, text, images, audio, and video. For ease of description, we will use video as an example of multimedia in the following text.

[0060] The target terminal is the terminal device used by the user. The search operation is the operation of searching for multimedia information, and the search operation includes search terms, which can be denoted as "query". The search operation can include either of the following two operations: the user enters a search term in the search box; or the user clicks the search control in the application. The search control is a control in the application that triggers the action of searching for multimedia information. Optionally, the search control can be a "recommended search terms" located below or next to the search box, or it can be a "search background text" located in the comment section interface. It should be understood that the specific form and location of the search control should not be construed as a limitation of this application.

[0061] The initial selection of multimedia refers to the multiple search results obtained by the server performing a search operation on the search terms.

[0062] 120. Obtain historical multimedia, wherein the historical multimedia refers to multimedia in which the target terminal has performed interactive behaviors within a historical time period.

[0063] The historical time period refers to the time period that occurred earlier than the search operation. Optionally, the historical time period can be a time period consisting of one hour prior to the time corresponding to the search operation. Alternatively, the historical time period can be a time period consisting of two hours prior to the time corresponding to the search operation. It should be understood that the specific length of the historical time period should not be construed as a limitation of this application.

[0064] Interactive behavior refers to the interaction between a user and multimedia content using a target terminal. Optionally, interactive behavior may include, but is not limited to, liking, disliking, commenting, saving, forwarding, sharing, and browsing time. It should be understood that the specific type of interactive behavior should not be construed as a limitation of this application.

[0065] 130. Based on the degree of matching between the historical multimedia and the search term, filter out related multimedia from the historical multimedia.

[0066] Historical multimedia may contain multimedia related to the search term. Therefore, historical multimedia can be filtered based on the degree of matching between it and the search term to obtain the filtered results, which are the related multimedia. Optionally, in one embodiment, step 130 may specifically include steps 131 to 133:

[0067] 131. Perform text extraction processing on each of the historical multimedia files to obtain the corresponding historical multimedia extraction results.

[0068] Text extraction processing refers to the process of extracting text from historical multimedia content. Continuing with the example above, if the historical multimedia is video, then the text extraction process for the video may include, but is not limited to: obtaining the video title, extracting text from the video cover image, performing Automatic Speech Recognition (ASR) on the audio in the video, and performing Optical Character Recognition (OCR) on the subtitles in the video. It should be understood that the specific processing steps of text extraction processing should not be construed as limiting this application.

[0069] The above text extraction process can be performed on multiple historical multimedia files to obtain the corresponding historical multimedia extraction results. A total of multiple historical multimedia extraction results were obtained.

[0070] 132. For each historical multimedia extraction result corresponding to the historical multimedia, determine the hit ratio between the number of hits and the total number of historical multimedia extraction results for that historical multimedia.

[0071] The number of hits refers to the number of times the historical multimedia extraction results corresponding to the historical multimedia content hit the search term.

[0072] Alternatively, in one implementation, the number of hits can be calculated as follows:

[0073] Core words are extracted from the search term to obtain at least one core word, which is denoted as the search core word. The extracted results of multiple historical multimedia files corresponding to a given historical multimedia file are compared with the search core word. The number of historical multimedia extraction results that match the search core word is recorded as the hit count.

[0074] Optionally, the above-described core word extraction process can be implemented in any of the following ways: By segmenting the text into words and selecting nouns, verbs, or adjectives as core words based on part-of-speech tagging. Alternatively, deep neural networks, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and their variants LSTM / GRU, can be used to learn feature representations from large amounts of data, followed by core word extraction. Another option is to use supervised learning algorithms, such as Support Vector Machines (SVMs), Random Forests, and neural networks, to train a model to automatically identify and extract core words.

[0075] After determining the number of hits, the total number of historical multimedia extraction results for that historical multimedia can be calculated. Then, the ratio of the number of hits to the total number of historical multimedia extraction results for that historical multimedia can be calculated and denoted as the hit ratio.

[0076] Alternatively, in one implementation, the hit ratio can be calculated using the following formula (1):

[0077]

[0078] Where term_hitrate represents the hit ratio; ∑ term is_hit represents the number of hits; term_num represents the total number of historical multimedia extraction results for the corresponding historical multimedia.

[0079] 133. Based on the hit ratio and the preset ratio threshold, filter out the associated multimedia from the historical multimedia.

[0080] The preset ratio threshold is a critical value for determining whether historical multimedia content is related to the search term. If the hit ratio is greater than or equal to the preset ratio threshold, the historical multimedia content corresponding to the hit ratio is determined to be related multimedia content; if the hit ratio is less than the preset ratio threshold, the historical multimedia content corresponding to the hit ratio is determined not to be related multimedia content. The specific value of the preset ratio threshold can be set by the developers based on their work experience; for example, it can be set to 0.1, or it can be set to other values, such as 0.3. It should be understood that the specific value of the preset ratio threshold should not be construed as a limitation of this application.

[0081] In the above implementation, for any historical multimedia file among multiple historical multimedia files, text extraction processing can be performed to obtain multiple historical multimedia extraction results corresponding to that historical multimedia file. Then, the number of times the multiple historical multimedia extraction results corresponding to that historical multimedia file are matched with the search term is calculated, and then the ratio of the number of matches to the total number of historical multimedia extraction results corresponding to that historical multimedia file is calculated; this ratio is the match ratio. Subsequently, the match ratio is compared with a preset ratio threshold to determine whether the historical multimedia file is a related multimedia file. The above process can be performed on each historical multimedia file to filter out related multimedia files from multiple historical multimedia files. Filtering out related multimedia files related to the search term from historical multimedia files can support subsequent search operations, thereby helping to improve search efficiency.

[0082] 140. Perform a first feature extraction process on the associated multimedia to obtain associated features.

[0083] Optionally, in one implementation, the association features include association extracted sequences and association multimedia features. Accordingly, step 140 may specifically include steps 141 to 143:

[0084] 141. Extract text from the associated multimedia to obtain the associated extraction results.

[0085] The text extraction process is the same as the text extraction process in step 131, so it will not be described again here.

[0086] 142. Arrange the multiple association extraction results in a set order to obtain the association extraction sequence.

[0087] Optionally, the associated extracted sequence can be represented by the string "10*pre_doc".

[0088] The set order is a pre-defined order. For example, the set order could be the chronological order in which the corresponding associated multimedia is viewed by the user corresponding to the target terminal; or it could be the chronological order in which the corresponding associated multimedia is published. It should be understood that the specific content of the set order should not be construed as a limitation of this application.

[0089] Optionally, in one embodiment, step 142 may further include the following steps:

[0090] Determine whether the number of associated extraction results reaches the sequence threshold;

[0091] If the number of association extraction results reaches the sequence threshold, then after arranging the multiple association extraction results in a set order, the first sequence threshold number of association extraction results are extracted, and the first sequence threshold number of association extraction results are the association extraction sequence.

[0092] If the number of associated extraction results does not reach the sequence threshold, then after arranging the multiple associated extraction results in a set order, the number of results that are insufficient to reach the sequence threshold is set to empty, thus obtaining the associated extraction sequence.

[0093] The sequence threshold is a pre-set threshold for the sequence. The sequence threshold can be set by the developers based on their experience. The sequence threshold can be represented by k, which can be 10 or other values, such as 15. It should be understood that the specific value of the sequence threshold should not be construed as a limitation of this application.

[0094] Let's take a sequence threshold of 10 as an example to illustrate the above steps:

[0095] If the number of association extraction results is 12, which exceeds the sequence threshold of 10, then after arranging the 12 association extraction results in a set order, the first 10 association extraction results are truncated and recorded as the association extraction sequence.

[0096] If the number of association extraction results is 6, which does not reach the sequence threshold of 10, then after arranging the 6 association extraction results in the set order, the number of fewer than 10 (i.e., 10-6=4) will be set to empty association extraction results, thus obtaining the association extraction sequence.

[0097] 143. Perform feature extraction on the associated extraction sequence to obtain associated multimedia features.

[0098] Alternatively, the associated multimedia features can be represented by the string "10*pre_visual_emb".

[0099] Optionally, in one implementation, TRANSFORMER's single-tower representation model can be used to perform feature extraction processing on each association extraction result in the association extraction sequence to obtain the corresponding feature vector. The multiple feature vectors obtained by the above process constitute the associated multimedia features.

[0100] Continuing with the example above, let's take multimedia video as an example to further illustrate the process of feature extraction from the association extraction results using TRANSFORMER's single-tower representation model, as follows:

[0101] A series of keyframes are extracted from the video. These frames can be uniformly distributed or selected based on a keyframe detection algorithm. A pre-trained convolutional neural network (CNN), such as ResNet or InceptionV3, is used to extract visual features from each video frame. Since video data has a temporal dimension, the features of different frames need to be arranged chronologically to form a feature sequence. This feature sequence is then used as input to a Transformer model. To accommodate the Transformer's structure, positional encoding is typically added to preserve the sequence's order information. Through the Transformer's self-attention layer, the model learns the relationships between different frames and which parts are most important for understanding the entire video. After each Transformer layer, a fully connected feed-forward neural network (FFN) is usually placed to further process the features. After multiple Transformer encoding layers, the final output is a feature vector.

[0102] In the above implementation, by performing a series of feature extraction processes on the associated multimedia, intermediate results can be obtained: the associated extraction sequence 10*pre_doc and the associated multimedia features 10*pre_visual_emb, which facilitates the subsequent correlation analysis process.

[0103] 150. Perform second feature extraction on the initially selected multimedia to be processed to obtain the features to be processed.

[0104] Wherein, the multimedia to be processed is any one of the initially selected multimedia. The features to be processed include the extraction results to be processed and the initial features to be processed. Accordingly, step 150 includes the following steps 151 to 152:

[0105] 151. Extract text from the initial multimedia files to be processed to obtain the extraction results.

[0106] The extracted result can be represented by the string "target_title". Taking a video as an example, the text extraction process for the initial multimedia selection can include: extracting text from the video title, cover image, audio, and subtitles, resulting in title extraction, cover image extraction, audio extraction, and subtitle extraction results, respectively; then concatenating these results in the order of title extraction, cover image extraction, audio extraction, and subtitle extraction to obtain the concatenated result, where adjacent extraction results can be separated by the delimiter [SEP]. Finally, the concatenated result is segmented at the character level to obtain the final extracted result.

[0107] 152. Perform feature extraction on the extraction results to be processed to obtain the initial selected features to be processed.

[0108] The extracted result to be processed can be represented by the string "target_visual_emb". The feature extraction process for the extracted result to be processed is the same as the feature extraction process in step 143, and will not be described again here.

[0109] In the above implementation, by performing a series of feature extraction processes on the initial multimedia to be processed, intermediate results can be obtained: the extracted result target_title and the extracted result target_visual_emb, which will facilitate the subsequent correlation analysis process.

[0110] 160. Perform correlation analysis on the search terms, the associated features, and the features to be processed to obtain the analysis results; and based on the analysis results, determine the display results of the initial multimedia selection to be processed.

[0111] Optionally, in one implementation, step 160 may specifically include steps 161 to 163:

[0112] 161. Perform a correlation analysis on the search term, the associated features, and the features to be processed to obtain multiple related weight values, wherein each related weight value reflects the weight of the corresponding correlation coefficient.

[0113] Optionally, in one implementation, relevance analysis can be performed using a trained scoring model. Continuing with the example above, the associated features include the associated extracted sequence 10*pre_doc and the associated multimedia features 10*pre_visual_emb, and the features to be processed include the extracted results target_title and target_visual_emb. Relevance analysis can then be performed on the search term query, the associated extracted sequence 10*pre_doc, the associated multimedia features 10*pre_visual_emb, the extracted results target_title and target_visual_emb, as follows: Figure 1c As shown.

[0114] Figure 1c This document illustrates a relevance analysis step in a specific implementation. The search term (query), the associated extraction sequence 10*pre_doc, and the target extraction result (target_title) are concatenated and input into a BERT (Bidirectional Encoder Representations from Transformers) model for natural language processing, yielding a 768-dimensional first feature vector. The associated multimedia features 10*pre_visual_emb are input into a first multilayer perceptron (MLP) to obtain a 256-dimensional second feature vector. Specifically, this process can be implemented using fully connected layers f = wx + b. The target extraction result (target_visual_emb) is then processed into a 256-dimensional third feature vector. Specifically, if the target extraction result (target_visual_emb) is greater than or equal to 256 dimensions, the first 256 dimensions are truncated; if the target extraction result (target_visual_emb) is less than 256 dimensions, the missing dimensions are padded with zeros.

[0115] The 768-dimensional first feature vector, the 256-dimensional second feature vector, and the 256-dimensional third feature vector are concatenated to obtain a 1280-dimensional concatenated result. This concatenated result is then input into a second multilayer perceptron for dimensionality reduction, yielding a 256-dimensional first dimensionality-reduced result. This 256-dimensional first dimensionality-reduced result is input into a third multilayer perceptron to obtain a 128-dimensional second dimensionality-reduced result. This 128-dimensional second dimensionality-reduced result is then input into a fourth multilayer perceptron to obtain five relevant weight values. These five relevant weight values ​​reflect the weight of their respective correlation coefficients. It should be understood that the number of relevant weight values ​​is the same as the number of correlation coefficients. In the above embodiment, the number of correlation coefficients can be set to five, or it can be set to another number. The number of relevant weight values ​​and the number of correlation coefficients should not be construed as a limitation of this application.

[0116] 162. Based on the multiple relevant weight values ​​and the correlation coefficient corresponding to each of the relevant weight values, a correlation score is obtained, and the correlation score is the analysis result.

[0117] The relevance score is used to characterize the degree of relevance between the initial multimedia selection and the search term. Specifically, the higher the relevance score, the higher the degree of relevance between the initial multimedia selection and the search term; the lower the relevance score, the lower the degree of relevance between the initial multimedia selection and the search term.

[0118] Optionally, in one implementation, a weighted average can be calculated for multiple relevant weight values ​​and the correlation coefficient corresponding to each relevant weight value to obtain the correlation score. Let's assume the five correlation coefficients are 1, 2, 3, 4, and 5. Then the above calculation process can be achieved using the following formula (2):

[0119]

[0120] Where x represents the correlation score, k represents the correlation coefficient, and p k Here is the relevant weight value corresponding to k.

[0121] 163. Display the initially selected multimedia in descending order of relevance scores.

[0122] In the above implementation, since the initial multimedia to be processed is any one of the initial multimedia options, the above steps can be performed for each initial multimedia option to obtain a relevance score for each. Then, the multiple initial multimedia options can be displayed sequentially in descending order of relevance scores, so that the initial multimedia options in the first position are multimedia information with a higher relevance to the search term.

[0123] For ease of description, continuing with the example above, let's take video as an example to illustrate the search term, initial multimedia selection, and related multimedia: Let's assume the search term is "Warriors." Initial multimedia selection could include movie-related videos, game-related videos, basketball team-related videos, etc., containing the term "Warriors." Related multimedia could be videos related to the "Warriors" basketball team. Therefore, when displaying the initial multimedia selection in step 163, video results related to the basketball team can be displayed first, while videos related to the "Warriors" movie and game can be displayed later. This allows for personalized search results based on the user's historical behavior, thereby improving search efficiency.

[0124] Optionally, in one specific embodiment, determining the display result of the preliminary multimedia selection based on the analysis results may further include: comparing the relevance score with a preset score threshold, using the preliminary multimedia selection with a relevance score exceeding the preset score threshold as the display result, and eliminating the preliminary multimedia selection with a relevance score less than or equal to the preset score threshold. It should be understood that the specific method for determining the display result of the preliminary multimedia selection based on the analysis results should not be construed as a limitation of this application.

[0125] Optionally, in one specific embodiment, step 160 is implemented using a trained scoring model. Accordingly, prior to step 160, this embodiment may further include a training process for the scoring model, which specifically may include the following steps S1 to S8:

[0126] S1. Obtain the initial training multimedia, wherein the initial training multimedia is the search result corresponding to the training search term.

[0127] S2. Obtain training history multimedia, wherein the training history multimedia is multimedia in which the training terminal has performed interactive behaviors during the training history time period.

[0128] S3. Based on the matching degree between the training history multimedia and the training search term, select training-related multimedia from the training history multimedia.

[0129] S4. Perform first feature extraction processing on the training associated multimedia to obtain training associated features.

[0130] S5. Perform a second feature extraction process on the initial multimedia to be processed for training to obtain the training features; wherein, the initial multimedia to be processed for training is any one of the initial multimedia to be processed for training.

[0131] Steps S1 to S5 correspond to the same steps 110 to 150 described above, and will not be repeated here.

[0132] S6. Use the training scoring model to perform correlation analysis on the training search terms, the training related features, and the training features to be processed to obtain the training analysis results.

[0133] S7. Based on the training analysis results, the initial selection of multimedia and labels, construct a loss function.

[0134] The loss function can be a contrastive loss function. For ease of description, let's assume the training search term is q. t The initial selection of multimedia training involves n elements, denoted as v1, v2, ..., v n The label is y, where +1 represents a positive sample pair and -1 represents a negative sample pair; the training analysis result is the initial selection of multimedia v to be processed. i With training search term q t The degree of correlation is denoted as s(q) t ,v i The tags can be set manually by staff or automatically generated based on historical data. Specifically, the tags for training the initial multimedia selection can be determined based on the click-through rate of the search results for each search behavior.

[0135] In one implementation, the loss function can be constructed through the following steps:

[0136] The formula for the contrast loss function is as follows (3):

[0137] L(y,s1,s2)=max(0,-y×(s1-s2)+margin)(3)

[0138] Where s1 represents the positive sample pair s(q) t ,v p ), where v p Represents the relationship between the training search term q t Related training initial selection multimedia; s2 represents negative sample pair s(q) t ,v n ), where v n Represents the relationship between the training search term q t Unrelated training initial selection multimedia.

[0139] For each pair of samples, the loss is calculated as follows:

[0140] Positive sample pairs (q) t ,v p The formula is as follows (4):

[0141] L(+1,s(q t ,v p ),s(q t,v n ))=max(0,-(s(q t ,v p )-s(q t ,v n ))+margin)(4)

[0142] Negative sample pairs (q) t ,v n The formula is as follows (5):

[0143] L(-1,s(q t ,v p ),s(q t ,v n ))=max(0,(s(q t ,v p )-s(q t ,v n ))+margin)(5)

[0144] By summing the losses of all sample pairs, we can obtain the overall loss, as shown in formula (6):

[0145]

[0146] The overall loss is the loss function constructed in step S7.

[0147] S8. Based on the loss function, update the parameters of the scoring model until convergence, and obtain the trained scoring model.

[0148] After constructing the loss function, the gradient is calculated using the backpropagation algorithm, and the parameters of the scoring model are updated using an optimizer (such as Adam, SGD, etc.) to minimize the loss function, resulting in a trained scoring model. The performance of the trained scoring model can be evaluated using accuracy and the area under the ROC curve (AUC).

[0149] In the multimedia search method provided in this application embodiment, in response to the search operation of the target terminal, the search results corresponding to the search term are obtained and denoted as the initial selected multimedia. Multimedia in which the target terminal has performed interactive behavior within a historical time period are obtained and denoted as historical multimedia. Based on the matching degree between historical multimedia and the search term, the historical multimedia is filtered to select related multimedia related to the search term. Subsequently, feature extraction is performed on the related multimedia to obtain related features. For any one of the initial selected multimedia (let's call it the initial selected multimedia to be processed), feature extraction is performed to obtain the feature to be processed. Relevance analysis is performed on the search term, related features, and the feature to be processed to obtain analysis results; and based on the analysis results, the display results of the initial selected multimedia to be processed are determined. In this application embodiment, when a user searches for multimedia information, this application embodiment first obtains the search results corresponding to the search term as the initial selected search results; then, related multimedia matching the search term is filtered from the historical multimedia; subsequently, based on the related multimedia and the search term, the initial selected search results are analyzed one by one regarding relevance to obtain the analysis results corresponding to each initial selected search result, and based on the analysis results, the display method of multiple initial selected search results is determined. This application uses historical multimedia interaction data as an important basis when displaying search results, so that the search results can better meet the personalized needs of users.

[0150] The embodiments of this application can improve search efficiency.

[0151] In this embodiment, video will be used as an example to describe the method of this application embodiment in detail. The multimedia search method provided in this application embodiment can be executed by an electronic device. In this embodiment, a terminal device will be used as an example for description, such as... Figure 2 As shown, the specific process of a multimedia search method is as follows:

[0152] 201. In response to a search operation from a target terminal, obtain a preliminary selection of videos, wherein the search operation includes search terms, and the preliminary selection of videos are the search results corresponding to the search terms.

[0153] 202. Obtain historical videos, wherein the historical videos are videos in which the target terminal performed interactive behaviors within a historical time period.

[0154] 203. Perform text extraction processing on each of the historical videos to obtain the corresponding historical video extraction results.

[0155] 204. For each historical video corresponding to a historical video, determine the hit ratio of the number of hits to the total number of historical video extraction results for that historical video, wherein the number of hits is the number of times the historical video extraction results corresponding to that historical video hit the search term.

[0156] 205. Based on the hit ratio and the preset ratio threshold, filter out the associated videos from the historical videos.

[0157] 206. Extract text from the associated video to obtain the associated extraction results.

[0158] 207. Arrange the multiple association extraction results in a set order to obtain the association extraction sequence.

[0159] 208. Perform feature extraction on the associated extraction sequence to obtain associated video features.

[0160] 209. Extract text from the initial selected videos to be processed to obtain the extraction result to be processed. The initial selected videos to be processed are any one of the initial selected videos.

[0161] 210. Perform feature extraction on the extraction results to be processed to obtain the initial selected features to be processed.

[0162] 211. Perform correlation analysis on search terms, associated extraction sequences, associated video features, extraction results to be processed, and preliminary selected features to be processed to obtain multiple related weight values, where each related weight value reflects the weight of the corresponding correlation coefficient.

[0163] Optionally, in one implementation, the relevance analysis of the search terms, associated extraction sequences, associated video features, extraction results to be processed, and preliminary selected features to be processed can be performed using a trained scoring model. Accordingly, prior to step 211, this embodiment may further include a training process for the scoring model, specifically including the following steps A1 to A8:

[0164] A1. Obtain the initial training videos, wherein the initial training videos are the search results corresponding to the training search terms.

[0165] A2. Obtain training history videos, wherein the training history videos are videos of interactive behaviors performed by the training terminal during the training history period.

[0166] A3. Based on the matching degree between the training history videos and the training search terms, select training-related videos from the training history videos.

[0167] A4. Perform first feature extraction processing on the training associated video to obtain training associated features.

[0168] A5. Perform a second feature extraction process on the initial selection video to be processed for training to obtain the training features; wherein, the initial selection video to be processed for training is any one of the initial selection videos.

[0169] A6. Use the training scoring model to perform correlation analysis on the training search terms, the training related features, and the training features to be processed to obtain the training analysis results.

[0170] A7. Based on the training analysis results, the initial training videos and labels, construct the loss function.

[0171] A8. Based on the loss function, update the parameters of the scoring model until convergence, and obtain the trained scoring model.

[0172] 212. Based on the multiple relevant weight values ​​and the correlation coefficient corresponding to each of the relevant weight values, obtain the correlation score.

[0173] 213. Display the preliminary selected videos in descending order of relevance scores.

[0174] The specific execution process of steps 201 to 213 has been explained in detail above, and will not be repeated here.

[0175] In the multimedia search method provided in this application embodiment, in response to the search operation of the target terminal, the search results corresponding to the search term are obtained and denoted as the initial selected multimedia. Multimedia in which the target terminal has performed interactive behavior within a historical time period are obtained and denoted as historical multimedia. Based on the matching degree between historical multimedia and the search term, the historical multimedia is filtered to select related multimedia related to the search term. Subsequently, feature extraction is performed on the related multimedia to obtain related features. For any one of the initial selected multimedia (let's call it the initial selected multimedia to be processed), feature extraction is performed to obtain the feature to be processed. Relevance analysis is performed on the search term, related features, and the feature to be processed to obtain analysis results; and based on the analysis results, the display results of the initial selected multimedia to be processed are determined. In this application embodiment, when a user searches for multimedia information, this application embodiment first obtains the search results corresponding to the search term as the initial selected search results; then, related multimedia matching the search term is filtered from the historical multimedia; subsequently, based on the related multimedia and the search term, the initial selected search results are analyzed one by one regarding relevance to obtain the analysis results corresponding to each initial selected search result, and based on the analysis results, the display method of multiple initial selected search results is determined. This application uses historical multimedia interaction data as an important basis when displaying search results, so that the search results can better meet the personalized needs of users.

[0176] The embodiments of this application can improve search efficiency.

[0177] To better implement the above methods, embodiments of this application also provide a multimedia search device, such as... Figure 3 As shown, the device includes:

[0178] The initial multimedia selection unit 301 is used to obtain initial multimedia in response to a search operation of the target terminal, wherein the search operation includes a search term, and the initial multimedia is the search result corresponding to the search term;

[0179] The historical multimedia unit 302 is used to acquire historical multimedia, wherein the historical multimedia is multimedia in which the target terminal has performed interactive behavior within a historical time period;

[0180] The associated multimedia unit 303 is used to filter associated multimedia from the historical multimedia based on the degree of matching between the historical multimedia and the search term;

[0181] The first extraction unit 304 is used to perform a first feature extraction process on the associated multimedia to obtain associated features;

[0182] The second extraction unit 305 is used to perform second feature extraction processing on the initial multimedia to be processed to obtain the feature to be processed; wherein, the initial multimedia to be processed is any one of the initial multimedia;

[0183] The result display unit 306 is used to perform correlation analysis on the search terms, the associated features, and the features to be processed to obtain analysis results; and based on the analysis results, to determine the display results of the preliminary multimedia to be processed.

[0184] In one embodiment, the associated features include an associated extraction sequence and associated multimedia features; correspondingly, the first extraction unit 304 includes:

[0185] The associated text subunit is used to extract text from the associated multimedia to obtain the associated extraction result;

[0186] The sequential arrangement subunit is used to arrange multiple association extraction results in a set order to obtain an association extraction sequence;

[0187] The associated feature subunit is used to extract features from the associated extraction sequence to obtain associated multimedia features.

[0188] In one embodiment, the features to be processed include the extraction results to be processed and the initial selected features to be processed; correspondingly, the second extraction unit includes:

[0189] The text to be processed subunit is used to extract text from the initially selected multimedia to obtain the extraction result.

[0190] The feature subunit to be processed is used to extract features from the extraction result to obtain the initial selected features to be processed.

[0191] In one embodiment, the associated multimedia unit 303 includes:

[0192] The historical text subunit is used to perform text extraction processing on each of the historical multimedia files to obtain the corresponding historical multimedia extraction results.

[0193] The ratio calculation subunit is used to determine the hit ratio of the number of hits to the total number of historical multimedia extraction results for each historical multimedia, wherein the number of hits is the number of times the historical multimedia extraction results for the historical multimedia hit the search term.

[0194] The multimedia filtering subunit is used to filter out the associated multimedia from the historical multimedia based on the hit ratio and a preset ratio threshold.

[0195] In one implementation, the result display unit 306 includes:

[0196] The relevance analysis subunit is used to perform relevance analysis on the search term, the associated features, and the features to be processed to obtain multiple relevance weight values, wherein each relevance weight value reflects the weight of the corresponding relevance coefficient;

[0197] The correlation score subunit is used to obtain a correlation score based on the multiple correlation weight values ​​and the correlation coefficient corresponding to each correlation weight value, wherein the correlation score is the analysis result.

[0198] In one embodiment, the result display unit 306 further includes:

[0199] The multimedia display subunit is used to display the initially selected multimedia in descending order of relevance scores.

[0200] In one implementation, the result display unit 306 is specifically used to: perform correlation analysis on the search term, the associated features, and the features to be processed using the trained scoring model to obtain analysis results.

[0201] In one embodiment, the device further includes:

[0202] A training initial selection unit is used to obtain training initial selection multimedia, wherein the training initial selection multimedia is the search result corresponding to the training search term;

[0203] The training history unit is used to acquire training history multimedia, wherein the training history multimedia is multimedia in which the training terminal has performed interactive behaviors during the training history time period;

[0204] A matching degree unit is used to filter out training-related multimedia from the training history multimedia based on the matching degree between the training history multimedia and the training search term;

[0205] The first feature training unit is used to perform first feature extraction processing on the training associated multimedia to obtain training associated features;

[0206] The second feature training unit is used to perform second feature extraction processing on the initial multimedia to be processed for training, so as to obtain the training features; wherein, the initial multimedia to be processed for training is any one of the initial multimedia to be processed for training.

[0207] The training relevance analysis unit is used to perform relevance analysis on the training search terms, the training related features, and the training features to be processed using the scoring model to be trained, so as to obtain the training analysis results.

[0208] The loss function unit is used to construct a loss function based on the training analysis results, the initial selection of multimedia and labels;

[0209] The parameter update unit is used to update the parameters of the scoring model based on the loss function until convergence, thereby obtaining the trained scoring model.

[0210] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0211] In the multimedia search method provided in this application embodiment, in response to the search operation of the target terminal, the search results corresponding to the search term are obtained and denoted as the initial selected multimedia. Multimedia in which the target terminal has performed interactive behavior within a historical time period are obtained and denoted as historical multimedia. Based on the matching degree between historical multimedia and the search term, the historical multimedia is filtered to select related multimedia related to the search term. Subsequently, feature extraction is performed on the related multimedia to obtain related features. For any one of the initial selected multimedia (let's call it the initial selected multimedia to be processed), feature extraction is performed to obtain the feature to be processed. Relevance analysis is performed on the search term, related features, and the feature to be processed to obtain analysis results; and based on the analysis results, the display results of the initial selected multimedia to be processed are determined. In this application embodiment, when a user searches for multimedia information, this application embodiment first obtains the search results corresponding to the search term as the initial selected search results; then, related multimedia matching the search term is filtered from the historical multimedia; subsequently, based on the related multimedia and the search term, the initial selected search results are analyzed one by one regarding relevance to obtain the analysis results corresponding to each initial selected search result, and based on the analysis results, the display method of multiple initial selected search results is determined. This application uses historical multimedia interaction data as an important basis when displaying search results, so that the search results can better meet the personalized needs of users.

[0212] The embodiments of this application can improve search efficiency.

[0213] This application also provides an electronic device. In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 4 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0214] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0215] The processor 401 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 401.

[0216] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and multimedia search model training by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0217] The electronic device also includes a power supply 403 that supplies power to the various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0218] The electronic device may also include an input module 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0219] The electronic device may also include a communication module 405. In some embodiments, the communication module 405 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 405 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0220] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402, thereby realizing the various functions in the various method embodiments of this application.

[0221] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0222] Therefore, embodiments of this application provide a computer-readable storage medium storing instructions that can be loaded by a processor to perform steps in any of the multimedia search methods provided in embodiments of this application. For example, the instructions can perform various steps in the various method embodiments of this application.

[0223] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0224] According to one aspect of this application, a computer program product or computer program is provided, comprising instructions stored in a computer-readable storage medium. A processor of a computer device reads the instructions from the computer-readable storage medium and executes the instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0225] Since the instructions stored in the storage medium can execute the steps of any of the multimedia search methods provided in the embodiments of this application, the beneficial effects that any of the multimedia search methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0226] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0227] The above provides a detailed description of a multimedia search method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multimedia search method, characterized in that, The method includes: In response to a search operation on a target terminal, a preliminary selection of multimedia is obtained, wherein the search operation includes a search term, and the preliminary selection of multimedia is the search result corresponding to the search term; Acquire historical multimedia, wherein the historical multimedia refers to multimedia in which the target terminal has performed interactive behaviors within a historical time period; Based on the degree of matching between the historical multimedia and the search term, related multimedia is filtered out from the historical multimedia. The associated multimedia is subjected to a first feature extraction process to obtain the associated features; The preliminary multimedia to be processed is subjected to a second feature extraction process to obtain the feature to be processed; wherein, the preliminary multimedia to be processed is any one of the preliminary multimedia; The search terms, the associated features, and the features to be processed are subjected to correlation analysis to obtain analysis results; and based on the analysis results, the display results of the initial multimedia selection to be processed are determined.

2. The method as described in claim 1, characterized in that, The associated features include associated extracted sequences and associated multimedia features; The first feature extraction process for the associated multimedia to obtain associated features includes: Text extraction is performed on the associated multimedia to obtain the associated extraction results; The multiple association extraction results are arranged in a set order to obtain an association extraction sequence; Feature extraction is performed on the associated extraction sequence to obtain associated multimedia features.

3. The method as described in claim 1, characterized in that, The features to be processed include the extraction results to be processed and the initial selected features to be processed; The second feature extraction process is performed on the initially selected multimedia to be processed to obtain the features to be processed, including: Text extraction is performed on the initially selected multimedia files to obtain the extracted results. Feature extraction is performed on the extraction results to obtain the initial selected features to be processed.

4. The method as described in claim 1, characterized in that, The step of filtering related multimedia from the historical multimedia based on the degree of matching between the historical multimedia and the search term includes: Each of the historical multimedia files is processed for text extraction to obtain the corresponding historical multimedia extraction result; For each historical multimedia extraction result corresponding to the historical multimedia, determine the hit ratio of the number of hits to the total number of historical multimedia extraction results for that historical multimedia, wherein the number of hits is the number of times the historical multimedia extraction results corresponding to that historical multimedia hit the search term; Based on the hit ratio and the preset ratio threshold, the associated multimedia is selected from the historical multimedia.

5. The method as described in claim 1, characterized in that, The relevance analysis of the search terms, the associated features, and the features to be processed yields the following results: A correlation analysis is performed on the search term, the associated features, and the features to be processed to obtain multiple related weight values, wherein each related weight value reflects the weight of the corresponding correlation coefficient; Based on the multiple relevant weight values ​​and the correlation coefficient corresponding to each relevant weight value, a correlation score is obtained, and the correlation score is the analysis result.

6. The method as described in claim 5, characterized in that, The step of determining the display result of the initially selected multimedia to be processed based on the analysis results includes: The initially selected multimedia resources are displayed in descending order of relevance scores.

7. The method as described in claim 1, characterized in that, The relevance analysis of the search terms, the associated features, and the features to be processed yields the following results: The trained scoring model is used to perform correlation analysis on the search terms, the associated features, and the features to be processed, and the analysis results are obtained.

8. The method as described in claim 7, characterized in that, Before performing relevance analysis on the search term, the associated features, and the features to be processed using the trained scoring model to obtain the analysis results, the method further includes: Obtain training initial multimedia, wherein the training initial multimedia is the search result corresponding to the training search term; Acquire training history multimedia, wherein the training history multimedia is multimedia in which the training terminal has performed interactive behaviors during the training history time period; Based on the degree of matching between the training history multimedia and the training search terms, training-related multimedia is selected from the training history multimedia. The training associated multimedia is subjected to a first feature extraction process to obtain training associated features; The initial multimedia to be processed for training is subjected to a second feature extraction process to obtain the training features; wherein, the initial multimedia to be processed for training is any one of the initial multimedia to be processed for training. The training search terms, training related features, and training features to be processed are analyzed using the scoring model to be trained to obtain the training analysis results. Based on the training analysis results, the initial selection of multimedia and labels, a loss function is constructed. Based on the loss function, the parameters of the scoring model are updated until convergence, resulting in a trained scoring model.

9. A multimedia search device, characterized in that, The device includes: The initial multimedia selection unit is used to respond to a search operation of the target terminal and obtain initial multimedia, wherein the search operation includes a search term, and the initial multimedia is the search result corresponding to the search term; A historical multimedia unit is used to acquire historical multimedia, wherein the historical multimedia is multimedia in which the target terminal has performed interactive behaviors within a historical time period; The associated multimedia unit is used to filter associated multimedia from the historical multimedia based on the degree of matching between the historical multimedia and the search term; The first extraction unit is used to perform a first feature extraction process on the associated multimedia to obtain associated features; The second extraction unit is used to perform second feature extraction processing on the initial multimedia to be processed to obtain the features to be processed; wherein, the initial multimedia to be processed is any one of the initial multimedia; The results display unit is used to perform relevance analysis on the search terms, the associated features, and the features to be processed to obtain analysis results; and based on the analysis results, to determine the display results of the initial multimedia selection to be processed.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing instructions; the processor loads instructions from the memory to perform the steps in the multimedia search method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions adapted for loading by a processor to perform the steps of the multimedia search method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, The method includes instructions that, when executed by a processor, implement the steps of the multimedia search method according to any one of claims 1 to 8.

Images