Video search method, device, electronic device and storage medium

Abstract

The present disclosure relates to a video search method, device, electronic device and storage medium. The method includes: receiving a search request carrying a search corpus; when the search corpus includes at least one target corpus segment indicating abnormal text, taking at least one target corpus segment as a search object to obtain a first candidate video sequence; determining at least one reference video in the first candidate video sequence; calculating an abnormal value corresponding to each reference video; calculating a filtering threshold according to the abnormal value corresponding to each reference video; taking the search corpus as a search object to obtain a second candidate video sequence; filtering the second candidate video sequence based on the filtering threshold to obtain a search result corresponding to the search request. The present disclosure combines the abnormal text attributes of the target corpus segment and the abnormal value of the reference video to determine the filtering threshold for the search corpus. The second candidate video sequence is filtered using the filtering threshold to effectively filter the abnormal content to reduce the aggregation of abnormal videos in the search results.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet communication technology, and in particular to a video search method, device, electronic device and storage medium. Background Art

[0002] With the development of Internet communication technology, based on the input search object, the video search system can return corresponding search feedback to the user, and the search feedback can include relevant one or all kinds of media content (such as video, short video, audio, image, text, etc.).

[0003] The returned search feedback often includes abnormal content, and even abnormal content may be concentrated. Abnormal content can be content that does not comply with the provisions of laws and administrative regulations, content that violates social morality, or content that damages national interests, social public interests, or the interests of third parties. In related technologies, when processing the returned search feedback, a fixed abnormal content evaluation threshold is often used to filter out content that is greater than the abnormal content evaluation threshold from the search feedback to obtain search results. In this way, when faced with search feedback corresponding to different search objects, it is easy for non-abnormal content to be filtered out and abnormal content to fail to be filtered out in the search results. Therefore, a video search solution that effectively filters abnormal content is needed. Summary of the invention

[0004] The present disclosure provides a video search method, device, electronic device and storage medium to at least solve the problem of lack of effective filtering of abnormal content in the related art. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a video search method is provided, the method comprising:

[0006] Receiving a search request carrying a search corpus;

[0007] When the search corpus includes at least one target corpus segment indicating abnormal text, taking the at least one target corpus segment as a search object to obtain a first candidate video sequence;

[0008] Determine at least one reference video in the first candidate video sequence;

[0009] Calculate the outlier value corresponding to each reference video;

[0010] Calculating a filtering threshold according to the outliers corresponding to each of the reference videos;

[0011] Taking the search corpus as a search object, obtaining a second candidate video sequence;

[0012] The second candidate video sequence is filtered based on the filtering threshold to obtain a search result corresponding to the search request.

[0013] In an exemplary implementation, the calculating the filtering threshold according to the abnormal values ​​corresponding to each of the reference videos includes:

[0014] Arranging the outlier values ​​corresponding to the reference videos in descending order to obtain an outlier value sequence;

[0015] Obtain a first target value M;

[0016] The Mth outlier in the outlier sequence is used as the filtering threshold.

[0017] In an exemplary embodiment, obtaining the first target value M includes:

[0018] Obtaining a scaling factor and the number of elements in the outlier sequence;

[0019] The product of the proportional factor and the number of elements in the abnormal value sequence is used as the first target value M.

[0020] In an exemplary implementation, filtering the second candidate video sequence based on the filtering threshold to obtain search results corresponding to the search request includes:

[0021] Calculating an outlier value corresponding to each second candidate video in the second candidate video sequence;

[0022] The second candidate videos whose corresponding abnormal values ​​in the second candidate video sequence are greater than the filtering threshold are filtered out to obtain search results corresponding to the search request.

[0023] In an exemplary embodiment, determining at least one reference video in the first candidate video sequence includes:

[0024] Obtain a second target value N;

[0025] The first N first candidate videos in the first candidate video sequence are determined as the reference videos.

[0026] In an exemplary implementation, the calculating of the outlier value corresponding to each of the reference videos includes:

[0027] Determine a reference image corresponding to each of the reference videos;

[0028] Calculating an outlier value of the reference image;

[0029] The abnormal value of the benchmark image is used as the abnormal value corresponding to the reference video.

[0030] In an exemplary embodiment, the calculating the abnormal value of the reference image includes:

[0031] Inputting the reference image into an abnormal image recognition model to obtain a first recognition result;

[0032] determining an abnormal value of the reference image according to the first recognition result;

[0033] The abnormal image recognition model is determined by machine learning training based on multiple sample images, and each of the sample images carries annotation information indicating its abnormal image attributes.

[0034] In an exemplary embodiment, the method further includes training the abnormal image recognition model, wherein the training the abnormal image recognition model includes:

[0035] acquiring the plurality of sample images;

[0036] A first preset machine learning model is trained according to the multiple sample images until the loss generated by the first preset machine learning model is less than a first loss threshold, so as to obtain the abnormal image recognition model.

[0037] In an exemplary embodiment, when the search corpus includes at least one target corpus segment indicating abnormal text, taking the at least one target corpus segment as a search object, before obtaining the first candidate video sequence, the method further includes the step of determining the at least one target corpus segment indicating abnormal text;

[0038] The step of determining at least one target corpus segment indicating abnormal text comprises:

[0039] Inputting the search corpus into an abnormal text recognition model to perform abnormal text recognition to obtain a second recognition result;

[0040] Determining at least one target corpus segment indicating abnormal text according to the second recognition result;

[0041] The abnormal text recognition model is determined by machine learning training based on multiple sample corpora, and each of the sample corpora carries annotation information indicating its abnormal text attributes.

[0042] In an exemplary embodiment, the method further includes a step of training the abnormal text recognition model, wherein the training of the abnormal text recognition model includes:

[0043] Acquire the plurality of sample corpora;

[0044] A second preset machine learning model is trained according to the multiple sample images until the loss generated by the second preset machine learning model is less than a second loss threshold, so as to obtain the abnormal text recognition model.

[0045] In an exemplary embodiment, taking the at least one target corpus segment as a search object to obtain a first candidate video sequence includes:

[0046] Taking each of the target corpus segments as a search object, respectively, to obtain a video sequence corresponding to each of the target corpus segments;

[0047] According to a preset sequence merging rule, the video sequences corresponding to the target corpus segments are merged to obtain the first candidate video sequence.

[0048] According to a second aspect of an embodiment of the present disclosure, a video search device is provided, the device comprising:

[0049] A search request receiving unit, configured to receive a search request carrying a search corpus;

[0050] A first candidate video sequence obtaining unit is configured to obtain a first candidate video sequence by taking the at least one target corpus segment as a search object when the search corpus includes at least one target corpus segment indicating abnormal text;

[0051] A reference video determining unit, configured to determine at least one reference video in the first candidate video sequence;

[0052] An outlier value calculation unit, configured to calculate an outlier value corresponding to each of the reference videos;

[0053] A filtering threshold calculation unit, configured to calculate a filtering threshold according to an abnormal value corresponding to each of the reference videos;

[0054] A second candidate video sequence obtaining unit is configured to perform a search using the search corpus as a search object to obtain a second candidate video sequence;

[0055] The filtering unit is configured to filter the second candidate video sequence based on the filtering threshold to obtain search results corresponding to the search request.

[0056] In an exemplary embodiment, the filtering threshold calculation unit includes:

[0057] An outlier sequence obtaining unit is configured to arrange the outliers corresponding to the reference videos in descending order to obtain an outlier sequence;

[0058] A first target value acquisition unit, configured to execute acquisition of a first target value M;

[0059] The filtering threshold determination unit is configured to use the Mth abnormal value in the abnormal value sequence as the filtering threshold.

[0060] In an exemplary embodiment, the first target value obtaining unit includes:

[0061] A parameter acquisition unit, configured to execute acquisition of a scale factor and the number of elements in the outlier sequence;

[0062] The first target value determination unit is configured to perform the multiplication of the proportional factor and the number of elements in the abnormal value sequence as the first target value M.

[0063] In an exemplary embodiment, the filtration unit comprises:

[0064] A first type of outlier calculation unit is configured to calculate an outlier value corresponding to each second candidate video in the second candidate video sequence;

[0065] The filtering subunit is configured to filter out the second candidate videos in the second candidate video sequence whose corresponding abnormal values ​​are greater than the filtering threshold, and obtain search results corresponding to the search request.

[0066] In an exemplary embodiment, the reference video determination includes:

[0067] A second target value determination unit, configured to execute acquisition of a second target value N;

[0068] The reference video determination subunit is configured to determine the first N first candidate videos in the first candidate video sequence as the reference videos.

[0069] In an exemplary embodiment, the outlier calculation includes:

[0070] A reference image determining unit, configured to determine a reference image corresponding to each of the reference videos;

[0071] A second type of outlier calculation unit, configured to calculate an outlier value of the reference image;

[0072] The outlier determination unit is configured to use the outlier value of the benchmark image as the outlier value corresponding to the reference video.

[0073] In an exemplary embodiment, the second type of outlier calculation unit includes:

[0074] A first recognition result obtaining unit is configured to input the reference image into an abnormal image recognition model to obtain a first recognition result;

[0075] A second type of outlier determination unit is configured to determine an outlier value of the reference image according to the first recognition result;

[0076] The abnormal image recognition model is determined by machine learning training based on multiple sample images, and each of the sample images carries annotation information indicating its abnormal image attributes.

[0077] In an exemplary embodiment, the device further includes a first model training unit for training the abnormal image recognition model, wherein the first model training unit includes:

[0078] A sample image acquisition unit, configured to acquire the plurality of sample images;

[0079] The first model training subunit is configured to execute training of a first preset machine learning model according to the multiple sample images until the loss generated by the first preset machine learning model is less than a first loss threshold, so as to obtain the abnormal image recognition model.

[0080] In an exemplary embodiment, the device further includes a target corpus segment determining unit for determining at least one target corpus segment indicating the abnormal text, the target corpus segment determining unit including:

[0081] A second recognition result obtaining unit is configured to input the search corpus into an abnormal text recognition model to perform abnormal text recognition to obtain a second recognition result;

[0082] a target corpus segment determination subunit, configured to determine at least one target corpus segment indicating abnormal text according to the second recognition result;

[0083] The abnormal text recognition model is determined by machine learning training based on multiple sample corpora, and each of the sample corpora carries annotation information indicating its abnormal text attributes.

[0084] In an exemplary embodiment, the device further includes a second model training unit for training the abnormal text recognition model, and the second model training unit includes:

[0085] A sample corpus acquisition unit, configured to acquire the plurality of sample corpora;

[0086] The second model training subunit is configured to execute training of a second preset machine learning model according to the multiple sample corpora until the loss generated by the second preset machine learning model is less than a second loss threshold, so as to obtain the abnormal text recognition model.

[0087] In an exemplary embodiment, the first candidate video sequence obtaining unit includes:

[0088] A video sequence obtaining unit is configured to perform a search using each of the target corpus segments as a search object to obtain a video sequence corresponding to each of the target corpus segments;

[0089] The merging unit is configured to execute a preset sequence merging rule to merge the video sequences corresponding to the target corpus segments to obtain the first candidate video sequence.

[0090] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0091] processor;

[0092] a memory for storing instructions executable by the processor;

[0093] The processor is configured to execute the instructions to implement the video search method as described in the first aspect.

[0094] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the video search method as described in the first aspect.

[0095] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided. When the computer program product is run on a computer, the computer executes the video search method described in the first aspect.

[0096] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0097] In video search, a first candidate video sequence is obtained based on a target corpus segment indicating abnormal text in the search corpus, and then an abnormal value is determined for a reference video determined from the first candidate video sequence to obtain a filtering threshold, and then a second candidate video sequence is obtained using the search corpus as a search object, and then the second candidate video sequence is filtered based on the filtering threshold to obtain search results corresponding to the search request. The present disclosure combines the abnormal text attributes of the target corpus segment and the abnormal value of the reference video to determine the filtering threshold for the search corpus. By using the filtering threshold to filter the second candidate video sequence as search feedback, the abnormal content therein can be effectively filtered, the misfiltering of non-abnormal videos can be reduced, and the aggregation of abnormal videos in the search results can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0099] Figure 1 The figure is a flowchart of a video search method according to an exemplary embodiment.

[0100] Figure 2 The invention is a flowchart showing a method of determining at least one target corpus segment indicating abnormal text according to an exemplary embodiment.

[0101] Figure 3 The invention is a flowchart showing a method of determining at least one reference video in a first candidate video sequence according to an exemplary embodiment.

[0102] Figure 4 The figure is a flow chart showing how to calculate an abnormal value corresponding to each reference video according to an exemplary embodiment.

[0103] Figure 5 It is a diagram of an application scenario of an abnormal image recognition model according to an exemplary embodiment.

[0104] Figure 6 The present invention is a flow chart showing how to calculate a filtering threshold value according to abnormal values ​​corresponding to each reference video according to an exemplary embodiment.

[0105] Figure 7 The figure is a block diagram of a video search device according to an exemplary embodiment.

[0106] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0107] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0108] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0109] The video search method provided by the present disclosure can be applied to a terminal or server installed with a video search system, and the terminal and the server can be connected through a wired network or a wireless network. The terminal can specifically be a smart phone, a desktop computer, a tablet computer, a laptop computer, an augmented reality (AR) / virtual reality (VR) device, a digital assistant, a smart speaker, a smart wearable device, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The video search method provided by the present disclosure can be executed independently by the client, can be executed independently by the server, or can be executed by the client interacting with the server.

[0110] Figure 1 is a flow chart of a video search method according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps S101 to S107.

[0111] In step S101, a search request carrying a search corpus is received.

[0112] The client or server receives a search request carrying a search corpus (query), and the search corpus can come from the input of the target object. The search corpus obtained by the client can come from the direct input of the target object. The search corpus obtained by the server can come from the direct input or indirect input of the target object, where the indirect input can be expressed as: the target object inputs the search corpus through the client running on the terminal, and the client sends the search corpus to the corresponding server (which can provide background services for the client). The target object can indicate a user, a user simulator, etc.

[0113] The search corpus may be natural corpus in text form, such as sentences, phrases, etc. The natural corpus in text form may be obtained by converting natural corpus in image form or voice form.

[0114] In step S102, when the search corpus includes at least one target corpus segment indicating abnormal text, the at least one target corpus segment is used as a search object to obtain a first candidate video sequence.

[0115] The search corpus includes at least one target corpus segment indicating abnormal text, which may indicate that all corpus segments of the search corpus have abnormal text attributes, or that some corpus segments of the search corpus have abnormal text attributes. The corpus segments may be obtained by segmenting the search corpus. Having abnormal text attributes may indicate that the target corpus segment represents text content that does not comply with laws and administrative regulations, violates social morality, or harms national interests, social public interests, or third-party interests. For example, the target corpus segment has a yellow suggestive attribute.

[0116] The client or server can use at least one target corpus segment as a search object and obtain a first candidate video sequence using a preset search engine. The first candidate video sequence can be obtained by sorting multiple first candidate videos in descending order according to a reference value. The reference value can be determined by a preset search engine. The reference value can be determined by a preset search engine based on the length of time a video is viewed, the number of times a video is clicked, the click-through rate of a video (the ratio of the number of times a certain content on a web page is clicked to the number of times it is displayed, i.e., clicks / views, which is a percentage), or the score of a video. The reference value can also be determined by a preset search engine based on any combination of the above-mentioned length of stay, number of clicks, click-through rate, and score (including a combination of two, three, and four).

[0117] In one embodiment, when the search corpus includes at least one target corpus segment indicating abnormal text, taking the at least one target corpus segment as the search object and obtaining the first candidate video sequence further includes the step of determining the at least one target corpus segment indicating abnormal text; Figure 2 As shown, the step includes:

[0118] S201: inputting the search corpus into an abnormal text recognition model to perform abnormal text recognition to obtain a second recognition result;

[0119] S202: Determine at least one target corpus segment indicating abnormal text according to the second recognition result;

[0120] The abnormal text recognition model is determined by machine learning training based on multiple sample corpora, each of which carries annotation information indicating its abnormal text attributes (such as yellow hint attributes). The abnormal text recognition model can be used as a text classifier to process the search corpus, which can include the following two processing methods:

[0121] 1) The client or server can directly input the search corpus into the abnormal text recognition model. The abnormal text recognition model performs word segmentation on the search corpus, performs abnormal text recognition (text classification) on the corpus segments obtained through the word segmentation, and outputs the recognition results of the corpus segments. The recognition results here are for all corpus segments of the search corpus. Accordingly, the client or server can use the recognition results here as the second recognition results, and then determine at least one target corpus segment indicating abnormal text according to the second recognition results.

[0122] 2) The client or server may perform word segmentation processing on the search corpus, and input the corpus segments obtained through the word segmentation processing into the abnormal text recognition model (for example, when at least two corpus segments are obtained, each corpus segment is input into the abnormal text recognition model respectively). The abnormal text recognition model outputs the recognition result of the corpus segment. The recognition result here is for a certain corpus segment of the search corpus. Accordingly, when the word segmentation result of the search corpus is a corpus segment, the client or server may use the recognition result here as the second recognition result, and then determine at least one target corpus segment indicating abnormal text according to the second recognition result. When the word segmentation result of the search corpus is at least two corpus segments, the client or server may synthesize the recognition results of each corpus segment to obtain a second recognition result, and then determine at least one target corpus segment indicating abnormal text according to the second recognition result.

[0123] Using the abnormal text recognition model to perform abnormal text recognition on the search corpus can improve the adaptability and accuracy of abnormal text recognition, and further improve the reliability and effectiveness of abnormal text recognition.

[0124] The steps of training the abnormal text recognition model are introduced below, which include: first, obtaining the multiple sample corpora; then, training a second preset machine learning model based on the multiple sample corpora until the loss generated by the second preset machine learning model is less than a second loss threshold, so as to obtain the abnormal text recognition model.

[0125] The sample corpus can come from the collection of online historical search corpora. The second preset machine learning model is trained based on multiple annotated sample corpora until the loss generated by the second preset machine learning model is less than the second loss threshold, which can also be manifested as the loss value almost no longer decreasing. This can indicate that the second preset machine learning model has met the convergence conditions, and the training can be terminated to obtain the abnormal text recognition model. The second preset machine learning model can use a transformer-based bidirectional encoding representation model (BERT model). For multiple sample corpora, the open source BERT model can be used to load pre-training parameters for model training. The above training steps can obtain an abnormal text recognition model with high generalization ability.

[0126] Regarding the above two methods of processing search corpora using abnormal text recognition models, there are also some differences in the training of abnormal text recognition models:

[0127] 1) For the first type of processing method: the annotation information carried by the sample corpus is in the corpus segment dimension. For example, sample corpus A includes corpus segment a1 and corpus segment a2. Then, the annotation information carried by the sample corpus includes first annotation information indicating the abnormal text attribute of corpus segment a1, and second annotation information indicating the abnormal text attribute of corpus segment a2. The abnormal text attributes indicated by the first annotation information and the second annotation information may be different. For example, if corpus segment a1 has abnormal text attributes, then the first annotation information may be flag=1 (positive incentive); if corpus segment a2 does not have abnormal text attributes, then the second annotation information may be flag=0 (negative incentive).

[0128] 2) For the second type of processing method: the annotation information carried by the sample corpus is corpus-dimensional. The sample corpus can be divided into positive sample corpus with abnormal text attributes and negative sample corpus without abnormal text attributes. The positive sample corpus carries the first type of annotation information indicating its abnormal text attributes (such as flag=1), and the negative sample corpus carries the second type of annotation information indicating its abnormal text attributes (such as flag=0). It should be noted that for the collected online historical search corpus, the online historical search corpus can be directly used as a sample corpus as a whole, or the online historical search corpus can be first segmented, and then the obtained corpus fragments can be used as sample corpus.

[0129] In another embodiment, step S102 may include step S1021, taking each of the target corpus segments as a search object to obtain a video sequence corresponding to each of the target corpus segments; and step S1022, merging the video sequences corresponding to each of the target corpus segments according to a preset sequence merging rule to obtain the first candidate video sequence.

[0130] When the search corpus includes a target corpus segment, the client or server can use the target corpus segment as the search object, use a preset search engine to obtain the video sequence corresponding to the target corpus segment, and then use the video sequence corresponding to the target corpus segment as the first candidate video sequence.

[0131] When the search corpus includes at least two target corpus segments, the client or server can first use each target corpus segment as a search object, and use a preset search engine to obtain the video sequence corresponding to each target corpus segment; then merge the video sequences corresponding to each target corpus segment according to the preset sequence merging rule to obtain the first candidate video sequence. The preset sequence merging rule here can refer to the relevant records of the aforementioned reference value, that is, the video sequences corresponding to each target corpus segment can be regarded as a video set as a whole, and then sorted from high to low according to the reference value of each video in the video set to obtain the first candidate video sequence. Of course, in the process of integrating and sorting videos, video deduplication can also be performed. According to the number of target corpus segments, different first candidate video sequence acquisition methods are formulated, and the acquisition path of the first candidate video sequence can be standardized with more fine-grained standards, especially for the case where the number of target corpus segments is greater than or equal to 2.

[0132] In step S103, at least one reference video is determined in the first candidate video sequence.

[0133] The client or the server may select all the first candidate videos in the first candidate video sequence as reference videos, or may select some of the first candidate videos in the first candidate video sequence as reference videos.

[0134] In one embodiment, Figure 3 As shown, step S103 may include step S1031, obtaining a second target value N; and step S1032, determining the first N first candidate videos in the first candidate video sequence as the reference videos.

[0135] The value of the second target value N is a positive integer, and is less than or equal to the number of elements in the first candidate video sequence. The value of the second target value N can be flexibly set as needed, and N is often set to a positive integer greater than or equal to 1, such as N is 10. Accordingly, the first 10 first candidate videos in the first candidate video sequence are determined as reference videos. Of course, the setting of the value of the second target value N can be determined in combination with historical relevant data, taking into account the accuracy of the reference video's reflection of the sorting rule of the first candidate video sequence, and the amount of calculation to determine the abnormal value of the reference video.

[0136] Compared with calculating the outliers of all first candidate videos, determining a reference video that can reflect the sorting rule of the first candidate video sequence to a certain extent and then calculating the outliers of the reference video can reduce the amount of calculation and reduce the computing resource consumption of the system.

[0137] In step S104, an outlier value corresponding to each of the reference videos is calculated.

[0138] The client or server determines the abnormal value corresponding to each reference video, and can detect whether there are abnormal video features in the reference video, and use the relevant information of the abnormal video features (including the number and type information of the abnormal video features, including the information that there are no abnormal video features) and preset determination rules to determine the abnormal value of the reference video. The abnormal video features can reflect that the video contains video content that does not comply with the provisions of laws and administrative regulations, video content that violates social morality, and video content that damages national interests, social public interests, and the interests of third parties.

[0139] In one embodiment, Figure 4 As shown, step S104 may include step S1041, determining a reference image corresponding to each of the reference videos; step S1042, calculating an outlier value of the reference image; and step S1043, using the outlier value of the reference image as an outlier value corresponding to the reference video.

[0140] The video cover of the reference video can be obtained, and the video cover of the reference video can be used as the reference video's base image. The key frame image (such as the first frame image) of the reference video can be obtained, and the key frame image of the reference video can be used as the reference video's base image.

[0141] like Figure 5 As shown, the abnormal value of the reference image can be calculated using the abnormal image recognition model, and then the abnormal value of the corresponding reference video can be obtained. The abnormal image recognition model is determined based on machine learning training of multiple sample images, and each of the sample images carries annotation information indicating its abnormal image attributes (such as yellow hint attributes). The client or server can input the reference image into the abnormal image recognition model, and the abnormal image recognition model can be used as an image classification model to process the reference image to output the recognition result of the reference image. When the reference image only takes the above-mentioned video cover or key frame image, the client or server can use the recognition result here as the first recognition result, and then determine the abnormal value of the reference image according to the first recognition result. When the reference image takes multiple images (which may include the above-mentioned video cover and key frame image), the client or server inputs each image into the abnormal image recognition model respectively, and obtains the first recognition result by combining the recognition results of each image, and then determines the abnormal value of the reference image according to the first recognition result.

[0142] Using the abnormal image recognition model to perform abnormal image recognition on the reference image can improve the adaptability and accuracy of abnormal image recognition, and further improve the reliability and effectiveness of abnormal image recognition.

[0143] The steps of training the abnormal image recognition model are introduced below, which include: first, obtaining the multiple sample images; then, training a first preset machine learning model based on the multiple sample images until the loss generated by the first preset machine learning model is less than a first loss threshold, so as to obtain the abnormal image recognition model.

[0144] The sample corpus can come from the collection of online images. The first preset machine learning model is trained according to multiple labeled sample images until the loss generated by the first preset machine learning model is less than the first loss threshold, which can also be manifested as the loss value almost no longer decreasing. This can indicate that the first preset machine learning model has met the convergence conditions, and the training can be terminated to obtain the abnormal image recognition model. The second preset machine learning model can use a residual network model (such as resnet101) or an Inception-v3 model (a convolutional neural network model), etc. For multiple sample corpora, the open source BERT model can be used to load pre-training parameters for model training. The above training steps can obtain an abnormal image recognition model with high generalization ability.

[0145] In step S105, a filtering threshold is calculated according to the abnormal values ​​corresponding to each of the reference videos.

[0146] When the number of reference videos is 1, the abnormal value of the reference video can be directly used as the filtering threshold, or an adjustment factor can be obtained, and the product of the adjustment factor and the abnormal value of the reference video can be used as the filtering threshold. The determination of the adjustment factor can refer to the feedback of the similar filtering threshold determination method in the historical related data.

[0147] When the number of reference videos is greater than or equal to 2, the outliers corresponding to these reference videos can be regarded as an outlier set as a whole, and the maximum value, minimum value or median value in the outlier set can be taken as the filtering threshold; or the outliers corresponding to these reference videos can be averaged, and the average value obtained can be used as the filtering threshold.

[0148] In one embodiment, Figure 6 As shown, step S105 may include step S1051, arranging the outliers corresponding to each of the reference videos in descending order to obtain an outlier sequence; step S1052, obtaining a first target value M; and step S1053, using the Mth outlier in the outlier sequence as the filtering threshold.

[0149] According to the outliers corresponding to each reference video, the outlier sequence is obtained by sorting from high to low. Since one reference video corresponds to one outlier, the value of the first target value M is a positive integer and is less than or equal to the number of reference videos, and the value of the first target value M can be flexibly set as needed. Combined with the records in steps S1031-S1032, the first target value M is less than or equal to the second target value N. For example, N is 10 and M is 2. It can be seen that the first 10 first candidate videos in the first candidate video sequence are determined as reference videos, and these first 10 first candidate videos are first candidate video 1-first candidate video 10 in sequence. The outliers of the first candidate video 1-first candidate video 10 are not necessarily sorted from high to low in the order of the first 10 first candidate videos. Therefore, the outlier sequence is obtained by integrating the outliers of the first 10 first candidate videos. When the first target value M is clear, the Mth outlier in the outlier sequence is used as the filtering threshold.

[0150] Furthermore, the above step S1052 may include: obtaining a proportional factor and the number of elements in the outlier sequence; and taking the product of the proportional factor and the number of elements in the outlier sequence as the first target value M. The value of the proportional factor is greater than or equal to 0 and less than or equal to 1. For example, if the proportional factor is 0.2 and the number of elements in the outlier sequence is 10, then the first target value M is 2 (=10*0.2).

[0151] The Mth outlier value serving as the filtering threshold is taken from the outlier sequence. Combined with the outlier sequence being taken from a reference video that can reflect the sorting rule of the first candidate video sequence to a certain extent, the filtering threshold can also reflect the outlier level of the first candidate video sequence under the sorting rule. The filtering threshold is then used to filter the second candidate video sequence under the same sorting rule to reduce the aggregation of abnormal videos caused by the sorting rule.

[0152] In step S106, the search corpus is used as a search object to obtain a second candidate video sequence.

[0153] The client or server can use the search corpus as the search object and use a preset search engine to obtain a second candidate video sequence. The second candidate video sequence can be obtained by sorting multiple second candidate videos in descending order according to the reference value. The reference value here can refer to the relevant record of the reference value in the aforementioned step S102, which will not be repeated here.

[0154] It should be noted that, when all the corpus segments of the search corpus indicate abnormal text, the first candidate video sequence obtained in the aforementioned step S102 can be directly used as the second candidate video sequence, without having to perform the step of "taking the search corpus as the search object to obtain the second candidate video sequence" again. In addition, although not all the corpus segments of the search corpus indicate abnormal text, if the corpus segments that do not indicate abnormal text are non-entity words such as "how" and "what", the first candidate video sequence obtained in the aforementioned step S102 can also be directly used as the second candidate video sequence, without having to perform the step of "taking the search corpus as the search object to obtain the second candidate video sequence" again.

[0155] In step S107, the second candidate video sequence is filtered based on the filtering threshold to obtain search results corresponding to the search request.

[0156] When filtering the second candidate video sequence, the client or server excludes the second candidate videos whose outliers are greater than the filtering threshold, and then obtains the search results corresponding to the search request. This process requires calculating the outliers corresponding to each second candidate video in the second candidate video sequence. Since the filtering threshold can reflect the outlier level of the first candidate video sequence under the sorting rule, filtering the second candidate video sequence under the same sorting rule based on the filtering threshold can reduce the aggregation of abnormal videos caused by the sorting rule, thereby providing a better video search experience.

[0157] In one embodiment, for the calculation of the outlier value corresponding to each second candidate video in the second candidate video sequence, reference may be made to the relevant records of the aforementioned steps S1041-S1043: determining the baseline image corresponding to each second candidate video in the second candidate video sequence; calculating the outlier value of the baseline image (of the second candidate video); and using the outlier value of the baseline image (of the second candidate video) as the outlier value corresponding to the second candidate video.

[0158] The video cover of the second candidate video can be obtained, and the video cover of the second candidate video can be used as the reference image of the second candidate video. The key frame image (such as the first frame image) of the second candidate video can be obtained, and the key frame image of the second candidate video can be used as the reference image of the second candidate video.

[0159] The abnormal image recognition model (corresponding to the model mentioned in the relevant records of the aforementioned steps S1041-S1043, see Figure 5) calculates the abnormal value of the reference image (of the second candidate video), and then obtains the abnormal value corresponding to the second candidate video. The client or server can input the reference image (of the second candidate video) into the abnormal image recognition model, and the abnormal image recognition model can be used as an image classification model to process the reference image (of the second candidate video) to output the recognition result of the reference image (of the second candidate video). When the reference image (of the second candidate video) only takes the above-mentioned video cover or key frame image, the client or server can use the recognition result here as the third recognition result, and then determine the abnormal value of the reference image (of the second candidate video) according to the third recognition result. When the reference image (of the second candidate video) takes multiple images (which may include the above-mentioned video cover and key frame images), the client or server inputs each image into the abnormal image recognition model respectively, obtains the third recognition result by combining the recognition results of each image, and then determines the abnormal value of the reference image (of the second candidate video) according to the third recognition result.

[0160] It should be noted that, in combination with the relevant records of the aforementioned step S106, when the first candidate video sequence obtained in the aforementioned step S102 can be directly used as the second candidate video sequence, step S107 is actually a step of "filtering the first candidate video sequence based on the filtering threshold to obtain the search result corresponding to the search request". Since the outlier value calculation has been performed on the reference video taken from the first candidate video sequence in the process of determining the filtering threshold, only the outlier value of the remaining first candidate video can be calculated here on this basis.

[0161] In actual applications, the reference value of the preset search engine is determined based on the click data of the video. Due to the clicks of users (some eye-catching abnormal videos are easily clicked by users), the search results corresponding to the partial search corpus as the search object are prone to abnormal video aggregation. This abnormal video aggregation situation is likely to cause bad social impact. The video search scheme provided by the present disclosure can be used to reduce the aggregation of abnormal videos in search scenarios. The scheme well determines the filtering threshold that is adaptive to the search corpus, avoiding a one-size-fits-all filtering method for search feedback of different search corpora. While reducing the aggregation of abnormal videos in the search results of the search corpus, it reduces the accidental damage to non-abnormal videos. When the search corpus with yellow-suggestive attributes is used as the suppression object, the technical effect guided by the scheme is to reduce the aggregation of videos with yellow-suggestive attributes in the search results.

[0162] The video search method provided in the above embodiment obtains a first candidate video sequence based on a target corpus segment indicating abnormal text in the search corpus, then determines the abnormal value of the reference video determined from the first candidate video sequence to obtain a filtering threshold, and then obtains a second candidate video sequence using the search corpus as the search object, and then filters the second candidate video sequence based on the filtering threshold to obtain the search result corresponding to the search request. The present disclosure combines the abnormal text attributes of the target corpus segment and the abnormal value of the reference video to determine the filtering threshold for the search corpus. By using the filtering threshold to filter the second candidate video sequence as search feedback, the abnormal content therein can be effectively filtered, which can reduce the misfiltering of non-abnormal videos and the aggregation of abnormal videos in the search results.

[0163] Figure 7 FIG. 1 is a block diagram of a video search device according to an exemplary embodiment. Figure 7 The device includes a search request receiving unit 710, a first candidate video sequence obtaining unit 720, a reference video determining unit 730, an abnormal value calculating unit 740, a filtering threshold calculating unit 750, a second candidate video sequence obtaining unit 760 and a filtering unit 770.

[0164] The search request receiving unit 710 is configured to receive a search request carrying a search corpus;

[0165] The first candidate video sequence obtaining unit 720 is configured to obtain a first candidate video sequence by taking the at least one target corpus segment as a search object when the search corpus includes at least one target corpus segment indicating abnormal text;

[0166] The reference video determination unit 730 is configured to determine at least one reference video in the first candidate video sequence;

[0167] The outlier value calculation unit 740 is configured to calculate an outlier value corresponding to each reference video;

[0168] The filtering threshold calculation unit 750 is configured to calculate the filtering threshold according to the abnormal values ​​corresponding to each of the reference videos;

[0169] The second candidate video sequence obtaining unit 760 is configured to perform a search using the search corpus as a search object to obtain a second candidate video sequence;

[0170] The filtering unit 770 is configured to perform filtering on the second candidate video sequence based on the filtering threshold to obtain search results corresponding to the search request.

[0171] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0172] In an exemplary embodiment, an electronic device is also provided, including a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the steps of any video search method in the above embodiments when executing the instructions stored in the memory.

[0173] The electronic device may be a terminal, a server or a similar computing device. For example, the electronic device is a server. Figure 8 It is a block diagram of an electronic device for video search according to an exemplary embodiment. The electronic device 800 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 810 (the processor 810 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 830 for storing data, and one or more storage media 820 (such as one or more mass storage devices) for storing application programs 823 or data 822. Among them, the memory 830 and the storage medium 820 can be short-term storage or permanent storage. The program stored in the storage medium 820 may include one or more modules, and each module may include a series of instruction operations in the electronic device. Furthermore, the central processing unit 810 can be configured to communicate with the storage medium 820 to execute a series of instruction operations in the storage medium 820 on the electronic device 800. The electronic device 800 may also include one or more power supplies 860, one or more wired or wireless network interfaces 850, one or more input and output interfaces 840, and / or one or more operating systems 821, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0174] The input / output interface 840 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the electronic device 800. In one example, the input / output interface 840 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In an exemplary embodiment, the input / output interface 80 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0175] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only and does not limit the structure of the above electronic device. Figure 8 More or fewer components as shown, or with Figure 8 Different configurations shown.

[0176] In an exemplary embodiment, a storage medium is further provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the steps of any video search method in the above embodiments.

[0177] In an exemplary embodiment, a computer program product is also provided, the computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the video search method provided in any of the above embodiments.

[0178] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0179] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0180] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video search method, characterized in that: The method comprises: Receiving a search request carrying a search corpus; When the search corpus includes at least one target corpus segment indicating abnormal text, taking the at least one target corpus segment as a search object to obtain a first candidate video sequence; Determine at least one reference video in the first candidate video sequence; Calculate the outlier value corresponding to each of the reference videos; Calculating a filtering threshold according to the outliers corresponding to each of the reference videos; Taking the search corpus as a search object, obtaining a second candidate video sequence; The second candidate video sequence is filtered based on the filtering threshold to obtain a search result corresponding to the search request.

2. The method according to claim 1, characterized in that The calculating of the filtering threshold according to the abnormal values ​​corresponding to each of the reference videos includes: Arranging the outlier values ​​corresponding to the reference videos in descending order to obtain an outlier value sequence; Obtain a first target value M; The Mth outlier in the outlier sequence is used as the filtering threshold.

3. The method according to claim 2, characterized in that The obtaining of the first target value M comprises: Obtaining a scaling factor and the number of elements in the outlier sequence; The product of the proportional factor and the number of elements in the abnormal value sequence is used as the first target value M.

4. The method according to any one of claims 1 to 3, characterized in that: The filtering the second candidate video sequence based on the filtering threshold to obtain search results corresponding to the search request includes: Calculating an outlier value corresponding to each second candidate video in the second candidate video sequence; The second candidate videos whose corresponding abnormal values ​​in the second candidate video sequence are greater than the filtering threshold are filtered out to obtain search results corresponding to the search request.

5. The method according to claim 1, characterized in that The determining at least one reference video in the first candidate video sequence comprises: Obtain a second target value N; The first N first candidate videos in the first candidate video sequence are determined as the reference videos.

6. The method according to claim 1, characterized in that The calculating the outlier value corresponding to each reference video includes: Determine a reference image corresponding to each of the reference videos; Calculating an outlier value of the reference image; The abnormal value of the benchmark image is used as the abnormal value corresponding to the reference video.

7. The method according to claim 6, characterized in that The calculating the abnormal value of the reference image comprises: Inputting the reference image into an abnormal image recognition model to obtain a first recognition result; determining an abnormal value of the reference image according to the first recognition result; The abnormal image recognition model is determined by machine learning training based on multiple sample images, and each of the sample images carries annotation information indicating its abnormal image attributes.

8. The method according to claim 7, characterized in that The method further includes training the abnormal image recognition model, wherein the training of the abnormal image recognition model includes: acquiring the plurality of sample images; A first preset machine learning model is trained according to the multiple sample images until the loss generated by the first preset machine learning model is less than a first loss threshold, so as to obtain the abnormal image recognition model.

9. The method according to claim 1, characterized in that: When the search corpus includes at least one target corpus segment indicating abnormal text, taking the at least one target corpus segment as a search object and obtaining the first candidate video sequence further includes the step of determining the at least one target corpus segment indicating abnormal text; The step of determining at least one target corpus segment indicating abnormal text comprises: Inputting the search corpus into an abnormal text recognition model to perform abnormal text recognition to obtain a second recognition result; Determining at least one target corpus segment indicating abnormal text according to the second recognition result; The abnormal text recognition model is determined by machine learning training based on multiple sample corpora, and each of the sample corpora carries annotation information indicating its abnormal text attributes.

10. The method according to claim 9, characterized in that The method further comprises the step of training the abnormal text recognition model, wherein the training of the abnormal text recognition model comprises: Acquire the plurality of sample corpora; A second preset machine learning model is trained according to the multiple sample corpora until the loss generated by the second preset machine learning model is less than a second loss threshold, so as to obtain the abnormal text recognition model.

11. The method according to claim 1, characterized in that: The step of taking the at least one target corpus segment as a search object to obtain a first candidate video sequence includes: Taking each of the target corpus segments as a search object, respectively, to obtain a video sequence corresponding to each of the target corpus segments; According to a preset sequence merging rule, the video sequences corresponding to the target corpus segments are merged to obtain the first candidate video sequence.

12. A video search device, characterized in that: The device comprises: A search request receiving unit, configured to receive a search request carrying a search corpus; A first candidate video sequence obtaining unit is configured to obtain a first candidate video sequence by taking the at least one target corpus segment as a search object when the search corpus includes at least one target corpus segment indicating abnormal text; A reference video determining unit, configured to determine at least one reference video in the first candidate video sequence; An outlier value calculation unit, configured to calculate an outlier value corresponding to each of the reference videos; A filtering threshold calculation unit, configured to calculate a filtering threshold according to an abnormal value corresponding to each of the reference videos; A second candidate video sequence obtaining unit is configured to perform a search using the search corpus as a search object to obtain a second candidate video sequence; The filtering unit is configured to filter the second candidate video sequence based on the filtering threshold to obtain search results corresponding to the search request.

13. The device according to claim 12, characterized in that The filtering threshold calculation unit comprises: An outlier sequence obtaining unit is configured to arrange the outliers corresponding to the reference videos in descending order to obtain an outlier sequence; A first target value acquisition unit, configured to execute acquisition of a first target value M; The filtering threshold determination unit is configured to use the Mth abnormal value in the abnormal value sequence as the filtering threshold.

14. The device according to claim 13, characterized in that The first target value acquisition unit includes: A parameter acquisition unit, configured to execute acquisition of a scale factor and the number of elements in the outlier sequence; The first target value determination unit is configured to perform the multiplication of the proportional factor and the number of elements in the abnormal value sequence as the first target value M.

15. The device according to any one of claims 12 to 14, characterized in that The filtering unit comprises: A first type of outlier calculation unit is configured to calculate an outlier value corresponding to each second candidate video in the second candidate video sequence; The filtering subunit is configured to filter out the second candidate videos in the second candidate video sequence whose corresponding abnormal values ​​are greater than the filtering threshold, and obtain search results corresponding to the search request.

16. The device according to claim 12, characterized in that The reference video determination includes: A second target value determination unit, configured to execute acquisition of a second target value N; The reference video determination subunit is configured to determine the first N first candidate videos in the first candidate video sequence as the reference videos.

17. The device according to claim 12, characterized in that The outlier calculation includes: A reference image determining unit, configured to determine a reference image corresponding to each of the reference videos; A second type of outlier calculation unit, configured to calculate an outlier value of the reference image; The outlier determination unit is configured to use the outlier value of the benchmark image as the outlier value corresponding to the reference video.

18. The device according to claim 17, characterized in that The second type of outlier calculation unit includes: A first recognition result obtaining unit is configured to input the reference image into an abnormal image recognition model to obtain a first recognition result; A second type of outlier determination unit is configured to determine an outlier value of the reference image according to the first recognition result; The abnormal image recognition model is determined by machine learning training based on multiple sample images, and each of the sample images carries annotation information indicating its abnormal image attributes.

19. The device according to claim 18, characterized in that The device also includes a first model training unit for training the abnormal image recognition model, wherein the first model training unit includes: A sample image acquisition unit, configured to acquire the plurality of sample images; The first model training subunit is configured to execute training of a first preset machine learning model according to the multiple sample images until the loss generated by the first preset machine learning model is less than a first loss threshold, so as to obtain the abnormal image recognition model.

20. The device according to claim 12, characterized in that The device further includes a target corpus segment determining unit for determining at least one target corpus segment indicating the abnormal text, the target corpus segment determining unit including: A second recognition result obtaining unit is configured to input the search corpus into an abnormal text recognition model to perform abnormal text recognition to obtain a second recognition result; a target corpus segment determination subunit, configured to determine at least one target corpus segment indicating abnormal text according to the second recognition result; The abnormal text recognition model is determined by machine learning training based on multiple sample corpora, and each of the sample corpora carries annotation information indicating its abnormal text attributes.

21. The device according to claim 20, characterized in that The device further includes a second model training unit for training the abnormal text recognition model, wherein the second model training unit includes: A sample corpus acquisition unit, configured to acquire the plurality of sample corpora; The second model training subunit is configured to execute training of a second preset machine learning model according to the multiple sample corpora until the loss generated by the second preset machine learning model is less than a second loss threshold, so as to obtain the abnormal text recognition model.

22. The device according to claim 12, characterized in that The first candidate video sequence obtaining unit comprises: A video sequence obtaining unit is configured to perform a search using each of the target corpus segments as a search object to obtain a video sequence corresponding to each of the target corpus segments; The merging unit is configured to execute a preset sequence merging rule to merge the video sequences corresponding to the target corpus segments to obtain the first candidate video sequence.

23. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video search method according to any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the video search method as described in any one of claims 1 to 11.

Images