Video feature extraction, model training method and device, equipment and storage medium

Abstract

The present disclosure relates to a video feature extraction method and device, a model training method and device, equipment and a storage medium, and relates to the technical field of computers. The present disclosure can not only ensure the accuracy of a video feature extraction model, but also improve the training efficiency of the video feature extraction model. The model training method comprises: obtaining video features and label information of a first video resource; determining a classification result of the video features of the first video resource based on a classification model, and determining a classification loss value according to difference information between the classification result and the label information; obtaining search term features of a second video resource; performing contrast learning on the video features of the first video resource and the search term features of the second video resource based on a contrast learning model to obtain a contrast loss value; and training a video feature extraction model to be trained based on the classification loss value and the contrast loss value to obtain the video feature extraction model.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a video feature extraction, model training method, apparatus, device, and storage medium. Background Technology

[0002] Video feature extraction technology refers to mapping a sequence of video images into a high-dimensional feature vector, and then using this high-dimensional feature vector to represent the content of the video frames. Video feature extraction technology can be applied to many scenarios, such as video recommendation or video search.

[0003] Currently, video feature extraction techniques typically employ unsupervised or supervised training methods to train models capable of extracting video features. However, unsupervised training methods often result in low accuracy for video feature extraction models. Supervised training, on the other hand, relies on supervisory information, which usually requires manual annotation, leading to a large amount of annotation work and significant time consumption, thus reducing the training efficiency of the video feature extraction models.

[0004] Therefore, how to improve the training efficiency of video feature extraction models while ensuring their accuracy is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This disclosure provides a video feature extraction and model training method, apparatus, device, and storage medium, which can not only ensure the accuracy of the video feature extraction model, but also improve the training efficiency of the video feature extraction model.

[0006] The technical solution of this disclosure is as follows:

[0007] According to a first aspect of the present disclosure, a method for training a video feature extraction model is provided, which can be applied to an electronic device. The method may include: acquiring video features and label information of a first video resource;

[0008] The classification result of the video features of the first video resource is determined based on the classification model, and the classification loss value is determined based on the difference between the classification result and the label information.

[0009] Obtain the search term features of the second video resource;

[0010] Based on the contrastive learning model, the video features of the first video resource and the search term features of the second video resource are compared and learned to obtain the contrastive loss value.

[0011] The video feature extraction model is trained based on classification loss and contrastive loss values ​​to obtain the video feature extraction model.

[0012] Optionally, when there are multiple first video resources and multiple second video resources, the multiple first video resources and multiple second video resources include both identical and different video resources; based on a contrastive learning model, the video features of the first video resources and the search term features of the second video resources are compared and learned to obtain a contrastive loss value, including:

[0013] The dot product of the video features and search term features of the first type of video resources is determined as the first classification target of the contrastive learning model; the first type of video resources is used to represent the same video resources among multiple first video resources and multiple second video resources;

[0014] The dot product of the video features and search term features of the second type of video resources is determined as the second classification target of the contrastive learning model; the second type of video resources is used to represent different video resources among multiple first video resources and multiple second video resources.

[0015] Based on the first and second classification objectives, the video features of the first video resource and the search term features of the second video resource are compared and learned to obtain the comparison loss value.

[0016] Optionally, based on the first and second classification objectives, comparative learning is performed on the video features of the first video resource and the search term features of the second video resource to obtain a comparative loss value, including:

[0017] Based on the first classification objective, a first type of feature set is determined; the first type of feature set is used to represent the video features and search term features of the first type of video resources.

[0018] Based on the second classification objective, a second type of feature set is determined; the second type of feature set is used to represent the video features and search term features of the second type of video resources.

[0019] The contrast loss value is determined based on the differences between the first and second feature sets.

[0020] Optionally, the video features of the first video resource are obtained, including:

[0021] Obtain the text and image features of the first video resource;

[0022] Based on a multimodal algorithm, text features and image features are fused to obtain video features.

[0023] Optionally, obtain the text features and image features of the first video resource, including:

[0024] Based on the image feature extraction algorithm, feature extraction is performed on the video images of the first video resource to obtain image features;

[0025] Based on speech recognition and text detection algorithms, text detection is performed on the first video resource to obtain text information;

[0026] Based on text feature extraction algorithms, features are extracted from text information to obtain text features.

[0027] Alternatively, video feature extraction model training methods may also include:

[0028] Obtain the initial video features of the first video resource and the initial search term features of the second video resource;

[0029] The initial video features and initial search term features are regularized to obtain the processed video features and processed search term features;

[0030] The processed video features are determined as the video features of the first video resource, and the processed search term features are determined as the search term features of the second video resource.

[0031] According to a second aspect of the present disclosure, a video feature extraction method is provided, which can be applied to an electronic device. The method may include:

[0032] Acquire the video resources to be processed;

[0033] The video resource to be processed is input into the video feature extraction model to obtain the video features of the video resource to be processed; the video feature extraction model is trained according to the video feature extraction model training method of any one of the first aspects.

[0034] According to a third aspect of the present disclosure, a video feature extraction model training apparatus is provided, which can be applied to an electronic device. The apparatus may include: an acquisition unit and a processing unit;

[0035] The acquisition unit is used to acquire the video features and tag information of the first video resource;

[0036] The processing unit is used to determine the classification result of the video features of the first video resource based on the classification model, and to determine the classification loss value based on the difference between the classification result and the label information.

[0037] The acquisition unit is also used to acquire the search term features of the second video resource;

[0038] The processing unit is also used to perform comparative learning on the video features of the first video resource and the search term features of the second video resource based on a comparative learning model to obtain a comparative loss value.

[0039] The processing unit is also used to train the video feature extraction model to be trained based on the classification loss value and the contrastive loss value, so as to obtain the video feature extraction model.

[0040] Optionally, when there are multiple first video resources and multiple second video resources, the multiple first video resources and multiple second video resources include both identical and different video resources; the processing unit is specifically used for:

[0041] The dot product of the video features and search term features of the first type of video resources is determined as the first classification target of the contrastive learning model; the first type of video resources is used to represent the same video resources among multiple first video resources and multiple second video resources;

[0042] The dot product of the video features and search term features of the second type of video resources is determined as the second classification target of the contrastive learning model; the second type of video resources is used to represent different video resources among multiple first video resources and multiple second video resources.

[0043] Based on the first and second classification objectives, the video features of the first video resource and the search term features of the second video resource are compared and learned to obtain the comparison loss value.

[0044] Optional, processing unit, specifically used for:

[0045] Based on the first classification objective, a first type of feature set is determined; the first type of feature set is used to represent the video features and search term features of the first type of video resources.

[0046] Based on the second classification objective, a second type of feature set is determined; the second type of feature set is used to represent the video features and search term features of the second type of video resources.

[0047] The contrast loss value is determined based on the differences between the first and second feature sets.

[0048] Optional, the acquisition unit is specifically used for:

[0049] Obtain the text and image features of the first video resource;

[0050] Based on a multimodal algorithm, text features and image features are fused to obtain video features.

[0051] Optional, the acquisition unit is specifically used for:

[0052] Based on the image feature extraction algorithm, feature extraction is performed on the video images of the first video resource to obtain image features;

[0053] Based on speech recognition and text detection algorithms, text detection is performed on the first video resource to obtain text information;

[0054] Based on text feature extraction algorithms, features are extracted from text information to obtain text features.

[0055] Optionally, the acquisition unit is also used to acquire the initial video features of the first video resource and the initial search term features of the second video resource;

[0056] The processing unit is also used to perform regularization processing on the initial video features and initial search term features to obtain the processed video features and processed search term features;

[0057] The processing unit is further configured to determine the processed video features as video features of the first video resource, and to determine the processed search term features as search term features of the second video resource.

[0058] According to a fourth aspect of the present disclosure, a video feature extraction apparatus is provided, which can be applied to an electronic device. The apparatus may include: an acquisition unit and a processing unit;

[0059] The acquisition unit is used to acquire the video resources to be processed.

[0060] The processing unit is used to input the video resource to be processed into the video feature extraction model to obtain the video features of the video resource to be processed; the video feature extraction model is trained according to the video feature extraction model training method of any one of the first aspects.

[0061] According to a fifth aspect of the present disclosure, an electronic device is provided, which may include: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the optional video feature extraction model training methods of the first aspect above, or the video feature extraction method of the second aspect above.

[0062] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform any of the optional video feature extraction model training methods of the first aspect above, or the video feature extraction method of the second aspect above.

[0063] According to a seventh aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions that, when executed on an electronic device, cause the electronic device to perform the video feature extraction model training method as described in any optional implementation of the first aspect, or the video feature extraction method of the second aspect described above.

[0064] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0065] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects:

[0066] Based on any of the above aspects, embodiments of this disclosure provide a video feature extraction model training method. After acquiring video features and tag information of a first video resource, the electronic device determines the classification result of the video features of the first video resource based on a classification model, and determines a classification loss value based on the difference between the classification result and the tag information. Then, after acquiring search term features of a second video resource, the electronic device can perform comparative learning on the video features of the first video resource and the search term features of the second video resource based on a contrastive learning model to obtain a contrastive loss value. Subsequently, the electronic device can train the video feature extraction model to be trained based on the classification loss value and the contrastive loss value to obtain the video feature extraction model.

[0067] Since the classification loss is derived from the difference between the classification result and the label information, and the contrastive loss is derived from video features and search term features, the video feature extraction model trained based on these two loss values ​​can be viewed as a video feature extraction model based on supervised information including label information and search term features (without the need for manual annotation of supervised information). In this way, when extracting video features from the video to be processed, the video feature extraction model can fully exploit the video features, improving both the accuracy and training efficiency of the target feature extraction model. Attached Figure Description

[0068] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0069] Figure 1 This diagram illustrates the structure of a video feature extraction model training system provided in an embodiment of the present disclosure.

[0070] Figure 2 A flowchart illustrating a video feature extraction model training method provided in an embodiment of this disclosure is shown.

[0071] Figure 3 A flowchart illustrating another video feature extraction model training method provided in this disclosure is shown.

[0072] Figure 4A flowchart illustrating another video feature extraction model training method provided in this disclosure is shown.

[0073] Figure 5 A flowchart illustrating another video feature extraction model training method provided in this disclosure is shown.

[0074] Figure 6 A flowchart illustrating another video feature extraction model training method provided in this disclosure is shown.

[0075] Figure 7 A flowchart illustrating another video feature extraction model training method provided in this disclosure is shown.

[0076] Figure 8 A flowchart illustrating another video feature extraction model training method provided in this disclosure is shown.

[0077] Figure 9 A flowchart illustrating a video feature extraction method provided in an embodiment of this disclosure is shown.

[0078] Figure 10 This diagram illustrates the structure of a video feature extraction model training device provided in an embodiment of the present disclosure.

[0079] Figure 11 A schematic diagram of the structure of a video feature extraction device provided in an embodiment of this disclosure is shown;

[0080] Figure 12 A schematic diagram of the structure of a terminal provided in an embodiment of this disclosure is shown;

[0081] Figure 13 A schematic diagram of the structure of a server provided in an embodiment of this disclosure is shown. Detailed Implementation

[0082] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0083] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0084] It should also be understood that the term "comprising" indicates the presence of the described feature, whole, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements and / or components.

[0085] The data disclosed herein may be data authorized by the user or fully authorized by all parties.

[0086] In general technology, video feature extraction techniques typically employ unsupervised or supervised training methods to train models capable of extracting video features. However, unsupervised training methods often result in lower accuracy for video feature extraction models. Supervised training, on the other hand, relies on supervisory information. This supervisory information usually requires manual annotation, which is time-consuming and involves a large amount of annotation, thus leading to low training efficiency for the video feature extraction model.

[0087] Based on this, this disclosure provides a video feature extraction model training method. After acquiring the video features and tag information of a first video resource, the electronic device determines the classification result of the video features of the first video resource based on a classification model, and determines a classification loss value based on the difference between the classification result and the tag information. Then, after acquiring the search term features of a second video resource, the electronic device can perform comparative learning on the video features of the first video resource and the search term features of the second video resource based on a contrastive learning model to obtain a contrastive loss value. Subsequently, the electronic device can train the video feature extraction model to be trained based on the classification loss value and the contrastive loss value to obtain the video feature extraction model.

[0088] Since the classification loss is derived from the difference between the classification result and the label information, and the contrastive loss is derived from video features and search term features, the video feature extraction model trained based on these two loss values ​​can be viewed as a video feature extraction model based on supervised information including label information and search term features (without the need for manual annotation of supervised information). In this way, when extracting video features from the video to be processed, the video feature extraction model can fully exploit the video features, improving both the accuracy and training efficiency of the target feature extraction model.

[0089] Figure 1 This is a schematic diagram of a video feature extraction model training system provided in an embodiment of the present disclosure, such as... Figure 1As shown, the video feature extraction model training system may include a server 110 and an electronic device 120. The server 110 can establish a connection with the electronic device 120 through a wired network or a wireless network.

[0090] Server 110 can be a data server for multimedia resource service platforms, used for storing and processing multimedia resources. For example, a multimedia resource service platform could be a short video application service platform, a news service platform, a live streaming service platform, a shopping service platform, a food delivery service platform, a sharing service platform, or a functional website. The multimedia resources provided by a short video application service platform can be short video works, the multimedia resources provided by a news service platform can be news information, and the multimedia resources provided by a live streaming service platform can be live broadcasts, etc., and so on. This disclosure does not limit the specific type of multimedia resource service platform.

[0091] In this disclosure, server 110 is mainly used to store data required for training the video feature extraction model, such as: a first video resource, video features and label information of the first video resource, a second video resource, and search term features of the second video resource. Server 110 can send the corresponding data to electronic device 120 when it receives a data acquisition request from electronic device 120.

[0092] In some embodiments, server 110 may also include or be connected to a database, and the multimedia resources of the multimedia resource service platform may be stored in the database. Electronic device 120 can access the multimedia resources in the database through server 110.

[0093] Electronic device 120 may be a server, a terminal, or other electronic devices used for training video feature extraction models; this disclosure does not limit the scope of the device.

[0094] When electronic device 120 is a server, electronic device 120 and server 110 can be two independent servers or integrated into the same server. This application does not make specific limitations in this regard.

[0095] It is easy to understand that when electronic device 120 and server 110 are integrated into the same server, the communication method between electronic device 120 and server 110 is the same as the communication between internal modules of the server. In this case, the communication process between the two is the same as that between electronic device 120 and server 110 when they are independent of each other.

[0096] For ease of understanding, this application mainly uses the independent configuration of electronic device 120 and server 110 as an example for illustration.

[0097] The aforementioned server can be a single server, or it can be a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. This disclosure does not limit the specific implementation method of the server.

[0098] When electronic device 120 is used as a terminal, it can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc., which can install and use content community applications (such as Kuaishou). This disclosure does not impose any special restrictions on the specific form of the terminal. It can interact with users through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting devices.

[0099] Optionally, the above Figure 1 In the video feature extraction model training system shown, electronic device 120 can be connected to at least one server 110. This disclosure does not limit the number or type of server 110.

[0100] The video feature extraction model training method provided in this disclosure can be applied to the aforementioned... Figure 1 Electronic device 120 in the application scenario shown.

[0101] The video feature extraction model training method provided in this disclosure will be described in detail below with reference to the accompanying drawings.

[0102] like Figure 2 As shown, when a video feature extraction model training method is applied to an electronic device, the video feature extraction model training method may include:

[0103] S201. The electronic device acquires the video features and tag information of the first video resource.

[0104] The tag information can be the topic tags of the first video resource.

[0105] Specifically, when training a video feature extraction model, to improve its accuracy, electronic devices can train the model using a supervised approach (supervised training generally yields higher accuracy than unsupervised training). Supervised training relies on supervisory information. In this case, the electronic device can acquire the video features and label information of the first video resource.

[0106] Since the tag information is matched with the first video resource, the electronic device can use the tag information as supervisory information. In this way, the electronic device can train a highly accurate video feature extraction model based on supervisory information with high relevance to the video resources.

[0107] For example, when the first video resource is a basketball video, the hashtag for the first video resource can be "#sports".

[0108] As another example, when the first video resource is a music video, the hashtag for the first video resource can be "#music".

[0109] In one feasible approach, since training the video feature extraction model requires a large amount of training data, this disclosure does not limit the number of first video resources. In practical applications, the number of first video resources can be 1000 or 10000. Correspondingly, the number of video features and the number of tag information are also multiple, meaning the electronic device can acquire the video features and tag information corresponding to each first video resource.

[0110] In one possible approach, the electronic device can access data from a server (e.g., a server storing data required for training a video feature extraction model). Figure 1 The video features and tag information of the first video resource can be obtained from the server 110 in the middle, or the first video resource and tag information can be obtained from the database storing the data required for training the video feature extraction model. Then, the video features of the first video resource are determined based on the feature extraction algorithm. This disclosure does not limit this.

[0111] It should be noted that the aforementioned database may be a database of an electronic device, or a database in other storage devices or storage systems (such as distributed storage systems), and this disclosure does not limit it in this regard.

[0112] S202. The electronic device determines the classification result of the video features of the first video resource based on the classification model, and determines the classification loss value based on the difference between the classification result and the label information.

[0113] Specifically, after acquiring the video features and label information of the first video resource, the electronic device can determine the difference between the video features and the label information in order to obtain the classification loss value used to train the video feature extraction model. However, since the video features are feature vectors, the electronic device can determine the classification result of the video features of the first video resource based on the classification model.

[0114] Electronic devices can learn and train classification models based on pre-classified category data, thereby obtaining classification results for video features.

[0115] In practical applications, common classification models include: logistic regression model, decision tree model, support vector machine model, and Naive Bayes model.

[0116] Optionally, the classification model can be a pre-trained model used to determine the classification result of video features.

[0117] Optionally, the electronic device can also determine the classification result of the video features of the first video resource based on a classification algorithm (such as k-nearest neighbor classification algorithm, decision tree classification algorithm, etc.).

[0118] After determining the classification result of the video features of the first video resource, the electronic device can determine the classification loss value based on the difference between the classification result and the label information.

[0119] Optionally, the classification loss value mentioned above can be the cross-entropy function value.

[0120] For example, when the first video resource is a basketball video, its hashtag could be "#sports". The electronic device determines the classification result of the video features of the basketball video as "basketball" based on a classification model. Then, the electronic device can determine the classification loss value based on the difference between "basketball" and "sports".

[0121] S203, Search term features of electronic devices acquiring second video resources.

[0122] Among them, the search term features can be the features of the search terms used when performing a search on the second video resource.

[0123] Specifically, when training a video feature extraction model, to improve its accuracy, electronic devices can train the model using a supervised approach (supervised training generally yields higher accuracy than unsupervised training). Supervised training relies on supervisory information. In this case, the electronic device can acquire search term features from a second video resource.

[0124] Since search term features are the characteristics of the search terms used when searching for the second video resource, electronic devices can use these search term features as supervisory information. In this way, electronic devices can train a highly accurate video feature extraction model based on supervisory information that is highly correlated with the video resource.

[0125] For example, when the second video resource is a basketball video, the search term for the second video resource could be "basketball".

[0126] As another example, when the second video resource is a music video, the search term for the second video resource can be "music".

[0127] In one feasible approach, since training the video feature extraction model requires a large amount of training data, this disclosure does not limit the number of second video resources. In practical applications, the number of second video resources can be 1000 or 10000. Correspondingly, the number of search term features for each second video resource is also multiple, meaning the electronic device can acquire the search term features corresponding to each second video resource.

[0128] In one possible approach, the electronic device can access data from a server (e.g., a server storing data required for training a video feature extraction model). Figure 1 The search term features of the second video resource can be obtained from the server 110 in the middle, or the second video resource can be obtained from the database storing the data required for training the video feature extraction model, and then the search term features of the second video resource can be determined based on the feature extraction algorithm. This disclosure does not limit this.

[0129] It should be noted that the aforementioned database may be a database of an electronic device, or a database in other storage devices or storage systems (such as distributed storage systems), and this disclosure does not limit it in this regard.

[0130] S204. The electronic device uses a contrastive learning model to compare and learn the video features of the first video resource and the search term features of the second video resource to obtain a contrastive loss value.

[0131] Specifically, after acquiring the video features of the first video resource and the search term features of the second video resource, in order to obtain the contrastive loss value for training the video feature extraction model, the electronic device can use a contrastive learning model to perform contrastive learning on the video features of the first video resource and the search term features of the second video resource to obtain the contrastive loss value.

[0132] Optionally, the contrastive learning model can be a pre-trained model used to contrast different features to obtain the loss value.

[0133] Optionally, the electronic device can also perform comparative learning on the video features of the first video resource and the search term features of the second video resource based on a contrastive learning algorithm to obtain a contrastive loss value.

[0134] Optionally, the comparison loss value mentioned above can be the cross-entropy function value.

[0135] S205. The electronic device trains the video feature extraction model to be trained based on the classification loss value and the contrast loss value to obtain the video feature extraction model.

[0136] Optionally, the electronic device can add the contrastive loss and the classification loss to obtain a joint loss, and then train the video feature extraction model based on this joint loss to obtain the video feature extraction model. As can be seen above, since the classification loss is obtained based on the difference between the classification result and the label information, and the contrastive loss is obtained based on the video features and search term features, the video feature extraction model trained based on the classification loss and contrastive loss can be considered as a video feature extraction model based on supervised information including label information and search term features (without the need for manual annotation of supervised information). In this way, when extracting video features from the video to be processed, the video feature extraction model can fully explore the video features of the video to be processed, improving both the accuracy and training efficiency of the target feature extraction model.

[0137] In one feasible approach, when there are multiple first video resources and multiple second video resources, the multiple first video resources and the multiple second video resources include both identical and different video resources. In this case, combining... Figure 2 ,like Figure 3 As shown, in S204 above, the method by which the electronic device performs comparative learning on the video features of the first video resource and the search term features of the second video resource based on a contrastive learning model to obtain the contrastive loss value specifically includes:

[0138] S301. The electronic device determines the dot product of the video features and search term features of the first type of video resources as the first classification target of the contrastive learning model.

[0139] Among them, the first type of video resource is used to represent the same video resource among multiple first video resources and multiple second video resources.

[0140] S302. The electronic device determines the dot product of the video features and search term features of the second type of video resources as the second classification target of the contrastive learning model.

[0141] The second type of video resource is used to represent different video resources among multiple first video resources and multiple second video resources.

[0142] Specifically, a contrastive learning model is a machine learning technique in which electronic devices can determine a contrastive loss value by learning similar, identical, or different data from multiple first video resources and multiple second video resources.

[0143] In contrastive learning models, it is necessary to set a learning optimization goal in advance so that the training data can approach this optimization goal, thereby obtaining a high-accuracy feature extraction model.

[0144] For the first type of video resources—that is, the same video resources among multiple first-class video resources and multiple second-class video resources—the electronic device can determine that the video features and search term features of the same video resources are similar or identical. In this case, the dot product of the video features and search term features of the same video resources is maximized. Therefore, the electronic device can determine the dot product of the video features and search term features of the first type of video resources as the first classification target of the contrastive learning model.

[0145] Correspondingly, for the second type of video resources—that is, different video resources among multiple first-type video resources and multiple second-type video resources—the electronic device can determine that the video features and search term features of different video resources are dissimilar. In this case, the dot product of the video features and search term features of different video resources is less than the dot product of the video features and search term features of the same video resource. Therefore, the electronic device can determine the dot product of the video features and search term features of the second type of video resources as the second classification target of the contrastive learning model.

[0146] For example, for multiple first video resources and multiple second video resources, the i-th first video resource among the multiple first video resources and the i-th second video resource among the multiple second video resources are the same. Correspondingly, the j-th first video resource among the multiple first video resources and the j-th second video resource among the multiple second video resources are also the same.

[0147] Where i is a positive integer; j is a positive integer.

[0148] However, the i-th first video resource among the multiple first video resources and the j-th second video resource among the multiple second video resources are different. Correspondingly, the j-th first video resource among the multiple first video resources and the i-th second video resource among the multiple second video resources are also different.

[0149] In this case, the dot product S of the video features of the i-th first video resource and the search term features of the i-th second video resource is... i，i And the dot product S of the video features of the i-th first video resource and the search term features of the j-th second video resource. i，j Satisfy the following formula:

[0150] S i，i >S i，j i≠j;

[0151] Similarly, the dot product S of the video features of the j-th first video resource and the search term features of the j-th second video resource is... j，j And the dot product S of the video features of the j-th first video resource and the search term features of the i-th second video resource. j，i Satisfy the following formula:

[0152] S j，j >S j，i ,i≠j.

[0153] In this scenario, the electronic device can perform a dot product S of the video features of the i-th first video resource and the search term features of the i-th second video resource. i，i The tags, and the dot product S of the video features of the j-th first video resource and the search term features of the j-th second video resource. j，j Set to 1, and calculate the dot product S of the video features of the i-th first video resource and the search term features of the j-th second video resource. i，j The tags, and the dot product S of the video features of the j-th first video resource and the search term features of the i-th second video resource. j，i The label is set to 0, thereby distinguishing the first category target from the second category target by different labels.

[0154] S303. The electronic device performs comparative learning on the video features of the first video resource and the search term features of the second video resource based on the first classification target and the second classification target to obtain a comparative loss value.

[0155] Specifically, after determining the first classification target and the second classification target, the electronic device can perform comparative learning on the video features of the first video resource and the search term features of the second video resource based on the first classification target and the second classification target to obtain the comparative loss value. A specific implementation method for determining the comparative loss value is given so that the video feature extraction model trained based on the comparative loss value can be used to improve the accuracy of the target feature extraction model and the training efficiency of the video feature extraction model.

[0156] In one feasible way, combining Figure 3 ,like Figure 4 As shown, in S303 above, the method by which the electronic device performs comparative learning on the video features of the first video resource and the search term features of the second video resource based on the first classification target and the second classification target to obtain the comparative loss value specifically includes:

[0157] S401. The electronic device determines the first type of feature set based on the first classification target.

[0158] The first type of feature set is used to represent the video features and search term features of the first type of video resources.

[0159] Specifically, after determining the first classification target, the electronic device can classify the video features of multiple first video resources and the search term features of multiple second video resources to obtain the video features and search term features of the first category of video resources.

[0160] Since the first classification objective is used to represent the dot product of video features and search term features of the same video resources among multiple first video resources and multiple second video resources, the electronic device can determine the dot product of the video features of each first video resource and the search term features of each second video resource to obtain multiple dot products. Then, the electronic device can determine the dot product of the video features and search term features of the same video resources from the multiple dot products, and determine the set composed of the video features and search term features of the same video resources as the first type of feature set.

[0161] S402. The electronic device determines the second set of features based on the second classification objective.

[0162] The second type of feature set is used to represent the video features and search term features of the second type of video resources.

[0163] Specifically, after determining the second classification target, the electronic device can classify the video features of multiple first video resources and the search term features of multiple second video resources to obtain the video features and search term features of the second type of video resources.

[0164] Since the second classification objective is used to represent the dot product of video features and search term features of different video resources among multiple first video resources and multiple second video resources, the electronic device can determine the dot product of the video features of each first video resource and the search term features of each second video resource to obtain multiple dot products. Then, the electronic device can determine the dot product of video features and search term features of different video resources from the multiple dot products, and determine the set composed of video features and search term features of different video resources as the second type feature set.

[0165] S403. The electronic device determines the contrast loss value based on the difference information between the first type of feature set and the second type of feature set.

[0166] For example, the plurality of first video resources includes first video resource A and first video resource B. The plurality of second video resources includes second video resource 1 and second video resource 2. First video resource A and second video resource 1 are the same video resource, and first video resource B and second video resource 2 are also the same video resource. First video resource A and second video resource 2 are different video resources, and first video resource B and second video resource 1 are also different video resources.

[0167] In this scenario, the electronic device can obtain two sets of first-class features based on the first classification objective. One set of first-class features includes: video features of the first video resource A and search term features of the second video resource 1. The other set of first-class features includes: video features of the second video resource B and search term features of the second video resource 2.

[0168] Accordingly, the electronic device can obtain two sets of second-class features based on the second classification objective. One set of second-class features includes: video features of the first video resource A and search term features of the second video resource 2. The other set of second-class features includes: video features of the second video resource A and search term features of the second video resource 2.

[0169] Next, the electronic device can determine the difference information between each first-class feature set and each second-class feature set, and determine the contrast loss value based on the obtained difference information.

[0170] Specifically, the aforementioned difference information can be obtained by weighted summation of feature vectors in each first-class feature set to obtain a first vector corresponding to each first-class feature set, and by weighted summation of feature vectors in each second-class feature set to obtain a second vector corresponding to each second-class feature set. The electronic device can then determine the difference between each first vector and each second vector as the difference information.

[0171] As shown above, after determining the first and second feature sets, the electronic device can determine the contrast loss value based on the difference information between the first and second feature sets. A specific implementation method for determining the contrast loss value is given, so that the video feature extraction model trained based on the contrast loss value can be used to improve the accuracy of the target feature extraction model and the training efficiency of the video feature extraction model.

[0172] In one feasible way, combining Figure 4 ,like Figure 5 As shown, in S201 above, the method for the electronic device to obtain the video features of the first video resource specifically includes:

[0173] S501, The electronic device acquires the text features and image features of the first video resource.

[0174] Specifically, the electronic device can acquire the text features and image features of the first video resource so that the video features of the first video resource can be obtained subsequently based on the text features and image features.

[0175] The aforementioned video features can be feature vectors, which are used to represent the resource content of the first video resource.

[0176] The aforementioned text features can also be feature vectors, which are used for the text content of the first video resource, such as the video title, subtitles in the video, and text converted from speech.

[0177] The aforementioned image features can also be feature vectors, which are used to represent the image content of the first video resource, such as video cover images, video frames, etc.

[0178] S502: The electronic device uses a multimodal algorithm to fuse text features and image features to obtain video features.

[0179] After acquiring the text and image features of the first video resource, given the sheer volume of such resources, using only two types of features to determine the classification loss value might reduce the efficiency of training the video feature extraction model. In this case, the electronic device can employ a multimodal algorithm to fuse the text and video features to obtain the video features. This allows the electronic device to determine the classification loss value based on a single multimodal feature (i.e., the video feature), improving the efficiency of training the video feature extraction model.

[0180] Among them, multimodal algorithms include: multi-headed self-attention algorithm.

[0181] In practical applications, electronic devices can also use other multimodal algorithms to fuse text features and video features to obtain video features, and this disclosure does not limit this.

[0182] As shown above, after obtaining the text and image features of the first video resource, the electronic device can perform feature fusion on the text and image features based on a multimodal algorithm to obtain video features. This provides a specific implementation method for determining video features, so that the classification loss value can be determined based on the video features. Then, the video feature extraction model can be trained based on the classification loss value, which improves the accuracy of the target feature extraction model and the training efficiency of the video feature extraction model.

[0183] In one feasible way, combining Figure 5 ,like Figure 6 As shown, in S501 above, the method by which the electronic device acquires the text features and image features of the first video resource specifically includes:

[0184] S601. The electronic device extracts features from the video image of the first video resource based on an image feature extraction algorithm to obtain image features.

[0185] Specifically, in order to quickly obtain video features, electronic devices can use image feature extraction algorithms to extract features from the video images of the first video resource to obtain image features.

[0186] The aforementioned image features can be feature vectors. These feature vectors are used to represent the image content of the video image of the first video resource.

[0187] In one possible implementation, the video image of the first video resource can be a cover image, video frame, etc., in the first video resource.

[0188] In one embodiment, the electronic device can perform video frame processing on the first video resource and obtain the cover image of the first video resource, thereby obtaining the video image of the first video resource.

[0189] For example, when the first video resource is a basketball video, the video image of the basketball video can be the cover image of the basketball video: "a picture of a basketball".

[0190] Next, the electronic device can use an image feature extraction algorithm to extract the image features of the first video resource from the video image of the basketball video.

[0191] Optionally, the image feature extraction algorithm can be a feature extraction algorithm based on residual networks (ResNet-50).

[0192] In practical applications, electronic devices can also use other image feature extraction techniques (such as image feature extraction models) to extract features from the video images of the first video resource to obtain image features, and this disclosure does not limit this.

[0193] S602. The electronic device performs text detection on the first video resource based on speech recognition algorithm and text detection algorithm to obtain text information.

[0194] S603. Electronic devices use a text feature extraction algorithm to extract features from text information to obtain text features.

[0195] Specifically, in order to quickly obtain video features, electronic devices can perform text detection on the first video resource based on speech recognition algorithms and text detection algorithms to obtain text information, and then extract features from the text information based on text feature extraction algorithms to obtain text features.

[0196] The aforementioned text features can be feature vectors. These feature vectors are used to represent the text content of the text information in the first video resource.

[0197] In one feasible approach, the text information of the first video resource can be text, title, audio content, etc., within the first video resource.

[0198] For example, when the first video resource is a basketball video, the text information of the basketball video can be the title of the basketball video: "How to play basketball".

[0199] Optionally, the text feature extraction algorithm can be a feature extraction algorithm based on bidirectional encoder representation from transformers (BERT).

[0200] In practical applications, electronic devices can also use other text feature extraction techniques (such as text feature extraction models) to extract text features from the text information of the first video resource to obtain text features, and this disclosure does not limit this.

[0201] As shown above, electronic devices can acquire text and image features of a first video resource based on various feature extraction algorithms. A specific implementation method for acquiring text and image features of a first video resource is given, so that video features can be determined based on the text and image features of the first video resource, and a classification loss value can be determined based on the video features. Then, a video feature extraction model is trained based on the classification loss value, which improves the accuracy of the target feature extraction model and also improves the training efficiency of the video feature extraction model.

[0202] In one feasible way, such as Figure 7 As shown, the training method for this video feature extraction model also includes:

[0203] S701, The electronic device acquires the initial video features of the first video resource and the initial search term features of the second video resource.

[0204] S702. The electronic device performs regularization processing on the initial video features and initial search term features to obtain the processed video features and processed search term features.

[0205] S703, the electronic device determines the processed video features as video features of the first video resource and the processed search term features as search term features of the second video resource.

[0206] Specifically, since the initial video features are the video features of the first video resource and the initial search term features are the search term features of the second video resource, in order to ensure that the video features and search term features used to determine the contrast loss value are on the same dimension, the electronic device can perform regularization processing on the initial video features and initial search term features to obtain processed video features and processed search term features. The processed video features are then determined as the video features of the first video resource, and the processed search term features are determined as the search term features of the second video resource.

[0207] In practical applications, the above regularization can be L2 regularization.

[0208] As shown above, electronic devices can determine the processed video features as the video features of the first video resource and the processed search term features as the search term features of the second video resource. A specific implementation method for obtaining the video features of the first video resource and the search term features of the second video resource is given, so as to facilitate subsequent comparative learning of the video features of the first video resource and the search term features of the second video resource to obtain a comparative loss value. Then, the video feature extraction model trained based on the comparative loss value improves the accuracy of the target feature extraction model and also improves the training efficiency of the video feature extraction model.

[0209] For example, Figure 8 This diagram illustrates a flowchart of a video feature extraction model training method provided in an embodiment of this application. Figure 8 As shown, electronic device 120 can obtain video images and text information of the first video resource from server 110.

[0210] Correspondingly, electronic device 120 can also obtain search terms for the second video resource from server 110.

[0211] Next, the electronic device 120 can determine the image features of the video image of the first video resource based on an image feature extraction algorithm (or an image encoder).

[0212] Accordingly, the electronic device 120 can determine the text features of the text information of the first video resource based on a text feature extraction algorithm (or a text encoder).

[0213] Accordingly, the electronic device 120 can determine the search term features of the search terms for the second video resource based on a text feature extraction algorithm (or a text encoder).

[0214] Next, the electronic device 120 can perform feature fusion on text features and image features based on a multimodal algorithm to obtain video features.

[0215] Next, the electronic device 120 can determine the classification result of the video features of the first video resource based on the classification model, and determine the classification loss value based on the difference between the classification result and the label information.

[0216] Next, the electronic device 120 can perform comparative learning on the video features of the first video resource and the search term features of the second video resource based on a contrastive learning model to obtain a contrastive loss value;

[0217] Next, the electronic device 120 can train the video feature extraction model to be trained based on the classification loss value and the contrast loss value to obtain the video feature extraction model.

[0218] Figure 9 A flowchart illustrating a video feature extraction method provided in an embodiment of this application is shown. Figure 9 As shown, the video feature extraction method includes:

[0219] S901, Electronic equipment acquires video resources to be processed.

[0220] In one feasible approach, electronic devices can obtain and process video resources from servers corresponding to various demanders (users or platforms that need to determine resource characteristics).

[0221] S902. The electronic device inputs the video resource to be processed into the video feature extraction model to obtain the video features of the video resource to be processed.

[0222] Among them, the video feature extraction model is based on Figures 2-8 The video feature extraction model is trained using any one of the following methods.

[0223] The aforementioned video features can be feature vectors. These feature vectors are used to represent the video content of the video resource to be processed.

[0224] The technical solution provided by the above embodiments brings at least the following beneficial effects: As shown in S901-S902, an application scenario is given for an electronic device to use a target feature extraction model to determine the video features of the video resource to be processed. The target feature extraction model can quickly and accurately obtain the video features of the video resource to be processed.

[0225] In one embodiment, when the video resources to be processed include a third video resource and a fourth video resource, the resource features of the video resources to be processed are input into the target feature extraction model to obtain video features. After that, the electronic device can also determine the similarity between the third video resource and the fourth video resource based on the content features of the third video resource and the content features of the fourth video resource, thereby providing an important basis for subsequent video resource association search.

[0226] It is understood that, in practical implementation, the terminal / server of this disclosure embodiment may include one or more hardware structures and / or software modules for implementing the aforementioned corresponding video feature extraction model training method. These hardware structures and / or software modules can constitute an electronic device. Those skilled in the art should readily recognize that, based on the algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0227] Based on this understanding, the present disclosure also provides a video feature extraction model training device that can be applied to electronic devices. Figure 10 A schematic diagram of the structure of the video feature extraction model training device provided in an embodiment of this disclosure is shown. Figure 10 As shown, the video feature extraction model training device may include: an acquisition unit 1001 and a processing unit 1002;

[0228] The acquisition unit 1001 is used to acquire the video features and tag information of the first video resource;

[0229] The processing unit 1002 is used to determine the classification result of the video features of the first video resource based on the classification model, and to determine the classification loss value based on the difference between the classification result and the label information.

[0230] The acquisition unit 1001 is also used to acquire the search term features of the second video resource;

[0231] The processing unit 1002 is also used to perform comparative learning on the video features of the first video resource and the search term features of the second video resource based on a comparative learning model, so as to obtain a comparative loss value.

[0232] The processing unit 1002 is also used to train the video feature extraction model to be trained based on the classification loss value and the contrast loss value, so as to obtain the video feature extraction model.

[0233] Optionally, when there are multiple first video resources and multiple second video resources, the multiple first video resources and multiple second video resources include both the same video resources and different video resources; the processing unit 1002 is specifically used for:

[0234] The dot product of the video features and search term features of the first type of video resources is determined as the first classification target of the contrastive learning model; the first type of video resources is used to represent the same video resources among multiple first video resources and multiple second video resources;

[0235] The dot product of the video features and search term features of the second type of video resources is determined as the second classification target of the contrastive learning model; the second type of video resources is used to represent different video resources among multiple first video resources and multiple second video resources.

[0236] Based on the first and second classification objectives, the video features of the first video resource and the search term features of the second video resource are compared and learned to obtain the comparison loss value.

[0237] Optionally, the processing unit 1002 is specifically used for:

[0238] Based on the first classification objective, a first type of feature set is determined; the first type of feature set is used to represent the video features and search term features of the first type of video resources.

[0239] Based on the second classification objective, a second type of feature set is determined; the second type of feature set is used to represent the video features and search term features of the second type of video resources.

[0240] The contrast loss value is determined based on the differences between the first and second feature sets.

[0241] Optionally, the acquisition unit 1001 is specifically used for:

[0242] Obtain the text and image features of the first video resource;

[0243] Based on a multimodal algorithm, text features and image features are fused to obtain video features.

[0244] Optionally, the acquisition unit 1001 is specifically used for:

[0245] Based on the image feature extraction algorithm, feature extraction is performed on the video images of the first video resource to obtain image features;

[0246] Based on speech recognition and text detection algorithms, text detection is performed on the first video resource to obtain text information;

[0247] Based on text feature extraction algorithms, features are extracted from text information to obtain text features.

[0248] Optionally, the acquisition unit 1001 is further configured to acquire the initial video features of the first video resource and the initial search term features of the second video resource;

[0249] The processing unit 1002 is also used to perform regularization processing on the initial video features and the initial search term features to obtain the processed video features and the processed search term features;

[0250] The processing unit 1002 is further configured to determine the processed video features as video features of the first video resource, and to determine the processed search term features as search term features of the second video resource.

[0251] Figure 11 A schematic diagram of the structure of the video feature extraction apparatus provided in an embodiment of this disclosure is shown. Figure 11 As shown, the video feature extraction device may include: an acquisition unit 1101 and a processing unit 1102;

[0252] The acquisition unit 1101 is used to acquire the video resources to be processed;

[0253] Processing unit 1102 is used to input the video resource to be processed into the video feature extraction model to obtain the video features of the video resource to be processed; the video feature extraction model is based on... Figures 2-8 It is obtained by training any one of the video feature extraction model training methods.

[0254] As described above, the embodiments of this disclosure can divide the electronic device into functional modules according to the above method examples. The integrated modules can be implemented in hardware or as software functional modules. Furthermore, it should be noted that the module division in these embodiments is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into a single processing module.

[0255] The specific methods of operation of each module and the beneficial effects of the video feature extraction model training device or video feature extraction device in the above embodiments have been described in detail in the foregoing method embodiments, and will not be repeated here.

[0256] This disclosure also provides a terminal, which may be a user terminal such as a mobile phone or a computer. Figure 12 A schematic diagram of the structure of a terminal provided in an embodiment of this disclosure is shown. The terminal may be a video feature extraction model training device or a video feature extraction device. The device may include at least one processor 61, a communication bus 62, a memory 63, and at least one communication interface 64.

[0257] Processor 61 can be a central processing unit (CPU), a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of programs according to the present disclosure. As an example, combined with... Figure 10 The processing unit 1002 in the electronic device performs the same functions as Figure 12 The processor 61 in it performs the same function.

[0258] The communication bus 62 may include a path for transmitting information between the aforementioned components.

[0259] Communication interface 64 uses any transceiver-like device for communicating with other devices or communication networks, such as servers, Ethernet, radio access networks (RAN), wireless local area networks (WLAN), etc. As an example,

[0260] Memory 63 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.

[0261] The memory 63 stores the application code that executes the present invention, and its execution is controlled by the processor 61. The processor 61 executes the application code stored in the memory 63 to implement the functions of the method of the present invention.

[0262] In a specific implementation, as one example, processor 61 may include one or more CPUs, for example... Figure 12 CPU0 and CPU1 in the CPU.

[0263] In a specific implementation, as one example, the terminal may include multiple processors, for example... Figure 12 Processors 61 and 65 are included. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0264] In a specific implementation, as one embodiment, the terminal may further include an input device 66 and an output device 67. The input device 66 and output device 67 communicate and can accept user input in various ways. For example, the input device 66 may be a mouse, keyboard, touchscreen device, or sensing device. The output device 67 communicates with the processor 61 and can display information in various ways. For example, the output device 61 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, etc.

[0265] Those skilled in the art will understand that Figure 12 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0266] This disclosure also provides a server. Figure 13 A schematic diagram of the server structure provided in this embodiment is shown. The server can be a video feature extraction model training device or a video feature extraction device. The server can vary significantly due to differences in configuration or performance, and may include one or more processors 71 and one or more memories 72. The memory 72 stores at least one instruction, which is loaded and executed by the processor 71 to implement the video feature extraction model training method or video feature extraction method provided in the above-described method embodiments. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated here.

[0267] This disclosure also provides a computer-readable storage medium including instructions stored thereon. When the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer is able to perform the video feature extraction model training method or the video feature extraction method provided in the embodiments described above. For example, the computer-readable storage medium can be a memory 63 including instructions, which can be executed by a processor 61 of a terminal to complete the above method. As another example, the computer-readable storage medium can be a memory 72 including instructions, which can be executed by a processor 71 of a server to complete the above method. Optionally, the computer-readable storage medium can be a non-transitory computer-readable storage medium, such as a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0268] This disclosure also provides a computer program product including computer instructions that, when executed on an electronic device, cause the electronic device to perform the above-described actions. Figures 2-8 The video feature extraction model training method shown in any of the attached figures, or the method described above. Figure 9 The video feature extraction method shown is illustrated.

[0269] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0270] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for training a video feature extraction model, characterized in that, include: Obtain video features and tag information from multiple primary video resources; The tag information is the topic tag of the first video resource; The classification results of the video features of the multiple first video resources are determined based on the classification model, and the classification loss value is determined based on the difference between the classification results and the label information. Obtain the search term features of multiple second video resources; The plurality of first video resources and the plurality of second video resources include both identical video resources and different video resources; The dot product of the video features and search term features of the first type of video resources is determined as the first classification target of the contrastive learning model; The first type of video resource is used to represent the same video resource among the plurality of first video resources and the plurality of second video resources; The dot product of the video features and search term features of the second type of video resources is determined as the second classification target of the contrastive learning model; The second type of video resources is used to represent different video resources among the plurality of first video resources and the plurality of second video resources; Based on the first classification target and the second classification target, the video features of the first video resource and the search term features of the second video resource are compared and learned to obtain a comparison loss value; Based on the classification loss value and the contrast loss value, the video feature extraction model to be trained is trained to obtain the video feature extraction model.

2. The video feature extraction model training method according to claim 1, characterized in that, The step of comparing and learning the video features of the first video resource and the search term features of the second video resource based on the first classification target and the second classification target to obtain the comparison loss value includes: Based on the first classification objective, a first type of feature set is determined; the first type of feature set is used to represent the video features and search term features of the first type of video resources; Based on the second classification objective, a second type of feature set is determined; the second type of feature set is used to represent the video features and search term features of the second type of video resources. The contrast loss value is determined based on the difference information between the first type of feature set and the second type of feature set.

3. The video feature extraction model training method according to claim 1, characterized in that, The video features of the first video resource to be acquired include: Obtain the text features and image features of the first video resource; Based on a multimodal algorithm, the text features and the image features are fused to obtain the video features.

4. The video feature extraction model training method according to claim 3, characterized in that, The acquisition of text features and image features of the first video resource includes: Based on the image feature extraction algorithm, feature extraction is performed on the video image of the first video resource to obtain the image features; Based on speech recognition and text detection algorithms, text detection is performed on the first video resource to obtain text information; Based on a text feature extraction algorithm, features are extracted from the text information to obtain the text features.

5. The video feature extraction model training method according to any one of claims 1-4, characterized in that, Also includes: Obtain the initial video features of the first video resource and the initial search term features of the second video resource; The initial video features and the initial search term features are regularized to obtain the processed video features and the processed search term features. The processed video features are determined as the video features of the first video resource, and the processed search term features are determined as the search term features of the second video resource.

6. A video feature extraction method, characterized in that, include: Acquire the video resources to be processed; The video resource to be processed is input into the video feature extraction model to obtain the video features of the video resource to be processed; The video feature extraction model is trained using the video feature extraction model training method according to any one of claims 1-5.

7. A video feature extraction model training device, characterized in that, include: Acquisition unit and processing unit; The acquisition unit is used to acquire video features and tag information of multiple first video resources; The tag information is the topic tag of the first video resource; The processing unit is configured to determine the classification results of the video features of the plurality of first video resources based on the classification model, and to determine the classification loss value based on the difference between the classification results and the label information. The acquisition unit is further configured to acquire search term features of multiple second video resources; the multiple first video resources and the multiple second video resources include the same video resources and different video resources; The processing unit is further configured to determine the dot product of the video features and search term features of the first type of video resources as the first classification target of the contrastive learning model; the first type of video resources is used to represent the same video resources among the plurality of first video resources and the plurality of second video resources; the dot product of the video features and search term features of the second type of video resources is determined as the second classification target of the contrastive learning model; the second type of video resources is used to represent the different video resources among the plurality of first video resources and the plurality of second video resources; Based on the first classification target and the second classification target, the video features of the first video resource and the search term features of the second video resource are compared and learned to obtain a comparison loss value; The processing unit is further configured to train the video feature extraction model to be trained based on the classification loss value and the contrast loss value, so as to obtain the video feature extraction model.

8. A video feature extraction device, characterized in that, include: Acquisition unit and processing unit; The acquisition unit is used to acquire the video resources to be processed; The processing unit is used to input the video resource to be processed into a video feature extraction model to obtain the video features of the video resource to be processed; the video feature extraction model is trained by the video feature extraction model training method according to any one of claims 1-5.

9. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the video feature extraction model training method as described in any one of claims 1-5, or to implement the video feature extraction method as described in claim 6.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the video feature extraction model training method as described in any one of claims 1-5, or to implement the video feature extraction method as described in claim 6.

Images