Model training method and system, data processing method and system

Abstract

The present application discloses a model training method and system, and a data processing method and system. The method comprises: receiving a model training request; obtaining target data and an initial model corresponding to the model training request, wherein the target data comprises multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample comprises: a first sample, a second sample, and a third sample, wherein the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; training the initial model using the multiple training samples to obtain a processing model; and outputting the processing model. The present application solves the technical problem in the related art that the training samples are transformed by artificially designed transformation patterns, which makes the training process prone to overfitting the image itself, resulting in poor generalization ability.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically, to a model training method and system, and a data processing method and system. Background Art

[0002] In the live streaming field, video search methods based on image features can be used to search and recommend live content. Over time, a large amount of video data has accumulated, much of which is unlabeled and unsupervised. Therefore, image self-supervised learning techniques can be used to train image feature extraction models and use the trained image feature extraction models to achieve video search.

[0003] Existing image self-supervised learning technology can utilize unsupervised data to generate positive sample image pairs by transforming a single image, and then randomly select other unrelated images as negative samples, and conduct comparative learning by constructing triplets to obtain a universal image feature expression.

[0004] Transformation operations based on a single image include random cropping, random scaling, random flipping, color change, etc. However, the above transformation operations are all manually designed transformation modes. In fact, the image content does not change in essence. Therefore, the training process is prone to overfitting the image itself, resulting in poor actual generalization ability.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] The embodiments of the present application provide a model training method and system, and a data processing method and system to at least solve the technical problem in the related art that the training samples are transformed through artificially designed transformation patterns, which makes the training process prone to overfitting the image itself and leads to poor generalization ability.

[0007] According to one aspect of an embodiment of the present application, a model training method is provided, including: receiving a model training request; obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; using the multiple training samples to train the initial model to obtain a processed model; and outputting the processed model.

[0008] According to another aspect of an embodiment of the present application, a model training method is also provided, including: obtaining target data, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; using multiple training samples to train the initial model to obtain a processed model.

[0009] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: receiving first data; processing the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first eigenvector of the first data and the second eigenvectors of multiple third data, determining target data among multiple third data; and outputting the target data.

[0010] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining first data; processing the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first eigenvector of the first data and the second eigenvectors of the multiple third data, determining the target data among the multiple third data.

[0011] According to another aspect of an embodiment of the present application, a model training method is also provided, including: obtaining a model training request by calling a first interface, wherein the first interface includes: a first parameter, and the parameter value of the first parameter is the model training request; obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; using multiple training samples to train the initial model to obtain a processing model; outputting the processing model by calling a second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the processing model.

[0012] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining first data by calling a first interface, wherein the first interface includes: a first parameter, and the parameter value of the first parameter is the first data; processing the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, and the second data contains multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; determining target data among multiple third data based on the first eigenvector of the first data and the second eigenvectors of multiple third data; outputting the target data by calling a second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the target data.

[0013] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned model training method and data processing method.

[0014] According to another aspect of an embodiment of the present application, a computer terminal is also provided, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the above-mentioned model training method and data processing method are executed when the program is running.

[0015] According to another aspect of an embodiment of the present application, a model training system is also provided, including: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: receiving a model training request; obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; using multiple training samples to train the initial model to obtain a processing model; and outputting the processing model.

[0016] According to another aspect of an embodiment of the present application, a data processing system is also provided, including: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: receiving first data; processing the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, the second data contains multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first eigenvector of the first data and the second eigenvectors of the multiple third data, determining the target data among the multiple third data; and outputting the target data.

[0017] In an embodiment of the present application, after receiving a model training request, the target data and the initial model corresponding to the model training request can be obtained, the target data can be further processed to obtain multiple training samples, and the initial model can be trained using the multiple training samples to achieve the purpose of processing model training. It is easy to notice that the first sample, the second sample, and the third sample in the training sample are all data frames extracted from the target data. Therefore, the difference between the first sample and the third sample comes from the natural change of the data content, thereby achieving the goal of constructing training samples and training models based on the video content and making full use of the relevance of the video context. Compared with the manually designed image transformation operation, the technical effect of improving the generalization ability of the self-supervised feature is achieved, thereby solving the technical problem in the related technology of using an artificially designed transformation mode to transform the training sample, which makes it easy to overfit the image itself during the training process and leads to poor generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a model training method and a data processing method according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a computer terminal serving as a receiving end according to an embodiment of the present application;

[0021] Figure 3 is a flowchart of a first model training method according to an embodiment of the present application;

[0022] Figure 4is a schematic diagram of an optional interactive interface according to an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of an optional processing model according to an embodiment of the present application;

[0024] Figure 6 is a flowchart of a second model training method according to an embodiment of the present application;

[0025] Figure 7 is a flowchart of a first data processing method according to an embodiment of the present application;

[0026] Figure 8 is a flowchart of a second data processing method according to an embodiment of the present application;

[0027] Figure 9 is a flowchart of a third model training method according to an embodiment of the present application;

[0028] Figure 10 is a schematic diagram of a first model training device according to an embodiment of the present application;

[0029] Figure 11 is a schematic diagram of a second model training device according to an embodiment of the present application;

[0030] Figure 12 is a schematic diagram of a first data processing device according to an embodiment of the present application;

[0031] Figure 13 is a schematic diagram of a second data processing device according to an embodiment of the present application;

[0032] Figure 14 is a schematic diagram of a third model training device according to an embodiment of the present application;

[0033] Figure 15 is a flowchart of a third data processing method according to an embodiment of the present application;

[0034] Figure 16 is a schematic diagram of a third data processing device according to an embodiment of the present application;

[0035] Figure 17 This is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0039] Video clips: Video data can be captured by cutting it into segments of fixed length.

[0040] Mini-batch stochastic gradient descent method: It can be a mini-batch based stochastic gradient descent (SGD) method, which can divide the samples into multiple sample sets and use one sample set for training in each iteration, where the number of sample sets is the same as the size of the mini-batch.

[0041] Image Encoder: Image encoding network, which can use CNN (Convolutional Neural Network) structure to extract feature vectors of images;

[0042] Projection head: Mapping head network, which can use MLP (Multilayer Perceptron) to map the input feature vector to a lower dimension.

[0043] Triplet loss: Triplet loss function, which makes the distance between similar frames smaller than the distance between dissimilar frames through learning.

[0044] Contrastive loss: Contrastive loss function, through which the matching success of paired samples can be better expressed.

[0045] NCE loss: Noise Contrastive Estimation, noise contrast estimation loss function, can distinguish real samples from noise samples by learning a classifier.

[0046] VLAD: Vector of Locally Aggregated Descriptors, local feature aggregation descriptor, can cluster local features into a vector.

[0047] In the application scenario of live broadcasting, image self-supervised learning technology can be used to realize the search and recommendation of live broadcast content. However, the existing image self-supervised learning technology requires a single image to be transformed through an artificially designed transformation pattern. In fact, the image content does not change in essence. Therefore, the training process is prone to overfitting the image itself, resulting in poor actual generalization ability.

[0048] To solve the above technical problems, this application provides an unsupervised image feature training method based on video context for searching similar videos. The specific implementation scheme is as follows:

[0049] Example 1

[0050] According to an embodiment of the present application, a model training method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing the model training method and data processing method is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more (illustrated as 102a, 102b, ..., 102n) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0052] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computer terminal 10 (or mobile device). The data processing circuitry may function as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0053] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage devices corresponding to the model training method and data processing method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned model training method and data processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0054] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0055] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0056] Figure 1 The hardware structure block diagram shown can be used not only as an exemplary block diagram of the computer terminal 10 (or mobile device), but also as an exemplary block diagram of the server. In an optional embodiment, Figure 2 The block diagram shows the use of the above Figure 1 The computer terminal 10 (or mobile device) shown in FIG. 1 is an embodiment of a receiving end. Figure 2 As shown, the computer terminal 10 (or mobile device) can be connected to one or more clients 20 via a data network connection or electronic connection. In an optional embodiment, the computer terminal 10 (or mobile device) can be any mobile computing terminal. The data network connection can be a local area network connection, a wide area network connection, an Internet connection, or other types of data network connections. The computer terminal 10 (or mobile device 10) can provide network-based user services for the client, such as social networks, cloud resources, email, online payment or other online applications.

[0057] Under the above operating environment, this application provides Figure 3 The model training method shown. Figure 3 This is a flow chart of the first model training method according to an embodiment of the present application. Figure 3 As shown, the method may include the following steps:

[0058] Step S302: Receive a model training request.

[0059] In order to train a model with high processing accuracy, it is often necessary to use a large number of training samples for multiple iterative training. The amount of data and computing power of the entire training process is large. In order to reduce the resource consumption of user terminals (such as smartphones, tablets, laptops, PDAs, personal computers, etc.), the server can perform model training, and the user terminal only deploys the trained model for user convenience. Furthermore, in order to significantly reduce the computing burden of user devices, the trained model can be directly deployed on the server. The user terminal connects to the server through a specific interface and sends the video data to the server. The server uses the deployed model to process the video data and feeds back the processing results to the user terminal.

[0060] The model training request in the above steps can be generated according to the user's model usage requirements. For example, in a video search scenario in the live broadcast field, the model training request can be a training request for an unsupervised image feature extraction model.

[0061] In an optional embodiment, an interactive interface may be provided on the user terminal, such as Figure 4 As shown, the user enters a model training request in the input area of ​​this interface, and the user terminal can then send the model training request to the server via the network. Because the model usage requirements in video search scenarios in the live broadcast field are relatively fixed, a selection button can be provided to the user to select whether to generate a model for video search, thereby generating the above-mentioned model training request.

[0062] Step S304: Obtain target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments.

[0063] The target data in the above steps can be audio data or video data. In the embodiment of the present application, the target data is video data as an example for explanation. The video data can be a certain amount of video data collected from a large amount of accumulated video data, which serves as the original data for model training. The video data is composed of video frames of a certain length. Therefore, the video data can be divided into multiple video segments (i.e., the above-mentioned data segments) according to a pre-set time interval, wherein the pre-set time interval can be 10s, but is not limited to this, and can be set according to actual needs to ensure that the video frames in the same video segment are similar.

[0064] It should be noted that due to the different content contained in the video data, it is possible that the same video clip contains two different contents, resulting in dissimilar video frames within the same video clip. However, in the video search scenario of the live broadcast field, the probability of this happening is low, and by collecting a large amount of video data, this situation can be ignored. Therefore, in the embodiments of the present application, it can be determined that video frames within the same video clip are similar, and video frames in different video clips are dissimilar.

[0065] The processing model in the above steps can be a model for extracting image features, and the specific structure of the model can adopt an existing structure. For example, the processing model can be composed of an Image Encoder and a Projection head, wherein the Image Encoder can use a structure such as ResNet (Residual Network) to extract the feature vector of the image, and the Projection head can use MLP to map the input original feature vector to a lower dimension.

[0066] Step S306 : Process the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample. The first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments.

[0067] Each training sample in the above steps can be composed of a triplet, namely a first sample, a second sample and a third sample, wherein the first sample is a video frame randomly extracted from the video data, the second sample is a positive sample of the first sample in the video data, and the third sample is a negative sample of the first sample in the video data.

[0068] For video data, video frames from the same video clip are adjacent frames and are considered similar frames, while video frames from different video clips are considered dissimilar frames. In an optional embodiment, a frame of image can be randomly extracted from the video data to obtain the above-mentioned first sample. Then, another frame of image can be randomly extracted from the same video clip to obtain the above-mentioned second sample. Finally, a frame of image can be randomly extracted from another video clip to obtain the above-mentioned third sample, thereby generating a training sample. By repeating the above operation, a large number of training samples can be obtained.

[0069] Step S308: train the initial model using multiple training samples to obtain a processed model.

[0070] Optionally, the initial model can be trained using multiple training samples based on the mini-batch SGD method to obtain a processing model. In each iteration, a batch-size number of training samples can be randomly selected for training, but is not limited to this.

[0071] In an optional embodiment, the size of the mini-batch can be set in advance, that is, the mini-batch-size is set, and multiple training samples can be divided into multiple sample sets based on the mini-batch-size. In each iteration, one sample set can be trained. During the training process, the training samples contained in the sample set can be input into the initial model to obtain the feature vector of each sample, and the network parameters of the initial model are updated through the similarity between the first sample and the second sample, and the similarity between the first sample and the third sample, until the two similarities meet the training requirements, and finally the above-mentioned processing model is obtained.

[0072] Step S310: output the processing model.

[0073] In an optional embodiment, if the processing model needs to be deployed in the user terminal, the server can transmit the processing model to the user terminal through the network; if the processing model needs to be deployed in the server, the processing model can be directly put online, so that the user can use the online processing model to extract image features and achieve the purpose of video search in the live broadcast field.

[0074] Through the technical solution provided by the above-mentioned embodiment of the present application, after receiving a model training request, the target data and the initial model corresponding to the model training request can be obtained, the target data can be further processed to obtain multiple training samples, and the initial model can be trained using the multiple training samples to achieve the purpose of processing model training. It is easy to notice that the first sample, the second sample, and the third sample in the training sample are all data frames extracted from the target data. Therefore, the difference between the first sample and the third sample comes from the natural change of the data content, thereby achieving the goal of constructing training samples and training models based on the video content and making full use of the relevance of the video context. Compared with the manually designed image transformation operation, the technical effect of improving the generalization ability of the self-supervised feature is achieved, thereby solving the technical problem in the related technology of using the artificially designed transformation mode to transform the training sample, which makes it easy to overfit the image itself during the training process and leads to poor generalization ability.

[0075] In the above embodiment of the present application, processing the target data to obtain multiple training samples includes: dividing the video data to obtain multiple video segments; extracting a preset number of target video frames from each video segment; and determining multiple training samples based on the target video frames.

[0076] The preset number in the above steps can be determined based on a preset sampling interval. The preset sampling interval can be a fixed interval, for example, 1s, or an interval determined based on the model processing accuracy required by the user. In this case, the preset number is a dynamically changing value.

[0077] In an optional embodiment, the video data can be divided according to a pre-set time interval to obtain multiple video segments, and several video segments can be randomly selected for video frame extraction. From the randomly selected video segments, video frames can be randomly extracted at a fixed sampling interval to obtain target video frames. Furthermore, triplets are constructed based on two video frames from the same video segment and a video frame from another video segment, thereby obtaining multiple training samples.

[0078] In the above embodiment of the present application, determining multiple training samples based on the target video frame includes: processing the target video frame according to a preset transformation mode to obtain multiple training samples, wherein the preset transformation mode includes at least one of the following: cropping operation, scaling operation, flipping operation, and color transformation operation.

[0079] In an optional embodiment, in order to further enhance the generalization performance and avoid model overfitting, the extracted video frames can be preprocessed by using data augmentation methods such as random cropping, random scaling, random flipping, and color transformation, and triplets can be constructed based on the processed video frames to obtain multiple training samples.

[0080] In the above embodiment of the present application, training the initial model using multiple training samples includes: inputting each training sample into the image encoding network of the initial model to obtain the first eigenvector of each training sample; inputting the first eigenvector of each training sample into the mapping head network of the initial model to obtain the second eigenvector of each training sample; based on the second eigenvector of each training sample, obtaining the loss value of each training sample; when the loss value of each training sample does not meet the training conditions, updating the network parameters of the initial model based on multiple training samples until the loss value of each training sample meets the training conditions.

[0081] The training condition in the above steps may be a pre-set fixed threshold, or a dynamically changing threshold determined according to the model processing accuracy required by the user, but is not limited thereto.

[0082] In an optional embodiment, in order to achieve the purpose of video search, the goal of model training is to increase the similarity between similar frame feature vectors and reduce the similarity between non-similar frame feature vectors. In this embodiment of the present application, the loss function of contrastive learning can be used for model training.

[0083] During each iterative training process, a certain amount of training samples can be input into the image coding network to extract the original feature vectors of the training samples, which include the original feature vectors of each of the three samples; the original feature vectors are input into the mapping head network to map the original feature vectors to a lower dimension to obtain the target feature vectors of the training samples, which include the target feature vectors of each of the three samples; based on the target feature vectors of each sample, the loss function of contrastive learning is used to obtain the corresponding loss value. If the loss value does not meet the training conditions, the next iterative training process is entered. If the loss value meets the training conditions, the training ends.

[0084] In the above embodiment of the present application, the second feature vector of each training sample includes: the first vector of the first sample, the second vector of the second sample, and the third vector of the third sample, wherein, based on the second feature vector of each training sample, obtaining the loss value of each training sample includes: obtaining a first similarity based on the first vector and the second vector; obtaining a second similarity based on the first vector and the third vector; inputting the first similarity and the second similarity into a preset loss function to obtain the loss value of each training sample.

[0085] In an optional embodiment, the similarity between two video frames can be measured using the cosine distance of the feature vectors, but is not limited thereto and can also be measured using the Euclidean distance. In this embodiment, the similarity between two video frames can be calculated using the following formula:

[0086] Among them, a and b represent the feature vectors of two video frames respectively.

[0087] Optionally, the above-mentioned preset loss function may include one of the following: Triplet loss, Contrastive loss, NCE loss, but is not limited thereto, and may also be other contrastive learning loss functions.

[0088] The above triplet loss formula is as follows:

[0089] Where i represents the first sample, c represents the second sample, represents the third sample, α is a preset difference between the distance between non-similar frames and the distance between similar frames, and can be a fixed value.

[0090] The formula for Contrastive loss is as follows:

[0091]

[0092] The formula for NCE loss is as follows:

[0093]

[0094] In an optional embodiment, the cosine distance between the first vector and the second vector can be calculated to obtain a first similarity, and the cosine distance between the first vector and the third vector can be calculated to obtain a second similarity. The first similarity and the second similarity are further input into the NCE loss formula to obtain the loss value of each training sample.

[0095] It should be noted that the specific application scenarios and implementation solutions of the processing models in the embodiments of the present application are detailed in the description of the data processing methods in the subsequent embodiments and will not be elaborated here.

[0096] The following combination Figure 5 Taking video search in the live broadcast field as an example, the unsupervised image feature training method based on video context provided by this application is described in detail. Figure 5 As shown in Figure 1, the unsupervised image feature model can be deployed on mobile devices or servers. The model consists of two parts: an image encoder and a projection head. The image encoder uses a basic CNN network such as ResNet to extract the image feature vector, while the projection head uses an MLP to map the original feature vector to a lower dimension.

[0097] The unsupervised image feature training process is as follows: first, a certain number of videos are collected, and several short video clips are randomly selected from each video, and each video clip is used as a piece of data; a mini-batch-based SGD training strategy can be adopted, and training is performed through multiple iterations until the loss converges. In each iteration, a batch-size number of video clips are randomly selected, and two frames of video frames are extracted from each video clip. Then, data augmentation methods such as random cropping, random scaling, random flipping, and color transformation are used to preprocess the video frames; based on data collection, the cosine distance of the feature vector is used to calculate the similarity between the two video frames, and the loss in this iteration is calculated using the NCE loss.

[0098] like Figure 5 As shown, we can randomly select a frame image x and select another frame image in the same segment as a positive sample Select another frame image in a different segment as a negative sample Positive samples and negative samples are input into the Image Encoder part respectively to obtain the original feature vector h i and h j , and further input them into the Projection head to obtain the final feature vector zi and z j , and finally through the eigenvector z i and z j Calculate the Contrastive loss to get the loss in this iteration. The two Image Encoders can share feature parameters with each other; similarly, the two Projection Heads can also share feature parameters with each other.

[0099] The application process of unsupervised image features in video search is as follows: Based on the trained unsupervised image feature model, some parameters of the Image Encoder can be isolated and used as the image feature model. Several frames are randomly extracted from the video to represent the video. The corresponding image feature vector is extracted for each video frame, and the video features are represented by a set of video frame features. Based on this set representation, the similarity between two sets can be calculated using video similarity metrics such as center similarity or VLAD vector similarity. Based on the similarity between the two sets, similar videos are further determined to obtain the final search results.

[0100] Through the above solution, we can fully utilize the relevance of video context to construct a self-supervised image learning task, which can obtain image features with better generalization performance. Based on the feature model obtained through training, we can further perform feature extraction and search on the video.

[0101] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0103] Example 2

[0104] According to an embodiment of the present application, a model training method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0105] Figure 6 This is a flow chart of the second model training method according to an embodiment of the present application. Figure 6 As shown, the method may include the following steps:

[0106] Step S602: Acquire target data, where the target data includes multiple data segments.

[0107] The target data in the above steps can be audio data or video data. In the embodiment of the present application, the target data is video data as an example for explanation. The video data can be a certain amount of video data collected from a large amount of accumulated video data, which serves as the original data for model training. The video data is composed of video frames of a certain length. Therefore, the video data can be divided into multiple video segments (i.e., the above-mentioned data segments) according to a pre-set time interval, wherein the pre-set time interval can be 10s, but is not limited to this, and can be set according to actual needs to ensure that the video frames in the same video segment are similar.

[0108] Step S604 : Process the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample. The first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments.

[0109] Each training sample in the above steps can be composed of a triplet, namely a first sample, a second sample and a third sample, wherein the first sample is a video frame randomly extracted from the video data, the second sample is a positive sample of the first sample in the video data, and the third sample is a negative sample of the first sample in the video data.

[0110] Step S606: train the initial model using multiple training samples to obtain a processed model.

[0111] The processing model in the above steps can be a model for extracting image features, and the specific structure of the model can adopt an existing structure. For example, the processing model can be composed of an Image Encoder and a Projection head, wherein the Image Encoder can use a structure such as ResNet (Residual Network) to extract the feature vector of the image, and the Projection head can use MLP to map the input original feature vector to a lower dimension.

[0112] Optionally, the initial model can be trained using multiple training samples based on the mini-batch SGD method to obtain a processing model. In each iteration, a batch-size number of training samples can be randomly selected for training, but is not limited to this.

[0113] In the above embodiment of the present application, processing the target data to obtain multiple training samples includes: dividing the video data to obtain multiple video segments; extracting a preset number of target video frames from each video segment; and determining multiple training samples based on the target video frames.

[0114] In the above embodiment of the present application, determining multiple training samples based on the target video frame includes: processing the target video frame according to a preset transformation mode to obtain multiple training samples, wherein the preset transformation mode includes at least one of the following: cropping operation, scaling operation, flipping operation, and color transformation operation.

[0115] In the above embodiment of the present application, training the initial model using multiple training samples includes: inputting each training sample into the image encoding network of the initial model to obtain the first eigenvector of each training sample; inputting the first eigenvector of each training sample into the mapping head network of the initial model to obtain the second eigenvector of each training sample; based on the second eigenvector of each training sample, obtaining the loss value of each training sample; when the loss value of each training sample does not meet the training conditions, updating the network parameters of the initial model based on multiple training samples until the loss value of each training sample meets the training conditions.

[0116] In the above embodiment of the present application, the second feature vector of each training sample includes: the first vector of the first sample, the second vector of the second sample, and the third vector of the third sample, wherein, based on the second feature vector of each training sample, obtaining the loss value of each training sample includes: obtaining a first similarity based on the first vector and the second vector; obtaining a second similarity based on the first vector and the third vector; inputting the first similarity and the second similarity into a preset loss function to obtain the loss value of each training sample.

[0117] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0118] Example 3

[0119] According to an embodiment of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0120] Figure 7 This is a flow chart of the first data processing method according to an embodiment of the present application. Figure 7 As shown, the method may include the following steps:

[0121] Step S702: Receive first data.

[0122] The first data in the above steps may be audio data or video data. In the embodiment of the present application, the first data is taken as the first video data as an example for explanation. The first video data may be a video provided by the user when performing a video search.

[0123] In an optional embodiment, the user may be provided with a Figure 4 In the interactive interface shown, the user uploads the first video data in the input area. The user can complete the uploading process of the first video data by directly dragging the first video data into the area or clicking the "Upload Video" button. If the processing model needs to be deployed on the user terminal, the user terminal can directly process the first video data. If the processing model needs to be deployed on the server, the user terminal can send the first video data to the server via the network for processing.

[0124] It should be noted that in order to significantly reduce the computing burden of user devices, the trained model can be directly deployed in the server. The user terminal connects to the server through a specific interface and sends the first video data to the server. The server uses the deployed model to process the first video data and feeds back the processing results to the user terminal.

[0125] Step S704: Use the processing model to process the first data to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are in the same data segment, and the first sample and the third sample are in different data segments.

[0126] The processing model in the above steps can be a model for extracting image features, and the specific structure of the model can adopt an existing structure. For example, the processing model can be composed of an Image Encoder and a Projection head, wherein the Image Encoder can use a structure such as ResNet (Residual Network) to extract the feature vector of the image, and the Projection head can use MLP to map the input original feature vector to a lower dimension.

[0127] The second data in the above steps can be audio data or video data. In the embodiment of the present application, the second data is taken as the second video data as an example for explanation. The second video data can be a certain amount of video data collected from a large amount of accumulated video data, which serves as the original data for model training. The second video data is composed of video frames of a certain length. Therefore, the second video data can be divided into multiple video segments (i.e., the above-mentioned data segments) according to a preset time interval, wherein the preset time interval can be 10s, but is not limited to this, and can be set according to actual needs to ensure that the video frames in the same video segment are similar.

[0128] Each training sample in the above steps can be composed of a triplet, namely a first sample, a second sample and a third sample, wherein the first sample is a video frame randomly extracted from the second video data, the second sample is a positive sample of the first sample in the second video data, and the third sample is a negative sample of the first sample in the second video data.

[0129] For video data, video frames from the same video clip are adjacent frames and are considered similar frames, while video frames from different video clips are considered dissimilar frames. In an optional embodiment, a frame of image can be randomly extracted from the video data to obtain the above-mentioned first sample. Then, another frame of image can be randomly extracted from the same video clip to obtain the above-mentioned second sample. Finally, a frame of image can be randomly extracted from another video clip to obtain the above-mentioned third sample, thereby generating a training sample. By repeating the above operation, a large number of training samples can be obtained.

[0130] It should be noted that due to the different content contained in the video data, it is possible that the same video clip contains two different contents, resulting in dissimilar video frames within the same video clip. However, in the video search scenario of the live broadcast field, the probability of this happening is low, and by collecting a large amount of video data, this situation can be ignored. Therefore, in the embodiments of the present application, it can be determined that video frames within the same video clip are similar, and video frames in different video clips are dissimilar.

[0131] Step S706 : determining target data from the plurality of third data based on the first eigenvector of the first data and the second eigenvectors of the plurality of third data.

[0132] The third data in the above steps can be audio data or video data. In the embodiment of the present application, the third data is described as third video data. The plurality of third video data can be all live videos stored in a database in the live broadcast field. The target data in the above steps can be audio data or video data. In the embodiment of the present application, the target data is described as target video data. The target video data can be a similar live video searched from the database. Therefore, the first video data is similar to the target video data.

[0133] Step S708: output target data.

[0134] In an optional embodiment, if the processing model needs to be deployed in the user terminal, the user terminal can directly display the target data on a Figure 4 If the processing model needs to be deployed in the server, the server can return the target video data to the user terminal through the network, and the user terminal will display it in the display area of ​​the interactive interface as shown in FIG. Figure 4 In the display area of ​​the interactive interface shown, the user can view the searched similar video data.

[0135] Through the technical solution provided by the above-mentioned embodiment of the present application, after receiving the first data, the first data can be processed using the processing model to obtain the first eigenvector of the first data, and further based on the first eigenvector of the first data and the second eigenvectors of the plurality of third data, the target data in the plurality of third data can be determined to achieve the purpose of similar data retrieval. It is easy to notice that the first sample, the second sample and the third sample in the training sample are all data frames extracted from the second data. Therefore, the difference between the first sample and the third sample comes from the natural change of the data content, thereby achieving the goal of constructing training samples and training models based on the video content and making full use of the relevance of the video context. Compared with the manually designed image transformation operation, the technical effect of improving the generalization ability of the self-supervised feature is achieved, thereby solving the technical problem in the related technology of using the artificially designed transformation mode to transform the training sample, which makes it easy to overfit the image itself during the training process and leads to poor generalization ability.

[0136] In the above embodiment of the present application, the first data is processed using a processing model to obtain a first feature vector of the first data, including: extracting multiple first video frames from the first video data; processing each first video frame using the image coding network in the processing model to obtain a third feature vector of each first video frame; and determining a set of the third feature vectors of multiple first video frames as the first feature vector of the first video data.

[0137] The third feature vector in the above step can be the image feature vector of each first video frame, the first feature vector can be the video feature vector of the first video data, and the video feature vector can be expressed as a set of image feature vectors of multiple first video frames, that is, V = {f1, f2, f3, ..., f n}.

[0138] In an optional embodiment, since the processing model is used to extract image feature vectors, and the first video data contains many video frames, the first video data cannot be processed directly using the processing model. Therefore, multiple first video frames can be randomly extracted from the first video data to represent the first video data, and the image feature vector of each first video frame can be extracted using the Image Encoder part to obtain the third feature vector of each first video frame, and the first feature quantity of the first video data can be represented by the set of the third feature vectors of all first video frames.

[0139] It should be noted that since the Projection head part in the processing model is used to map the original feature vector to a lower dimension, in actual video search applications, based on the trained processing model, some parameters of the Image Encoder can be separated as an image feature model, and the extracted video frames can be processed through the image feature model to obtain the image feature vector.

[0140] It should also be noted that, in order to facilitate video search, the modulus length of each extracted image feature vector may be normalized.

[0141] In the above embodiment of the present application, based on the first eigenvector of the first data and the second eigenvector of multiple third data, determining the target data among the multiple third data includes: obtaining the similarity between the first eigenvector of the first video data and the second eigenvector of each third video data to obtain multiple similarities; determining the third video data corresponding to the similarity greater than a preset value among the multiple similarities as the target video data.

[0142] The preset value in the above steps may be a minimum threshold of similarity set according to the accuracy requirement of the video search. The user may modify it according to his or her own needs, and this application does not impose any specific limitation on this.

[0143] In an optional embodiment, after determining the first feature vector of the first video data and the second feature vector of each third video data, the similarity between the feature vectors can be calculated to determine the similarity between the first video data and each third video data, and the third video data with a similarity greater than a preset value can be further obtained as the searched similar video data, that is, the above-mentioned target video data.

[0144] It should be noted that the number of video data with similarity greater than the preset value may be greater than 1. On this basis, the third video data with the greatest similarity can be determined as the searched similar video data.

[0145] In the above embodiment of the present application, obtaining the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtaining multiple similarities includes: obtaining the average vector of all feature vectors contained in the first feature vector of the first video data to obtain a first average vector; obtaining the average vector of all feature vectors contained in the second feature vector of each third video data to obtain multiple second average vectors; obtaining the distance between the first average vector and each second average vector to obtain multiple similarities.

[0146] In an optional embodiment, since both the first eigenvector and the second eigenvector are sets of image eigenvectors of multiple video frames, in order to determine the similarity between the first eigenvector and the second eigenvector, the center similarity metric can be used: where f c represents the average vector of all feature vectors in the set, and s represents the cosine distance function.

[0147] In the above embodiment of the present application, obtaining the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtaining multiple similarities includes: using the local feature aggregation descriptor algorithm to process the first feature vector of the first video data and the second feature vector of each third video data respectively, to obtain the first aggregation vector of the first video data and the second aggregation vector of each third video data; obtaining the similarity between the first aggregation vector of the first video data and the second aggregation vector of each third video data, to obtain multiple similarities.

[0148] In an optional embodiment, since the first feature vector and the second feature vector are both sets of image feature vectors of multiple video frames, in order to determine the similarity between the first feature vector and the second feature vector, it can be achieved through the VLAD vector similarity measurement method, and all feature vectors in the set can be expressed as a VLAD vector (that is, the first aggregation vector and the second aggregation vector mentioned above), and the similarity of the VLAD vector is further used to represent the similarity between the two sets.

[0149] It should be noted that the training method for the processing model in the embodiment of the present application is detailed in the description of the model training method in the above embodiment, and will not be repeated here.

[0150] Example 4

[0151] According to an embodiment of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0152] Figure 8 This is a flow chart of a second data processing method according to an embodiment of the present application. Figure 8 As shown, the method may include the following steps:

[0153] Step S802: Acquire first data.

[0154] The first data in the above steps may be audio data or video data. In the embodiment of the present application, the first data is taken as the first video data as an example for explanation. The first video data may be a video provided by the user when performing a video search.

[0155] Step S804: Use the processing model to process the first data to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are in the same data segment, and the first sample and the third sample are in different data segments.

[0156] The processing model in the above steps can be a model for extracting image features, and the specific structure of the model can adopt an existing structure. For example, the processing model can be composed of an Image Encoder and a Projection head, wherein the Image Encoder can use a structure such as ResNet (Residual Network) to extract the feature vector of the image, and the Projection head can use MLP to map the input original feature vector to a lower dimension.

[0157] The second data in the above steps can be audio data or video data. In the embodiment of the present application, the second data is taken as the second video data as an example for explanation. The second video data can be a certain amount of video data collected from a large amount of accumulated video data, which serves as the original data for model training. The second video data is composed of video frames of a certain length. Therefore, the second video data can be divided into multiple video segments (i.e., the above-mentioned data segments) according to a preset time interval, wherein the preset time interval can be 10s, but is not limited to this, and can be set according to actual needs to ensure that the video frames in the same video segment are similar.

[0158] Each training sample in the above steps can be composed of a triplet, namely a first sample, a second sample and a third sample, wherein the first sample is a video frame randomly extracted from the second video data, the second sample is a positive sample of the first sample in the second video data, and the third sample is a negative sample of the first sample in the second video data.

[0159] Step S806 : determining target data from the plurality of third data based on the first eigenvector of the first data and the second eigenvectors of the plurality of third data.

[0160] The third data in the above steps can be audio data or video data. In the embodiment of the present application, the third data is described as third video data. The plurality of third video data can be all live videos stored in a database in the live broadcast field. The target data in the above steps can be audio data or video data. In the embodiment of the present application, the target data is described as target video data. The target video data can be a similar live video searched from the database. Therefore, the first video data is similar to the target video data.

[0161] In the above embodiment of the present application, the first data is processed using a processing model to obtain a first feature vector of the first data, including: extracting multiple first video frames from the first video data; processing each first video frame using the image coding network in the processing model to obtain a third feature vector of each first video frame; and determining a set of the third feature vectors of multiple first video frames as the first feature vector of the first video data.

[0162] In the above embodiment of the present application, based on the first eigenvector of the first data and the second eigenvector of multiple third data, determining the target data among the multiple third data includes: obtaining the similarity between the first eigenvector of the first video data and the second eigenvector of each third video data to obtain multiple similarities; determining the third video data corresponding to the similarity greater than a preset value among the multiple similarities as the target video data.

[0163] In the above embodiment of the present application, obtaining the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtaining multiple similarities includes: obtaining the average vector of all feature vectors contained in the first feature vector of the first video data to obtain a first average vector; obtaining the average vector of all feature vectors contained in the second feature vector of each third video data to obtain multiple second average vectors; obtaining the distance between the first average vector and each second average vector to obtain multiple similarities.

[0164] In the above embodiment of the present application, obtaining the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtaining multiple similarities includes: using the local feature aggregation descriptor algorithm to process the first feature vector of the first video data and the second feature vector of each third video data respectively, to obtain the first aggregation vector of the first video data and the second aggregation vector of each third video data; obtaining the similarity between the first aggregation vector of the first video data and the second aggregation vector of each third video data, to obtain multiple similarities.

[0165] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 3, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 3.

[0166] Example 5

[0167] According to an embodiment of the present application, a model training method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0168] Figure 9 This is a flow chart of the third model training method according to an embodiment of the present application. Figure 9 As shown, the method may include the following steps:

[0169] Step S902: Obtain a model training request by calling a first interface, wherein the first interface includes: a first parameter, and a parameter value of the first parameter is the model training request.

[0170] The first interface in the above steps can be an interface for data interaction between the server and the user terminal. The user terminal can pass the model training request into the interface function as a parameter of the interface function to achieve the purpose of uploading the model training request to the server.

[0171] The model training request in the above steps can be generated according to the user's model usage requirements. For example, in a video search scenario in the live broadcast field, the model training request can be a training request for an unsupervised image feature extraction model.

[0172] Step S904: Obtain target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments.

[0173] The target data in the above steps can be audio data or video data. In the embodiment of the present application, the target data is video data as an example for explanation. The video data can be a certain amount of video data collected from a large amount of accumulated video data, which serves as the original data for model training. The video data is composed of video frames of a certain length. Therefore, the video data can be divided into multiple video segments (i.e., the above-mentioned data segments) according to a pre-set time interval, wherein the pre-set time interval can be 10s, but is not limited to this, and can be set according to actual needs to ensure that the video frames in the same video segment are similar.

[0174] The processing model in the above steps can be a model for extracting image features, and the specific structure of the model can adopt an existing structure. For example, the processing model can be composed of an Image Encoder and a Projection head, wherein the Image Encoder can use a structure such as ResNet (Residual Network) to extract the feature vector of the image, and the Projection head can use MLP to map the input original feature vector to a lower dimension.

[0175] Step S906 : Process the target data to obtain multiple training samples, where each training sample includes: a first sample, a second sample, and a third sample. The first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments.

[0176] Each training sample in the above steps can be composed of a triplet, namely a first sample, a second sample and a third sample, wherein the first sample is a video frame randomly extracted from the video data, the second sample is a positive sample of the first sample in the video data, and the third sample is a negative sample of the first sample in the video data.

[0177] Step S908: train the initial model using multiple training samples to obtain a processed model.

[0178] Optionally, the initial model can be trained using multiple training samples based on the mini-batch SGD method to obtain a processing model. In each iteration, a batch-size number of training samples can be randomly selected for training, but is not limited to this.

[0179] Step S910: Output the processing model by calling the second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the processing model.

[0180] The second interface in the above steps may be an interface for data interaction between the server and the user terminal. The server may pass the processing model into the interface function as a parameter of the interface function to achieve the purpose of sending the processing model to the client.

[0181] In the above embodiment of the present application, processing the target data to obtain multiple training samples includes: dividing the first video data to obtain multiple video segments; extracting a preset number of target video frames from each video segment; and determining multiple training samples based on the target video frames.

[0182] In the above embodiment of the present application, determining multiple training samples based on the target video frame includes: processing the target video frame according to a preset transformation mode to obtain multiple training samples, wherein the preset transformation mode includes at least one of the following: cropping operation, scaling operation, flipping operation, and color transformation operation.

[0183] In the above embodiment of the present application, training the initial model using multiple training samples includes: inputting each training sample into the image encoding network of the initial model to obtain the first eigenvector of each training sample; inputting the first eigenvector of each training sample into the mapping head network of the initial model to obtain the second eigenvector of each training sample; based on the second eigenvector of each training sample, obtaining the loss value of each training sample; when the loss value of each training sample does not meet the training conditions, updating the network parameters of the initial model based on multiple training samples until the loss value of each training sample meets the training conditions.

[0184] In the above embodiment of the present application, the second feature vector of each training sample includes: the first vector of the first sample, the second vector of the second sample, and the third vector of the third sample, wherein, based on the second feature vector of each training sample, obtaining the loss value of each training sample includes: obtaining a first similarity based on the first vector and the second vector; obtaining a second similarity based on the first vector and the third vector; inputting the first similarity and the second similarity into a preset loss function to obtain the loss value of each training sample.

[0185] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0186] Example 6

[0187] According to an embodiment of the present application, a model training device for implementing the above-mentioned model training method is also provided. Figure 10 As shown, the device 1000 includes: a receiving module 1002 , an acquisition module 1004 , a processing module 1006 , a training module 1008 and an output module 1010 .

[0188] Among them, the receiving module 1002 is used to receive a model training request; the acquisition module 1004 is used to obtain the target data and the initial model corresponding to the model training request, wherein the target data includes multiple data segments; the processing module 1006 is used to process the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; the training module 1008 is used to train the initial model using multiple training samples to obtain a processed model; the output module 1010 is used to output the processed model.

[0189] It should be noted that the receiving module 1002, acquiring module 1004, processing module 1006, training module 1008, and output module 1010 described above correspond to steps S302 to S310 in Example 1. The examples and application scenarios implemented by the five modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0190] In the above embodiment of the present application, the processing module includes: a division unit, used to divide the first video data to obtain multiple video segments; a first extraction unit, used to extract a preset number of target video frames from each video segment; and a first determination unit, used to determine multiple training samples based on the target video frames.

[0191] In the above embodiment of the present application, the first determination unit is further used to process the target video frame according to a preset transformation mode to obtain multiple training samples, wherein the preset transformation mode includes at least one of the following: cropping operation, scaling operation, flipping operation, and color transformation operation.

[0192] In the above embodiment of the present application, the training module includes: a first input unit, used to input each training sample into the image encoding network of the initial model to obtain the first eigenvector of each training sample; a second input unit, used to input the first eigenvector of each training sample into the mapping head network of the initial model to obtain the second eigenvector of each training sample; a first processing unit, used to obtain the loss value of each training sample based on the second eigenvector of each training sample; an updating unit, used to update the network parameters of the initial model based on multiple training samples when the loss value of each training sample does not meet the training conditions, until the loss value of each training sample meets the training conditions.

[0193] In the above embodiment of the present application, the second feature vector of each training sample includes: the first vector of the first sample, the second vector of the second sample and the third vector of the third sample, wherein the processing unit includes: a first processing subunit, for obtaining a first similarity based on the first vector and the second vector; a second processing subunit, for obtaining a second similarity based on the first vector and the third vector; and an input subunit, for inputting the first similarity and the second similarity into a preset loss function to obtain a loss value for each training sample.

[0194] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0195] Example 7

[0196] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 11 As shown, the device 1100 includes: an acquisition module 1102 , a processing module 1104 and a training module 1106 .

[0197] Among them, the acquisition module 1102 is used to acquire target data, wherein the target data includes multiple data segments; the processing module 1104 is used to process the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; the training module 1106 is used to use multiple training samples to train the initial model to obtain a processed model.

[0198] It should be noted that the acquisition module 1102, processing module 1104, and training module 1106 correspond to steps S602 to S606 in Example 2. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0199] In the above embodiment of the present application, the processing module includes: a division unit, used to divide the first video data to obtain multiple video segments; a first extraction unit, used to extract a preset number of target video frames from each video segment; and a first determination unit, used to determine multiple training samples based on the target video frames.

[0200] In the above embodiment of the present application, the first determination unit is further used to process the target video frame according to a preset transformation mode to obtain multiple training samples, wherein the preset transformation mode includes at least one of the following: cropping operation, scaling operation, flipping operation, and color transformation operation.

[0201] In the above embodiment of the present application, the training module includes: a first input unit, used to input each training sample into the image encoding network of the initial model to obtain the first eigenvector of each training sample; a second input unit, used to input the first eigenvector of each training sample into the mapping head network of the initial model to obtain the second eigenvector of each training sample; a first processing unit, used to obtain the loss value of each training sample based on the second eigenvector of each training sample; an updating unit, used to update the network parameters of the initial model based on multiple training samples when the loss value of each training sample does not meet the training conditions, until the loss value of each training sample meets the training conditions.

[0202] In the above embodiment of the present application, the second feature vector of each training sample includes: the first vector of the first sample, the second vector of the second sample and the third vector of the third sample, wherein the processing unit includes: a first processing subunit, for obtaining a first similarity based on the first vector and the second vector; a second processing subunit, for obtaining a second similarity based on the first vector and the third vector; and an input subunit, for inputting the first similarity and the second similarity into a preset loss function to obtain a loss value for each training sample.

[0203] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0204] Example 8

[0205] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 12 As shown, the device 1200 includes: a receiving module 1202 , a processing module 1204 , a determining module 1206 and an output module 1208 .

[0206] Among them, the receiving module 1202 is used to receive the first data; the processing module 1204 is used to process the first data using the processing model to obtain the first eigenvector of the first data, wherein the processing model is a model obtained by training the initial model using multiple training samples obtained by processing the second data, and the second data includes multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; the determination module 1206 is used to determine the target data among multiple third data based on the first eigenvector of the first data and the second eigenvectors of multiple third data; the output module 1208 is used to output the target data.

[0207] It should be noted that the receiving module 1202, processing module 1204, determination module 1206, and output module 1208 correspond to steps S702 to S708 in Example 3. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0208] In the above embodiment of the present application, the processing module includes: a first extraction unit, used to extract multiple first video frames from the first video data; a first processing unit, used to process each first video frame using the image coding network in the processing model to obtain a third feature vector of each first video frame; a first determination unit, used to determine the set of third feature vectors of multiple first video frames as the first feature vector of the first video data.

[0209] In the above embodiment of the present application, the determination module includes: an acquisition unit, used to obtain the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtain multiple similarities; a second determination unit, used to determine that the third video data corresponding to the similarity greater than a preset value among the multiple similarities is the target video data.

[0210] In the above embodiment of the present application, the acquisition unit includes: a first acquisition subunit, used to obtain the average vector of all feature vectors contained in the first feature vector of the first video data, to obtain a first average vector; a second acquisition subunit, used to obtain the average vector of all feature vectors contained in the second feature vector of each third video data, to obtain multiple second average vectors; a third acquisition subunit, used to obtain the distance between the first average vector and each second average vector, to obtain multiple similarities.

[0211] In the above embodiment of the present application, the acquisition unit includes: a first processing subunit, used to use the local feature aggregation descriptor algorithm to process the first feature vector of the first video data and the second feature vector of each third video data respectively, to obtain the first aggregation vector of the first video data and the second aggregation vector of each third video data; a fourth acquisition subunit, used to obtain the similarity between the first aggregation vector of the first video data and the second aggregation vector of each third video data, to obtain multiple similarities.

[0212] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 3, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 3.

[0213] Example 9

[0214] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 13 As shown, the device 1300 includes: an acquisition module 1302 , a processing module 1304 and a determination module 1306 .

[0215] Among them, the acquisition module 1302 is used to acquire the first data; the processing module 1304 is used to process the first data using the processing model to obtain the first eigenvector of the first data, wherein the processing model is a model obtained by training the initial model using multiple training samples obtained by processing the second data, the second data includes multiple video clips, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; the determination module 1306 is used to determine the target data among multiple third data based on the first eigenvector of the first data and the second eigenvectors of multiple third data.

[0216] It should be noted that the acquisition module 1302, processing module 1304, and determination module 1306 correspond to steps S802 to S806 in Example 4. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0217] In the above embodiment of the present application, the processing module includes: a first extraction unit, used to extract multiple first video frames from the first video data; a first processing unit, used to process each first video frame using the image coding network in the processing model to obtain a third feature vector of each first video frame; a first determination unit, used to determine the set of third feature vectors of multiple first video frames as the first feature vector of the first video data.

[0218] In the above embodiment of the present application, the determination module includes: an acquisition unit, used to obtain the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtain multiple similarities; a second determination unit, used to determine that the third video data corresponding to the similarity greater than a preset value among the multiple similarities is the target video data.

[0219] In the above embodiment of the present application, the acquisition unit includes: a first acquisition subunit, used to obtain the average vector of all feature vectors contained in the first feature vector of the first video data, to obtain a first average vector; a second acquisition subunit, used to obtain the average vector of all feature vectors contained in the second feature vector of each third video data, to obtain multiple second average vectors; a third acquisition subunit, used to obtain the distance between the first average vector and each second average vector, to obtain multiple similarities.

[0220] In the above embodiment of the present application, the acquisition unit includes: a first processing subunit, used to use the local feature aggregation descriptor algorithm to process the first feature vector of the first video data and the second feature vector of each third video data respectively, to obtain the first aggregation vector of the first video data and the second aggregation vector of each third video data; a fourth acquisition subunit, used to obtain the similarity between the first aggregation vector of the first video data and the second aggregation vector of each third video data, to obtain multiple similarities.

[0221] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 3, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 3.

[0222] Example 10

[0223] According to an embodiment of the present application, a model training device for implementing the above-mentioned model training method is also provided. Figure 14 As shown, the device 1400 includes: a first acquisition module 1402 , a second acquisition module 1404 , a processing module 1406 , a training module 1408 and an output module 1410 .

[0224] Among them, the first acquisition module 1402 is used to obtain a model training request by calling a first interface, wherein the first interface includes: a first parameter, and the parameter value of the first parameter is the model training request; the second acquisition module 1404 is used to obtain the target data and initial model corresponding to the model training request, wherein the target data includes multiple data segments; the processing module 1406 is used to process the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; the training module 1408 is used to train the initial model using multiple training samples to obtain a processing model; the output module 1410 is used to output the processing model by calling the second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the processing model.

[0225] It should be noted that the first acquisition module 1402, the second acquisition module 1404, the processing module 1406, the training module 1408, and the output module 1410 described above correspond to steps S902 to S910 in Example 5. The examples and application scenarios implemented by the five modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0226] In the above embodiment of the present application, the processing module includes: a division unit, used to divide the first video data to obtain multiple video segments; a first extraction unit, used to extract a preset number of target video frames from each video segment; and a first determination unit, used to determine multiple training samples based on the target video frames.

[0227] In the above embodiment of the present application, the first determination unit is further used to process the target video frame according to a preset transformation mode to obtain multiple training samples, wherein the preset transformation mode includes at least one of the following: cropping operation, scaling operation, flipping operation, and color transformation operation.

[0228] In the above embodiment of the present application, the training module includes: a first input unit, used to input each training sample into the image encoding network of the initial model to obtain the first eigenvector of each training sample; a second input unit, used to input the first eigenvector of each training sample into the mapping head network of the initial model to obtain the second eigenvector of each training sample; a first processing unit, used to obtain the loss value of each training sample based on the second eigenvector of each training sample; an updating unit, used to update the network parameters of the initial model based on multiple training samples when the loss value of each training sample does not meet the training conditions, until the loss value of each training sample meets the training conditions.

[0229] In the above embodiment of the present application, the second feature vector of each training sample includes: the first vector of the first sample, the second vector of the second sample and the third vector of the third sample, wherein the processing unit includes: a first processing subunit, for obtaining a first similarity based on the first vector and the second vector; a second processing subunit, for obtaining a second similarity based on the first vector and the third vector; and an input subunit, for inputting the first similarity and the second similarity into a preset loss function to obtain a loss value for each training sample.

[0230] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0231] Example 11

[0232] According to an embodiment of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0233] Figure 15 : is a flow chart of the third data processing method according to an embodiment of the present application. Figure 15 As shown, the method may include the following steps:

[0234] Step S1502: Acquire first data by calling a first interface, wherein the first interface includes: a first parameter, and a parameter value of the first parameter is the first data.

[0235] The first data in the above steps may be audio data or video data. In the embodiment of the present application, the first data is taken as the first video data as an example for explanation. The first video data may be a video provided by the user when performing a video search.

[0236] Step S1504: Use the processing model to process the first data to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are in the same data segment, and the first sample and the third sample are in different data segments.

[0237] The processing model in the above steps can be a model for extracting image features, and the specific structure of the model can adopt an existing structure. For example, the processing model can be composed of an Image Encoder and a Projection head, wherein the Image Encoder can use a structure such as ResNet (Residual Network) to extract the feature vector of the image, and the Projection head can use MLP to map the input original feature vector to a lower dimension.

[0238] The second data in the above steps can be audio data or video data. In the embodiment of the present application, the second data is taken as the second video data as an example for explanation. The second video data can be a certain amount of video data collected from a large amount of accumulated video data, which serves as the original data for model training. The second video data is composed of video frames of a certain length. Therefore, the second video data can be divided into multiple video segments (i.e., the above-mentioned data segments) according to a preset time interval, wherein the preset time interval can be 10s, but is not limited to this, and can be set according to actual needs to ensure that the video frames in the same video segment are similar.

[0239] Each training sample in the above steps can be composed of a triplet, namely a first sample, a second sample and a third sample, wherein the first sample is a video frame randomly extracted from the second video data, the second sample is a positive sample of the first sample in the second video data, and the third sample is a negative sample of the first sample in the second video data.

[0240] Step S1506 : determining target data from the plurality of third data based on the first eigenvector of the first data and the second eigenvectors of the plurality of third data.

[0241] The third data in the above steps can be audio data or video data. In the embodiment of the present application, the third data is described as third video data. The plurality of third video data can be all live videos stored in a database in the live broadcast field. The target data in the above steps can be audio data or video data. In the embodiment of the present application, the target data is described as target video data. The target video data can be a similar live video searched from the database. Therefore, the first video data is similar to the target video data.

[0242] Step S1508: Output the target data by calling the second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the target data.

[0243] In the above embodiment of the present application, the first data is processed using a processing model to obtain a first feature vector of the first data, including: extracting multiple first video frames from the first video data; processing each first video frame using the image coding network in the processing model to obtain a third feature vector of each first video frame; and determining a set of the third feature vectors of multiple first video frames as the first feature vector of the first video data.

[0244] In the above embodiment of the present application, based on the first eigenvector of the first data and the second eigenvector of multiple third data, determining the target data among the multiple third data includes: obtaining the similarity between the first eigenvector of the first video data and the second eigenvector of each third video data to obtain multiple similarities; determining the third video data corresponding to the similarity greater than a preset value among the multiple similarities as the target video data.

[0245] In the above embodiment of the present application, obtaining the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtaining multiple similarities includes: obtaining the average vector of all feature vectors contained in the first feature vector of the first video data to obtain a first average vector; obtaining the average vector of all feature vectors contained in the second feature vector of each third video data to obtain multiple second average vectors; obtaining the distance between the first average vector and each second average vector to obtain multiple similarities.

[0246] In the above embodiment of the present application, obtaining the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtaining multiple similarities includes: using the local feature aggregation descriptor algorithm to process the first feature vector of the first video data and the second feature vector of each third video data respectively, to obtain the first aggregation vector of the first video data and the second aggregation vector of each third video data; obtaining the similarity between the first aggregation vector of the first video data and the second aggregation vector of each third video data, to obtain multiple similarities.

[0247] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 3, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 3.

[0248] Example 12

[0249] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 16 As shown, the device 1600 includes: an acquisition module 1602 , a processing module 1604 , a determination module 1606 and an output module 1608 .

[0250] Among them, the acquisition module 1602 is used to obtain the first data by calling the first interface, wherein the first interface includes: a first parameter, and the parameter value of the first parameter is the first data; the processing module 1604 is used to process the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training the initial model using multiple training samples obtained by processing the second data, and the second data contains multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; the determination module 1606 is used to determine the target data among multiple third data based on the first eigenvector of the first data and the second eigenvectors of multiple third data; the output module 1608 is used to output the target data by calling the second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the target data.

[0251] It should be noted that the acquisition module 1602, processing module 1604, determination module 1606, and output module 1608 correspond to steps 1502 to 1508 in Example 11. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0252] In the above embodiment of the present application, the processing module includes: a first extraction unit, used to extract multiple first video frames from the first video data; a first processing unit, used to process each first video frame using the image coding network in the processing model to obtain a third feature vector of each first video frame; a first determination unit, used to determine the set of third feature vectors of multiple first video frames as the first feature vector of the first video data.

[0253] In the above embodiment of the present application, the determination module includes: an acquisition unit, used to obtain the similarity between the first feature vector of the first video data and the second feature vector of each third video data, and obtain multiple similarities; a second determination unit, used to determine that the third video data corresponding to the similarity greater than a preset value among the multiple similarities is the target video data.

[0254] In the above embodiment of the present application, the acquisition unit includes: a first acquisition subunit, used to obtain the average vector of all feature vectors contained in the first feature vector of the first video data, to obtain a first average vector; a second acquisition subunit, used to obtain the average vector of all feature vectors contained in the second feature vector of each third video data, to obtain multiple second average vectors; a third acquisition subunit, used to obtain the distance between the first average vector and each second average vector, to obtain multiple similarities.

[0255] In the above embodiment of the present application, the acquisition unit includes: a first processing subunit, used to use the local feature aggregation descriptor algorithm to process the first feature vector of the first video data and the second feature vector of each third video data respectively, to obtain the first aggregation vector of the first video data and the second aggregation vector of each third video data; a fourth acquisition subunit, used to obtain the similarity between the first aggregation vector of the first video data and the second aggregation vector of each third video data, to obtain multiple similarities.

[0256] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 3, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 3.

[0257] Example 13

[0258] According to an embodiment of the present application, a video processing system is further provided, including:

[0259] processor; and

[0260] The memory is connected to the processor and is used to provide the processor with instructions for processing the following processing steps: receiving a model training request; obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; using the multiple training samples to train the initial model to obtain a processing model; and outputting the processing model.

[0261] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0262] Example 14

[0263] According to an embodiment of the present application, a video processing system is further provided, including:

[0264] processor; and

[0265] A memory is connected to the processor and is used to provide the processor with instructions for processing the following processing steps: receiving first data; processing the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first eigenvector of the first data and the second eigenvectors of the multiple third data, determining target data among the multiple third data; and outputting the target data.

[0266] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 3, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 3.

[0267] Example 15

[0268] The embodiment of the present application can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0269] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0270] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the model training method: receiving a model training request; obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; using multiple training samples to train the initial model to obtain a processed model; and outputting the processed model.

[0271] Optionally, Figure 17 This is a structural block diagram of a computer terminal according to an embodiment of the present application. Figure 17 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1702 and a memory 1704.

[0272] Among them, the memory can be used to store software programs and modules, such as the model training method and device in the embodiment of the present application, and the program instructions / modules corresponding to the data processing method and device. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned model training method and data processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0273] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receive a model training request; obtain the target data and initial model corresponding to the model training request, wherein the target data includes multiple data segments; process the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; use multiple training samples to train the initial model to obtain a processed model; and output the processed model.

[0274] Optionally, the processor may further execute program code of the following steps: dividing the video data to obtain multiple video segments; extracting a preset number of target video frames from each video segment; and determining multiple training samples based on the target video frames.

[0275] Optionally, the processor may further execute the program code of the following steps: processing the target video frame according to a preset transformation mode to obtain multiple training samples, wherein the preset transformation mode includes at least one of the following: cropping operation, scaling operation, flipping operation, and color transformation operation.

[0276] Optionally, the processor may also execute the following steps of program code: input each training sample into the image encoding network of the initial model to obtain the first eigenvector of each training sample; input the first eigenvector of each training sample into the mapping head network of the initial model to obtain the second eigenvector of each training sample; based on the second eigenvector of each training sample, obtain the loss value of each training sample; if the loss value of each training sample does not meet the training conditions, update the network parameters of the initial model based on multiple training samples until the loss value of each training sample meets the training conditions

[0277] Optionally, the second feature vector of each training sample includes: the first vector of the first sample, the second vector of the second sample, and the third vector of the third sample. The above-mentioned processor can also execute the program code of the following steps: obtain a first similarity based on the first vector and the second vector; obtain a second similarity based on the first vector and the third vector; input the first similarity and the second similarity into a preset loss function to obtain the loss value of each training sample.

[0278] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain target data, where the target data includes multiple data segments; process the target data to obtain multiple training samples, where each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; use multiple training samples to train the initial model to obtain a processing model.

[0279] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receive first data; use the processing model to process the first data to obtain the first eigenvector of the first data, wherein the processing model is a model obtained by training the initial model using multiple training samples obtained by processing the second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first eigenvector of the first data and the second eigenvectors of the multiple third data, determine the target data among the multiple third data; output the target data.

[0280] Optionally, the processor may also execute the program code of the following steps: extracting multiple first video frames from the first video data; processing each first video frame using the image coding network in the processing model to obtain a third eigenvector of each first video frame; and determining a set of third eigenvectors of multiple first video frames as the first eigenvector of the first video data.

[0281] Optionally, the processor may also execute the program code of the following steps: obtaining the similarity between the first feature vector of the first video data and the second feature vector of each third video data to obtain multiple similarities; and determining that the third video data corresponding to the similarity greater than a preset value among the multiple similarities is the target video data.

[0282] Optionally, the processor may also execute the program code of the following steps: obtaining the average vector of all feature vectors contained in the first feature vector of the first video data to obtain a first average vector; obtaining the average vector of all feature vectors contained in the second feature vector of each third video data to obtain multiple second average vectors; obtaining the distance between the first average vector and each second average vector to obtain multiple similarities.

[0283] Optionally, the processor may also execute the program code of the following steps: using the local feature aggregation descriptor algorithm to process the first feature vector of the first video data and the second feature vector of each third video data respectively to obtain the first aggregation vector of the first video data and the second aggregation vector of each third video data; obtaining the similarity between the first aggregation vector of the first video data and the second aggregation vector of each third video data to obtain multiple similarities.

[0284] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain first data; use the processing model to process the first data to obtain the first feature vector of the first data, wherein the processing model is a model obtained by training the initial model using multiple training samples obtained by processing the second data, the second data contains multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first feature vector of the first data and the second feature vectors of the multiple third data, determine the target data among the multiple third data.

[0285] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a model training request by calling the first interface, wherein the first interface includes: a first parameter, and the parameter value of the first parameter is the model training request; obtain the target data and the initial model corresponding to the model training request, wherein the target data includes multiple data segments; process the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; use the multiple training samples to train the initial model to obtain a processing model; output the processing model by calling the second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the processing model.

[0286] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain the first data by calling the first interface, wherein the first interface includes: a first parameter, and the parameter value of the first parameter is the first data; use the processing model to process the first data to obtain the first feature vector of the first data, wherein the processing model is a model obtained by training the initial model using multiple training samples obtained by processing the second data, and the second data contains multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first feature vector of the first data and the second feature vectors of the multiple third data, determine the target data among the multiple third data; output the target data by calling the second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the target data.

[0287] By adopting the embodiment of the present application, a solution for model training and data processing is provided. By extracting video frames from video data as the first sample, the second sample and the third sample in the training sample, the difference between the first sample and the third sample comes from the natural change of the video content, thereby achieving the goal of fully utilizing the relevance of the video context for the video content, constructing training samples and training the model. Compared with the manually designed image transformation operation, the technical effect of improving the generalization ability of the self-supervised feature is achieved, thereby solving the technical problem in the related art that the training sample is transformed through the manually designed transformation mode, which makes the training process prone to overfitting the image itself and leads to poor generalization ability.

[0288] It can be understood by those skilled in the art that Figure 17 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 17 It does not limit the structure of the above electronic device. For example, the computer terminal A may also include Figure 17 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 17 Different configurations shown.

[0289] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0290] Example 16

[0291] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the model training method and data processing method provided in the above embodiment.

[0292] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0293] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving a model training request; obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; training the initial model using multiple training samples to obtain a processed model; and outputting the processed model.

[0294] Optionally, the storage medium is further configured to store program codes for executing the following steps: dividing the video data to obtain multiple video segments; extracting a preset number of target video frames from each video segment; and determining multiple training samples based on the target video frames.

[0295] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: processing the target video frame according to a preset transformation mode to obtain multiple training samples, wherein the preset transformation mode includes at least one of the following: cropping operation, scaling operation, flipping operation, and color transformation operation.

[0296] Optionally, the storage medium is further configured to store program codes for executing the following steps: inputting each training sample into the image encoding network of the initial model to obtain the first eigenvector of each training sample; inputting the first eigenvector of each training sample into the mapping head network of the initial model to obtain the second eigenvector of each training sample; obtaining the loss value of each training sample based on the second eigenvector of each training sample; and updating the network parameters of the initial model based on multiple training samples until the loss value of each training sample meets the training condition when the loss value of each training sample does not meet the training condition.

[0297] Optionally, the second feature vector of each training sample includes: the first vector of the first sample, the second vector of the second sample, and the third vector of the third sample. The above-mentioned storage medium is also configured to store program code for performing the following steps: obtaining a first similarity based on the first vector and the second vector; obtaining a second similarity based on the first vector and the third vector; inputting the first similarity and the second similarity into a preset loss function to obtain a loss value for each training sample.

[0298] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining target data, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; using multiple training samples to train the initial model to obtain a processed model.

[0299] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving first data; processing the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first eigenvector of the first data and the second eigenvectors of the multiple third data, determining the target data among the multiple third data; and outputting the target data.

[0300] Optionally, the above-mentioned storage medium is also configured to store program code for performing the following steps: extracting multiple first video frames from the first video data; processing each first video frame using the image coding network in the processing model to obtain a third eigenvector of each first video frame; and determining a set of third eigenvectors of multiple first video frames as the first eigenvector of the first video data.

[0301] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: obtaining the similarity between the first feature vector of the first video data and the second feature vector of each third video data to obtain multiple similarities; determining that the third video data corresponding to the similarity greater than a preset value among the multiple similarities is the target video data.

[0302] Optionally, the above-mentioned storage medium is also configured to store program code for performing the following steps: obtaining the average vector of all feature vectors contained in the first feature vector of the first video data to obtain a first average vector; obtaining the average vector of all feature vectors contained in the second feature vector of each third video data to obtain multiple second average vectors; obtaining the distance between the first average vector and each second average vector to obtain multiple similarities.

[0303] Optionally, the storage medium is also configured to store program code for executing the following steps: using a local feature aggregation descriptor algorithm to process the first feature vector of the first video data and the second feature vector of each third video data respectively to obtain a first aggregation vector of the first video data and a second aggregation vector of each third video data; obtaining the similarity between the first aggregation vector of the first video data and the second aggregation vector of each third video data to obtain multiple similarities.

[0304] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining first data; processing the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; based on the first eigenvector of the first data and the second eigenvectors of the multiple third data, determine the target data among the multiple third data.

[0305] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a model training request by calling a first interface, wherein the first interface includes: a first parameter, and the parameter value of the first parameter is the model training request; obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; training the initial model using multiple training samples to obtain a processing model; outputting the processing model by calling a second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the processing model.

[0306] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining first data by calling a first interface, wherein the first interface includes: a first parameter, and the parameter value of the first parameter is the first data; processing the first data using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, and the second data contains multiple data segments, each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment, and the first sample and the third sample are located in different data segments; determining target data among multiple third data based on the first eigenvector of the first data and the second eigenvectors of multiple third data; outputting the target data by calling a second interface, wherein the second interface includes: a second parameter, and the parameter value of the second parameter is the target data.

[0307] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0308] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0309] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0310] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0311] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0312] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0313] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A model training method, comprising: Receive model training requests; Obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; Processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment of the target data, the first sample and the third sample are located in different data segments of the target data, the second sample is a positive sample of the first sample in each training sample, the third sample is a negative sample of the first sample in each training sample, the first sample, the second sample, and the third sample in each training sample are all data frames extracted from the target data, and the difference between the first sample and the third sample comes from natural variation in data content of the target data; Training the initial model using the multiple training samples to obtain a processing model; outputting the processing model; The step of training the initial model using the plurality of training samples to obtain the processing model comprises: using the image encoding network and the mapping head network of the initial model, based on each training sample, obtaining a loss value for each training sample, wherein the loss value is obtained using a loss function of contrastive learning; and updating the network parameters of the initial model based on the loss value and the plurality of training samples to obtain the processing model. Among them, in the video search scenario in the live broadcast field, the model training request is a training request for an unsupervised image feature extraction model.

2. The method according to claim 1, wherein The target data is video data, wherein processing the target data to obtain the plurality of training samples comprises: Dividing the video data to obtain multiple video segments; Extracting a preset number of target video frames from each video clip; The plurality of training samples are determined based on the target video frame.

3. The method according to claim 2, wherein: Determining the plurality of training samples based on the target video frame includes: The target video frame is processed according to a preset transformation mode to obtain the multiple training samples, wherein the preset transformation mode includes at least one of the following: a cropping operation, a scaling operation, a flipping operation, and a color transformation operation.

4. The method according to claim 1, wherein Based on a small batch stochastic gradient descent method, the initial model is trained using the multiple training samples to obtain the processing model.

5. The method according to claim 4, wherein Obtaining a loss value for each training sample based on each training sample using the image encoding network and the mapping head network of the initial model; Updating the network parameters of the initial model based on the loss value and the multiple training samples to obtain the processing model includes: Inputting each training sample into the image coding network to obtain a first feature vector of each training sample; Inputting the first feature vector of each training sample into the mapping head network to obtain the second feature vector of each training sample; Obtaining a loss value of each training sample based on the second eigenvector of each training sample; When the loss value of each training sample does not meet the training condition, the network parameters of the initial model are updated based on the multiple training samples until the loss value of each training sample meets the training condition.

6. The method according to claim 5, wherein: The second feature vector of each training sample includes: a first vector of the first sample, a second vector of the second sample, and a third vector of the third sample, wherein obtaining the loss value of each training sample based on the second feature vector of each training sample includes: Obtaining a first similarity based on the first vector and the second vector; Obtaining a second similarity based on the first vector and the third vector; The first similarity and the second similarity are input into a preset loss function to obtain a loss value of each training sample.

7. The method according to claim 6, wherein: The preset loss function includes one of the following: a triplet loss function, a contrast loss function, and a noise contrast estimation loss function.

8. A model training method comprising: Acquiring target data, wherein the target data includes multiple data segments; Processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment of the target data, the first sample and the third sample are located in different data segments of the target data, the second sample is a positive sample of the first sample in each training sample, the third sample is a negative sample of the first sample in each training sample, the first sample, the second sample, and the third sample in each training sample are all data frames extracted from the target data, and the difference between the first sample and the third sample comes from natural variation in data content of the target data; Training the initial model using the multiple training samples to obtain a processing model; The step of training the initial model using the plurality of training samples to obtain the processing model comprises: using the image encoding network and the mapping head network of the initial model, based on each training sample, obtaining a loss value for each training sample, wherein the loss value is obtained using a loss function of contrastive learning; and updating the network parameters of the initial model based on the loss value and the plurality of training samples to obtain the processing model. In the video search scenario in the live broadcast field, the model training request is a training request for an unsupervised image feature extraction model.

9. The method according to claim 8, wherein The target data is video data, wherein processing the target data to obtain the plurality of training samples comprises: Dividing the video data to obtain multiple video segments; Extracting a preset number of target video frames from each video clip; The plurality of training samples are determined based on the target video frame.

10. A data processing method, comprising: receiving first data; Processing the first data using a processing model to obtain a first feature vector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment of the second data, the first sample and the third sample are located in different data segments of the second data, the difference between the first sample and the third sample comes from natural variation in data content of the second data, and the processing model is trained using the model training method according to any one of claims 1 to 7; determining target data among the plurality of third data based on a first feature vector of the first data and a second feature vector of a plurality of third data; The target data is output.

11. The method according to claim 10, wherein: The first data is first video data, and the second data is second video data. The first data is processed using the processing model to obtain a first feature vector of the first data, including: extracting a plurality of first video frames from the first video data; Processing each first video frame using the image coding network in the processing model to obtain a third eigenvector of each first video frame; A set of third feature vectors of the plurality of first video frames is determined as a first feature vector of the first video data.

12. The method according to claim 11, wherein The third data is third video data, and the target data is target video data, wherein, based on a first feature vector of the first data and second feature vectors of a plurality of third data, determining the target data among the plurality of third data comprises: Obtaining a similarity between a first feature vector of the first video data and a second feature vector of each third video data to obtain a plurality of similarities; The third video data corresponding to the similarity greater than a preset value among the multiple similarities is determined as the target video data.

13. The method according to claim 12, wherein: Obtaining a similarity between a first feature vector of the first video data and a second feature vector of each third video data, where the obtained multiple similarities include: Obtaining an average vector of all feature vectors included in the first feature vector of the first video data to obtain a first average vector; Obtaining an average vector of all feature vectors included in the second feature vector of each third video data to obtain a plurality of second average vectors; Obtain the distance between the first average vector and each second average vector to obtain the multiple similarities.

14. The method according to claim 12, wherein: Obtaining a similarity between a first feature vector of the first video data and a second feature vector of each third video data, where the obtained multiple similarities include: Using a local feature aggregation descriptor algorithm, the first feature vector of the first video data and the second feature vector of each of the third video data are processed to obtain a first aggregation vector of the first video data and a second aggregation vector of each of the third video data; The similarities between the first aggregation vector of the first video data and the second aggregation vector of each of the third video data are obtained to obtain the multiple similarities.

15. A data processing method, comprising: obtaining first data; Processing the first data using a processing model to obtain a first feature vector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment of the second data, the first sample and the third sample are located in different data segments of the second data, the difference between the first sample and the third sample comes from natural variation in data content of the second data, and the processing model is trained using the model training method according to any one of claims 1 to 7; Target data among the plurality of third data is determined based on a first eigenvector of the first data and a second eigenvector of a plurality of third data.

16. The method according to claim 15, wherein The first data is first video data, and the second data is second video data. The first data is processed using the processing model to obtain a first feature vector of the first data, including: extracting a plurality of first video frames from the first video data; Processing each first video frame using the image coding network in the processing model to obtain a third eigenvector of each first video frame; A set of third feature vectors of the plurality of first video frames is determined as a first feature vector of the first video data.

17. A model training method comprising: Obtaining a model training request by calling a first interface, wherein the first interface includes: a first parameter, a parameter value of the first parameter being the model training request; Obtaining target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; Processing the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment of the target data, the first sample and the third sample are located in different data segments of the target data, the second sample is a positive sample of the first sample in each training sample, the third sample is a negative sample of the first sample in each training sample, the first sample, the second sample, and the third sample in each training sample are all data frames extracted from the target data, and the difference between the first sample and the third sample comes from natural variation in data content of the target data; Training the initial model using the multiple training samples to obtain a processing model; Outputting the processing model by calling a second interface, wherein the second interface includes: a second parameter, a parameter value of the second parameter being the processing model; The step of training the initial model using the plurality of training samples to obtain the processing model comprises: using the image encoding network and the mapping head network of the initial model, based on each training sample, obtaining a loss value for each training sample, wherein the loss value is obtained using a loss function of contrastive learning; and updating the network parameters of the initial model based on the loss value and the plurality of training samples to obtain the processing model. Among them, in the video search scenario in the live broadcast field, the model training request is a training request for an unsupervised image feature extraction model.

18. A data processing method comprising: Acquire first data by calling a first interface, wherein the first interface includes: a first parameter, a parameter value of the first parameter being the first data; Processing the first data using a processing model to obtain a first feature vector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing second data, the second data includes multiple data segments, each training sample includes: a first sample, a second sample, and a third sample, the first sample and the second sample are located in the same data segment of the second data, the first sample and the third sample are located in different data segments of the second data, the difference between the first sample and the third sample comes from natural variation in data content of the second data, and the processing model is trained using the model training method according to any one of claims 1 to 7; determining target data among the plurality of third data based on a first feature vector of the first data and a second feature vector of a plurality of third data; The target data is output by calling a second interface, wherein the second interface includes: a second parameter, and a parameter value of the second parameter is the target data.

19. A computer-readable storage medium comprising a stored program, wherein: When the program is running, the device where the computer-readable storage medium is located is controlled to execute the model training method described in any one of claims 1 to 9 and 17, or the data processing method described in any one of claims 10 to 16 and 18.

20. A computer terminal comprising: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein when the program is run, the model training method described in any one of claims 1 to 9 and 17, or the data processing method described in any one of claims 10 to 16 and 18 is executed.

21. A model training system comprising: processor; as well as A memory, connected to the processor, is configured to provide the processor with instructions for processing the following processing steps: receiving a model training request; Obtain target data and an initial model corresponding to the model training request, wherein the target data includes multiple data segments; process the target data to obtain multiple training samples, wherein each training sample includes: a first sample, a second sample, and a third sample, wherein the first sample and the second sample are located in the same data segment of the target data, and the first sample and the third sample are located in different data segments of the target data, the second sample is a positive sample of the first sample in each training sample, and the third sample is a negative sample of the first sample in each training sample, the first sample, the second sample, and the third sample in each training sample are all data frames extracted from the target data, and the difference between the first sample and the third sample comes from the natural variation of the data content of the target data; use the multiple training samples to train the initial model to obtain a processed model; and output the processed model; The step of training the initial model using the plurality of training samples to obtain the processing model comprises: using the image encoding network and the mapping head network of the initial model, based on each training sample, obtaining a loss value for each training sample, wherein the loss value is obtained using a loss function of contrastive learning; and updating the network parameters of the initial model based on the loss value and the plurality of training samples to obtain the processing model. Among them, in the video search scenario in the live broadcast field, the model training request is a training request for an unsupervised image feature extraction model.

22. A data processing system comprising: processor; as well as A memory, connected to the processor, is configured to provide the processor with instructions for processing the following processing steps: receiving first data; The first data is processed by using a processing model to obtain a first eigenvector of the first data, wherein the processing model is a model obtained by training an initial model using multiple training samples obtained by processing the second data, the second data contains multiple data segments, and each training sample includes: a first sample, a second sample and a third sample, the first sample and the second sample are located in the same data segment of the second data, the first sample and the third sample are located in different data segments of the second data, and the difference between the first sample and the third sample comes from the natural change of the data content of the second data; based on the first eigenvector of the first data and the second eigenvectors of multiple third data, the target data among the multiple third data is determined; and the target data is output, and the processing model is trained using the model training method described in any one of claims 1 to 7.

Images