Method and apparatus for training a neural network model

Abstract

Embodiments of the present specification provide a method and device for training a neural network model. The method comprises: obtaining a first training sample set, wherein any first training sample comprises a first query video and a first reference video, and label positioning information of a video clip of infringement; training a first neural network model using the first training sample set; performing iterative updating of the neural network model, comprising: obtaining a second training sample set, wherein any second training sample comprises a second query video and a second reference video, and does not comprise label positioning information; inputting the second query video and the second reference video into the first neural network model to obtain predicted positioning information; updating the second training sample set according to the predicted positioning information, so that each second training sample has label positioning information; and training a second neural network model using the first training sample set and the updated second training sample set. The effect of the neural network model in video infringement detection can be improved.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of computers, and more particularly to methods and apparatus for training neural network models. Background Technology

[0002] Currently, rampant video piracy not only causes significant losses to video websites but also inflicts immeasurable economic and psychological damage on content creators. In recent years, infringement methods, including derivative works and video editing, have proliferated. Examples of video piracy are no longer limited to easily identifiable methods such as simple filming or adding watermarks. Therefore, video infringement detection for copyright protection has become particularly important. Detection systems need to be robust to various increasingly complex video transformations, including photometric transformations, geometric transformations, editing transformations, encoding formats, and time-series editing operations. Addressing these challenges, multimedia comparison algorithms based on artificial intelligence (AI) can significantly reduce the cost of manual review, improve the efficiency of infringement evidence collection, and achieve accurate identification in large-scale searches, making it an effective solution for video infringement detection.

[0003] With increased emphasis on privacy protection, videos may become the private data of their owners. Video infringement detection is divided into video-level and segment-level detection. Video-level infringement detection only needs to output detection information on whether two videos infringe on each other, while segment-level infringement detection requires outputting the location information of the corresponding infringing video segments from the two videos. This location information is used to indicate the temporal position of the infringing segments in the two videos.

[0004] In existing technologies, for infringement detection at the fragment level, it is usually impossible to obtain a sufficient amount of labeled data for training neural network models, which consequently affects the performance of neural network models in video infringement detection. Summary of the Invention

[0005] This specification describes one or more embodiments of a method and apparatus for training a neural network model, which can improve the performance of the neural network model in video infringement detection when the amount of labeled data is insufficient.

[0006] Firstly, a method for training a neural network model is provided, the method including:

[0007] Obtain a first training sample set, wherein any first training sample includes a first query video and a first reference video, as well as the tag location information of the video segment infringing on the first query video relative to the first reference video;

[0008] Using the first training sample set, a first neural network model is trained.

[0009] Perform iterative updates to the neural network model, the iterative updates including:

[0010] Obtain a second training sample set, wherein any second training sample includes a second query video and a second reference video, but does not include the tag positioning information;

[0011] The second query video and the second reference video are input into the first neural network model to obtain the predicted location information of the infringing video segment of the second query video relative to the second reference video in the second training sample;

[0012] The second training sample set is updated according to the predicted positioning information so that each second training sample in the updated second training sample set has label positioning information;

[0013] The second neural network model is trained using the first training sample set and the updated second training sample set.

[0014] In one possible implementation, updating the second training sample set based on the predicted positioning information includes:

[0015] If the confidence level corresponding to the predicted location information is higher than a preset threshold, then the predicted location information is used as the label location information of the corresponding second training sample.

[0016] If the confidence level corresponding to the predicted location information is less than or equal to the preset threshold, then the second training sample corresponding to the predicted location information is deleted from the second training sample set.

[0017] In one possible implementation, the second training sample set includes a first type of training sample and a second type of training sample; the first type of training sample includes tag detection information on whether the query video infringes on the reference video; the second type of training sample does not include the tag detection information.

[0018] Further, updating the second training sample set based on the predicted positioning information includes:

[0019] If the predicted location information corresponds to the first type of training sample and is consistent with the label detection information, then the predicted location information is used as the label location information of the corresponding second training sample.

[0020] If the predicted location information corresponds to the first type of training sample and is inconsistent with the label detection information, then the second training sample corresponding to the predicted location information is deleted from the second training sample set.

[0021] In one possible implementation, training a second neural network model using the first training sample set and the updated second training sample set includes:

[0022] Based on the first training sample set, determine the first prediction loss;

[0023] Based on the updated second training sample set, determine the second prediction loss;

[0024] The total loss is obtained by weighted summation of the first predicted loss and the second predicted loss;

[0025] The second neural network model is trained with the goal of minimizing the total loss.

[0026] In one possible implementation, the method includes an offline phase and an online phase; updating the second training sample set belongs to the offline phase, and training the second neural network model belongs to the online phase.

[0027] In one possible implementation, the method includes an online phase; updating the second training sample set and training the second neural network model both belong to the online phase.

[0028] Furthermore, after the iterative update, the method further includes:

[0029] The obtained second neural network model is used as the first neural network model in this round, and the iterative update steps are continued.

[0030] Secondly, an apparatus for training a neural network model is provided, the apparatus comprising:

[0031] The acquisition unit is used to acquire a first training sample set, wherein any first training sample includes a first query video and a first reference video, as well as the tag positioning information of the video segment infringing on the first query video relative to the first reference video.

[0032] A training unit is used to train a first neural network model using the first training sample set obtained by the acquisition unit.

[0033] An iterative unit, used to perform iterative updates of the neural network model, the iterative unit comprising:

[0034] A sub-unit is used to acquire a second training sample set, wherein any second training sample includes a second query video and a second reference video, but does not include the tag positioning information;

[0035] The prediction subunit is used to input the second query video and the second reference video obtained by the acquisition subunit into the first neural network model to obtain the predicted location information of the infringing video segment of the second query video relative to the second reference video in the second training sample.

[0036] An update subunit is used to update the second training sample set according to the predicted positioning information obtained by the prediction subunit, so that each second training sample in the updated second training sample set has label positioning information.

[0037] The training subunit is used to train a second neural network model using the first training sample set and the updated second training sample set obtained by the updating subunit.

[0038] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of the first aspect.

[0039] Fourthly, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method of the first aspect.

[0040] The method and apparatus provided in the embodiments of this specification utilize both first training samples with labeled location information and second training samples without labeled location information when training a neural network model. First, a first neural network model is trained using a first training sample set composed of the first training samples. Then, the neural network model is iteratively updated using a second training sample set composed of the second training samples. During the iterative update process, the predicted location information of the second query video relative to the infringing video segment of the second reference video is obtained using the first neural network model. The second training sample set is updated based on the predicted location information, so that each second training sample in the updated second training sample set has labeled location information. Finally, a second neural network model is trained using the first training sample set and the updated second training sample set. As can be seen from the above, the embodiments of this specification first use labeled training samples to initially train the first neural network model, and then use the trained first neural network model to update the second training sample set, so that each second training sample in the updated second training sample set has labeled location information. This increases the number of samples with labeled location information, improving the performance of the neural network model in video infringement detection when the amount of labeled data is insufficient. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification;

[0043] Figure 2 A flowchart illustrating a method for training a neural network model according to one embodiment is shown.

[0044] Figure 3 A schematic diagram showing a second neural network model obtained through training according to one embodiment is shown;

[0045] Figure 4 A schematic block diagram of an apparatus for training a neural network model according to one embodiment is shown. Detailed Implementation

[0046] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0047] Figure 1This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification. This implementation scenario involves training a neural network model for video infringement detection at the segment level. The model outputs location information for infringing video segments corresponding to two videos. This location information indicates the temporal position of the infringing segments in the two videos. This segment-level video infringement detection is also known as video infringement localization. Typically, the infringing video segment contains only a small portion of the original video and is embedded within the content of another video. Video infringement localization refers to finding the infringing time segment of the infringing video and the infringed time segment of the original video, effectively pointing to the infringing video segment in actual copyright protection operations, and improving the efficiency and reliability of the detection system.

[0048] Reference Figure 1 Training the aforementioned neural network model requires labeled training samples, specifically fragment-level infringement annotations. These labeled training samples typically include query videos, reference videos, and tag location information, for example... Figure 1 The tag location information in the video is t1, t2, t3, t4. It can be understood that F0, F1...Fn in the query video and F0, F1...Fm in the reference video represent video frames. Multiple consecutive video frames in a video constitute a video segment. Each video frame in the video has its own corresponding time. The corresponding video frame can be determined by the time, and the corresponding video segment can be determined by the time interval between two times. t1, t2, t3, t4 are used to represent the start and end times of the video segments of the two videos, respectively. t1 and t2 are used to indicate the infringing time segment of the query video, and t3 and t4 are used to indicate the infringed time segment of the reference video.

[0049] The aforementioned label location information is usually difficult to obtain in practice. Consequently, only a small number of labeled training samples can be obtained, and the training effect of the above neural network model using conventional supervised learning is not good.

[0050] In this embodiment, in addition to a small number of labeled training samples, unlabeled training samples and / or weakly labeled training samples are also used. That is, semi-supervised learning and / or weakly supervised learning are employed to train the neural network model. It is understood that neither the unlabeled nor the weakly labeled training samples contain label localization information. The unlabeled training samples include the query video and the reference video, while the weakly labeled training samples include the query video, the reference video, and label detection information. This labeling is a video-level infringement label. The label detection information indicates whether the query video infringes on the reference video. If the query video contains infringing segments relative to the reference video, the label detection information indicates infringement; otherwise, if the query video does not contain infringing segments relative to the reference video, the label detection information indicates no infringement.

[0051] Furthermore, the video infringement detection in the embodiments of this specification can have a broader meaning, such as detecting whether there are identical or similar video segments in two videos. A video segment can also be broadly understood as at least one video frame in a video, and video infringement detection can also be used to detect whether there are identical or similar video frames in two videos.

[0052] Figure 2 This diagram illustrates a method flowchart for training a neural network model according to one embodiment, which can be based on... Figure 1 The implementation scenario is shown. For example... Figure 2 As shown, the method for training a neural network model in this embodiment includes the following steps: Step 21, obtaining a first training sample set, wherein any first training sample includes a first query video and a first reference video, and tag positioning information of the video segment infringing on the first reference video relative to the first query video; Step 22, using the first training sample set, training a first neural network model; Step 23, performing iterative updates to the neural network model, wherein the iterative updates include: Step 231, obtaining a second training sample set, wherein any second training sample includes a second query video and a second reference video, excluding the tag positioning information; Step 232, inputting the second query video and the second reference video into the first neural network model to obtain the predicted positioning information of the video segment infringing on the second reference video relative to the second query video of the second training sample; Step 233, updating the second training sample set according to the predicted positioning information, so that each second training sample in the updated second training sample set has tag positioning information; Step 234, using the first training sample set and the updated second training sample set, training a second neural network model. The specific execution methods of the above steps are described below.

[0053] First, in step 21, a first training sample set is obtained, wherein any first training sample includes a first query video and a first reference video, as well as tag location information of the infringing video segment in the first query video relative to the first reference video. It is understood that this tag location information includes both the time information of the infringing video segment in the first query video and the time information of the infringed video segment in the first reference video.

[0054] In the embodiments described in this specification, the aforementioned infringing video clips and the infringed video clips satisfy a certain degree of similarity. Since video clips are composed of video frames, the video frames of the aforementioned infringing video clips and the video frames of the infringed video clips satisfy a certain degree of similarity.

[0055] The first query video and the first reference video can be characterized by the frame-level features of each of their respective video frames.

[0056] Among them, the tag location information can be obtained through manual annotation.

[0057] Then, in step 22, the first neural network model is trained using the first training sample set. It is understood that all training samples in the first training sample set are labeled training samples.

[0058] The embodiments in this specification can be trained using conventional supervised learning methods to obtain the first neural network model.

[0059] Finally, in step 23, an iterative update of the neural network model is performed, which includes:

[0060] Step 231: Obtain the second training sample set, wherein any second training sample includes the second query video and the second reference video, but does not include the tag positioning information. It is understood that the second training sample can be either the aforementioned unlabeled training sample or the aforementioned weakly labeled training sample.

[0061] In one example, the second training sample set includes a first type of training sample and a second type of training sample; the first type of training sample includes tag detection information on whether the query video infringes on the reference video; the second type of training sample does not include the tag detection information.

[0062] In this example, the first type of training sample is equivalent to the aforementioned weakly labeled training sample, and the second type of training sample is equivalent to the aforementioned unlabeled training sample.

[0063] Furthermore, the second training sample set may include only the first type of training samples, or only the second type of training samples.

[0064] The embodiments in this specification are applicable to scenarios involving continuous reflow of training data in actual online environments with unlabeled or weakly labeled training samples.

[0065] Step 232: Input the second query video and the second reference video into the first neural network model to obtain the predicted location information of the infringing video segment of the second query video relative to the second reference video in the second training sample. It is understood that the first neural network model is a trained model, and the predicted location information it obtains has a certain degree of accuracy.

[0066] In one example, the confidence level corresponding to the predicted location information can also be obtained through the first neural network model. The higher the confidence level, the more accurate the predicted location information.

[0067] Step 233: Update the second training sample set according to the predicted positioning information, so that each second training sample in the updated second training sample set has label positioning information. It is understood that the second training samples in the second training sample set initially do not have label positioning information, and the predicted positioning information can be used as their label positioning information.

[0068] In this embodiment of the specification, the predicted positioning information obtained in step 232 can be filtered in order to control the quality of the obtained tag positioning information.

[0069] In one example, updating the second training sample set based on the predicted location information includes:

[0070] If the confidence level corresponding to the predicted location information is higher than a preset threshold, then the predicted location information is used as the label location information of the corresponding second training sample.

[0071] If the confidence level corresponding to the predicted location information is less than or equal to the preset threshold, then the second training sample corresponding to the predicted location information is deleted from the second training sample set.

[0072] In this example, the quality of the obtained tag location information is controlled by setting a relatively high confidence threshold, for example, setting the confidence threshold to 0.9.

[0073] In one example, the second training sample set includes a first type of training sample and a second type of training sample; the first type of training sample includes tag detection information on whether the query video infringes on the reference video; the second type of training sample does not include the tag detection information.

[0074] Further, updating the second training sample set based on the predicted positioning information includes:

[0075] If the predicted location information corresponds to the first type of training sample and is consistent with the label detection information, then the predicted location information is used as the label location information of the corresponding second training sample.

[0076] If the predicted location information corresponds to the first type of training sample and is inconsistent with the label detection information, then the second training sample corresponding to the predicted location information is deleted from the second training sample set.

[0077] In this example, training samples with weak annotations containing video-level infringement information help filter out low-quality predicted location information, thus controlling the quality of the obtained tag location information. It is understood that the predicted location information can also indicate whether the query video infringes on the reference video.

[0078] Step 234: Using the first training sample set and the updated second training sample set, a second neural network model is trained. It is understood that each first training sample in the first training sample set has label localization information, and each second training sample in the updated second training sample set also has label localization information, but its accuracy is not as good as that of the first training samples in the first training sample set.

[0079] In one example, training a second neural network model using the first training sample set and the updated second training sample set includes:

[0080] Based on the first training sample set, determine the first prediction loss;

[0081] Based on the updated second training sample set, determine the second prediction loss;

[0082] The total loss is obtained by weighted summation of the first predicted loss and the second predicted loss;

[0083] The second neural network model is trained with the goal of minimizing the total loss.

[0084] In this example, the first training sample set is labeled data, and the second training sample set is pseudo-labeled data. By using labeled data and pseudo-labeled data, the neural network model is trained by jointly minimizing supervised, unsupervised, or weakly supervised loss. The losses of the two parts are weighted and summed by controlling the weights, which can achieve better training results.

[0085] Understandably, a second neural network model can be trained based on the first neural network model obtained through the aforementioned training. When training the second neural network model, the total training sample set can be divided into multiple batches, and the neural network model can be trained on each batch of training samples. Each batch of training samples may include both training samples from the first training sample set and training samples from the updated second training sample set.

[0086] Figure 3 This diagram illustrates a second neural network model obtained through training according to one embodiment. (Refer to...) Figure 3 When training the second neural network model, the total training sample set consists of two parts: the first training sample set and the updated second training sample set. These two parts of the training sample set are different when calculating the total loss. For example, the aforementioned total loss can be expressed as L. s +λ u L u , where L sL represents the aforementioned first prediction loss, which applies to the training samples in the first training sample set. u Representing the aforementioned second prediction loss, which applies to the training samples in the updated second training sample set, λ u The weight representing the second prediction loss mentioned above is usually less than 1, so that the first training sample set plays a stronger role in training.

[0087] In one example, the method includes an offline phase and an online phase; updating the second training sample set belongs to the offline phase, and training the second neural network model belongs to the online phase.

[0088] In one example, the method includes an online phase; updating the second training sample set and training the second neural network model both belong to the online phase.

[0089] Furthermore, after the iterative update, the method further includes:

[0090] The obtained second neural network model is used as the first neural network model in this round, and the iterative update steps are continued.

[0091] In the embodiments described in this specification, the processes of updating the second training sample set and training the second neural network model can be separate, and updating the second training sample set can be performed offline; alternatively, the processes of updating the second training sample set and training the second neural network model can be jointly completed online. Furthermore, the training from the first neural network model to the second neural network model can be iterated multiple times to achieve further data utilization efficiency.

[0092] It is understandable that the first neural network model plays a guiding role in the subsequent training to obtain the second neural network model. Therefore, the first neural network model can be called the teacher model, and the second neural network model can be called the student model.

[0093] The embodiments in this specification demonstrate how to obtain a better algorithm model by adding weakly supervised or unsupervised data to a teacher-based model. Regarding technical performance, two scenarios were tested, both yielding excellent results. The first scenario involved a single dataset, where a portion of the data was extracted as fully supervised labeled data, and the remainder as unlabeled or weakly labeled data. The second scenario involved cross-dataset training; after training on a fully supervised labeled dataset, other weakly labeled datasets were added to observe the algorithm's performance. It can be seen that compared to the model performance obtained with only labeled data, adding a large amount of weakly supervised or unsupervised data based on the embodiments in this specification significantly improves the model's training performance.

[0094] The method provided in the embodiments of this specification utilizes both first training samples with labeled location information and second training samples without labeled location information when training the neural network model. First, a first neural network model is trained using a first training sample set composed of the first training samples. Then, the neural network model is iteratively updated using a second training sample set composed of the second training samples. During the iterative update process, the predicted location information of the infringing video segment of the second query video relative to the second reference video is obtained using the first neural network model. The second training sample set is updated based on the predicted location information, so that each second training sample in the updated second training sample set has labeled location information. Finally, a second neural network model is trained using the first training sample set and the updated second training sample set. As can be seen from the above, the embodiments of this specification first use labeled training samples to initially train the first neural network model, and then use the trained first neural network model to update the second training sample set, so that each second training sample in the updated second training sample set has labeled location information. This increases the number of samples with labeled location information, improving the performance of the neural network model in video infringement detection when the amount of labeled data is insufficient.

[0095] According to another embodiment, an apparatus for training a neural network model is also provided, which is used to perform the methods provided in the embodiments of this specification. Figure 4 A schematic block diagram of an apparatus for training a neural network model according to one embodiment is shown. Figure 4 As shown, the device 400 includes:

[0096] The acquisition unit 41 is used to acquire a first training sample set, wherein any first training sample includes a first query video and a first reference video, as well as the tag positioning information of the video segment infringing on the first reference video relative to the first query video.

[0097] Training unit 42 is used to train a first neural network model using the first training sample set obtained by acquisition unit 41;

[0098] Iteration unit 43, used to perform iterative updates of the neural network model, the iteration unit 43 includes:

[0099] Acquisition subunit 431 is used to acquire a second training sample set, wherein any second training sample includes a second query video and a second reference video, but does not include the tag positioning information;

[0100] The prediction subunit 432 is used to input the second query video and the second reference video acquired by the acquisition subunit 431 into the first neural network model to obtain the predicted location information of the infringing video segment of the second query video relative to the second reference video in the second training sample.

[0101] The update subunit 433 is used to update the second training sample set according to the predicted positioning information obtained by the prediction subunit 432, so that each second training sample in the updated second training sample set has label positioning information.

[0102] The training subunit 434 is used to train a second neural network model using the first training sample set and the updated second training sample set obtained by the updating subunit 433.

[0103] Optionally, as an embodiment, the update subunit 433 includes:

[0104] The first label adding module is used to use the predicted positioning information as the label positioning information of the corresponding second training sample if the confidence level corresponding to the predicted positioning information is higher than a preset threshold.

[0105] The first sample deletion module is used to delete the second training sample corresponding to the predicted positioning information from the second training sample set if the confidence level corresponding to the predicted positioning information is less than or equal to the preset threshold.

[0106] Optionally, as an embodiment, the second training sample set includes a first type of training sample and a second type of training sample; the first type of training sample includes tag detection information on whether the query video infringes on the reference video; the second type of training sample does not include the tag detection information.

[0107] Furthermore, the update subunit 433 includes:

[0108] The second label adding module is used to use the predicted positioning information as the label positioning information of the corresponding second training sample if the predicted positioning information corresponds to the first type of training sample and is consistent with the label detection information.

[0109] The second sample deletion module is used to delete the second training sample corresponding to the predicted positioning information from the second training sample set if the predicted positioning information corresponds to the first type of training sample and is inconsistent with the label detection information.

[0110] Optionally, as an embodiment, the training subunit 434 includes:

[0111] The first determining module is used to determine a first prediction loss based on the first training sample set;

[0112] The second determining module is used to determine the second prediction loss based on the updated second training sample set;

[0113] The weighting module is used to obtain the total loss by weighted summation of the first prediction loss obtained by the first determining module and the second prediction loss obtained by the second determining module;

[0114] The training module is used to train the second neural network model with the goal of minimizing the total loss obtained by the weighting module.

[0115] Optionally, as an embodiment, the device includes a subunit for an offline phase and a subunit for an online phase; the update subunit 433 belongs to the subunit for the offline phase, and the training subunit 434 belongs to the subunit for the online phase.

[0116] Optionally, as an embodiment, the apparatus includes a subunit for the online phase; both the update subunit 433 and the training subunit 434 belong to the subunit for the online phase.

[0117] Furthermore, the device also includes:

[0118] The loop processing unit is used to take the obtained second neural network model as the first neural network model in this round after the iteration unit 43 performs the iterative update, and continue to have the iteration unit 43 perform the iterative update steps.

[0119] The apparatus provided in the embodiments of this specification utilizes both first training samples with tag positioning information and second training samples without tag positioning information when training the neural network model. First, the training unit 42 uses the first training sample set composed of the first training samples to train a first neural network model. Then, the iteration unit 43 uses the second training sample set composed of the second training samples to perform iterative updates of the neural network model. During the iterative update process, the prediction subunit 432 uses the first neural network model to obtain the predicted positioning information of the second query video relative to the infringing video segment of the second reference video. The update subunit 433 updates the second training sample set according to the predicted positioning information so that each second training sample in the updated second training sample set has tag positioning information. The training subunit 434 then uses the first training sample set and the updated second training sample set to train a second neural network model. As can be seen from the above, in the embodiments of this specification, the first neural network model is initially trained using labeled training samples, and then the second training sample set is updated using the trained first neural network model, so that each second training sample in the updated second training sample set has label positioning information, thereby increasing the number of samples with label positioning information. This can improve the effect of the neural network model in video infringement detection when the amount of labeled data is insufficient.

[0120] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 2 The method described.

[0121] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements a combination... Figure 2 The method described.

[0122] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0123] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training a neural network model, the method comprising: obtaining a first training sample set, wherein any first training sample comprises a first query video and a first reference video, and label location information of a video segment of the first query video that infringes on the first reference video; the label location information is used to indicate a time location of the video segment that infringes; training a first neural network model based on the first training sample set; performing iterative update of the neural network model, the iterative update comprising: obtaining a second training sample set, wherein any second training sample comprises a second query video and a second reference video, and does not comprise the label location information; inputting the second query video and the second reference video into the first neural network model to obtain predicted location information of a video segment of the second query video that infringes on the second reference video for the second training sample; updating the second training sample set based on the predicted location information, so that each second training sample in the updated second training sample set has label location information; training a second neural network model based on the first training sample set and the updated second training sample set; wherein the second training sample set comprises first type training samples; the first type training samples comprise label detection information of whether the second query video infringes on the second reference video; wherein the updating the second training sample set based on the predicted location information comprises: if the predicted location information corresponds to the first type training sample and is consistent with the label detection information, taking the predicted location information as label location information of the corresponding second training sample; if the predicted location information corresponds to the first type training sample and is inconsistent with the label detection information, deleting the second training sample corresponding to the predicted location information from the second training sample set.

2. The method of claim 1, wherein, wherein the updating the second training sample set based on the predicted location information comprises: if a confidence level corresponding to the predicted location information is higher than a preset threshold, taking the predicted location information as label location information of the corresponding second training sample; if the confidence level corresponding to the predicted location information is less than or equal to the preset threshold, deleting the second training sample corresponding to the predicted location information from the second training sample set.

3. The method of claim 1, wherein, the second training sample set further comprises second type training samples; the second type training samples do not comprise the label detection information.

4. The method of claim 1, wherein, the training the second neural network model based on the first training sample set and the updated second training sample set comprises: determining a first prediction loss based on the first training sample set; determining a second prediction loss based on the updated second training sample set; obtaining a total loss by performing weighted summation on the first prediction loss and the second prediction loss; training the second neural network model with a training target of minimizing the total loss.

5. The method of claim 1, wherein, The method comprises an offline stage and an online stage; the updating of the second training sample set belongs to the offline stage, and the training of the second neural network model belongs to the online stage.

6. The method of claim 1, wherein, The method comprises an online stage; the updating of the second training sample set and the training of the second neural network model both belong to the online stage.

7. The method of claim 6, wherein, After the iterative updating, the method further comprises: The obtained second neural network model is taken as the first neural network model of the current round, and the step of iterative updating is continuously performed.

8. An apparatus for training a neural network model, the apparatus comprising: an acquisition unit configured to acquire a first training sample set, wherein any first training sample comprises a first query video and a first reference video, and label positioning information of a video segment infringing on the first query video relative to the first reference video; the positioning information is used to indicate a time position of the video segment infringing; a training unit configured to train a first neural network model by using the first training sample set acquired by the acquisition unit; an iteration unit configured to perform iterative updating of the neural network model, the iteration unit comprising: an acquisition subunit configured to acquire a second training sample set, wherein any second training sample comprises a second query video and a second reference video, and does not comprise the label positioning information; a prediction subunit configured to input the second query video and the second reference video of the second training sample acquired by the acquisition subunit into the first neural network model to obtain predicted positioning information of a video segment infringing on the second query video relative to the second reference video; an updating subunit configured to update the second training sample set according to the predicted positioning information obtained by the prediction subunit, so that each second training sample in the updated second training sample set has label positioning information; a training subunit configured to train a second neural network model by using the first training sample set and the updated second training sample set obtained by the updating subunit; wherein the second training sample set comprises first-type training samples; the first-type training samples comprise label detection information of whether the second query video infringes on the second reference video; wherein the updating subunit comprises: a second label adding module configured to, if the predicted positioning information corresponds to the first-type training sample and is consistent with the label detection information, take the predicted positioning information as label positioning information of the corresponding second training sample; a second sample deleting module configured to, if the predicted positioning information corresponds to the first-type training sample and is inconsistent with the label detection information, delete the second training sample corresponding to the predicted positioning information from the second training sample set.

9. The apparatus of claim 8, wherein, The updating subunit comprises: a first label adding module configured to, if a confidence level of the predicted positioning information is higher than a preset threshold, take the predicted positioning information as label positioning information of the corresponding second training sample. The first sample deletion module is configured to delete the second training sample corresponding to the predicted positioning information from the second training sample set if the confidence corresponding to the predicted positioning information is less than or equal to the preset threshold.

10. The apparatus of claim 8, wherein, The second training sample set further includes second type training samples; and the second type training samples do not include the label detection information.

11. The apparatus of claim 8, wherein, The training subunit includes: The first determination module is configured to determine a first prediction loss based on the first training sample set; The second determination module is configured to determine a second prediction loss based on the updated second training sample set; The weighting module is configured to obtain a total loss by performing weighted summation on the first prediction loss obtained by the first determination module and the second prediction loss obtained by the second determination module; The training module is configured to train the second neural network model by taking minimization of the total loss obtained by the weighting module as a training target.

12. The apparatus of claim 8, wherein, The device includes a subunit for an offline stage and a subunit for an online stage; the updating subunit belongs to the subunit for the offline stage, and the training subunit belongs to the subunit for the online stage.

13. The apparatus of claim 8, wherein, The device includes a subunit for an online stage; the updating subunit and the training subunit both belong to the subunit for the online stage.

14. The apparatus of claim 13, wherein, The device further includes: The loop processing unit is configured to, after the iteration unit performs the iterative updating, take the obtained second neural network model as a first neural network model of this round, and continue the step of performing the iterative updating by the iteration unit. 15.A computer readable storage medium having stored thereon a computer program, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-7. 16.A computing device comprising a memory and a processor, the memory having stored therein executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-7.

Images