Model training method and device, nonvolatile storage medium and electronic device

Abstract

The application discloses a model training method and device, a nonvolatile storage medium and an electronic device. The method comprises the following steps: acquiring a preset video set with quality defects, wherein the preset video set comprises labeled videos and unlabeled videos; using a preset generator to add pseudo-defect labels to the quality defects in the unlabeled videos to generate pseudo-labeled videos, and extracting defect features in the labeled videos and the pseudo-labeled videos to generate defect feature videos, wherein the defect feature videos indicate defect features through preset defect labels, and the preset defect labels comprise true defect labels and pseudo-defect labels; using a preset discriminator to perform label discrimination on the preset defect labels in the defect feature videos to generate a discrimination result; and using a preset loss function and the discrimination result to train an initial video detection model to generate a target video detection model. The application solves the technical problem of low model training efficiency.

Description

Technical Field

[0001] This invention relates to the field of video processing, and more specifically, to a model training method, apparatus, non-volatile storage medium, and electronic device. Background Technology

[0002] With the continuous development of technology, digital video has become mainstream, making life more convenient and comfortable. As the performance of intelligent visual electronic devices improves, users have higher and higher requirements for video playback quality. However, due to factors such as shooting, post-editing or transcoding, videos may have some defects, such as color bars, missing frames, abnormal brightness, etc. This requires operators to analyze and monitor video sources in advance and optimize them to provide a better user viewing experience.

[0003] When analyzing and monitoring video sources, video detection models are typically used for defect analysis. However, training these models requires a large number of videos with pre-added defect labels, which is done manually, resulting in low efficiency. Consequently, the training speed of the video detection model is affected by the rate at which labels are added, leading to low training efficiency.

[0004] There is currently no effective solution to the problem of low training efficiency of the aforementioned models. Summary of the Invention

[0005] The present invention provides a model training method, apparatus, non-volatile storage medium, and electronic device to at least solve the technical problem of low model training efficiency.

[0006] According to one aspect of the present invention, a model training method is provided, comprising: acquiring a preset set of videos with image quality defects, wherein the preset video set includes labeled videos and unlabeled videos, the labeled videos indicating defect features by true defect labels; using a preset generator to extract defect features from the labeled videos and the unlabeled videos to generate defect feature videos, wherein the preset generator is trained by machine learning using multiple sets of data, each set of data including: the labeled videos and the true defect labels of the labeled videos, the preset generator being used to add pseudo-defect labels to the image quality defects in the unlabeled videos to generate pseudo-label videos, and extracting defect features from the labeled videos and the unlabeled videos. The defect feature video is generated from the defect features in the tagged video and the pseudo-tagged video. The defect feature video indicates the defect features through preset defect labels, which include: the true defect label and the pseudo defect label. A preset discriminator is used to perform label discrimination on the preset defect labels in the defect feature video to generate a discrimination result. The discrimination result is used to indicate the defect feature video that passes the discrimination of the true defect label and the pseudo defect label. An initial video detection model is trained using a preset loss function and the discrimination result to generate a target video detection model. The preset loss function is used to estimate the degree of difference between the predicted value and the true value of the target video detection model.

[0007] Optionally, a preset generator is used to extract defect features from the labeled video and the unlabeled video to generate a defect feature video. This includes: extracting multiple labeled video frames and a true defect label corresponding to each labeled video frame from the labeled video; determining each labeled video frame and its corresponding true defect label as a set of training data; training an initial generator using multiple sets of the training data to generate the preset generator; adding pseudo-defect labels to video frames with image quality defects in the unlabeled video using the preset generator to generate pseudo-labeled videos; and generating the defect feature video based on the pseudo-labeled videos and the labeled videos.

[0008] Optionally, generating the defect feature video based on the pseudo-labeled video and the labeled video includes: extracting defect features indicated by the true defect label from the labeled video; extracting defect features indicated by the pseudo-defect label from the pseudo-labeled video; and concatenating the defect features indicated by the true defect label and the defect features indicated by the pseudo-defect label to generate the defect feature video.

[0009] Optionally, a preset generator is used to extract defect features from the labeled video and the unlabeled video. Generating a defect feature video includes: processing the labeled video and the pseudo-labeled video based on the convolutional layer, self-attention module, and residual module in the preset generator to determine high-dimensional temporal features; and processing the high-dimensional temporal features based on the dilated convolutional bridge, transposed convolutional layer, and convolutional layer in the preset generator to determine the defect feature video.

[0010] Optionally, the preset discriminator includes at least a first discriminator, and the discrimination result includes at least a first discrimination result. Using the preset discriminator to perform label discrimination on the preset defect labels in the defect feature video and generating the discrimination result includes: using the first discriminator to perform label discrimination on the preset defect labels in the defect feature video and generating the first discrimination result.

[0011] Optionally, the preset discriminator includes at least a second discriminator, and the discrimination result includes at least a second discrimination result. After using the first discriminator to perform tag discrimination on the defect feature video and generating the first discrimination result, the method further includes: performing weak augmentation on the defect feature video to obtain a weakly augmented video, wherein the weak augmentation includes at least one of the following: translation, rotation, and flipping; and using the second discriminator to perform tag discrimination on the weakly augmented video to generate the second discrimination result.

[0012] Optionally, the preset discriminator includes at least a third discriminator, and the discrimination result includes at least a third discrimination result. After using the second discriminator to perform tag discrimination on the weakly amplified video and generating the second discrimination result, the method further includes: performing strong amplification on the weakly amplified video to obtain a strongly amplified video, wherein the strong amplification refers to performing color enhancement and shape enhancement on the weakly amplified video, and the strong amplification includes at least one of the following: brightness change, color change, contrast change, equalization, tone separation, horizontal translation, rotation, sharpening, horizontal cropping, vertical cropping, exposure, vertical translation, and cropping; using the third discriminator to perform tag discrimination on the strongly amplified video to generate the third discrimination result.

[0013] According to another aspect of the present invention, a model training apparatus is also provided, comprising: an acquisition module, configured to acquire a preset video set with image quality defects, wherein the preset video set includes labeled videos and unlabeled videos, and the labeled videos are indicated by true defect labels; and a generation module, configured to use a preset generator to extract defect features from the labeled videos and the unlabeled videos to generate defect feature videos, wherein the preset generator is trained using multiple sets of data through machine learning, each set of data includes: the labeled videos and the true defect labels of the labeled videos, and the preset generator is configured to add pseudo-defect labels to the image quality defects in the unlabeled videos to generate pseudo-label videos, and to extract the defects from the labeled videos. A defect feature video is generated from the defect features in the labeled video and the pseudo-labeled video. The defect feature video indicates the defect features through preset defect labels, which include: the true defect label and the pseudo defect label; an identification module is used to identify the preset defect labels in the defect feature video using a preset discriminator and generate an identification result, wherein the identification result is used to indicate the defect feature video that passes the identification of the true defect label and the pseudo defect label; a training module is used to train an initial video detection model using a preset loss function and the identification result to generate a target video detection model, wherein the preset loss function is used to estimate the degree of difference between the predicted value and the true value of the target video detection model.

[0014] According to another aspect of the present invention, a non-volatile storage medium is also provided, wherein a program is stored in the non-volatile storage medium, wherein the program controls the device where the non-volatile storage medium is located to execute the model training method described above when the program is running.

[0015] According to another aspect of the present invention, a processor is also provided, the processor being used to run a program, wherein the program executes the model training method described above when it runs.

[0016] In this embodiment of the invention, a preset set of videos with image quality defects is obtained. The preset video set includes labeled videos and unlabeled videos. Labeled videos are identified by true defect labels. A preset generator is used to extract defect features from the labeled and unlabeled videos to generate defect feature videos. The preset generator is trained using machine learning with multiple sets of data. Each set of data includes labeled videos and their true defect labels. The preset generator is used to add pseudo-defect labels to the image quality defects in unlabeled videos to generate pseudo-labeled videos, and to extract defect features from labeled and pseudo-labeled videos to generate defect feature videos. The defect feature videos are identified by preset defect labels, which include both true and pseudo defect labels. A preset discriminator is used to identify the preset defect labels in the defect feature videos. The system generates identification results, which indicate the defect feature videos that pass the identification of true and false defect labels. An initial video detection model is trained using a preset loss function and the identification results to generate a target video detection model. The preset loss function estimates the difference between the predicted and true values ​​of the target video detection model. A preset generator can add false defect labels to unlabeled videos based on labeled videos, achieving the goal of quickly adding defect labels. A preset discriminator identifies the added false defect labels, making them more consistent with the requirements of true defect labels. This ensures that the videos with added false labels meet the training requirements of the target video detection model, achieving the goal of ensuring the accuracy of the target video detection model. This improves the technical effect of model training efficiency and solves the technical problem of low model training efficiency. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of a model training method according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of a semi-supervised model framework for an adversarial network according to an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of a self-attention module framework according to an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of a model training device according to an embodiment of the present invention;

[0022] Figure 5 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] According to an embodiment of the present invention, a model training method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] Figure 1 This is a flowchart of a model training method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0027] Step S102: Obtain a preset video set with image quality defects. The preset video set includes labeled videos and unlabeled videos. Labeled videos are indicated by true defect labels to indicate defect features.

[0028] Step S104: Use a preset generator to extract defect features from labeled and unlabeled videos to generate defect feature videos. The preset generator is trained by machine learning using multiple sets of data. Each set of data includes labeled videos and true defect labels for labeled videos. The preset generator is used to add pseudo-defect labels to the image quality defects in unlabeled videos to generate pseudo-labeled videos, and to extract defect features from labeled and pseudo-labeled videos to generate defect feature videos. The defect feature videos indicate defect features through preset defect labels, which include true defect labels and pseudo-defect labels.

[0029] Step S106: Use a preset discriminator to perform label discrimination on preset defect labels in the defect feature video and generate discrimination results. The discrimination results are used to indicate the defect feature videos that pass the discrimination of true defect labels and false defect labels.

[0030] Step S108: Train the initial video detection model using a preset loss function and the identification results to generate a target video detection model. The preset loss function is used to estimate the degree of difference between the predicted value and the true value of the target video detection model.

[0031] In this embodiment of the invention, a preset set of videos with image quality defects is obtained. The preset video set includes labeled videos and unlabeled videos. Labeled videos are identified by true defect labels. A preset generator is used to extract defect features from the labeled and unlabeled videos to generate defect feature videos. The preset generator is trained using machine learning with multiple sets of data. Each set of data includes labeled videos and their true defect labels. The preset generator is used to add pseudo-defect labels to the image quality defects in unlabeled videos to generate pseudo-labeled videos, and to extract defect features from labeled and pseudo-labeled videos to generate defect feature videos. The defect feature videos are identified by preset defect labels, which include both true and pseudo defect labels. A preset discriminator is used to identify the preset defect labels in the defect feature videos. The system generates identification results, which indicate the defect feature videos that pass the identification of true and false defect labels. An initial video detection model is trained using a preset loss function and the identification results to generate a target video detection model. The preset loss function estimates the difference between the predicted and true values ​​of the target video detection model. A preset generator can add false defect labels to unlabeled videos based on labeled videos, achieving the goal of quickly adding defect labels. A preset discriminator identifies the added false defect labels, making them more consistent with the requirements of true defect labels. This ensures that the videos with added false labels meet the training requirements of the target video detection model, achieving the goal of ensuring the accuracy of the target video detection model. This improves the technical effect of model training efficiency and solves the technical problem of low model training efficiency.

[0032] In step S102 above, image quality defects include common video problems such as stuttering, black screen, mosaic, distortion, abnormal brightness, color loss, color bars, missing frames, and flickering.

[0033] As an optional embodiment, a preset generator is used to extract defect features from labeled and unlabeled videos. The process of generating defect feature videos includes: processing labeled and pseudo-labeled videos based on convolutional layers, self-attention modules, and residual modules in the preset generator to determine high-dimensional temporal features; and processing the high-dimensional temporal features based on dilated convolutional bridges, transposed convolutional layers, and convolutional layers in the preset generator to determine defect feature videos.

[0034] In the above embodiments of the present invention, the preset generator includes: a convolutional layer, a self-attention module, a residual module, a dilated convolutional bridge, and a convolutional layer; labeled and unlabeled videos are processed through the convolutional layer, the self-attention module, and the residual module to obtain high-dimensional temporal features, and the high-dimensional temporal features are then processed through the dilated convolutional bridge and a transposed convolutional layer to obtain a defect feature video.

[0035] In step S104 above, the structures of the preset generator acquire low-dimensional and high-dimensional information (such as high-order time series features) by using a skip connection method.

[0036] It should be noted that the self-attention module extracts features from both labeled and unlabeled videos. The final feature map is a weighted sum of features at all locations and the original features, which helps to improve the separability and variability of features.

[0037] In step S104 above, the preset generator can be trained using labeled videos. Based on the trained preset generator, it can classify unlabeled videos, thus generating pseudo-defect labels.

[0038] Optionally, after classifying unlabeled videos and adding pseudo-defect labels to them, the unlabeled videos that have been correctly classified can be used as samples to continue training the preset generator, thereby improving the understanding ability of the preset generator.

[0039] In step S104 above, the preset generator can add pseudo-defect labels to the image quality defects in the unlabeled video based on the training results of the labeled video, and generate pseudo-labeled videos.

[0040] In step S104 above, in order for the pseudo-labeled video with added pseudo-defect labels to be used to train the target video detection model, the added pseudo-defect labels need to be able to fool the preset discriminator. If the pseudo-defect labels can fool the preset discriminator, it means that the added pseudo-defect labels are close to the real defect labels, ensuring that the pseudo-defect video using pseudo-defect labels can meet the training requirements of the target video detection model, and ensuring that the trained target video detection model is accurate enough.

[0041] In step S106 above, the preset discriminator is a conventional UNet structure.

[0042] In step S106 above, the preset loss function is used to estimate the degree of difference between the predicted value and the true value of the target video detection model, which is used to reflect the gap between the model and the actual data. The smaller the loss function, the better the robustness of the model.

[0043] It should be noted that due to factors such as shooting, post-production editing, or transcoding, the video may have some defects, such as color bars, missing frames, abnormal brightness, etc. The target video detection model can analyze and monitor the video source and promptly remind staff to optimize the defective image quality.

[0044] In step S108 above, the identification results include: the defect feature video of the real label video corresponding to the real defect label, and the defect feature video of the pseudo label video corresponding to the pseudo defect label. The identification results are used as training samples to train the initial video detection model to obtain the preset video detection model. Then, the preset loss function is used to verify the trained preset video detection model. If the value of the preset loss function of the preset video detection model is lower than the preset threshold, it indicates that the preset video detection model has good robustness, and thus the preset video detection model is determined to be the target video detection model.

[0045] As an optional embodiment, a preset generator is used to extract defect features from labeled and unlabeled videos. The process of generating a defect feature video includes: extracting multiple labeled video frames and a true defect label corresponding to each labeled video frame from the labeled video; determining each labeled video frame and its corresponding true defect label as a set of training data; training an initial generator using multiple sets of training data to generate a preset generator; adding pseudo-defect labels to video frames with image quality defects in the unlabeled video using the preset generator to generate pseudo-labeled videos; and generating a defect feature video based on the pseudo-labeled video and the labeled video.

[0046] In the above embodiments of the present invention, the preset generator can be trained based on multiple labeled video frames and corresponding true defect labels. Based on the trained preset generator, pseudo-defect labels can be added to video frames with image quality defects in unlabeled videos to generate pseudo-labeled videos. Then, based on the defect features of labeled videos and pseudo-labeled videos, defect feature videos can be generated for training target video detection models.

[0047] It should be noted that a video consists of multiple video frames. Therefore, the true defect label in a labeled video indicates the defective image in each video frame. The process of adding pseudo-labels to an unlabeled video requires identifying whether there is an image defect in each video frame of the unlabeled video. If the video frame has an image defect, a pseudo-defect label is added to that video frame; if the video frame does not have an image defect, no defect label is added to that video frame. By traversing multiple video frames of the unlabeled video, pseudo-defect labels can be added to the unlabeled video, generating a pseudo-labeled video.

[0048] As an optional embodiment, generating a defect feature video based on pseudo-labeled video and labeled video includes: extracting defect features indicated by genuine defect labels from labeled video; extracting defect features indicated by pseudo-defect labels from pseudo-labeled video; and concatenating the defect features indicated by genuine defect labels and the defect features indicated by pseudo-defect labels to generate a defect feature video.

[0049] In the above embodiments of the present invention, the target video detection model is trained based on defective videos and pseudo-defective videos. The main feature used is the defect feature in the defective video and pseudo-defective video. In order to improve the training efficiency of the target video detection model, it is necessary to use the most accurate and the smallest number of samples for training, and reduce the impact of irrelevant data on the efficiency of the training process. Therefore, concatenating the defect features in the labeled video and the defect features in the pseudo-labeled video can reduce the amount of sample data used to train the target video detection model and improve the model training efficiency.

[0050] As an optional embodiment, the preset discriminator includes at least: a first discriminator, and the discrimination result includes at least: a first discrimination result. Using the preset discriminator to perform label discrimination on the preset defect labels in the defect feature video and generating the discrimination result includes: using the first discriminator to perform label discrimination on the preset defect labels in the defect feature video and generating the first discrimination result.

[0051] In the above embodiments of the present invention, the authenticity of the fake defect labels can be identified by the first discriminator. If the authenticity of the fake defect label is high, the fake defect label will pass the identification of the first discriminator. If the authenticity of the fake defect label is poor, the fake defect label will not pass the identification of the first discriminator. Therefore, the first discriminator filters out more authentic fake defect labels and fake defect videos with added fake defect labels.

[0052] Optionally, the defect feature video of the tagged video and the defect feature video of the pseudo-tagged video are input into the first discriminator; the first discriminator performs the identification of whether there is a label or not. If the pseudo-defect label fools the first discriminator, the first discriminator considers that there is a label and outputs the first identification result.

[0053] Optionally, for tagged videos, the first discrimination result output by the first discriminator is: output a video frame with a height and width value of 1; for tagged videos, the first discrimination result output by the first discriminator is: output a video frame with a height and width value of 0.

[0054] As an optional embodiment, the preset discriminator includes at least a second discriminator, and the discrimination result includes at least a second discrimination result. After using the first discriminator to perform label discrimination on the defect feature video and generate the first discrimination result, the method further includes: performing weak augmentation on the defect feature video to obtain a weakly augmented video, wherein the weak augmentation includes at least one of the following: translation, rotation, and flipping; and using the second discriminator to perform label discrimination on the weakly augmented video to generate the second discrimination result.

[0055] In the above embodiments of the present invention, the main purpose of weak amplification is to obtain stable and high-quality pseudo-labeled videos, improve the semantic understanding ability of the model, and make the model's detection of videos more accurate.

[0056] Optionally, for tagged videos, the second discrimination result output by the second discriminator is: output a video frame with a height and width value of 1; for tagged videos, the second discrimination result output by the second discriminator is: output a video frame with a height and width value of 0.

[0057] Optionally, weak augmentation includes translation, rotation, and flipping, which can be combined in different ways to improve the semantic understanding ability of the model.

[0058] As an optional embodiment, the preset discriminator includes at least a third discriminator, and the discrimination result includes at least a third discrimination result. After using the second discriminator to perform tag discrimination on the weakly amplified video and generating the second discrimination result, the method further includes: performing strong amplification on the weakly amplified video to obtain a strongly amplified video, wherein strong amplification refers to performing color enhancement and shape enhancement on the weakly amplified video, and strong amplification includes at least one of the following: brightness change, color change, contrast change, equalization, tone separation, horizontal translation, rotation, sharpening, horizontal cropping, vertical cropping, exposure, vertical translation, and cropping; using the third discriminator to perform tag discrimination on the strongly amplified video and generating the third discrimination result.

[0059] Optionally, for tagged videos, the third discrimination result output by the third discriminator is: output a video frame with a height and width value of 1; for tagged videos, the third discrimination result output by the third discriminator is: output a video frame with a height and width value of 0.

[0060] In the above embodiments of the present invention, the purpose of strong amplification is to avoid incorrectly fitting unlabeled videos to pseudo-labeled videos.

[0061] Optionally, color enhancement and shape enhancement mainly include several enhancement methods such as brightness change, color change, contrast change, equalization, tone separation, lateral translation, rotation, sharpening, lateral cropping, vertical cropping, exposure, vertical translation, and cropping. Through different combinations of enhancements, the semantic understanding ability of the model can be improved.

[0062] It should be noted that in adversarial networks, the pseudo-defect labels assigned to video frames by the pre-generator attempt to deceive three pre-generator discriminators (such as the first discriminator, the second discriminator, and the third discriminator) at the same time. The pre-generator discriminators will assign a pre-defined label type to the defect labels (such as the real defect label and the pseudo-defect label). Using an appropriate pre-generator discriminator can minimize the dispersion between the real defect label and the pseudo defect label generated by the pre-generator.

[0063] This invention also provides an optional embodiment, a preferred embodiment of which offers a semi-supervised video monitoring scheme based on adversarial networks. Using a dataset (i.e., a pre-defined video set), an initial semi-supervised adversarial network model (i.e., an initial video detection model) is constructed. This initial model is then trained for accuracy using a loss function. During training, in addition to using a discriminator to predict the results of feature-extracted video frames, two additional discriminators are used to predict the results of weakly augmented and strongly augmented video frames in the dataset (i.e., the pre-defined video set). Weak augmentation is used to obtain stable, high-quality pseudo-labeled video frames. Strong augmentation is used to prevent unlabeled video frames from overfitting to incorrectly labeled video frames, both aimed at improving the semantic understanding ability of the semi-supervised adversarial network model. This scheme effectively improves the identification of defective video quality, increases work efficiency, and significantly enhances recognition accuracy as the model learns through analysis and monitoring. Video monitoring based on the trained semi-supervised adversarial network model (i.e., the target video detection model) can overcome the problem of missing video source quality.

[0064] As an optional embodiment, a semi-supervised video monitoring device based on adversarial networks includes the following steps:

[0065] 1) Obtain the dataset (i.e., obtain the preset video set).

[0066] Optionally, the dataset (i.e., obtaining a preset set of videos) includes a small number of labeled videos and a large number of unlabeled videos.

[0067] Optionally, the dataset (i.e., the preset set of videos) consists of videos with image quality defects, including common video problems such as stuttering, black screen, mosaic, distortion, abnormal brightness, color loss, color bars, missing frames, and flickering.

[0068] It should be noted that semi-supervised learning aims to improve the training and generalization performance of a model by using a small amount of labeled data and a large amount of unlabeled data. Compared with self-supervised solutions, semi-supervised learning is a more reliable and effective solution, while also saving the time and manpower costs required for labeled data.

[0069] 2) Construct an initial semi-supervised adversarial network model (i.e., an initial video detection model), which includes a pre-defined generator, a first discriminator, a second discriminator, and a third discriminator.

[0070] Figure 2 This is a schematic diagram of a semi-supervised model framework for adversarial networks according to an embodiment of the present invention, as shown below. Figure 2As shown, the preset generator includes convolutional layers, self-attention modules, residual modules, and dilated convolutional bridges. Labeled and unlabeled videos are processed through convolutional layers, self-attention modules, and residual modules to obtain high-dimensional temporal features. The high-dimensional temporal features are then processed through dilated convolutional bridges, transposed convolutional layers, and convolutional layers to obtain defect feature videos.

[0071] Optionally, the pre-defined generator uses skip connections between its various structures to obtain low-dimensional and high-dimensional information (such as high-dimensional temporal features).

[0072] Alternatively, the discriminator can be a conventional UNet structure.

[0073] Figure 3 This is a schematic diagram of a self-attention module framework according to an embodiment of the present invention, as shown below. Figure 3 As shown, the self-attention module is used to extract features from labeled and unlabeled videos. The final feature map is a weighted sum of features at all locations and the original features, which helps to improve the separability of features.

[0074] 3) Input labeled and unlabeled videos into the initial generator for feature extraction to obtain defect feature videos. The defect feature videos include defect feature videos of labeled videos and defect feature videos of pseudo-labeled videos.

[0075] Optionally, feature extraction refers to: performing defect labeling (i.e., adding pseudo-defect labels) and defect-free labeling on each frame in the unlabeled video.

[0076] Optionally, video frames of labeled and unlabeled videos are sequentially passed through convolutional layers, self-attention modules, and residual modules to obtain high-dimensional temporal features. These high-dimensional temporal features are then passed through dilated convolutional bridges and convolutional layers to obtain defect feature videos.

[0077] It should be noted that before determining the defect feature video, a preset generator is obtained by training the initial generator with labeled videos. Then, the preset generator is used to classify unlabeled video frames, which will generate pseudo defect labels.

[0078] Optionally, select unlabeled videos that are considered to be correctly classified as unlabeled samples, and use the selected unlabeled samples to continue training the preset generator to improve the preset generator's understanding ability.

[0079] 4) The defect features of the tagged video and the pseudo-tagged video are concatenated into a defect feature video and then input into the first discriminator to obtain the first discrimination result.

[0080] Optionally, for tagged videos, the first discrimination result output by the first discriminator is: output a video frame with a height and width value of 1; for tagged videos, the first discrimination result output by the first discriminator is: output a video frame with a height and width value of 0.

[0081] It should be noted that in adversarial networks, the pseudo-defect labels assigned to video frames by the pre-generated generator attempt to deceive three discriminators simultaneously. The discriminators assign a pre-defined label type to the data that matches the label. Using appropriate discriminators can minimize the dispersion between the real data and the pseudo-label data generated by the generator.

[0082] 5) Perform weak amplification on the defect feature video. After weak amplification, input the video into the second discriminator to obtain the second discrimination result.

[0083] Optionally, similar to the first discriminator, for tagged videos, the second discriminator outputs a second discrimination result of: outputting a video frame with a height and width value of 1; for tagged videos, the second discriminator outputs a second discrimination result of: outputting a video frame with a height and width value of 0.

[0084] Optionally, weak augmentation of video frames is performed, mainly to obtain stable and high-quality video frames with pseudo-defect labels, improve the model's semantic understanding ability, and make the model's detection of videos more accurate.

[0085] Optionally, weak augmentation includes translation, rotation, and flipping, which can be combined in different ways to improve the semantic understanding ability of the model.

[0086] 6) After performing strong amplification on the weakly amplified video, input it into the third discriminator to obtain the third discrimination result.

[0087] Optionally, similar to the first discriminator, for tagged videos, the second discriminator outputs a second discrimination result of: outputting a video frame with a height and width value of 1; for tagged videos, the second discriminator outputs a second discrimination result of: outputting a video frame with a height and width value of 0.

[0088] Alternatively, strong amplification can be used to prevent video frames without defective labels from overfitting to video frames with incorrect pseudo-defective labels.

[0089] Optionally, color enhancement and shape enhancement mainly include several enhancement methods such as brightness change, color change, contrast change, equalization, tone separation, lateral translation, rotation, sharpening, lateral cropping, vertical cropping, exposure, vertical translation, and cropping. Through different combinations of enhancements, the semantic understanding ability of the model can be improved.

[0090] 7) Train the initial semi-supervised adversarial network model (such as the initial video detection model) based on the pre-loss function, the first discrimination result, the second discrimination result, and the third discrimination result to obtain the trained semi-supervised adversarial network model (i.e., the target video detection model).

[0091] Optionally, the preset loss function specifically includes: label loss function, adversarial loss function, first loss function, second loss function, and third loss function.

[0092] It should be noted that a preset loss function is used to estimate the degree of inconsistency between the model's predicted values ​​and the true values, reflecting the gap between the model and the actual data. The smaller the loss function, the better the robustness of the model.

[0093] Optionally, the label loss function is: L s =L ce (y l y u )+L dice (y l y u Among them, L ce Let L be the cross-entropy loss function. dice This is the dice loss function.

[0094] Optionally, the cross-entropy loss function L ce and dice loss function L dice They are respectively:

[0095]

[0096]

[0097] The dataset (i.e., the preset video set) is Z = (x, y), and the labeled videos are Z0. l =(x l y l Untagged videos are: Z u =(x u ); y represents a true defect label, y u The pseudo-defect label is represented by u = (1, 2, ..., H × W), where H represents the height and W represents the width.

[0098] Optionally, the adversarial loss function is:

[0099] L GAN (G,D)=E x,y [logD(x,y)]+E x [log(1-D(x,G(x)))].

[0100] Optionally, the first loss function is: L1 = L 1G +L1D Among them, the preset generator loss L 1G and the loss L of the first discriminator 1D They are respectively:

[0101]

[0102]

[0103] Where G represents the preset generator, D1 represents the first discriminator, and G(x l G(x) represents a labeled video with defect features (hereinafter referred to as: labeled defect feature video), u ) represents a video with defect features of a pseudo-labeled video (hereinafter referred to as: pseudo-labeled defect feature video).

[0104] Optionally, the second loss function is: L2 = L 2G +L 2D Among them, the preset generator loss L 2G and the loss L of the first discriminator 2D They are respectively:

[0105]

[0106]

[0107] Where G represents the preset generator, D2 represents the second discriminator, and G(x l G(x) represents a video that undergoes weak augmentation of labeled defective features (hereinafter referred to as: labeled weakly augmented video), where G(x) represents the video that undergoes weak augmentation of labeled defective features. u This refers to a video that has undergone weak augmentation of the video with the defective features of the pseudo-label (hereinafter referred to as: pseudo-label weak augmentation video).

[0108] Optionally, the second loss function is: L3 = L 3G +L 3D Among them, the preset generator loss L 3G and the loss L of the first discriminator 3D They are respectively:

[0109]

[0110]

[0111] Where G represents the preset generator and D3 represents the third discriminator. This refers to videos that have been strongly amplified from weakly amplified labeled videos. This refers to videos that have undergone strong amplification of weakly amplified pseudo-labeled videos.

[0112] 8) Perform video detection using a trained semi-supervised adversarial network model (i.e., a target video detection model).

[0113] According to an embodiment of the present invention, a model training device embodiment is also provided. It should be noted that the model training device can be used to execute the model training method in the embodiment of the present invention, and the model training method in the embodiment of the present invention can be executed in the model training device.

[0114] Figure 4 This is a schematic diagram of a model training device according to an embodiment of the present invention, such as... Figure 4 As shown, the device may include: an acquisition module 41, configured to acquire a preset set of videos with image quality defects, wherein the preset video set includes labeled videos and unlabeled videos, and the labeled videos are indicated by true defect labels; and a generation module 43, configured to use a preset generator to extract defect features from the labeled videos and the unlabeled videos to generate defect feature videos, wherein the preset generator is trained using multiple sets of data through machine learning, and each set of data includes: the labeled videos and the true defect labels of the labeled videos; the preset generator is used to add pseudo-defect labels to the image quality defects in the unlabeled videos to generate pseudo-labeled videos, and to extract the label features from the labeled videos and the pseudo-defect labels. The defect feature video is generated by identifying defect features in the video. The defect feature video is indicated by preset defect labels, which include: true defect labels and false defect labels. The identification module 44 is used to identify the preset defect labels in the defect feature video using a preset discriminator and generate an identification result. The identification result is used to indicate the defect feature video that passes the identification of the true defect labels and the false defect labels. The training module 46 is used to train the initial video detection model using a preset loss function and the identification result to generate a target video detection model. The preset loss function is used to estimate the degree of difference between the predicted value and the true value of the target video detection model.

[0115] It should be noted that the acquisition module 41 in this embodiment can be used to execute step S102 in this application embodiment, the generation module 43 in this embodiment can be used to execute step S104 in this application embodiment, the identification module 44 in this embodiment can be used to execute step S106 in this application embodiment, and the training module 46 in this embodiment can be used to execute step S108 in this application embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments.

[0116] In this embodiment of the invention, a preset set of videos with image quality defects is obtained. The preset video set includes labeled videos and unlabeled videos. Labeled videos are identified by true defect labels. A preset generator is used to extract defect features from the labeled and unlabeled videos to generate defect feature videos. The preset generator is trained using machine learning with multiple sets of data. Each set of data includes labeled videos and their true defect labels. The preset generator is used to add pseudo-defect labels to the image quality defects in unlabeled videos to generate pseudo-labeled videos, and to extract defect features from labeled and pseudo-labeled videos to generate defect feature videos. The defect feature videos are identified by preset defect labels, which include both true and pseudo defect labels. A preset discriminator is used to identify the preset defect labels in the defect feature videos. The system generates identification results, which indicate the defect feature videos that pass the identification of true and false defect labels. An initial video detection model is trained using a preset loss function and the identification results to generate a target video detection model. The preset loss function estimates the difference between the predicted and true values ​​of the target video detection model. A preset generator can add false defect labels to unlabeled videos based on labeled videos, achieving the goal of quickly adding defect labels. A preset discriminator identifies the added false defect labels, making them more consistent with the requirements of true defect labels. This ensures that the videos with added false labels meet the training requirements of the target video detection model, achieving the goal of ensuring the accuracy of the target video detection model. This improves the technical effect of model training efficiency and solves the technical problem of low model training efficiency.

[0117] As an optional embodiment, the generation module includes: an extraction unit, configured to extract multiple labeled video frames and a true defect label corresponding to each labeled video frame from the labeled video; a determination unit, configured to determine each labeled video frame and its corresponding true defect label as a set of training data; a training unit, configured to train an initial generator using multiple sets of the training data to generate the preset generator; an adding unit, configured to add pseudo-defect labels to video frames with image quality defects in the unlabeled video using the preset generator to generate pseudo-labeled videos; and a generation unit, configured to generate the defect feature video based on the pseudo-labeled video and the labeled video.

[0118] As an optional embodiment, the generation unit includes: a first extraction subunit, used to extract the defect features indicated by the true defect label from the tagged video; a second extraction subunit, used to extract the defect features indicated by the pseudo defect label from the pseudo-labeled video; and a generation unit, used to splice the defect features indicated by the true defect label and the defect features indicated by the pseudo defect label to generate a defect feature video.

[0119] As an optional embodiment, the generation module includes: a feature determination unit, used to process the labeled video and the pseudo-labeled video based on the convolutional layer, self-attention module and residual module in the preset generator to determine high-dimensional temporal features; and a video determination unit, used to process the high-dimensional temporal features based on the dilated convolutional bridge, transposed convolutional layer and convolutional layer in the preset generator to determine the defect feature video.

[0120] As an optional embodiment, the preset discriminator includes at least a first discriminator, the discrimination result includes at least a first discrimination result, and the discrimination module includes a first discrimination unit, used to use the first discriminator to perform tag discrimination on the preset defect tags in the defect feature video, and generate the first discrimination result.

[0121] As an optional embodiment, the preset discriminator includes at least a second discriminator, and the discrimination result includes at least a second discrimination result. The device further includes a weak amplification unit, configured to perform tag discrimination on the defect feature video using the first discriminator, generate the first discrimination result, and then perform weak amplification on the defect feature video to obtain a weakly amplified video, wherein the weak amplification includes at least one of the following: translation, rotation, and flipping; and a second discrimination unit, configured to perform tag discrimination on the weakly amplified video using the second discriminator to generate the second discrimination result.

[0122] As an optional embodiment, the preset discriminator includes at least a third discriminator, and the discrimination result includes at least a third discrimination result. The device further includes a strong amplification unit, configured to perform tag discrimination on the weakly amplified video using the second discriminator, generate the second discrimination result, and then perform strong amplification on the weakly amplified video to obtain a strongly amplified video. The strong amplification refers to color enhancement and shape enhancement on the weakly amplified video, and the strong amplification includes at least one of the following: brightness change, color change, contrast change, equalization, tone separation, horizontal translation, rotation, sharpening, horizontal cropping, vertical cropping, exposure adjustment, vertical translation, and cropping. The third discrimination unit is configured to perform tag discrimination on the strongly amplified video using the third discriminator and generate the third discrimination result.

[0123] Embodiments of the present invention can provide a computer terminal, which can be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the computer terminal can also be replaced by a mobile terminal or other terminal device.

[0124] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0125] In this embodiment, the computer terminal described above can execute the program code for the following steps in the model training method: obtaining a preset set of videos with image quality defects, wherein the preset video set includes labeled videos and unlabeled videos, and labeled videos indicate defect features through true defect labels; using a preset generator to extract defect features from labeled videos and unlabeled videos to generate defect feature videos, wherein the preset generator is trained using multiple sets of data through machine learning, and each set of data includes: labeled videos and true defect labels for labeled videos, and the preset generator is used to add pseudo-defect labels to image quality defects in unlabeled videos to generate pseudo-label videos. The process involves several steps: first, analyzing video data and extracting defect features from labeled and pseudo-labeled videos to generate defect feature videos. These videos are labeled with predefined defect tags, including both genuine and pseudo-defect tags. A predefined discriminator is then used to identify these tags, generating a discriminator result. This result indicates which defect feature videos have passed the discriminator for identifying genuine and pseudo-defect tags. Finally, a predefined loss function and the discriminator result are used to train an initial video detection model, generating a target video detection model. This predefined loss function measures the difference between the predicted and actual values ​​of the target video detection model.

[0126] Optionally, Figure 5 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 5 As shown, the computer terminal 50 may include one or more (only one is shown in the figure) processors 52 and memory 54.

[0127] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the model training method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned model training method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal 50 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0128] The processor can access information and applications stored in memory via a transmission device to perform the following steps: acquiring a preset set of videos with image quality defects, wherein the preset video set includes labeled videos and unlabeled videos, and labeled videos are indicated by true defect labels; using a preset generator to extract defect features from the labeled and unlabeled videos to generate defect feature videos, wherein the preset generator is trained using machine learning with multiple sets of data, each set of data including: labeled videos and true defect labels for the labeled videos; the preset generator is used to add pseudo-defect labels to image quality defects in unlabeled videos to generate pseudo-label videos. The process involves several steps: first, analyzing video data and extracting defect features from labeled and pseudo-labeled videos to generate defect feature videos. These videos are labeled with predefined defect tags, including both genuine and pseudo-defect tags. A predefined discriminator is then used to identify these tags, generating a discriminator result. This result indicates which defect feature videos have passed the discriminator for identifying genuine and pseudo-defect tags. Finally, a predefined loss function and the discriminator result are used to train an initial video detection model, generating a target video detection model. This predefined loss function measures the difference between the predicted and actual values ​​of the target video detection model.

[0129] Optionally, the processor may also execute program code for the following steps: extracting multiple labeled video frames and the true defect label corresponding to each labeled video frame from the labeled video; determining each labeled video frame and the corresponding true defect label as a set of training data; training the initial generator using multiple sets of training data to generate a preset generator; adding pseudo-defect labels to video frames with image quality defects in unlabeled videos using the preset generator to generate pseudo-labeled videos; and generating a defect feature video based on the pseudo-labeled video and the labeled video.

[0130] Optionally, the processor may also execute program code that performs the following steps: extracting defect features indicated by genuine defect labels from labeled videos; extracting defect features indicated by pseudo-defect labels from pseudo-labeled videos; and concatenating the defect features indicated by genuine defect labels and the defect features indicated by pseudo-defect labels to generate a defect feature video.

[0131] Optionally, the processor may also execute program code for the following steps: processing labeled and pseudo-labeled videos based on convolutional layers, self-attention modules, and residual modules in a preset generator to determine high-dimensional temporal features; and processing high-dimensional temporal features based on dilated convolutional bridges, transposed convolutional layers, and convolutional layers in a preset generator to determine defective feature videos.

[0132] Optionally, the preset discriminator includes at least a first discriminator, and the discrimination result includes at least a first discrimination result. The processor may also execute program code that performs the following steps: using the first discriminator to perform label discrimination on the preset defect labels in the defect feature video and generating a first discrimination result.

[0133] Optionally, the preset discriminator includes at least a second discriminator, and the discrimination result includes at least a second discrimination result. The processor may also execute program code for the following steps: after using the first discriminator to perform label discrimination on the defect feature video and generating a first discrimination result, the defect feature video is weakly augmented to obtain a weakly augmented video, wherein the weak augmentation includes at least one of the following: translation, rotation, and flipping; the second discriminator is used to perform label discrimination on the weakly augmented video to generate a second discrimination result.

[0134] Optionally, the preset discriminator includes at least a third discriminator, and the discrimination result includes at least a third discrimination result. The processor can also execute program code for the following steps: after using the second discriminator to perform tag discrimination on the weakly amplified video and generating a second discrimination result, the weakly amplified video is strongly amplified to obtain a strongly amplified video. Here, strong amplification refers to color enhancement and shape enhancement on the weakly amplified video. Strong amplification includes at least one of the following: brightness change, color change, contrast change, equalization, tone separation, horizontal translation, rotation, sharpening, horizontal cropping, vertical cropping, exposure, vertical translation, and cropping; the third discriminator is used to perform tag discrimination on the strongly amplified video to generate a third discrimination result.

[0135] This invention provides a model training scheme. It involves acquiring a preset set of videos with image quality defects, including labeled and unlabeled videos. Labeled videos are identified by true defect labels. A preset generator is used to extract defect features from both labeled and unlabeled videos, generating defect feature videos. The preset generator is trained using machine learning on multiple datasets, each dataset including labeled videos and their true defect labels. The preset generator is used to add pseudo-defect labels to unlabeled videos to generate pseudo-labeled videos, and to extract defect features from labeled and pseudo-labeled videos to generate defect feature videos. These defect feature videos are identified by preset defect labels, which include both true and pseudo-defect labels. A preset discriminator is used to identify the preset defect labels in the defect feature videos, generating... The identification results indicate which videos have passed the identification of true and false defect labels. An initial video detection model is trained using a preset loss function and the identification results to generate a target video detection model. The preset loss function estimates the difference between the predicted and actual values ​​of the target video detection model. A preset generator adds false defect labels to unlabeled videos based on labeled videos, achieving rapid label addition. A preset discriminator identifies the added false defect labels, making them more consistent with the requirements of true defect labels. This ensures that the false defect videos with added labels meet the training requirements of the target video detection model, thus ensuring its accuracy and improving training efficiency. This solves the problem of low training efficiency in traditional models.

[0136] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, computer terminal 50 may also include components that are more advanced than those described above. Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0137] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0138] Embodiments of the present invention also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the model training method provided in the above embodiments.

[0139] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0140] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a preset set of videos with image quality defects, wherein the preset video set includes labeled videos and unlabeled videos, and labeled videos are indicated by true defect labels; using a preset generator to extract defect features from the labeled videos and unlabeled videos to generate defect feature videos, wherein the preset generator is trained using multiple sets of data through machine learning, and each set of data includes: labeled videos and true defect labels for the labeled videos, and the preset generator is used to add pseudo-defect labels to the image quality defects in the unlabeled videos to generate pseudo-label videos. The process involves several steps: first, analyzing video data and extracting defect features from labeled and pseudo-labeled videos to generate defect feature videos. These videos are labeled with predefined defect tags, including both genuine and pseudo-defect tags. A predefined discriminator is then used to identify these tags, generating a discriminator result. This result indicates which defect feature videos have passed the discriminator for identifying genuine and pseudo-defect tags. Finally, a predefined loss function and the discriminator result are used to train an initial video detection model, generating a target video detection model. This predefined loss function measures the difference between the predicted and actual values ​​of the target video detection model.

[0141] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: extracting multiple labeled video frames and true defect labels corresponding to each labeled video frame from the labeled video; determining each labeled video frame and its corresponding true defect label as a set of training data; training an initial generator using multiple sets of training data to generate a preset generator; adding pseudo-defect labels to video frames with image quality defects in unlabeled videos using the preset generator to generate pseudo-labeled videos; and generating a defect feature video based on the pseudo-labeled video and the labeled video.

[0142] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: extracting defect features indicated by genuine defect tags from tagged videos; extracting defect features indicated by pseudo-defect tags from pseudo-tagged videos; and concatenating the defect features indicated by genuine defect tags and the defect features indicated by pseudo-defect tags to generate a defect feature video.

[0143] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: processing labeled and pseudo-labeled videos based on convolutional layers, self-attention modules, and residual modules in a preset generator to determine high-dimensional temporal features; and processing the high-dimensional temporal features based on dilated convolutional bridges, transposed convolutional layers, and convolutional layers in a preset generator to determine defective feature videos.

[0144] Optionally, in this embodiment, the preset discriminator includes at least a first discriminator, the discrimination result includes at least a first discrimination result, and the storage medium is configured to store program code for performing the following steps: using the first discriminator to perform label discrimination on preset defect labels in the defect feature video, and generating a first discrimination result.

[0145] Optionally, in this embodiment, the preset discriminator includes at least a second discriminator, and the discrimination result includes at least a second discrimination result. The storage medium is configured to store program code for performing the following steps: after using the first discriminator to perform tag discrimination on the defect feature video and generating a first discrimination result, the defect feature video is weakly amplified to obtain a weakly amplified video, wherein the weak amplification includes at least one of the following: translation, rotation, and flipping; the second discriminator is used to perform tag discrimination on the weakly amplified video to generate a second discrimination result.

[0146] Optionally, in this embodiment, the preset discriminator includes at least a third discriminator, and the discrimination result includes at least a third discrimination result. The storage medium is configured to store program code for performing the following steps: after using the second discriminator to perform tag discrimination on the weakly amplified video and generating a second discrimination result, the weakly amplified video is strongly amplified to obtain a strongly amplified video. The strongly amplified video refers to color enhancement and shape enhancement on the weakly amplified video. The strongly amplified video includes at least one of the following: brightness change, color change, contrast change, equalization, tone separation, horizontal translation, rotation, sharpening, horizontal cropping, vertical cropping, exposure, vertical translation, and cropping; the third discriminator is used to perform tag discrimination on the strongly amplified video to generate a third discrimination result.

[0147] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0148] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0152] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0153] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A model training method, characterized in that, include: Obtain a preset set of videos with image quality defects, wherein the preset set of videos includes tagged videos and untagged videos, and the tagged videos are indicated by true defect labels to indicate defect features; A preset generator is used to extract defect features from the labeled and unlabeled videos to generate a defect feature video. The preset generator is trained using machine learning with multiple sets of data. Each set of data includes the labeled video and the true defect labels of the labeled video. The preset generator is used to add pseudo-defect labels to the image quality defects in the unlabeled video to generate a pseudo-labeled video, and to extract defect features from the labeled and pseudo-labeled videos to generate the defect feature video. The defect feature video indicates the defect features through preset defect labels, which include the true defect labels and the pseudo-defect labels. The preset discriminator is used to perform label identification on the preset defect labels in the defect feature video, and an identification result is generated, wherein the identification result is used to indicate the defect feature video that passes the identification of the true defect label and the false defect label; The initial video detection model is trained using a preset loss function and the identification result to generate a target video detection model, wherein the preset loss function is used to estimate the degree of difference between the predicted value and the true value of the target video detection model. Specifically, a preset generator is used to extract defect features from the labeled and unlabeled videos, generating defect feature videos including: Extract multiple labeled video frames and the true defect label corresponding to each labeled video frame from the labeled video; Each labeled video frame and its corresponding true defect label are determined as a set of training data; The initial generator is trained using multiple sets of the training data to generate the preset generator; The preset generator is used to add pseudo-defect tags to video frames with image quality defects in the unlabeled video to generate pseudo-labeled videos. The defect feature video is generated based on the pseudo-labeled video and the labeled video; The generation of the defect feature video based on the pseudo-labeled video and the labeled video includes: Extract the defect features indicated by the true defect labels from the labeled video; Extract the defect features indicated by the pseudo-defect labels from the pseudo-labeled video; The defect features indicated by the true defect label and the defect features indicated by the false defect label are spliced ​​together to generate the defect feature video.

2. The method according to claim 1, characterized in that, Defect features are extracted from the labeled and unlabeled videos using a preset generator, resulting in defect feature videos including: The labeled video and the pseudo-labeled video are processed based on the convolutional layer, self-attention module and residual module in the preset generator to determine high-dimensional temporal features; The high-dimensional temporal features are processed based on the dilated convolutional bridge, transposed convolutional layer, and convolutional layer in the preset generator to determine the defect feature video.

3. The method according to claim 1, characterized in that, The preset discriminator includes at least one first discriminator, and the discrimination result includes at least one first discrimination result. The preset discriminator is used to perform label discrimination on the preset defect labels in the defect feature video, generating discrimination results including: The first discriminator is used to identify the preset defect labels in the defect feature video, and the first identification result is generated.

4. The method according to claim 3, characterized in that, The preset discriminator includes at least a second discriminator, and the discrimination result includes at least a second discrimination result. After using the first discriminator to perform tag discrimination on the defect feature video and generating the first discrimination result, the method further includes: The defect feature video is weakly augmented to obtain a weakly augmented video, wherein the weak augmentation includes at least one of the following: translation, rotation, and flipping; The second discriminator is used to identify the tags of the weakly amplified video, generating the second identification result.

5. The method according to claim 4, characterized in that, The preset discriminator includes at least a third discriminator, and the discrimination result includes at least a third discrimination result. After using the second discriminator to perform tag discrimination on the weakly amplified video and generating the second discrimination result, the method further includes: The weakly amplified video is subjected to strong amplification to obtain a strongly amplified video. The strong amplification refers to color enhancement and shape enhancement of the weakly amplified video. The strong amplification includes at least one of the following: brightness change, color change, contrast change, equalization, tone separation, horizontal translation, rotation, sharpening, horizontal cropping, vertical cropping, exposure adjustment, vertical translation, and cropping. The third discriminator is used to identify the tags of the strongly amplified video, and the third identification result is generated.

6. A model training device, characterized in that, include: The acquisition module is used to acquire a preset set of videos with image quality defects, wherein the preset set of videos includes tagged videos and untagged videos, and the tagged videos are indicated by true defect labels to indicate defect features; A generation module is used to extract defect features from the labeled video and the unlabeled video using a preset generator to generate a defect feature video. The preset generator is trained using multiple sets of data through machine learning. Each set of data includes the labeled video and its true defect labels. The preset generator is used to add pseudo-defect labels to the image quality defects in the unlabeled video to generate a pseudo-labeled video, and to extract defect features from the labeled video and the pseudo-labeled video to generate the defect feature video. The defect feature video indicates defect features through preset defect labels, which include both true defect labels and pseudo-defect labels. An identification module is used to identify the preset defect labels in the defect feature video using a preset discriminator and generate an identification result, wherein the identification result is used to indicate the defect feature video that passes the identification of the true defect labels and the false defect labels; The training module is used to train the initial video detection model using a preset loss function and the discrimination result to generate a target video detection model, wherein the preset loss function is used to estimate the degree of difference between the predicted value and the true value of the target video detection model. The generation module includes: An extraction unit is used to extract multiple labeled video frames and a true defect label corresponding to each labeled video frame from the labeled video. The determining unit is used to determine each labeled video frame and its corresponding true defect label as a set of training data; A training unit is used to train the initial generator using multiple sets of the training data to generate the preset generator; The addition unit is used to add pseudo-defect tags to video frames with image quality defects in the unlabeled video using the preset generator, thereby generating pseudo-labeled videos. The generation unit is used to generate the defect feature video based on the pseudo-labeled video and the labeled video; The generation unit includes: The first extraction subunit is used to extract the defect features indicated by the true defect label from the labeled video; The second extraction subunit is used to extract the defect features indicated by the pseudo-defect label from the pseudo-label video; The generation unit is used to stitch together the defect features indicated by the true defect label and the defect features indicated by the false defect label to generate the defect feature video.

7. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a program, wherein when the program is executed, it controls the device where the non-volatile storage medium is located to execute the model training method according to any one of claims 1 to 5.

8. A processor, characterized in that, The processor is used to run a program, wherein the program executes the model training method according to any one of claims 1 to 5 when it runs.

Images