A method, device, equipment and storage medium for identifying a flower screen image

Abstract

The present application discloses a method, device, equipment and storage medium for identifying a flower screen image, which can be applied to the field of maps. By obtaining an image sequence corresponding to a target video; inputting the video image in the image sequence into a target network model to obtain the flower screen confidence corresponding to the video image, the target network model is trained based on labeled data and unlabeled data, and the unlabeled data is configured with a pseudo-label determined based on the first image data and the second image data; and then the flower screen image in the target video is determined according to the flower screen confidence. Thus, a process of model training with only a small number of labeled images and unlabeled images is achieved. Since the model prediction consistency of unlabeled data and the generation of pseudo-labels are utilized, the labeled data set is expanded, and the accuracy of flower screen image recognition is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for identifying a flower screen image. Background Art

[0002] With the rapid development of Internet technology, people have higher and higher requirements for video content. Screen distortion detection is a way to improve video quality.

[0003] Generally, for the process of screen distortion detection, manual features can be extracted from the input image first, and then a classifier can be trained based on the extracted manual features to perform binary classification on whether the current image is a screen distortion image.

[0004] However, the generalization of manual features is relatively poor, and for complex scenes, it may not be able to cover all features, affecting the accuracy of flower screen image recognition. Summary of the invention

[0005] In view of this, the present application provides a method for recognizing a flower screen image, which can effectively improve the accuracy of flower screen image recognition.

[0006] The first aspect of the present application provides a method for identifying a distorted screen image, which can be applied to a system or program including a distorted screen image identification function in a terminal device, and specifically includes:

[0007] Get the image sequence corresponding to the target video;

[0008] Inputting the video image in the image sequence into a target network model to obtain a screen distortion confidence corresponding to the video image, wherein the target network model is trained based on labeled data and unlabeled data, the unlabeled data is configured with a pseudo label determined based on first image data and second image data, the first image data includes a plurality of first processed images in series adjusted based on the training image, the second image data includes a second processed image adjusted based on the training image, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image;

[0009] The distorted screen image in the target video is determined according to the distorted screen confidence.

[0010] Optionally, in some possible implementations of the present application, the method further includes:

[0011] Acquire the unlabeled training image;

[0012] Performing a first adjustment operation on the training image to obtain a first serial image;

[0013] performing the first adjustment operation on the first serial images to obtain a second serial image;

[0014] Associating the first serial image and the second serial image to form a plurality of first processed images in series, wherein the first adjustment operation is used to indicate an adjustment direction that is similar to the training image;

[0015] performing a second adjustment operation on the training image to obtain the second processed image, wherein the first adjustment operation is used to indicate an adjustment direction different from that of the training image;

[0016] Inputting a plurality of the first processed images into a first model to obtain first training parameters;

[0017] Inputting the second processed image into a second model to obtain second training parameters, wherein the first model and the second model share weight parameters;

[0018] Weighting the first training parameter and the second training parameter to obtain a target parameter;

[0019] Perform a sharpening operation based on the target parameter to determine a pseudo label corresponding to the training image;

[0020] The first model is trained according to the training image configured with the pseudo label and the annotation data to obtain the target network model.

[0021] Optionally, in some possible implementations of the present application, inputting the plurality of first processed images into a first model to obtain a first training parameter includes:

[0022] Inputting a plurality of the first processed images into a first model to obtain a parameter sequence for indicating a screen distortion probability corresponding to the first processed images;

[0023] An averaging operation is performed on the parameter sequence to obtain the first training parameter.

[0024] Optionally, in some possible implementations of the present application, after performing an averaging operation on the parameter sequence to obtain the first training parameter, the method further includes:

[0025] Acquire consistency loss information corresponding to the first model based on the parameter sequence;

[0026] The parameters of the first model are updated based on the consistency loss information.

[0027] Optionally, in some possible implementations of the present application, performing a sharpening operation based on the target parameter to determine a pseudo label corresponding to the training image includes:

[0028] Obtaining distribution information corresponding to the one-hot encoding method adopted by the first model;

[0029] determining a sharpening factor based on the distribution information;

[0030] A sharpening operation is performed on the target parameter according to the sharpening coefficient to determine a pseudo label corresponding to the training image.

[0031] Optionally, in some possible implementations of the present application, the training of the first model according to the training image configured with the pseudo label and the annotated data to obtain the target network model includes:

[0032] Acquire corresponding first loss information based on the labeled data;

[0033] Analyzing the consistency of the probability estimation corresponding to the pseudo-label to obtain second loss information;

[0034] Perform supervised training based on the pseudo-label to obtain third loss information;

[0035] weighting the first loss information, the second loss information, and the third loss information to obtain target loss information;

[0036] The first model is trained based on the target loss information to obtain the target network model.

[0037] Optionally, in some possible implementations of the present application, the method further includes:

[0038] Acquire the data volume corresponding to the annotation data associated with the target video;

[0039] If the data amount corresponding to the labeled data does not reach a preset ratio, calling the video frame image in the target video;

[0040] Using the video frame image as associated unlabeled data;

[0041] The target network model is trained based on the labeled data and the associated unlabeled data, the associated unlabeled data is configured with a pseudo label determined based on third image data and fourth image data, the third image data is a serial plurality of third processed images obtained by successively performing image adjustment operations on the video frame image, the fourth image data includes a fourth processed image obtained based on the adjustment of the video frame image, and the adjustment amplitude corresponding to the fourth processed image is greater than the adjustment amplitude corresponding to the third processed image.

[0042] The second aspect of the present application provides a device for identifying a distorted screen image, comprising:

[0043] An acquisition unit, used for acquiring an image sequence corresponding to a target video;

[0044] An input unit is used to input the video image in the image sequence into a target network model to obtain a screen distortion confidence corresponding to the video image, wherein the target network model is trained based on labeled data and unlabeled data, the unlabeled data is configured with a pseudo label determined based on first image data and second image data, the first image data is a serial plurality of first processed images obtained by successively performing image adjustment operations on the training image, the second image data includes a second processed image obtained by adjusting the training image, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image;

[0045] The recognition unit is used to determine the distorted screen image in the target video according to the distorted screen confidence.

[0046] Optionally, in some possible implementations of the present application, the recognition unit is specifically used to obtain the unlabeled training image;

[0047] The recognition unit is specifically configured to perform a first adjustment operation on the training image to obtain a first serial image;

[0048] The identification unit is specifically configured to perform the first adjustment operation on the first serial image to obtain a second serial image;

[0049] The recognition unit is specifically used to associate the first serial image and the second serial image to form a plurality of first processed images in series, and the first adjustment operation is used to indicate an adjustment direction that is similar to the training image;

[0050] The recognition unit is specifically configured to perform a second adjustment operation on the training image to obtain the second processed image, wherein the first adjustment operation is used to indicate an adjustment direction different from that of the training image;

[0051] The recognition unit is specifically configured to input the plurality of the first processed images into a first model to obtain a first training parameter;

[0052] The recognition unit is specifically configured to input the second processed image into a second model to obtain second training parameters, and the first model and the second model share weight parameters;

[0053] The identification unit is specifically configured to weight the first training parameter and the second training parameter to obtain a target parameter;

[0054] The recognition unit is specifically configured to perform a sharpening operation based on the target parameter to determine a pseudo label corresponding to the training image;

[0055] The recognition unit is specifically used to train the first model according to the training image configured with the pseudo label and the annotation data to obtain the target network model.

[0056] Optionally, in some possible implementations of the present application, the recognition unit is specifically used to input the plurality of the first processed images into a first model to obtain a parameter sequence indicating a screen distortion probability corresponding to the first processed images;

[0057] The identification unit is specifically configured to perform an averaging operation on the parameter sequence to obtain the first training parameter.

[0058] Optionally, in some possible implementations of the present application, the identification unit is specifically used to obtain consistency loss information corresponding to the first model based on the parameter sequence;

[0059] The identification unit is specifically used to update the parameters of the first model based on the consistency loss information.

[0060] Optionally, in some possible implementations of the present application, the identification unit is specifically used to obtain distribution information corresponding to the one-hot encoding method adopted by the first model;

[0061] The identification unit is specifically configured to determine a sharpening coefficient based on the distribution information;

[0062] The recognition unit is specifically configured to perform a sharpening operation on the target parameter according to the sharpening coefficient to determine a pseudo label corresponding to the training image.

[0063] Optionally, in some possible implementations of the present application, the identification unit is specifically configured to obtain corresponding first loss information based on the labeled data;

[0064] The identification unit is specifically used to analyze the consistency of the probability estimation corresponding to the pseudo label to obtain the second loss information;

[0065] The recognition unit is specifically used to perform supervised training based on the pseudo label to obtain third loss information;

[0066] The identification unit is specifically configured to weight the first loss information, the second loss information and the third loss information to obtain target loss information;

[0067] The identification unit is specifically used to train the first model based on the target loss information to obtain the target network model.

[0068] Optionally, in some possible implementations of the present application, the identification unit is specifically used to obtain a data volume corresponding to the annotation data associated with the target video;

[0069] The recognition unit is specifically configured to call a video frame image in the target video if the amount of data corresponding to the labeled data does not reach a preset ratio;

[0070] The recognition unit is specifically configured to use the video frame image as associated unlabeled data;

[0071] The recognition unit is specifically used to train the target network model based on the labeled data and the associated unlabeled data, the associated unlabeled data is configured with a pseudo label determined based on third image data and fourth image data, the third image data is a serial plurality of third processed images obtained by successively performing image adjustment operations on the video frame image, the fourth image data includes a fourth processed image obtained based on the adjustment of the video frame image, and the adjustment amplitude corresponding to the fourth processed image is greater than the adjustment amplitude corresponding to the third processed image.

[0072] The third aspect of the present application provides a computer device, including: a memory, a processor and a bus system; the memory is used to store program code; the processor is used to execute the method for recognizing a flower screen image as described in the first aspect or any one of the first aspects according to the instructions in the program code.

[0073] The fourth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the method for recognizing a flower screen image as described in the first aspect or any one of the first aspects.

[0074] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for recognizing a distorted screen image provided in the first aspect or various optional implementations of the first aspect.

[0075] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0076] By obtaining an image sequence corresponding to a target video; then inputting the video images in the image sequence into a target network model to obtain the screen noise confidence corresponding to the video images, wherein the target network model is trained based on labeled data and unlabeled data, the unlabeled data is configured with a pseudo-label determined based on the first image data and the second image data, the first image data includes a plurality of first processed images in series adjusted based on the training image, the second image data includes a second processed image adjusted based on the training image, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image; and then determining the screen noise image in the target video according to the screen noise confidence. Thus, a process of model training can be realized with only a small amount of manually labeled images and a large amount of unlabeled images, and the model prediction consistency of the unlabeled data and the generation of pseudo-labels are utilized to expand the labeled data set and improve the accuracy of screen noise image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0078] Figure 1 The network architecture diagram for the recognition system of flower screen images;

[0079] Figure 2 A flow chart of the recognition of a flower screen image provided in an embodiment of the present application;

[0080] Figure 3 A flowchart of a method for identifying a flower screen image provided in an embodiment of the present application;

[0081] Figure 4 A schematic diagram of a scene of a method for identifying a flower screen image provided in an embodiment of the present application;

[0082] Figure 5 A schematic diagram of another scene of a method for identifying a flower screen image provided in an embodiment of the present application;

[0083] Figure 6 A schematic diagram of another scene of a method for identifying a flower screen image provided in an embodiment of the present application;

[0084] Figure 7 A flowchart of another method for identifying a flower screen image provided in an embodiment of the present application;

[0085] Figure 8A schematic diagram of the structure of a device for identifying a flower screen image provided in an embodiment of the present application;

[0086] Fig. 9 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0087] Fig.10 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0088] The embodiment of the present application provides a method for identifying a distorted screen image and a related device, which can be applied to a system or program in a terminal device that includes a distorted screen image recognition function, by obtaining an image sequence corresponding to a target video; then inputting the video image in the image sequence into a target network model to obtain a distorted screen confidence corresponding to the video image, wherein the target network model is trained based on labeled data and unlabeled data, the unlabeled data is configured with a pseudo-label determined based on the first image data and the second image data, the first image data includes a serial plurality of first processed images adjusted based on the training image, the second image data includes a second processed image adjusted based on the training image, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image; and then determining the distorted screen image in the target video according to the distorted screen confidence. Thus, a process of model training can be achieved with only a small number of manually labeled images and a large number of unlabeled images, and the accuracy of distorted screen image recognition is improved by utilizing the model prediction consistency of the unlabeled data and the generation of pseudo-labels, thereby expanding the labeled data set.

[0089] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0090] First, some terms that may appear in the embodiments of the present application are explained.

[0091] One-Hot Encoding: Also known as one-bit effective encoding, the method is to use an N-bit state register to encode N states. Each state has its own independent register bit, and at any time, only one of them is valid.

[0092] It should be understood that the method for identifying a distorted screen image provided in the present application can be applied to a system or program in a terminal device that includes a distorted screen image recognition function, such as a video player. Specifically, the distorted screen image recognition system can be run on a system such as a video player. Figure 1 In the network architecture shown in Figure 1 As shown in the figure, it is a network architecture diagram of the operation of the flower screen image recognition system. As can be seen from the figure, the flower screen image recognition system can provide a flower screen image recognition process with multiple information sources, that is, through the trigger operation on the terminal side, the flower screen recognition is performed on the video sent by the server, thereby improving the video playback quality; it can be understood that Figure 1 A variety of terminal devices are shown in FIG. 1 . The terminal device may be a computer device. In actual scenarios, more or fewer types of terminal devices may be involved in the process of identifying the flower screen image. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 One server is shown in the figure, but in actual scenarios, multiple servers may be involved, especially in scenarios of multi-video interaction, and the specific number of servers depends on the actual scenario.

[0093] In this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, smart voice interaction device, smart home appliance, car terminal, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, and the terminal and the server can be connected to form a blockchain network, which is not limited in this application.

[0094] It can be understood that the above-mentioned flower screen image recognition system can run on a personal mobile terminal, for example: as an application such as a video player, it can also run on a server, and it can also run on a third-party device to provide flower screen image recognition to obtain the recognition processing result of the flower screen image of the information source; the specific flower screen image recognition system can be run in the above-mentioned device in the form of a program, or it can be run as a system component in the above-mentioned device, and it can also be used as a cloud service program. This embodiment can be applied to cloud technology, autonomous driving and other scenarios. The specific operation mode depends on the actual scenario and is not limited here.

[0095] With the rapid development of Internet technology, people have higher and higher requirements for video content. Screen distortion detection is a way to improve video quality.

[0096] Generally, for the process of screen distortion detection, manual features can be extracted from the input image first, and then a classifier can be trained based on the extracted manual features to perform binary classification on whether the current image is a screen distortion image.

[0097] However, the generalization of manual features is relatively poor, and for complex scenes, it may not be able to cover all features, affecting the accuracy of flower screen image recognition.

[0098] In order to solve the above problems, this application proposes a method for identifying a flower screen image. Figure 2 In the process framework of the recognition of the flower screen image shown in Figure 2 As shown, it is a flow architecture diagram of the recognition of a flower screen image provided by an embodiment of the present application. The user determines the corresponding video through the playback operation of the terminal, so that the server performs flower screen detection on the video. Flower screen detection refers to the detection of whether a picture has a flower screen phenomenon. The final output result is the probability that the current image is a flower screen image, and further, the binary result of whether the current image is a flower screen image can be output. This task is a necessary step in the quality analysis of pictures and videos, and can be used to evaluate the quality of the current picture or video.

[0099] This embodiment expands the training images to configure pseudo labels and then trains the target network model. It is a semi-supervised flower screen detection algorithm based on serial image enhancement. Only a small amount of flower screen labeled data and a certain amount of training data without flower screen labeling are needed to complete the model training. The trained model can directly classify flower screens or normal images with high precision and achieve good classification accuracy.

[0100] It can be understood that the method provided by the present application can be written as a program to serve as a processing logic in a hardware system, or as a device for identifying a flower screen image, and the above processing logic is implemented in an integrated or external manner. As an implementation method, the device for identifying a flower screen image obtains an image sequence corresponding to a target video; then the video image in the image sequence is input into a target network model to obtain a flower screen confidence corresponding to the video image, wherein the target network model is trained based on labeled data and unlabeled data, the unlabeled data is configured with a pseudo label determined based on the first image data and the second image data, the first image data includes a plurality of first processed images in series adjusted based on the training image, the second image data includes a second processed image adjusted based on the training image, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image; and then the flower screen image in the target video is determined according to the flower screen confidence. Thus, a process of model training can be realized with only a small number of manually labeled images and a large number of unlabeled images. Since the model prediction consistency of the unlabeled data and the generation of pseudo labels are utilized, the labeled data set is expanded, and the accuracy of flower screen image recognition is improved.

[0101] The solution provided in the embodiments of the present application involves artificial intelligence computer vision technology, which is specifically described by the following embodiments:

[0102] Combined with the above process architecture, the following will introduce the method for identifying the flower screen image in this application. Please refer to Figure 3 , Figure 3 A flowchart of a method for identifying a distorted screen image provided in an embodiment of the present application. The management method can be executed by a server or a terminal. The embodiment of the present application includes at least the following steps:

[0103] 301. Obtain an image sequence corresponding to a target video.

[0104] In this embodiment, the target video may be a video in the user's playlist, a video in a cloud video database, or a video played in real time. The specific video type depends on the actual scenario.

[0105] Specifically, the image sequence may be images obtained by parsing the target video frames. The specific video frame interval may depend on the playback configuration of the player. For example, the higher the playback frame rate, the smaller the interval between the video frames, and the greater the number of images in the obtained image sequence.

[0106] 302. Input the video images in the image sequence into the target network model to obtain the screen noise confidence corresponding to the video images, the target network model is trained based on the labeled data and the unlabeled data, the unlabeled data is configured with a pseudo label determined based on the first image data and the second image data, the first image data is a serial of multiple first processed images obtained by successively performing image adjustment operations on the training images, the second image data includes a second processed image obtained by adjusting the training images, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image.

[0107] In this embodiment, the training process of sample expansion is based on unlabeled data. Since the image of the flower screen has a gradual characteristic, the specific example is Figure 4 As shown, Figure 4 A scene schematic diagram of a method for identifying a flower screen image provided in an embodiment of the present application; the figure shows flower screen images of different degrees, and it can be seen that the flower screen image is obtained by gradually adjusting and processing the normal image, so the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image, which simulates the sample generation process of the flower screen image.

[0108] Specifically, the semi-supervised screen blur detection algorithm framework based on serial image enhancement is as follows: Figure 5 As shown, Figure 5 A schematic diagram of a scenario of another method for identifying a flower screen image provided by an embodiment of the present application; that is, for an input image x without an artificial label, two types of data enhancement modes are used respectively, that is, firstly, an unlabeled training image is obtained; then a first adjustment operation (simple enhancement) is performed on the training image to obtain a first serial image; further, a first adjustment operation is performed on the first serial image to obtain a second serial image; and the first serial image and the second serial image are associated to serve as a plurality of first processed images in a serial manner, and the first adjustment operation is used to indicate an adjustment direction that is close to the training image; then a second adjustment operation (complex enhancement) is performed on the training image. , to obtain a second processed image, the second adjustment operation is used to indicate an adjustment direction different from the training image; then multiple first processed images are input into the first model to obtain first training parameters; and the second processed images are input into the second model to obtain second training parameters, and the first model and the second model share weight parameters to ensure the effect of training different samples; further weight the first training parameters and the second training parameters to obtain target parameters; and perform a sharpening operation based on the target parameters to determine the pseudo-label corresponding to the training image; and then train the first model according to the training images and annotation data configured with pseudo-labels to obtain a target network model.

[0109] It can be understood that, in the process of performing the first adjustment operation, the above example performs the first adjustment operation twice; in actual scenarios, more times of the first adjustment operation may be performed, for example, if there are 5 serial first processed images, the specific processing process is to perform a first simple enhancement on the training image to obtain a serial image A1; further perform a second simple enhancement on the serial image A1 to obtain a serial image A2; then perform a second simple enhancement on the serial image A2 to obtain a serial image A3; and perform a second simple enhancement on the serial image A3 to obtain a serial image A4, and then perform a second simple enhancement on the serial image A4 to obtain a serial image A5, so that the multiple first processed images obtained are A1-A5, and the specific number depends on the actual scenario and is not limited here.

[0110] It should be noted that the specific methods used in the simple enhancement process can be the same or different. The specific simple enhancement can include: linear transformation, contrast transformation, brightness transformation, image equalization, image flipping (horizontal and vertical directions) and other operations. In actual scenarios, it can be a combination of any of the above methods, which is not limited here.

[0111] Specifically, the first adjustment operation is used to indicate the adjustment direction that is similar to the training image; wherein, the adjustment direction that is similar to the training image means that the texture, spatial arrangement, content and other information of the image are relatively intact, and the adjusted image is similar to the image before the adjustment; in a possible scenario, the first adjustment operation is a simple enhancement, which may include: linear transformation, contrast transformation, brightness transformation, image equalization, image flipping (horizontal and vertical directions) and other operations; correspondingly, the second adjustment operation is used to indicate the adjustment direction that is different from the training image, that is, a large-scale adjustment of the original image, and the complex enhancement may include: random erasing, large-angle image rotation, adding complex noise, etc. The main difference between the two is that the degree of image enhancement is inconsistent. For simple image enhancement, the texture, spatial arrangement, content and other information of the image are relatively intact, while complex image enhancement often means a large-scale adjustment of the original image, such as random erasing, which will assign values ​​to image pixels in a certain area, so that the pixel values ​​(image content) of the assigned area are lost. Simple enhancement is performed N times, while complex enhancement is performed only once. The purpose of this is to enhance the consistency supervision of the features after N simple enhancements, which is equivalent to a regularization term that can stabilize the network training and improve the generalization ability of the model. For the N+1 inputs after both enhancements, through a deep neural network CNN model with shared weight values, the final output probabilities P1, P2…P N and P c , where P1, P2…P NThe consistency loss is calculated. The principle is to hope that after N times of simple enhancement of the same image, the same CNN model can obtain relatively consistent prediction results. N An average operation is performed, that is, a plurality of first processed images are input into the first model to obtain a parameter sequence; then an average operation is performed on the parameter sequence to obtain a first training parameter. Then, the first training parameter is obtained by adding P c By performing weighted fusion, we can obtain the output probability fusion result corresponding to all enhanced input data, and then perform probability sharpening operation on the fused probability. The purpose of this operation is to sharpen the probability to a unique hot encoding distribution closer to one-hot (more consistent with the result of manual labeling). The sharpened probability is directly used as the pseudo label of the current image. Finally, the input image x without manual labels obtains a reliable pseudo label, which is equivalent to giving manual labels to some unlabeled data. Therefore, the CNN model can be further trained using supervised learning until the network converges. Therefore, a small amount of labeled data and a large amount of unlabeled data can be used for training, and finally a robust CNN model can be obtained.

[0112] It can be understood that in this embodiment, the target network model is a CNN model for example only, and the specific network structure is not limited. The more mature CNN model in the academic community can be used as the structure of the deep network.

[0113] In addition, the simple enhancement and complex enhancement methods are not limited to the ones described above, and more complex enhancement methods can be designed to further improve the effect.

[0114] In one possible scenario, for the process of generating pseudo labels for simple and complex image enhancement, the consistency loss calculation process can be performed, that is, the consistency loss information corresponding to the first model is obtained based on the parameter sequence; then the parameters of the first model are updated based on the consistency loss information. Among them, simple image enhancement will be performed serially N times to obtain x1, x2...x N , while complex image enhancement is performed only once to obtain x c , the input after N simple enhancements will pass through the CNN model to obtain N probability outputs P1, P2…P N , a consistency loss is imposed on these N probability outputs:

[0115]

[0116] Among them, x1, x2,…x N A sequence of images representing simple image augmentation.

[0117] It is understandable that this consistency loss will cause the network to move towards P1, P2…P NThe principle is that for simple image enhancement, the output of the CNN model should be highly consistent, which provides the CNN model with a supervisory signal for parameter update. For complex image enhancement, the probability output P obtained by the CNN model is as close as possible because the image texture, spatial information, content information, etc. are destroyed. c Possibly with P1, P2…P N The difference is relatively large and is not suitable for consistency loss. However, the complex enhanced images are helpful for the generalization of the network and can prevent the CNN model from overfitting to the local optimal solution.

[0118] In addition, the sharpening operation is intended to make the target parameters more representative, that is, firstly obtain the distribution information corresponding to the one-hot encoding method (one-hot) adopted by the first model, so as to match the probability distribution information after the sharpening operation with the encoding method, that is, to sharpen the probability to be closer to the one-hot encoding distribution, which is more in line with the result of manual labeling; therefore, the sharpening coefficient can be determined based on the distribution information; and the sharpening operation is performed on the target parameter according to the sharpening coefficient to determine the pseudo-label corresponding to the training image.

[0119] It is understandable that when configuring pseudo labels, the probabilities P1, P2, ... P after simple enhancement are considered at the same time. N And the probability P after complex enhancement c :

[0120] P sofy =((P1+P2+…+P N ) / N+P c ) / 2

[0121] Among them, P soft The probability distribution information of the image after simple enhancement and complex enhancement is integrated, which can be used to reflect the potential label of the current input image x, but P soft The distribution is often not sharp enough, for example (0.6, 0.4), which is inconsistent with the one-hot label (1, 0) given by the manual label. soft Perform probabilistic sharpening:

[0122]

[0123] Among them, p i P soft The corresponding probability distribution; p j is the probability distribution after the sharpening operation; T∈(0,1) is the sharpening coefficient, which indicates the degree of sharpening. The smaller T is, the higher the P hard The closer it is to the one-hot form. The final P hardIt serves as the pseudo label of the current image and is used for training the CNN model in the subsequent stage.

[0124] Below, the configuration of the loss function of the target network model is explained, including the loss function configuration of three dimensions: labeled data, parameter sequence consistency, and unlabeled data. That is, first, the corresponding first loss information is obtained based on the labeled data; then the consistency of the probability estimate corresponding to the pseudo-label is analyzed to obtain the second loss information; and supervised training is performed based on the pseudo-label to obtain the third loss information; then the first loss information, the second loss information, and the third loss information are weighted to obtain the target loss information; and then the first model is trained based on the target loss information to obtain the target network model.

[0125] Specifically, for labeled data (annotated data), the corresponding loss function uses cross entropy loss:

[0126] L S (x s ,y s )=-[y s *log(f(x s ,θ))+(1-y s )*log(1-(f(x s ,θ))

[0127] Among them, x s represents the labeled samples, y s is its corresponding label, f represents the nonlinear operation of CNN, and θ is the parameter of f, that is, the weight value that needs to be updated.

[0128] In addition, for unlabeled data, in addition to the above consistency loss L e , using the generated pseudo-label P hard The training of the network can also be supervised in the following form:

[0129] L u (x u , P hard )=-[P hard *log(f(x u ,θ))+(1-P hard )*log(1-(f(x u ,θ))

[0130] Therefore, the objective loss function can be expressed as:

[0131] L=L S +αL u +βL e

[0132] Among them, α and β are used to control the weights of the corresponding two loss functions.

[0133] It is understandable that the choice of the above loss function is not necessarily limited to the cross entropy loss. Other loss functions, including L1 loss, L2 loss or JS divergence, can be used as specific implementation forms of the loss function.

[0134] This embodiment only relies on a training method with a small number of annotated images. The semi-supervised flower screen detection algorithm based on serial image enhancement proposed by the present invention only requires a small number of manually annotated images and a large number of unannotated images for training, reducing the cost of manual annotation. Its principle is to use the CNN model of unannotated data to predict consistency and generate pseudo labels, thereby expanding the annotated data set. Due to the sufficient number of training samples, the training effect of the CNN model can be improved.

[0135] 303. Determine the distorted screen image in the target video according to the distorted screen confidence.

[0136] In this embodiment, the screen distorting confidence is used to indicate the probability that the image belongs to the screen distorting image. The specific recognition result can be as follows: Figure 6 As shown, Figure 6 A scene diagram of another method for identifying a flower screen image provided in an embodiment of the present application; the figure shows that the flower screen confidence of the identified flower screen image is 0.99, and the flower screen confidence of the normal image is 0.05.

[0137] This embodiment can be used as a video analysis function to directly output the confidence level of a single image being a flower screen, and then obtain the final detection result of whether it is a flower screen (flower screen or normal), thereby indicating further video processing operations to improve the quality of the video.

[0138] In combination with the above embodiments, it can be known that by obtaining an image sequence corresponding to a target video; then inputting the video images in the image sequence into a target network model to obtain the screen noise confidence corresponding to the video images, wherein the target network model is trained based on labeled data and unlabeled data, the unlabeled data is configured with a pseudo-label determined based on the first image data and the second image data, the first image data includes a plurality of first processed images in series adjusted based on the training image, the second image data includes a second processed image adjusted based on the training image, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image; and then determining the screen noise image in the target video according to the screen noise confidence. Thus, a process of model training can be realized with only a small amount of manually labeled images and a large amount of unlabeled images, and the accuracy of screen noise image recognition is improved by utilizing the model prediction consistency of unlabeled data and the generation of pseudo-labels, thereby expanding the labeled data set.

[0139] In one possible scenario, it is possible to determine whether the annotated data corresponding to the target video meets the requirements, so as to supplement the data. The following is an explanation of this scenario. Figure 7 , Figure 7 A flowchart of another method for identifying a distorted screen image provided in an embodiment of the present application, wherein the embodiment of the present application at least includes the following steps:

[0140] 701. Obtain a data volume corresponding to the annotation data associated with the target video.

[0141] In this embodiment, during the recognition process of the target video, the data amount corresponding to the labeled data may be judged to determine whether it is necessary to supplement the unlabeled data to ensure the accuracy of the recognition model.

[0142] 702. If the data volume corresponding to the labeled data does not reach a preset ratio, the video frame image in the target video is called, and the video frame image is used as the associated unlabeled data.

[0143] In this embodiment, the preset ratio may be the ratio of the amount of data corresponding to the labeled data to all training data, or may be a ratio parameter dynamically configured according to the bit rate of the target video. For example, the higher the bit rate, the higher the configuration of the preset ratio.

[0144] In a possible scenario, the video frame images in the target video may be video frame images in the target video that are marked as frequently displaying blur, such as plot key points, the beginning, etc., thereby improving the effectiveness of associating unlabeled data.

[0145] 703. Train the target network model based on the labeled data and the associated unlabeled data.

[0146] In this embodiment, the associated unlabeled data is the unlabeled data associated with the target video, such as the unlabeled data obtained based on the video frames in the target video. The video frames in the target video can be automatically selected or can be popular video frames, such as video frames with a large number of barrages, thereby improving the effectiveness of the unlabeled data.

[0147] In addition, the specific training process is as follows Figure 3 As shown in the description of step 302, no further details are given here. That is, the associated unlabeled data is configured with a pseudo label determined based on the third image data and the fourth image data, the third image data is a serial plurality of third processed images obtained by successively performing image adjustment operations on the video frame image (that is, the third image data is a serial image after simple enhancement of the video frame image), the fourth image data includes a fourth processed image obtained based on the video frame image adjustment (that is, the fourth image data is an image after complex enhancement of the video frame image), and the adjustment amplitude corresponding to the fourth processed image is greater than the adjustment amplitude corresponding to the third processed image.

[0148] This embodiment proposes a semi-supervised flower screen detection algorithm based on serial image enhancement, which only requires a small amount of flower screen labeled data and a certain amount of training data without flower screen labeling to complete the model training. The trained model can directly classify flower screens or normal images with high precision and achieve good classification accuracy, greatly reducing the manpower cost consumed by labeling.

[0149] In order to better implement the above solution of the embodiment of the present application, the following also provides related devices for implementing the above solution. Figure 8 , Figure 8 The structural diagram of a device for identifying a distorted screen image provided in an embodiment of the present application is as follows. The device for identifying a distorted screen image 800 includes:

[0150] An acquisition unit 801 is used to acquire an image sequence corresponding to a target video;

[0151] An input unit 802 is used to input the video image in the image sequence into a target network model to obtain a screen distortion confidence corresponding to the video image, wherein the target network model is trained based on labeled data and unlabeled data, the unlabeled data is configured with a pseudo label determined based on first image data and second image data, the first image data includes a serial plurality of first processed images adjusted based on the training image, the second image data includes a second processed image adjusted based on the training image, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image;

[0152] The identification unit 803 is used to determine the distorted screen image in the target video according to the distorted screen confidence.

[0153] Optionally, in some possible implementations of the present application, the recognition unit 803 is specifically configured to obtain the unlabeled training image;

[0154] The recognition unit 803 is specifically configured to perform a plurality of first adjustment operations on the training image to obtain a plurality of first processed images in series, wherein the first adjustment operation is used to indicate an adjustment direction that is similar to the training image;

[0155] The recognition unit 803 is specifically configured to perform a second adjustment operation on the training image to obtain the second processed image, wherein the first adjustment operation is used to indicate an adjustment direction different from that of the training image;

[0156] The recognition unit 803 is specifically configured to input the plurality of the first processed images into a first model to obtain a first training parameter;

[0157] The recognition unit 803 is specifically configured to input the second processed image into a second model to obtain a second training parameter, wherein the first model and the second model share a weight parameter;

[0158] The identification unit 803 is specifically configured to weight the first training parameter and the second training parameter to obtain a target parameter;

[0159] The recognition unit 803 is specifically configured to perform a sharpening operation based on the target parameter to determine a pseudo label corresponding to the training image;

[0160] The recognition unit 803 is specifically configured to train the first model according to the training image configured with the pseudo label and the annotation data to obtain the target network model.

[0161] Optionally, in some possible implementations of the present application, the recognition unit 803 is specifically configured to input the plurality of the first processed images into a first model to obtain a parameter sequence;

[0162] The identification unit 803 is specifically configured to perform an averaging operation on the parameter sequence to obtain the first training parameter.

[0163] Optionally, in some possible implementations of the present application, the identification unit 803 is specifically configured to obtain consistency loss information corresponding to the first model based on the parameter sequence;

[0164] The identification unit 803 is specifically configured to update the parameters of the first model based on the consistency loss information.

[0165] Optionally, in some possible implementations of the present application, the identification unit 803 is specifically used to obtain distribution information corresponding to the one-hot encoding method adopted by the first model;

[0166] The identification unit 803 is specifically configured to determine a sharpening coefficient based on the distribution information;

[0167] The identification unit 803 is specifically configured to perform a sharpening operation on the target parameter according to the sharpening coefficient to determine a pseudo label corresponding to the training image.

[0168] Optionally, in some possible implementations of the present application, the identification unit 803 is specifically configured to obtain corresponding first loss information based on the labeled data;

[0169] The identification unit 803 is specifically used to analyze the consistency of the probability estimation corresponding to the pseudo label to obtain the second loss information;

[0170] The recognition unit 803 is specifically configured to perform supervised training based on the pseudo label to obtain third loss information;

[0171] The identification unit 803 is specifically configured to weight the first loss information, the second loss information and the third loss information to obtain target loss information;

[0172] The identification unit 803 is specifically configured to train the first model based on the target loss information to obtain the target network model.

[0173] Optionally, in some possible implementations of the present application, the identification unit 803 is specifically configured to obtain a data volume corresponding to the annotation data associated with the target video;

[0174] The recognition unit 803 is specifically configured to call the video frame image in the target video if the data amount corresponding to the labeled data does not reach a preset ratio;

[0175] The recognition unit 803 is specifically configured to use the video frame image as associated unlabeled data;

[0176] The recognition unit 803 is specifically used to train the target network model based on the labeled data and the associated unlabeled data, the associated unlabeled data is configured with a pseudo label determined based on third image data and fourth image data, the third image data is a serial plurality of third processed images obtained by successively performing image adjustment operations on the video frame image, the fourth image data includes a fourth processed image obtained based on the adjustment of the video frame image, and the adjustment amplitude corresponding to the fourth processed image is greater than the adjustment amplitude corresponding to the third processed image.

[0177] By obtaining an image sequence corresponding to a target video; then inputting the video images in the image sequence into a target network model to obtain the screen noise confidence corresponding to the video images, wherein the target network model is trained based on labeled data and unlabeled data, the unlabeled data is configured with a pseudo-label determined based on the first image data and the second image data, the first image data includes a plurality of first processed images in series adjusted based on the training image, the second image data includes a second processed image adjusted based on the training image, and the adjustment amplitude corresponding to the second processed image is greater than the adjustment amplitude corresponding to the first processed image; and then determining the screen noise image in the target video according to the screen noise confidence. Thus, a process of model training can be realized with only a small amount of manually labeled images and a large amount of unlabeled images, and the model prediction consistency of the unlabeled data and the generation of pseudo-labels are utilized to expand the labeled data set and improve the accuracy of screen noise image recognition.

[0178] The present application also provides a terminal device, such as Fig. 9 The figure is a schematic diagram of the structure of another terminal device provided by the embodiment of the present application. For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking the mobile phone as an example:

[0179] Fig. 9 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal provided in an embodiment of the present application. Fig. 9 The mobile phone includes: a radio frequency (RF) circuit 910, a memory 920, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a wireless fidelity (WiFi) module 970, a processor 980, and a power supply 990. Those skilled in the art will understand that Fig. 9 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0180] Combine the following Fig. 9 A detailed introduction to the various components of the mobile phone:

[0181] The RF circuit 910 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 980 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 910 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 910 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0182] The memory 920 can be used to store software programs and modules. The processor 980 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 920. The memory 920 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 920 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0183] The input unit 930 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 930 may include a touch panel 931 and other input devices 932. The touch panel 931, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 931 or near the touch panel 931 using any suitable object or accessory such as a finger, stylus, etc., and the air touch operation within a certain range on the touch panel 931), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 931 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 980, and can receive and execute commands sent by the processor 980. In addition, the touch panel 931 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 931, the input unit 930 may further include other input devices 932. Specifically, the other input devices 932 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0184] The display unit 940 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 940 may include a display panel 941. Optionally, the display panel 941 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 931 may cover the display panel 941. When the touch panel 931 detects a touch operation on or near it, it is transmitted to the processor 980 to determine the type of touch event. Subsequently, the processor 980 provides a corresponding visual output on the display panel 941 according to the type of touch event. Although in Fig. 9 In the embodiment, the touch panel 931 and the display panel 941 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 931 and the display panel 941 can be integrated to realize the input and output functions of the mobile phone.

[0185] The mobile phone may also include at least one sensor 950, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 941 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 941 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0186] The audio circuit 960, the speaker 961, and the microphone 962 can provide an audio interface between the user and the mobile phone. The audio circuit 960 can transmit the received audio data to the speaker 961 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 962 converts the collected sound signal into an electrical signal, which is received by the audio circuit 960 and converted into audio data, and then the audio data is output to the processor 980 for processing, and then sent to another mobile phone through the RF circuit 910, or the audio data is output to the memory 920 for further processing.

[0187] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 970. It provides users with wireless broadband Internet access. Fig. 9 A WiFi module 970 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0188] The processor 980 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 920, and calling data stored in the memory 920, so as to monitor the mobile phone as a whole. Optionally, the processor 980 may include one or more processing units; optionally, the processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 980.

[0189] The mobile phone also includes a power supply 990 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 980 through a power management system, thereby managing functions such as charging, discharging, and power consumption management through the power management system.

[0190] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0191] In the embodiment of the present application, the processor 980 included in the terminal also has the function of executing each step of the above-mentioned page processing method.

[0192] The present application also provides a server. Fig.10 , Fig.10 1 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1000 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1022 (for example, one or more processors) and a memory 1032, and one or more storage media 1030 (for example, one or more mass storage devices) storing application programs 1042 or data 1044. Among them, the memory 1032 and the storage medium 1030 may be short-term storage or permanent storage. The program stored in the storage medium 1030 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1022 may be configured to communicate with the storage medium 1030 to execute a series of instruction operations in the storage medium 1030 on the server 1000.

[0193] The server 1000 may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input and output interfaces 1058, and / or one or more operating systems 1041, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0194] The steps performed by the management device in the above embodiment can be based on the Fig.10 The server structure shown.

[0195] In an embodiment of the present application, a computer-readable storage medium is also provided, in which a recognition instruction of a flower screen image is stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned Figures 3 to 7 The illustrated embodiment describes the steps performed by the device for identifying a flower screen image in the method.

[0196] The present application also provides a computer program product including instructions for identifying a distorted screen image. When the computer program product is run on a computer, the computer executes the above-mentioned Figures 3 to 7 The illustrated embodiment describes the steps performed by the device for identifying a flower screen image in the method.

[0197] The embodiment of the present application also provides a system for identifying a distorted screen image. The system for identifying a distorted screen image may include: Figure 8 The device for identifying the flower screen image in the described embodiment, or Fig. 9 The terminal device in the described embodiment, or Fig.10 The server being described.

[0198] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0199] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0200] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0202] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a device for identifying a flower screen image, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0203] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying a flower screen image. It is characterized in that include: Get the image sequence corresponding to the target video; Inputting the video image in the image sequence into the target network model to obtain the screen distortion confidence corresponding to the video image; Determine the flower screen image in the target video according to the flower screen confidence; The target network model is trained in the following way: Obtain unlabeled training images; Performing a first adjustment operation on the training image one by one to obtain a plurality of first processed images in series, wherein the first adjustment operation is used to indicate an adjustment direction that is similar to the training image; Performing a second adjustment operation on the training image to obtain a second processed image, wherein the second adjustment operation is used to indicate an adjustment direction different from that of the training image; and an adjustment amplitude corresponding to the second processed image is greater than an adjustment amplitude corresponding to the first processed image; Inputting a plurality of the first processed images into a first model to obtain first training parameters; Inputting the second processed image into a second model to obtain second training parameters, wherein the first model and the second model share weight parameters; Weighting the first training parameter and the second training parameter to obtain a target parameter; Perform a sharpening operation based on the target parameter to determine a pseudo label corresponding to the training image; The first model is trained according to the training images and annotation data configured with the pseudo labels to obtain the target network model.

2. The method according to claim 1, It is characterized in that The step of successively performing the first adjustment operation on the training images to obtain a plurality of serial first processed images comprises: Performing a first adjustment operation on the training image to obtain a first serial image; performing the first adjustment operation on the first serial images to obtain a second serial image; The first serial image and the second serial image are associated to form a plurality of first processed images in series.

3. The method according to claim 2, It is characterized in that The step of inputting the plurality of the first processed images into a first model to obtain a first training parameter comprises: Inputting a plurality of the first processed images into a first model to obtain a parameter sequence indicating a screen distortion probability corresponding to the first processed images; An averaging operation is performed on the parameter sequence to obtain the first training parameter.

4. The method according to claim 3, It is characterized in that After performing an averaging operation on the parameter sequence to obtain the first training parameter, the method further includes: Acquire consistency loss information corresponding to the first model based on the parameter sequence; The parameters of the first model are updated based on the consistency loss information.

5. The method according to claim 2, It is characterized in that The performing a sharpening operation based on the target parameter to determine a pseudo label corresponding to the training image includes: Obtaining distribution information corresponding to the one-hot encoding method adopted by the first model; determining a sharpening factor based on the distribution information; A sharpening operation is performed on the target parameter according to the sharpening coefficient to determine a pseudo label corresponding to the training image.

6. The method according to claim 2, It is characterized in that The step of training the first model according to the training image and the annotation data configured with the pseudo-label to obtain the target network model includes: Acquire corresponding first loss information based on the labeled data; Analyzing the consistency of the probability estimation corresponding to the pseudo-label to obtain second loss information; Perform supervised training based on the pseudo-label to obtain third loss information; weighting the first loss information, the second loss information, and the third loss information to obtain target loss information; The first model is trained based on the target loss information to obtain the target network model.

7. The method according to any one of claims 1 to 6, It is characterized in that The method further comprises: Acquire the data volume corresponding to the annotation data associated with the target video; If the data amount corresponding to the labeled data does not reach a preset ratio, calling the video frame image in the target video; Using the video frame image as associated unlabeled data; The target network model is trained based on the labeled data and the associated unlabeled data, the associated unlabeled data is configured with a pseudo label determined based on third image data and fourth image data, the third image data is a serial plurality of third processed images obtained by successively performing image adjustment operations on the video frame image, the fourth image data includes a fourth processed image obtained based on the adjustment of the video frame image, and the adjustment amplitude corresponding to the fourth processed image is greater than the adjustment amplitude corresponding to the third processed image.

8. A device for identifying a flower screen image, It is characterized in that include: An acquisition unit, used for acquiring an image sequence corresponding to a target video; An input unit, used for inputting the video image in the image sequence into the target network model to obtain the screen distortion confidence corresponding to the video image; An identification unit, used for determining a distorted screen image in the target video according to the distorted screen confidence; The recognition unit is further used to train the target network model in the following manner: Obtain unlabeled training images; Performing a first adjustment operation on the training image one by one to obtain a plurality of first processed images in series, wherein the first adjustment operation is used to indicate an adjustment direction that is similar to the training image; Performing a second adjustment operation on the training image to obtain a second processed image, wherein the second adjustment operation is used to indicate an adjustment direction different from that of the training image; and an adjustment amplitude corresponding to the second processed image is greater than an adjustment amplitude corresponding to the first processed image; Inputting a plurality of the first processed images into a first model to obtain first training parameters; Inputting the second processed image into a second model to obtain second training parameters, wherein the first model and the second model share weight parameters; Weighting the first training parameter and the second training parameter to obtain a target parameter; Perform a sharpening operation based on the target parameter to determine a pseudo label corresponding to the training image; The first model is trained according to the training images and annotation data configured with the pseudo labels to obtain the target network model.

9. The device according to claim 8, It is characterized in that The identification unit is specifically used for: Performing a first adjustment operation on the training image to obtain a first serial image; performing the first adjustment operation on the first serial images to obtain a second serial image; The first serial image and the second serial image are associated to form a plurality of first processed images in series.

10. The device according to claim 9, It is characterized in that The identification unit is specifically used for: Inputting a plurality of the first processed images into a first model to obtain a parameter sequence indicating a screen distortion probability corresponding to the first processed images; An averaging operation is performed on the parameter sequence to obtain the first training parameter.

11. The device according to claim 10, It is characterized in that The identification unit is further used for: Acquire consistency loss information corresponding to the first model based on the parameter sequence; The parameters of the first model are updated based on the consistency loss information.

12. The device according to claim 9, It is characterized in that The identification unit is specifically used for: Obtaining distribution information corresponding to the one-hot encoding method adopted by the first model; determining a sharpening factor based on the distribution information; A sharpening operation is performed on the target parameter according to the sharpening coefficient to determine a pseudo label corresponding to the training image.

13. The device according to claim 9, It is characterized in that The identification unit is specifically used for: Acquire corresponding first loss information based on the labeled data; Analyzing the consistency of the probability estimation corresponding to the pseudo-label to obtain second loss information; Perform supervised training based on the pseudo-label to obtain third loss information; weighting the first loss information, the second loss information, and the third loss information to obtain target loss information; The first model is trained based on the target loss information to obtain the target network model.

14. The device according to any one of claims 8 to 13, It is characterized in that The identification unit is further used for: Acquire the data volume corresponding to the annotation data associated with the target video; If the data amount corresponding to the labeled data does not reach a preset ratio, calling the video frame image in the target video; Using the video frame image as associated unlabeled data; The target network model is trained based on the labeled data and the associated unlabeled data, the associated unlabeled data is configured with a pseudo label determined based on third image data and fourth image data, the third image data is a serial plurality of third processed images obtained by successively performing image adjustment operations on the video frame image, the fourth image data includes a fourth processed image obtained based on the adjustment of the video frame image, and the adjustment amplitude corresponding to the fourth processed image is greater than the adjustment amplitude corresponding to the third processed image.

15. A computer device, It is characterized in that The computer device comprises a processor and a memory: The memory is used to store program codes; the processor is used to execute the method for recognizing a flower screen image according to any one of claims 1 to 7 according to instructions in the program codes.

16. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the method for recognizing a distorted screen image as described in any one of claims 1 to 7.

17. A computer program product, It is characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for identifying a flower screen image as described in any one of claims 1 to 7.

Images