Blurred screen image processing method and device with long-tail distribution characteristic and product
By constructing multi-type flower screen image sets and performing feature extraction and loss function construction, the problem of difficult to subclassify flower screen image recognition in the prior art is solved, and the accuracy of flower screen image recognition and the generalization ability of model are improved.
Patent Information
- Application Number
- CN202410994312.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-23
AI Technical Summary
The existing flower screen image recognition scheme can only realize binary classification detection, but cannot realize subclassification detection of flower screen type, which affects the identification effect of subsequent visual analysis models.
Build an initial image set, including multiple types of flower screen images, expand color distortion and noise image samples, extract intermediate features and remix and enhance, build target loss function, iterative training of flower screen recognition model, and realize the type of flower screen image Subdivided classification detection.
The subclassification detection of the flower screen image type is realized, the generalization ability and classification accuracy of the model are improved, and the generalization error caused by uneven sample size is reduced.
Smart Images

Figure CN119068233A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method, device and product for processing a flower screen image with a long-tail distribution characteristic. Background Art
[0002] With the improvement of the level of information technology development, video surveillance systems based on computer vision analysis are widely used in engineering construction. Video surveillance systems are mostly installed in multiple points in outdoor construction sites. The number, behavior and posture of target objects such as construction machinery and construction personnel in the picture are identified and analyzed through visual analysis models. Due to the influence of bandwidth, network transmission, environmental interference sources or unstable video frame rate, video decoding format, some data may be lost during video transmission, resulting in abnormal color blocks in part or the entire image, that is, flower screen images. Flower screen images will affect the visual analysis model's recognition of target objects in video surveillance images. Therefore, before performing visual analysis on video surveillance images, it is necessary to conduct discriminant analysis of flower screen images.
[0003] The current flower screen image recognition scheme can only achieve binary classification detection of flower screen images, and the current flower screen model training scheme mostly focuses on reducing the model's dependence on the training data set, or improving the accuracy of binary classification detection of flower screen images, and cannot achieve detailed classification detection of flower screen types. However, different types of flower screen images have different degrees of impact on whether the subsequent visual analysis model can identify the number and behavior of target objects in the image, and also directly affect the training effect of the subsequent visual analysis model for the target object. Therefore, it is very important to achieve detailed detection of specific types of flower screen images. Summary of the invention
[0004] In view of this, the present application aims to propose a method, device and product for processing flower screen images with long-tail distribution characteristics, so as to achieve segmented detection of the types of flower screen images.
[0005] To achieve the above objectives, the technical solution of this application is as follows:
[0006] A first aspect of an embodiment of the present application provides a method for processing a distorted screen image with a long-tail distribution characteristic, the method comprising:
[0007] Constructing an initial image set, wherein the initial image set includes the following types of flower screen images: large-area pure color flower screen images, small-area pure color flower screen images, large-area striped flower screen images, small-area striped flower screen images, color-distorted flower screen images, and noise flower screen images;
[0008] Expanding the number of color-distorted distorted screen images and noise-distorted screen images in the initial image set to generate a first sample set;
[0009] Randomly extracting a plurality of different types of flower screen images from the first sample set, and extracting intermediate features of each flower screen image;
[0010] Remix and enhance the intermediate features of different types of flower screen images to generate mixed features;
[0011] Based on the intermediate features and the mixed features, construct a target loss function;
[0012] Based on the target loss function and the first sample set, iteratively train the flower screen recognition model to obtain a trained flower screen recognition model;
[0013] The image to be identified is input into the flower screen identification model, and the flower screen identification model is used to determine whether the image to be identified is a flower screen image and the type of the flower screen image.
[0014] Optionally, construct an initial image set, including:
[0015] Extract frames from a sample video at a first time interval to obtain an initial image set; the sample video is a video with a distorted screen;
[0016] Remove non-flowered screen images from the initial image set;
[0017] Add labels corresponding to the type of screen distortion to all images.
[0018] Optionally, the number of color distorted distorted screen images and noise distorted screen images in the initial image set is expanded to generate a first sample set, including:
[0019] An image with color distortion and flower screen is collected, and the flower screen pixel area in the image is cropped to obtain a plurality of small-sized regional images; the plurality of small-sized regional images are randomly mixed with the non-flower screen image to generate a new color distortion and flower screen image;
[0020] Add noise to the non-flowering image to generate a new noisy flowering image;
[0021] The new color distorted distorted screen image and the new noise distorted screen image are added to the initial image set to generate a first sample set.
[0022] Optionally, before adding labels of corresponding screen distortion types to all images, the following is also included:
[0023] In a flower screen image of a pure color flower screen type, when the flower screen pixel coverage area is not less than a first threshold, the flower screen image is determined to be a large-area pure color flower screen;
[0024] In a flower screen image of a pure color flower screen type, when the flower screen pixel coverage area is less than a first threshold, the flower screen image is determined to be a small-area pure color flower screen;
[0025] In a striped flower screen type flower screen image, when the flower screen pixel coverage area is not less than a first threshold, determining that the flower screen image is a large-area striped flower screen;
[0026] In a striped screen type distorted image, when the distorted screen pixel coverage area is smaller than a first threshold, the distorted screen image is determined to be a small-area striped screen distorted image.
[0027] Optionally, the flower screen recognition model includes: a first feature extraction network, a second feature extraction network and a classification module; the first feature extraction network is used to extract intermediate features of each flower screen image;
[0028] Based on the intermediate features and the mixed features, a target loss function is constructed, including:
[0029] Processing each intermediate feature through the second feature extraction network and the classification module to obtain a first probability distribution of the flower screen image corresponding to each intermediate feature belonging to each type of flower screen image;
[0030] Remix and enhance the intermediate features of images of different flower screen types to obtain mixed features; process each mixed feature through the second feature extraction network and the classification module to obtain a second probability distribution of the flower screen image corresponding to each mixed feature belonging to each type of flower screen image;
[0031] Based on the first probability distribution and the second probability distribution, a target loss function is constructed.
[0032] Optionally, the method for processing a distorted screen image with a long-tail distribution characteristic further includes:
[0033] For each mixed feature, obtain the label of the flower screen image corresponding to the intermediate feature that generates the mixed feature;
[0034] According to the label of the flower screen image corresponding to the intermediate feature, the label of the flower screen image corresponding to the mixed feature is determined.
[0035] Optionally, the method for processing a distorted screen image with a long-tail distribution characteristic further includes:
[0036] In the case where the image to be identified is a non-flowered screen image, adding the image to be identified to the second sample set;
[0037] In a case where the image to be identified is a distorted screen image and the type is a noise distorted screen image, adding the image to be identified to the second sample set;
[0038] In the case where the image to be identified is a flower screen image, and the type is a small area pure color flower screen, a small area stripe flower screen or a color distortion flower screen, further determining whether the target object in the image to be identified is completely covered by the flower screen pixels; if the target object is not completely covered by the flower screen pixels, adding the image to be identified to the second sample set;
[0039] Based on the second sample set, a visual analysis model is trained; the visual analysis model is used to perform a visual analysis task on a target object in an image to be identified.
[0040] Optionally, the method for processing a distorted screen image with a long-tail distribution characteristic further includes:
[0041] Calculate the ratio of the number of the distorted screen images to the number of the images to be identified as a first ratio;
[0042] The first proportion is compared with a second threshold, and a first warning prompt is generated when the first proportion is not less than the second threshold.
[0043] Optionally, calculating the ratio of the number of the distorted screen images to the number of the images to be identified as a first ratio includes:
[0044] Determine the number of images to be identified within the current most recent target time period;
[0045] Determine the number of distorted screen images within the target time period;
[0046] The ratio of the number of the distorted screen images to the number of the to-be-recognized images within the target time period is calculated as a first ratio.
[0047] Optionally, the method for processing a distorted screen image with a long-tail distribution characteristic further includes:
[0048] Calculate the ratio of the number of target type flower screen images to the number of the to-be-recognized images as the second ratio; the target types include: large area pure color flower screen, large area striped flower screen, color distortion flower screen and noise flower screen;
[0049] The second proportion is compared with a second threshold, and a second warning prompt is generated when the second proportion is not less than the second threshold.
[0050] Optionally, calculating the ratio of the number of the target type of distorted screen images to the number of the to-be-recognized images as a second ratio includes:
[0051] Determine the number of images to be identified within the current most recent target time period;
[0052] Determine the number of the target type of distorted screen images within the target time period;
[0053] The ratio of the number of the target type of distorted screen images to the number of the to-be-recognized images within the target time period is calculated as a second ratio.
[0054] According to a second aspect of an embodiment of the present application, a device for processing a distorted screen image with a long-tail distribution characteristic is provided, which is used to implement the method for processing a distorted screen image with a long-tail distribution characteristic provided in the first aspect of the embodiment of the present application, and the device includes:
[0055] The first sample construction module is configured to construct an initial image set, wherein the initial image set includes the following types of flower screen images: large-area pure color flower screen images, small-area pure color flower screen images, large-area striped flower screen images, small-area striped flower screen images, color-distorted flower screen images, and noise flower screen images; the number of color-distorted flower screen images and noise flower screen images in the initial image set is expanded to generate a first sample set;
[0056] The first training module is configured to randomly extract a plurality of different types of flower screen images from the first sample set, extract intermediate features of each flower screen image; remix and enhance the intermediate features of different types of flower screen images to generate mixed features; construct a target loss function based on the intermediate features and the mixed features; iteratively train the flower screen recognition model based on the target loss function and the first sample set to obtain a trained flower screen recognition model;
[0057] The detection module is configured to input the image to be identified into the flower screen recognition model, and determine whether the image to be identified is a flower screen image and the type of the flower screen image through the flower screen recognition model.
[0058] According to a third aspect of an embodiment of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect of the embodiment of the present application.
[0059] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect of the embodiment of the present application are implemented.
[0060] According to the fifth aspect of the embodiments of the present application, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method described in the first aspect of the embodiments of the present application are implemented.
[0061] The flower screen image processing method with long-tail distribution characteristics provided by the present application is used to pre-train a flower screen recognition model to detect whether the image to be recognized is a flower screen image and to determine the specific type of the flower screen image. In the process of training the flower screen recognition model, an initial image set with multiple types of flower screen images is first constructed, and then the number of flower screen image samples of two types with fewer samples in the initial image set is expanded to generate a first sample set. Multiple different types of flower screen images are randomly selected from the first sample set, and intermediate features are extracted. Remixing and enhancement are performed based on the intermediate features of different types to obtain mixed features. A target loss function is constructed based on the intermediate features and mixed features, and the flower screen recognition model is iteratively trained.
[0062] This method can be used to perform fine classification detection of the image flower screen type and determine the specific type of the flower screen image. In addition, when constructing the model training samples, this method expands the minority class samples (color distortion flower screen, noise flower screen) of the flower screen type by synthesizing samples, so as to move the decision boundary to the majority class, reduce the generalization error caused by the imbalance of sample quantity, and improve the classification accuracy of the model. In addition, during the training process, the intermediate features extracted from different types of flower screen images are remixed and enhanced to further improve the generalization ability of the model and improve the classification accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0064] Figure 1 It is a flow chart of a method for processing a flower screen image with a long-tail distribution characteristic proposed in an embodiment of the present application;
[0065] Figure 2 is a schematic diagram of some types of flower screen images in an embodiment of the present application;
[0066] Figure 3 It is a schematic diagram of training a flower screen recognition model in one embodiment of the present application;
[0067] Figure 4 is a schematic diagram of a flower screen image processing device with a long-tail distribution characteristic proposed in an embodiment of the present application;
[0068] Figure 5 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION
[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0070] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0071] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0072] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects as detailed in the present application.
[0073] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.
[0074] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.
[0075] Figure 1 1 is a flow chart of a method for processing a distorted screen image with a long-tail distribution characteristic proposed in one embodiment of the present application. Figure 1 As shown, the method includes:
[0076] S1: construct an initial image set, wherein the initial image set includes the following types of flower screen images: large-area pure color flower screen images, small-area pure color flower screen images, large-area striped flower screen images, small-area striped flower screen images, color-distorted flower screen images, and noise flower screen images;
[0077] Expanding the number of color-distorted distorted screen images and noise-distorted screen images in the initial image set to generate a first sample set;
[0078] Randomly extracting a plurality of different types of flower screen images from the first sample set, and extracting intermediate features of each flower screen image;
[0079] Remix and enhance the intermediate features of different types of flower screen images to generate mixed features;
[0080] Based on the intermediate features and the mixed features, construct a target loss function;
[0081] Based on the target loss function and the first sample set, iteratively train the flower screen recognition model to obtain a trained flower screen recognition model;
[0082] S2: Input the image to be identified into the flower screen identification model, and determine whether the image to be identified is a flower screen image and the type of the flower screen image through the flower screen identification model.
[0083] In this embodiment, by pre-building and training a flower screen recognition model, detailed classification detection of the flower screen type of the image is achieved. In view of the uneven distribution of sample images of various types of flower screen images, which have long-tail distribution characteristics, for example, color distorted flower screen images and noise flower screen images rarely appear in actual application scenarios, resulting in a small number of training samples for these two types of flower screen images. The long-tail distribution of training samples will cause the model to have a higher classification accuracy in the head category (category with a larger sample size), while the classification accuracy in the tail category (category with a smaller sample size) decreases, resulting in insufficient overall recognition accuracy of the model. In order to reduce the generalization error caused by the imbalance of sample numbers, in this embodiment, when constructing the sample set, the flower screen image samples of the type with a smaller sample size are expanded to generate a first sample set.
[0084] In this embodiment, an initial image set is first created, specifically including the following six types of flower screen images: large-area pure color flower screen images, small-area pure color flower screen images, large-area striped flower screen images, small-area striped flower screen images, color-distorted flower screen images, and noise flower screen images. Further, the number of samples of two types of flower screen types with a small number of samples (i.e., color-distorted flower screen images and noise flower screen images) is expanded to obtain a first sample set. Different types of flower screen images are randomly extracted from the first sample set, and the flower screen recognition model is iteratively trained. Specifically, the intermediate features of each flower screen image are extracted through the flower screen recognition model, and then the intermediate features of different types of flower screen images are further re-mixed and enhanced to generate mixed features. A target loss function is constructed based on the intermediate features and the mixed features, and then the target loss function and the flower screen images randomly extracted from the first sample set are used to iteratively train the flower screen recognition model, calculate the optimal model parameters, and obtain a high-precision flower screen recognition model. The image to be identified is input into the trained flower screen recognition model to determine whether the image to be identified is a flower screen image and the specific flower screen type of the flower screen image, thereby realizing detailed classification detection of the flower screen type of the image to be identified.
[0085] By adopting this method, it is possible to implement fine classification detection of flower screen types for images, and the flower screen recognition model trained in this method has strong generalization ability, thereby improving the accuracy of flower screen fine classification detection.
[0086] As an implementation of the present application, constructing an initial image set includes:
[0087] Extract frames from a sample video at a first time interval to obtain an initial image set; the sample video is a video with a distorted screen;
[0088] Remove non-flowered screen images from the initial image set;
[0089] Add labels corresponding to the type of screen distortion to all images.
[0090] In this embodiment, an initial image set is created based on a sample video with a distorted screen. Specifically, a video containing a distorted screen is collected, and frames are extracted from the video at a first time interval to obtain an initial image set. Images without distorted screens (i.e., non-distorted screen images) in the initial image set are removed, and images with distorted screens are retained. Optionally, the sample video can be a video used for computer vision analysis tasks. For example, a video for visual analysis tasks such as target detection, target analysis, and action classification can be used as a sample video.
[0091] In this embodiment, possible screen distortion types are divided into six types, and when constructing the initial image set, it is ensured that all types of screen distortion images are included in the initial image set. Specifically, the screen distortion types include the following six types: large area solid color screen distortion, small area solid color screen distortion, large area stripe screen distortion, small area stripe screen distortion, color distortion screen distortion, and noise screen distortion. Figure 2 Schematic diagram of some types of flower screen images in an embodiment of the present application. Figure 2 As shown, (a) is a large-area pure color flower screen image, (b) is a small-area pure color flower screen image, (c) is a large-area striped flower screen image, (d) is a small-area striped flower screen image, and (e) is a color distorted flower screen image. In this embodiment, according to the above six flower screen types, labels of the flower screen types are added to all images in the initial image set.
[0092] As an implementation manner of the present application, before adding labels corresponding to the screen distortion type to all images, the method further includes:
[0093] In a flower screen image of a pure color flower screen type, when the flower screen pixel coverage area is not less than a first threshold, the flower screen image is determined to be a large-area pure color flower screen;
[0094] In a flower screen image of a pure color flower screen type, when the flower screen pixel coverage area is less than a first threshold, the flower screen image is determined to be a small-area pure color flower screen;
[0095] In a striped flower screen type flower screen image, when the flower screen pixel coverage area is not less than a first threshold, determining that the flower screen image is a large-area striped flower screen;
[0096] In a striped screen type distorted image, when the distorted screen pixel coverage area is smaller than a first threshold, the distorted screen image is determined to be a small-area striped screen distorted image.
[0097] In this embodiment, when adding a label of the flower screen type to the image in the initial image set, it is necessary to determine the large-area flower screen type and the small-area flower screen type in the solid-color flower screen image and the striped flower screen image. Specifically, for the solid-color flower screen image or the striped flower screen image, the flower screen pixel coverage area in the image is compared with the first threshold. If the flower screen pixel coverage area reaches the first threshold, it is determined to be a large-area flower screen image. If the flower screen pixel coverage area is less than the first threshold, it is determined to be a small-area flower screen image. The first threshold can be set according to the needs of the actual application. In this embodiment, the first threshold can be set to 1 / 3 of the image area. For example, for a striped flower screen image, if the flower screen pixel coverage area reaches 1 / 3 of the image area, the image is determined to be a large-area striped flower screen image.
[0098] As an implementation manner of the present application, the number of color distorted distorted screen images and noise distorted screen images in the initial image set is expanded to generate a first sample set, including:
[0099] An image with color distortion and flower screen is collected, and the flower screen pixel area in the image is cropped to obtain a plurality of small-sized regional images; the plurality of small-sized regional images are randomly mixed with the non-flower screen image to generate a new color distortion and flower screen image;
[0100] Add noise to the non-flowering image to generate a new noisy flowering image;
[0101] The new color distorted distorted screen image and the new noise distorted screen image are added to the initial image set to generate a first sample set.
[0102] In this embodiment, in order to reduce the generalization error caused by the imbalance of sample numbers, the number of image samples of two flower screen types with a small number of samples in the initial image set is expanded to generate a first sample set. The number of image samples of each flower screen type in the first sample set is more balanced than that in the initial image set. Training the flower screen recognition model based on the first image set can improve the generalization ability of the model.
[0103] Specifically, in the initial image set, the number of samples of color distorted distorted screen images and noise distorted screen images is relatively small. In this embodiment, different methods are used to expand these two types of sample images.
[0104] In the case where the proportion of color distorted and distorted images is relatively small, image samples with color distorted and distorted images are collected, and the color distorted and distorted pixel areas in the images are cropped to obtain multiple small-sized regional images. These small-sized regional images are then randomly filled and mixed with normal images without distorted images to generate new color distorted and distorted image samples, and labels are added to identify them as color distorted and distorted images. It is worth noting that the normal images used to randomly alleviate the small-sized distorted pixel area images can be selected from video frames without distorted images extracted from sample videos, or other images without distorted images.
[0105] In the case where the proportion of noise-prone images is small, noise is added to normal images without noise to generate new noise-prone image samples, and a label is added to identify them as noise-prone images. The added noise can be salt and pepper noise (impulse noise) or noise that follows a Gaussian distribution.
[0106] In this embodiment, by expanding the sample of the minority flower screen type image, the long-tail distribution characteristic of the flower screen type samples in the initial image set is alleviated, the imbalance of the training samples is reduced, and the generalization ability of the flower screen recognition model is improved.
[0107] As an implementation mode of the present application, the flower screen recognition model includes: a first feature extraction network, a second feature extraction network and a classification module; the first feature extraction network is used to extract the intermediate features of each flower screen image;
[0108] Based on the intermediate features and the mixed features, a target loss function is constructed, including:
[0109] Processing each intermediate feature through the second feature extraction network and the classification module to obtain a first probability distribution of the flower screen image corresponding to each intermediate feature belonging to each type of flower screen image;
[0110] Remix and enhance the intermediate features of images of different flower screen types to obtain mixed features; process each mixed feature through the second feature extraction network and the classification module to obtain a second probability distribution of the flower screen image corresponding to each mixed feature belonging to each type of flower screen image;
[0111] Based on the first probability distribution and the second probability distribution, a target loss function is constructed.
[0112] In one embodiment, a flower screen recognition model is constructed, including: a first feature extraction network, a second feature extraction network and a classification module. Figure 3 Schematic diagram of training a flower screen recognition model in one embodiment of the present application. Figure 3 As shown, each time the model is trained, a training image set is randomly sampled from the first sample set, and the training image set is input into the first feature extraction network to obtain the intermediate features of each flower screen image. The intermediate features of different flower screen images are remixed and enhanced to obtain mixed features, and then the intermediate features and mixed features are respectively passed through the second feature extraction network and the classification module to obtain the first probability distribution and the second probability distribution of the images corresponding to the intermediate features and the mixed features belonging to each type of flower screen image. The target loss function is constructed based on the first probability distribution and the second probability distribution, and the parameters of the model are updated based on the target loss function using the back propagation algorithm. After multiple iterative training, the optimal model parameters are obtained.
[0113] Specifically, in this embodiment, the six types of flower screen samples are marked with integers from 1 to 6: large-area solid color flower screen is marked as 1; small-area solid color flower screen is marked as 2; large-area striped flower screen is marked as 3; small-area striped flower screen is marked as 4; color distortion flower screen is marked as 5; and noise flower screen is marked as 6. The images in the training image set are input into the first feature extraction network to obtain the intermediate feature feature, and the expression is as follows:
[0114] feature i =f(x i );
[0115] feature j =f(x j );
[0116] Among them, i and j are different integers from 1 to 6, and x i 、x j They represent the flower screen image samples of the i-th and j-th types respectively; f() represents the feature extraction function of the first feature extraction network, feature i 、feature j represents the intermediate feature representation obtained after calculation by the first feature extraction network. Optionally, the first feature extraction network can be constructed based on a convolutional neural network such as an MBConv (Mobile Inverted Bottleneck Convolution) network or a Resnet residual network.
[0117] The intermediate features of different types of flower screen images are remixed and enhanced to obtain the mixed feature mixFeature as follows:
[0118] mixFeature=λ x feature i +(1-λ x )feature j ;
[0119] Among them, λ x Represents the feature mixing factor, and its value range is [-1,1].
[0120] The remixed and enhanced mixed feature mixFeature and the intermediate feature feature of the original flower screen image are respectively passed through the second feature extraction network and the classification module, and the remixed and enhanced flower screen category feature vector mixZ and the flower screen category feature vector Z of the original input flower screen image are obtained based on the following expression:
[0121] Z = G(F(feature));
[0122] mixZ = G(F(mixFeature));
[0123] Wherein, F() represents the feature extraction function of the second feature extraction network, and G() represents the classifier function of the classification module. Optionally, the second feature extraction network in this embodiment can be constructed based on a convolutional neural network such as an MBConv module and a residual network Resnet, and the classification module can be constructed based on a fully connected layer.
[0124] Based on the flower screen category feature vector, the probability distribution of the flower screen image belonging to each flower screen type is calculated and predicted to be used to analyze the possibility of the prediction result of the flower screen type. The calculation expression is as follows:
[0125]
[0126] Wherein, p represents the probability distribution (i.e., the first probability distribution, the second probability distribution) of the type of the flower screen image calculated based on the intermediate features / mixed features. Indicates the predicted label of the image sample's screen blur type. Z i , Z j Respectively represent the values of the i-th and j-th items in the feature vector of the original input training image, mixZ i 、mixZ j They represent the values of the i-th and j-th items in the feature vector of the flower screen category after remixing and enhancement; C represents the total number of flower screen types, n i 、n j Indicates the number of training image samples of the i-th and j-th types of flower screen, n max Indicates the number of flower screen image samples of the flower screen type with the largest image sample number.
[0127] Based on the first probability distribution and the second probability distribution, the target loss function L for the flower screen recognition model training is constructed:
[0128]
[0129] In this embodiment, each time the flower screen recognition model is trained, the flower screen images in the first sample set are randomly selected to generate a training image set. The flower screen recognition model is iteratively trained through the training image set and the target loss function. During the iterative training process, the target loss function is continuously optimized, and the optimal model parameters are calculated to finally obtain a flower screen recognition model for performing flower screen type subclassification detection.
[0130] After the flower screen recognition model training is completed, the image to be recognized is input into the model, the flower screen category feature vector is calculated by the model, and the flower screen type with the highest probability corresponding to the predicted image to be recognized is output, so as to determine whether the image to be recognized is a flower screen image and the specific flower screen type it belongs to. Optionally, the image to be recognized can be a video frame image extracted from a real-time video stream at a second time interval. In this embodiment, the flower screen recognition model can be used for flower screen detection of images or flower screen detection of video streams.
[0131] As an implementation manner of the present application, the method for processing a distorted screen image with a long-tail distribution characteristic further includes:
[0132] For each mixed feature, obtain the label of the flower screen image corresponding to the intermediate feature that generates the mixed feature;
[0133] According to the label of the flower screen image corresponding to the intermediate feature, the label of the flower screen image corresponding to the mixed feature is determined.
[0134] In this embodiment, after the intermediate features are mixed to obtain the mixed features, the labels corresponding to the mixed features need to be determined. i 、x j The labels are remixed and enhanced at the label level. First, the label mixing factor λ is determined by the following expression: y :
[0135]
[0136] Among them, n i 、n j They represent the number of samples of the i-th and j-th types of the flower screen images, respectively. k is the set category ratio threshold, and τ is the set feature level mixing factor threshold, with a value range of (0,1). For example, k is 3 and τ is 0.5.
[0137] Based on label mixing factor λ y , calculate the label mixY corresponding to the mixed feature according to the following expression:
[0138] mixY=λ y y i +(1-λ y )y j ;
[0139] Among them, y i ,y j Respectively represent the flower screen type as the i-th and j-th training image samples x i 、x j The corresponding flower screen type label (i.e. the intermediate feature feature i 、feature j Corresponding label), mixY represents the label of the mixed feature mixFeature obtained after the heavy mixing enhancement calculation (that is, the label of the flower screen type corresponding to the flower screen image after the i-th and j-th training images are mixed).
[0140] In this embodiment, the intermediate features of different types of flower screen images are mixed to obtain mixed features, and the mixed features are processed by a second feature extraction network and a classification module, so that the flower screen recognition model can be used to detect mixed flower screen images in which multiple flower screen types exist at the same time, further improving the generalization ability of the model.
[0141] As an implementation manner of the present application, the method for processing a distorted screen image with a long-tail distribution characteristic further includes:
[0142] In the case where the image to be identified is a non-flowered screen image, adding the image to be identified to the second sample set;
[0143] In a case where the image to be identified is a distorted screen image and the type is a noise distorted screen image, adding the image to be identified to the second sample set;
[0144] In the case where the image to be identified is a flower screen image, and the type is a small area pure color flower screen, a small area stripe flower screen or a color distortion flower screen, further determining whether the target object in the image to be identified is completely covered by the flower screen pixels; if the target object is not completely covered by the flower screen pixels, adding the image to be identified to the second sample set;
[0145] Based on the second sample set, a visual analysis model is trained; the visual analysis model is used to perform a visual analysis task on a target object in an image to be identified.
[0146] In one embodiment, the image to be identified is detected using a flower screen recognition model, and a second sample set is constructed based on the detection result. The second sample set can be used for training
[0147] Specifically, according to the detection result of the image to be identified by the flower screen recognition model, when the recognition result of the image to be detected is a normal image (i.e., a non-flower screen image) or a noisy flower screen image, the image is added to the second image set. For small-area pure color flower screen images, small-area striped flower screen images, or color-distorted flower screen images, it is necessary to further determine whether the target object in the image (for example:) is completely covered by flower screen pixels (i.e., pure color pixels, striped pixels, color-distorted pixels). If it is not completely covered, the image is added to the second sample set. When a certain amount of flower screen image samples are included in the second sample set, the visual analysis model can be trained to identify and analyze the target object in the presence of a flower screen without affecting the feature learning of the target object, thereby improving the robustness of the visual analysis model applied to actual scenes. If the target object in the image is completely covered by flower screen pixels, the image is removed. For large-area solid-color flower screen images or large-area striped flower screen images, since large-area solid-color flower screen images or large-area striped flower screen images cover a large area of the image, they have a great impact on the visual analysis model's recognition of target objects and actions in the image. Therefore, large-area solid-color flower screen images and large-area striped flower screen images are removed when constructing the second sample set.
[0148] As an implementation manner of the present application, the method for processing a distorted screen image with a long-tail distribution characteristic further includes:
[0149] Calculate the ratio of the number of the distorted screen images to the number of the images to be identified as a first ratio;
[0150] The first proportion is compared with a second threshold, and a first warning prompt is generated when the first proportion is not less than the second threshold.
[0151] In one embodiment, a trained flower screen recognition model is used to monitor and warn the video picture quality. In this embodiment, the flower screen recognition model can be used to detect flower screens on video frame images extracted from historical record videos, and can also be used to detect flower screens on video frames extracted in real time from real-time video streams. Specifically, video frame images are periodically extracted from the video, and it is detected whether they are flower screen images, and the proportion of the number of flower screen images to all images is recorded as the first proportion. The first proportion is compared with the second threshold. When the second threshold is reached, the video surveillance system may determine that the quality of the collected video does not meet the actual supervision needs due to factors such as the environment or signal transmission, and then generate a first warning prompt to warn about the picture quality. The second threshold can be set according to the needs of the actual application. In this embodiment, the second threshold can be set to 20%-30%.
[0152] By using the flower screen recognition model to detect flower screens in video frames, the video picture quality can be monitored. For flower screen detection of real-time video streams, timely alarms can be issued when flower screens appear frequently. For flower screen detection of historical videos, the video picture quality can be analyzed and evaluated as a whole, which is convenient for operation and maintenance personnel to carry out corresponding maintenance and improve the quality of video streams collected by the video surveillance system.
[0153] As an implementation manner of the present application, calculating the ratio of the number of the distorted screen images to the number of the images to be identified as a first ratio includes:
[0154] Determine the number of images to be identified within the current most recent target time period;
[0155] Determine the number of distorted screen images within the target time period;
[0156] The ratio of the number of the distorted screen images to the number of the to-be-recognized images within the target time period is calculated as a first ratio.
[0157] In one embodiment, when monitoring the video quality of a real-time video stream, according to a preset time period length, a video frame extracted within a target time period closest to the current one is subjected to a screen distortion detection, and it is determined whether the ratio of the number of screen distortion images appearing within the target time period to the total number of all video frame images within the target time period (i.e., the first ratio) reaches a second threshold. When the second threshold is reached, a first warning prompt is generated.
[0158] By judging the video quality within the nearest target time period, timely alarms can be issued when a large number of video artifacts suddenly appear in the real-time video stream within a short period of time, facilitating timely maintenance by operation and maintenance personnel.
[0159] As an implementation manner of the present application, the method for processing a distorted screen image with a long-tail distribution characteristic includes:
[0160] Calculate the ratio of the number of target type flower screen images to the number of the to-be-recognized images as the second ratio; the target types include: large area pure color flower screen, large area striped flower screen, color distortion flower screen and noise flower screen;
[0161] The second proportion is compared with a second threshold, and a second warning prompt is generated when the second proportion is not less than the second threshold.
[0162] In one embodiment, considering that real-time video streams often produce distorted images due to environmental or network signal factors, and in most cases small-area distorted images will not affect the identification of target objects and behaviors in the image, therefore, when counting the ratio of distorted images to all video frame images, small-area distorted images can be ignored.
[0163] In order to avoid frequent alarms caused by small-area flower screen images, which affects the normal maintenance of the video surveillance system, in this embodiment, only the proportion of the target type of flower screen images in all video frame images is counted, that is, the proportion of the total number of large-area pure color flower screens, large-area striped flower screens, color distorted flower screens and noise flower screen images other than small-area pure color flower screen images and small-area striped flower screen images in the total number of video frames is counted as the second proportion. When the second proportion reaches the second threshold, it is determined that the quality of the collected video does not meet the actual supervision needs, and then the second warning information is generated to facilitate the operation and maintenance personnel to carry out corresponding maintenance.
[0164] As an implementation manner of the present application, calculating the ratio of the number of target type distorted screen images to the number of images to be identified as a second ratio includes:
[0165] Determine the number of images to be identified within the current most recent target time period;
[0166] Determine the number of the target type of distorted screen images within the target time period;
[0167] The ratio of the number of the target type of distorted screen images to the number of the to-be-recognized images within the target time period is calculated as a second ratio.
[0168] In one embodiment, when monitoring the video quality of a real-time video stream, according to a preset time period length, a video frame extracted within a target time period closest to the current one is subjected to a screen distortion detection, and it is determined whether the ratio of the number of screen distortion images of a target type appearing within the target time period to the total number of all video frame images within the target time period (i.e., a second ratio) reaches a second threshold. If the second threshold is reached, a second warning prompt is generated.
[0169] By judging the video quality within the nearest target time period, timely alarms can be issued when a large number of video artifacts suddenly appear in the real-time video stream within a short period of time, facilitating timely maintenance by operation and maintenance personnel.
[0170] Based on the same inventive concept, an embodiment of the present application provides a device for processing a distorted screen image with a long-tail distribution characteristic. Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a distorted screen image processing device 100 with a long-tail distribution characteristic proposed in an embodiment of the present application. Figure 4 As shown, the device comprises:
[0171] The first sample construction module 101 is configured to construct an initial image set, wherein the initial image set includes the following types of flower screen images: large-area pure color flower screen images, small-area pure color flower screen images, large-area striped flower screen images, small-area striped flower screen images, color-distorted flower screen images, and noise flower screen images; the number of color-distorted flower screen images and noise flower screen images in the initial image set is expanded to generate a first sample set;
[0172] The first training module 102 is configured to randomly extract a plurality of different types of flower screen images from the first sample set, extract intermediate features of each flower screen image; remix and enhance the intermediate features of different types of flower screen images to generate mixed features; construct a target loss function based on the intermediate features and the mixed features; iteratively train the flower screen recognition model based on the target loss function and the first sample set to obtain a trained flower screen recognition model;
[0173] The detection module 103 is configured to input the image to be identified into the distorted screen recognition model, and determine whether the image to be identified is a distorted screen image and the type of the distorted screen image through the distorted screen recognition model.
[0174] As an implementation manner of the present application, the first sample construction module 101 is configured to construct an initial image set, specifically including:
[0175] Extract frames from a sample video at a first time interval to obtain an initial image set; the sample video is a video with a distorted screen;
[0176] Remove non-flowered screen images from the initial image set;
[0177] Add labels corresponding to the type of screen distortion to all images.
[0178] As an implementation manner of the present application, the first sample construction module 101 is configured to expand the number of color distorted distorted screen images and noise distorted screen images in the initial image set to generate a first sample set, specifically including:
[0179] An image with color distortion and flower screen is collected, and the flower screen pixel area in the image is cropped to obtain a plurality of small-sized regional images; the plurality of small-sized regional images are randomly mixed with the non-flower screen image to generate a new color distortion and flower screen image;
[0180] Add noise to the non-flowering image to generate a new noisy flowering image;
[0181] The new color distorted distorted screen image and the new noise distorted screen image are added to the initial image set to generate a first sample set.
[0182] As an implementation manner of the present application, the first sample construction module 101 is configured to, before adding labels corresponding to the distorted screen types to all images, further include:
[0183] In a flower screen image of a pure color flower screen type, when the flower screen pixel coverage area is not less than a first threshold, the flower screen image is determined to be a large-area pure color flower screen;
[0184] In a flower screen image of a pure color flower screen type, when the flower screen pixel coverage area is less than a first threshold, the flower screen image is determined to be a small-area pure color flower screen;
[0185] In a striped flower screen type flower screen image, when the flower screen pixel coverage area is not less than a first threshold, determining that the flower screen image is a large-area striped flower screen;
[0186] In a striped screen type distorted image, when the distorted screen pixel coverage area is smaller than a first threshold, the distorted screen image is determined to be a small-area striped screen distorted image.
[0187] As an implementation mode of the present application, the flower screen recognition model includes: a first feature extraction network, a second feature extraction network and a classification module; the first feature extraction network is used to extract the intermediate features of each flower screen image;
[0188] The first training module 102 is configured to construct a target loss function based on the intermediate features and the mixed features, specifically including:
[0189] Processing each intermediate feature through the second feature extraction network and the classification module to obtain a first probability distribution of the flower screen image corresponding to each intermediate feature belonging to each type of flower screen image;
[0190] Remix and enhance the intermediate features of images of different flower screen types to obtain mixed features; process each mixed feature through the second feature extraction network and the classification module to obtain a second probability distribution of the flower screen image corresponding to each mixed feature belonging to each type of flower screen image;
[0191] Based on the first probability distribution and the second probability distribution, a target loss function is constructed.
[0192] As an implementation manner of the present application, the first training module 102 is further configured to perform the following steps:
[0193] For each mixed feature, obtain the label of the flower screen image corresponding to the intermediate feature that generates the mixed feature;
[0194] According to the label of the flower screen image corresponding to the intermediate feature, the label of the flower screen image corresponding to the mixed feature is determined.
[0195] As an implementation manner of the present application, the device 100 for processing a distorted screen image with a long-tail distribution characteristic further includes:
[0196] The second sample construction module is configured to add the image to be identified to the second sample set when the image to be identified is a non-flowered screen image; add the image to be identified to the second sample set when the image to be identified is a flowered screen image and the type is a noise flowered screen; further determine whether the target object in the image to be identified is completely covered by flowered screen pixels when the image to be identified is a flowered screen image and the type is a small-area pure color flowered screen, a small-area striped flowered screen or a color-distorted flowered screen; if the target object is not completely covered by flowered screen pixels, add the image to be identified to the second sample set;
[0197] The second training module is configured to train a visual analysis model based on the second sample set; the visual analysis model is used to perform a visual analysis task on a target object in an image to be identified.
[0198] As an implementation manner of the present application, the device 100 for processing a distorted screen image with a long-tail distribution characteristic further includes an early warning module configured to perform the following steps:
[0199] Calculate the ratio of the number of the distorted screen images to the number of the images to be identified as a first ratio;
[0200] The first proportion is compared with a second threshold, and a first warning prompt is generated when the first proportion is not less than the second threshold.
[0201] As an implementation manner of the present application, the early warning module is configured to calculate the ratio of the number of distorted screen images to the number of images to be identified as the first ratio, specifically including:
[0202] Determine the number of images to be identified within the current most recent target time period;
[0203] Determine the number of distorted screen images within the target time period;
[0204] The ratio of the number of the distorted screen images to the number of the to-be-recognized images within the target time period is calculated as a first ratio.
[0205] As an implementation manner of the present application, the early warning module is configured to perform the following steps:
[0206] Calculate the ratio of the number of target type flower screen images to the number of the to-be-recognized images as the second ratio; the target types include: large area pure color flower screen, large area striped flower screen, color distortion flower screen and noise flower screen;
[0207] The second proportion is compared with a second threshold, and a second warning prompt is generated when the second proportion is not less than the second threshold.
[0208] As an implementation manner of the present application, the early warning module is configured to calculate the ratio of the number of target type distorted screen images to the number of images to be identified as the second ratio, specifically including:
[0209] Determine the number of images to be identified within the current most recent target time period;
[0210] Determine the number of the target type of distorted screen images within the target time period;
[0211] The ratio of the number of the target type of distorted screen images to the number of the to-be-recognized images within the target time period is calculated as a second ratio.
[0212] Based on the same inventive concept, an embodiment of the present application provides a computer program product. The computer program product includes a computer program, which, when executed by a processor, implements the steps in the method for processing a distorted screen image with a long-tail distribution characteristic as described in any of the above embodiments of the present application.
[0213] Based on the same inventive concept, an embodiment of the present application provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method for processing a flower screen image with long-tail distribution characteristics as described in any of the above embodiments of the present application are implemented.
[0214] Based on the same inventive concept, an embodiment of the present application provides an electronic device, referring to Figure 5 , Figure 5 2 is a schematic diagram of an electronic device 200 proposed in an embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps in the method for processing a distorted screen image with a long-tail distribution characteristic as described in any of the above embodiments of the present application are implemented.
[0215] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0216] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
[0217] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and components involved are not necessarily required by the present application.
[0218] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0219] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0220] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0221] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0222] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the present application is interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.
[0223] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0224] The above is a detailed introduction to the flower screen image processing method, device and product with long-tail distribution characteristics provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A method for processing a distorted screen image with a long-tail distribution characteristic, characterized in that: include: Constructing an initial image set, wherein the initial image set includes the following types of flower screen images: large-area pure color flower screen images, small-area pure color flower screen images, large-area striped flower screen images, small-area striped flower screen images, color-distorted flower screen images, and noise flower screen images; Expanding the number of color-distorted distorted screen images and noise-distorted screen images in the initial image set to generate a first sample set; Randomly extracting a plurality of different types of flower screen images from the first sample set, and extracting intermediate features of each flower screen image; The intermediate features of different types of flower screen images are remixed and enhanced to generate mixed features; Based on the intermediate features and the mixed features, construct a target loss function; Based on the target loss function and the first sample set, iteratively train the flower screen recognition model to obtain a trained flower screen recognition model; The image to be identified is input into the flower screen identification model, and the flower screen identification model is used to determine whether the image to be identified is a flower screen image and the type of the flower screen image.
2. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 1, characterized in that: Construct an initial set of images, including: Extract frames from a sample video at a first time interval to obtain an initial image set; the sample video is a video with a distorted screen; Remove non-flowered screen images from the initial image set; Add labels corresponding to the type of screen distortion to all images.
3. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 2, characterized in that: The number of color distorted distorted screen images and noise distorted screen images in the initial image set is expanded to generate a first sample set, including: An image with color distortion and flower screen is collected, and the flower screen pixel area in the image is cropped to obtain a plurality of small-sized regional images; the plurality of small-sized regional images are randomly mixed with the non-flower screen image to generate a new color distortion and flower screen image; Add noise to the non-flowering image to generate a new noisy flowering image; The new color distorted distorted screen image and the new noise distorted screen image are added to the initial image set to generate a first sample set.
4. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 2, characterized in that: Before adding labels of corresponding screen distortion types to all images, it also includes: In a flower screen image of a pure color flower screen type, when the flower screen pixel coverage area is not less than a first threshold, the flower screen image is determined to be a large-area pure color flower screen; In a flower screen image of a pure color flower screen type, when the flower screen pixel coverage area is less than a first threshold, the flower screen image is determined to be a small-area pure color flower screen; In a striped flower screen type flower screen image, when the flower screen pixel coverage area is not less than a first threshold, determining that the flower screen image is a large-area striped flower screen; In a striped screen type distorted image, when the distorted screen pixel coverage area is smaller than a first threshold, the distorted screen image is determined to be a small-area striped screen distorted image.
5. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 1, characterized in that: The flower screen recognition model includes: a first feature extraction network, a second feature extraction network and a classification module; the first feature extraction network is used to extract the intermediate features of each flower screen image; Based on the intermediate features and the mixed features, a target loss function is constructed, including: Processing each intermediate feature through the second feature extraction network and the classification module to obtain a first probability distribution of the flower screen image corresponding to each intermediate feature belonging to each type of flower screen image; Remix and enhance the intermediate features of images of different flower screen types to obtain mixed features; process each mixed feature through the second feature extraction network and the classification module to obtain a second probability distribution of the flower screen image corresponding to each mixed feature belonging to each type of flower screen image; Based on the first probability distribution and the second probability distribution, a target loss function is constructed.
6. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 1, characterized in that: Also includes: For each mixed feature, obtain the label of the flower screen image corresponding to the intermediate feature that generates the mixed feature; According to the label of the flower screen image corresponding to the intermediate feature, the label of the flower screen image corresponding to the mixed feature is determined.
7. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 1, characterized in that: Also includes: In the case where the image to be identified is a non-flowered screen image, adding the image to be identified to the second sample set; In a case where the image to be identified is a distorted screen image and the type is a noise distorted screen image, adding the image to be identified to the second sample set; In the case where the image to be identified is a flower screen image, and the type is a small area pure color flower screen, a small area stripe flower screen or a color distortion flower screen, further determining whether the target object in the image to be identified is completely covered by the flower screen pixels; if the target object is not completely covered by the flower screen pixels, adding the image to be identified to the second sample set; Based on the second sample set, training a visual analysis model; The visual analysis model is used to perform a visual analysis task on a target object in an image to be identified.
8. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 1, characterized in that: Also includes: Calculate the ratio of the number of the distorted screen images to the number of the images to be identified as a first ratio; The first proportion is compared with a second threshold, and a first warning prompt is generated when the first proportion is not less than the second threshold.
9. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 8, characterized in that: Calculating the ratio of the number of the distorted screen images to the number of the images to be identified as a first ratio includes: Determine the number of images to be identified within the current most recent target time period; Determine the number of distorted screen images within the target time period; The ratio of the number of the distorted screen images to the number of the to-be-recognized images within the target time period is calculated as a first ratio.
10. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 1, characterized in that: Also includes: Calculate the ratio of the number of the target type of distorted screen images to the number of the to-be-recognized images as a second ratio; The target types include: large-area pure color flower screen, large-area striped flower screen, color distortion flower screen and noise flower screen; The second proportion is compared with a second threshold, and a second warning prompt is generated when the second proportion is not less than the second threshold.
11. The method for processing a distorted screen image with a long-tail distribution characteristic according to claim 10, characterized in that: Calculating the ratio of the number of the target type of distorted screen images to the number of the to-be-recognized images as a second ratio includes: Determine the number of images to be identified within the current most recent target time period; Determine the number of the target type of distorted screen images within the target time period; The ratio of the number of the target type of distorted screen images to the number of the to-be-recognized images within the target time period is calculated as a second ratio.
12. A device for processing a distorted screen image with a long-tail distribution characteristic, characterized in that: Used to perform the method according to any of claims 1 to 11, comprising: The first sample construction module is configured to construct an initial image set, wherein the initial image set includes the following types of flower screen images: large-area pure color flower screen images, small-area pure color flower screen images, large-area striped flower screen images, small-area striped flower screen images, color-distorted flower screen images, and noise flower screen images; the number of color-distorted flower screen images and noise flower screen images in the initial image set is expanded to generate a first sample set; The first training module is configured to randomly extract a plurality of different types of flower screen images from the first sample set, extract intermediate features of each flower screen image; remix and enhance the intermediate features of different types of flower screen images to generate mixed features; construct a target loss function based on the intermediate features and the mixed features; iteratively train the flower screen recognition model based on the target loss function and the first sample set to obtain a trained flower screen recognition model; The detection module is configured to input the image to be identified into the flower screen recognition model, and determine whether the image to be identified is a flower screen image and the type of the flower screen image through the flower screen recognition model.
13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.
15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.
Citation Information
Patent Citations
Target image recognition method and device, storage medium and electronic equipment
CN114627085A
Long-tail image recognition method based on representation data enhancement and loss rebalance
CN116030302A
Method and system for adaptation of a trained object detection model to account for domain shift
US20230281974A1
Biological feature image recognition method and apparatus, and electronic device and readable storage medium
WO2022105179A1
Cited By
VDI (Virtual Desktop Infrastructure) picture anomaly identification method, system and equipment and computer storage medium
CN121437957A
