Training method and device for pedestrian image quality assessment model

Through the training method of the pedestrian image quality evaluation model, the model parameters are updated using convolution processing and feature similarity, which solves the problem of inaccurate pedestrian image quality evaluation in the prior art, and improves the accuracy and credibility of image quality evaluation.

CN116844026BActive Publication Date: 2024-11-08GRG BANKING EQUIPMENT CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202310818271.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-07-04
Publication Date
2024-11-08
Estimated Expiration
2043-07-04

AI Technical Summary

Technical Problem

The prior art is difficult to effectively evaluate and improve the quality of pedestrian images, resulting in the inability to follow-up personnel analysis and attribute analysis of low-quality images.

Method used

A training method for pedestrian image quality evaluation model is proposed. By acquiring pedestrian image data sets, using backbone network to perform convolution processing, mining the global semantic features and local information features of the image, calculating feature similarity, updating model parameters, and improving evaluation accuracy Sex and credibility.

Benefits of technology

A more comprehensive and objective evaluation of pedestrian image quality is achieved, and the accuracy and reliability of pedestrian image quality evaluation is improved, making it easier to analyze subsequent attributes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116844026B_ABST
    Figure CN116844026B_ABST
Patent Text Reader

Abstract

The present application discloses a training method and device for a pedestrian image quality assessment model, which belongs to the field of image processing technology. The method includes: obtaining a pedestrian image dataset; performing convolution processing through the backbone network of the pedestrian image quality assessment model to obtain feature information of pedestrian sample images and feature information of pedestrian label images output by the backbone network, wherein the feature information of the pedestrian sample images includes sample fine-grained features, sample sharpness features and sample texture features, and the feature information of the pedestrian label images includes label fine-grained features, label sharpness features and label texture features; obtaining feature similarity between the feature information of the pedestrian sample images and the feature information of the pedestrian label images; based on the feature similarity, updating the network parameters of the pedestrian image quality assessment model to obtain a trained pedestrian image quality assessment model. The method can improve the accuracy and credibility of pedestrian image quality assessment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a training method and device for a pedestrian image quality assessment model. Background Art

[0002] With the continuous improvement of video surveillance technology, the application scenarios of video surveillance are becoming more and more extensive. Pedestrian images extracted from video streams can be effectively used in security prevention, personnel statistics and other aspects.

[0003] When extracting pedestrian images, the quality of the obtained pedestrian images often varies greatly due to factors such as complex video surveillance background, pedestrian occlusion, and limited video surveillance resolution. Low-quality images cannot be directly used for subsequent personnel analysis and attribute analysis. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a training method and device for a pedestrian image quality assessment model to improve the accuracy and reliability of pedestrian image quality assessment.

[0005] In a first aspect, the present application provides a method for training a pedestrian image quality assessment model, the method comprising:

[0006] Acquire a pedestrian image dataset, wherein the pedestrian image dataset includes a plurality of pedestrian sample images and a pedestrian label image corresponding to each of the pedestrian sample images;

[0007] The pedestrian image data set is input into the pedestrian image quality assessment model to be trained, and convolution processing is performed through the backbone network of the pedestrian image quality assessment model to obtain feature information of the pedestrian sample image and feature information of the pedestrian label image output by the backbone network, wherein the feature information of the pedestrian sample image includes sample fine-grained features, sample sharpness features and sample texture features, and the feature information of the pedestrian label image includes label fine-grained features, label sharpness features and label texture features;

[0008] Acquire feature similarity between feature information of the pedestrian sample image and feature information of the pedestrian label image;

[0009] Based on the feature similarity, the network parameters of the pedestrian image quality assessment model are updated to obtain the trained pedestrian image quality assessment model.

[0010] According to the training method of the pedestrian image quality evaluation model of the present application, the acquired pedestrian image data set is input into the pedestrian image quality evaluation model to be trained, and convolution processing is performed through the backbone network of the pedestrian image quality evaluation model to mine the global semantic features and local information features of the pedestrian image, and the feature information such as fine-grainedness, sharpness and texture of the pedestrian sample image and the pedestrian label image is obtained, so as to evaluate the quality of the pedestrian image more comprehensively and objectively, and based on the feature similarity, the network parameters of the pedestrian image quality evaluation model are trained and updated to improve the accuracy and credibility of the model in evaluating the quality of pedestrian images.

[0011] According to one embodiment of the present application, the convolution processing is performed through the backbone network of the pedestrian image quality assessment model, including:

[0012] Multi-order central difference convolution is performed on the images of the pedestrian image dataset through the backbone network.

[0013] According to one embodiment of the present application, performing multi-order center difference convolution on the images of the pedestrian image dataset through the backbone network includes:

[0014] Apply the formula

[0015]

[0016] Among them, p0 is the center position of the convolution, θ and λ are hyperparameters, and ω n is the convolution kernel at position p n The weight of is the feature layer at position p n The value of, R is the range of the convolution kernel, is the feature output at position p0 after the central difference convolution, for The corresponding multi-order central differences.

[0017] According to one embodiment of the present application, the backbone network is ResNet18, and the backbone network includes a convolution layer, a central difference convolution layer, a batch normalization layer, an activation function layer, a global pooling layer and an average pooling layer, and the central difference convolution layer is used to perform multi-order central difference convolution.

[0018] According to an embodiment of the present application, the acquiring the feature similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image includes:

[0019] The feature similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image is calculated by using a bulldozer distance loss function.

[0020] According to one embodiment of the present application, the training strategy of the pedestrian image quality assessment model is a cosine annealing learning rate strategy.

[0021] In a second aspect, the present application provides a method for evaluating the quality of pedestrian images, the method comprising:

[0022] Obtain the pedestrian image to be evaluated;

[0023] Inputting the pedestrian image to be evaluated into the pedestrian image quality evaluation model, and obtaining the quality evaluation score of the pedestrian image to be evaluated output by the pedestrian image quality evaluation model;

[0024] The pedestrian image quality assessment model is trained based on the training method of the pedestrian image quality assessment model described in the first aspect.

[0025] According to the pedestrian image quality evaluation method of the present application, by inputting the acquired pedestrian image to be evaluated into a trained pedestrian image quality evaluation model, the fine-grained features, sharpness features and texture features of the pedestrian image can be accurately extracted, and the quality evaluation score of the pedestrian image to be evaluated can be output. This can improve the accuracy of the pedestrian image quality evaluation and facilitate the subsequent analysis of pedestrian attributes.

[0026] In a third aspect, the present application provides a training device for a pedestrian image quality assessment model, the device comprising:

[0027] A first acquisition module is used to acquire a pedestrian image dataset, wherein the pedestrian image dataset includes a plurality of pedestrian sample images and a pedestrian label image corresponding to each of the pedestrian sample images;

[0028] a first processing module, configured to input the pedestrian image data set into a pedestrian image quality assessment model to be trained, and perform convolution processing through a backbone network of the pedestrian image quality assessment model to obtain feature information of the pedestrian sample image and feature information of the pedestrian label image output by the backbone network, wherein the feature information of the pedestrian sample image includes sample fine-grained features, sample sharpness features, and sample texture features, and the feature information of the pedestrian label image includes label fine-grained features, label sharpness features, and label texture features;

[0029] A second acquisition module, used to acquire feature similarity between feature information of the pedestrian sample image and feature information of the pedestrian label image;

[0030] The second processing module is used to update the network parameters of the pedestrian image quality assessment model based on the feature similarity to obtain the trained pedestrian image quality assessment model.

[0031] According to the training device of the pedestrian image quality assessment model of the present application, the acquired pedestrian image data set is input into the pedestrian image quality assessment model to be trained, and convolution processing is performed through the backbone network of the pedestrian image quality assessment model to mine the global semantic features and local information features of the pedestrian image, and the feature information such as fine-grainedness, sharpness and texture of the pedestrian sample image and the pedestrian label image is obtained, so as to evaluate the quality of the pedestrian image more comprehensively and objectively, and update the network parameters of the pedestrian image quality assessment model based on the feature similarity, so as to improve the accuracy and credibility of the model in evaluating the quality of the pedestrian image.

[0032] In a fourth aspect, the present application provides a pedestrian image quality assessment device, the device comprising:

[0033] The third acquisition module is used to acquire the image of the pedestrian to be evaluated;

[0034] A third processing module is used to input the pedestrian image to be evaluated into the pedestrian image quality evaluation model to obtain the quality evaluation score of the pedestrian image to be evaluated output by the pedestrian image quality evaluation model;

[0035] The pedestrian image quality assessment model is obtained by training based on the training method of the pedestrian image quality assessment model described in the first aspect.

[0036] According to the pedestrian image quality evaluation device of the present application, by inputting the acquired pedestrian image to be evaluated into a trained pedestrian image quality evaluation model, the fine-grained features, sharpness features and texture features of the pedestrian image can be accurately extracted, and the quality evaluation score of the pedestrian image to be evaluated can be output. This can improve the accuracy of the pedestrian image quality evaluation and facilitate the subsequent analysis of pedestrian attributes.

[0037] In a fifth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the pedestrian image quality assessment model as described in the first aspect above is implemented.

[0038] In a sixth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method for the pedestrian image quality assessment model as described in the first aspect above.

[0039] In a seventh aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the training method of the pedestrian image quality assessment model as described in the first aspect above.

[0040] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0042] Figure 1 This is one of the flow charts of the training method of the pedestrian image quality assessment model provided in the embodiment of the present application;

[0043] Figure 2 This is the second flow chart of the training method of the pedestrian image quality assessment model provided in the embodiment of the present application;

[0044] Figure 3 is a flow chart of a pedestrian image quality assessment method provided in an embodiment of the present application;

[0045] Figure 4 It is a structural schematic diagram of a training device for a pedestrian image quality assessment model provided in an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of the structure of a pedestrian image quality assessment device provided in an embodiment of the present application;

[0047] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0049] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0050] In combination with the accompanying drawings, the training method of the pedestrian image quality assessment model, the pedestrian image quality assessment method, the training device of the pedestrian image quality assessment model, the pedestrian image quality assessment device, the electronic device and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0051] The training method of the pedestrian image quality assessment model can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0052] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0053] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0054] The training method of the pedestrian image quality assessment model provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the training method of the pedestrian image quality assessment model. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The training method of the pedestrian image quality assessment model provided in the embodiment of the present application is described below using an electronic device as an example of the execution subject.

[0055] The pedestrian image quality assessment model is used to evaluate the quality of pedestrian images. By training the pedestrian image quality assessment model, accurate pedestrian image quality assessment can be obtained.

[0056] like Figure 1 As shown, the training method of the pedestrian image quality assessment model includes: steps 110 to 140.

[0057] Step 110: Obtain a pedestrian image dataset.

[0058] The pedestrian image dataset includes a plurality of pedestrian sample images and a pedestrian label image corresponding to each pedestrian sample image.

[0059] It should be noted that the pedestrian image dataset is a preprocessed dataset, and the dataset for pedestrian image quality evaluation collected by an image acquisition device can be preprocessed to obtain a pedestrian image dataset.

[0060] In actual implementation, the original data of the video surveillance scene can be collected and screened, and the video of the corresponding scene can be saved frame by frame. For example, an image is saved every 10 frames to obtain a data set for pedestrian image quality evaluation.

[0061] The dataset used for pedestrian image quality evaluation is preprocessed by pedestrian detection and data expansion to obtain a pedestrian image dataset.

[0062] In this embodiment, a pedestrian detection model may be used to extract valid pedestrian images from images in a data set for pedestrian image quality assessment, and then data expansion may be performed on the valid pedestrian images by methods such as image rotation and image noise addition.

[0063] Among them, the pedestrian detection model can be a detection model such as YOLOv7, YOLOv5, SSD, etc., and a valid pedestrian image is extracted through the pedestrian detection model.

[0064] It should be noted that when video surveillance shoots pedestrians, the pedestrians and the surrounding scenes are captured. Before obtaining the preprocessed data set, the collected images are passed through the pedestrian detection model to eliminate useless data such as the surrounding scenes, and retain partial images of pedestrians, that is, effective pedestrian images, which can effectively reduce the interference of complex backgrounds on the quality evaluation of pedestrian images.

[0065] It can be understood that the obtained pedestrian image dataset includes two types of images: pedestrian sample images and pedestrian label images corresponding to the pedestrian sample images, wherein the pedestrian label images include label information of pedestrian image quality evaluation of the corresponding pedestrian sample images.

[0066] Step 120: input the pedestrian image data set into the pedestrian image quality assessment model to be trained, perform convolution processing through the backbone network of the pedestrian image quality assessment model, and obtain feature information of the pedestrian sample image and feature information of the pedestrian label image output by the backbone network.

[0067] Among them, the backbone network refers to the backbone part of the neural network model, which is used to extract feature information from the image.

[0068] The feature information of the pedestrian sample image includes sample fine-grained feature information, sample sharpness feature information and sample texture feature information; the feature information of the pedestrian label image includes label fine-grained feature information, label sharpness feature information and label texture feature information.

[0069] In this embodiment, the acquired pedestrian image data set is input into the pedestrian image quality assessment model to be trained, and convolution processing is performed in the backbone network to obtain the feature information of the output pedestrian sample image and the feature information such as fine-grainedness, sharpness and texture of the pedestrian label image, which is conducive to training the pedestrian image quality assessment model.

[0070] It can be understood that the backbone network processes the images in the pedestrian image dataset by using convolution processing to extract feature information such as image granularity, sharpness, and texture.

[0071] After the pedestrian image dataset is acquired in step 110, before the pedestrian image dataset is input into the pedestrian image quality assessment model to be trained in step 120, the images in the pedestrian image dataset can be set to a unified image specification to facilitate the pedestrian image quality assessment model to process the images.

[0072] For example, the size of all images in the pedestrian image dataset can be set to 192x64 resolution, and the batch size of the pedestrian image dataset can be set to 64.

[0073] The image pixel range of the pedestrian image dataset is normalized by subtracting the mean [0, 0, 0] and dividing it by the standard deviation [255, 255, 255]. This unifies the image specifications of the pedestrian image dataset, which is beneficial for the pedestrian image quality evaluation model to process the image.

[0074] Step 130: Obtain feature similarity between feature information of the pedestrian sample image and feature information of the pedestrian label image.

[0075] Among them, feature similarity is the similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image. Feature similarity can reflect the gap between the predicted output of the pedestrian image quality evaluation model and the label.

[0076] In actual implementation, after obtaining the similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image, the loss function in the pedestrian image quality evaluation model is used to calculate the feature similarity.

[0077] For example, by using the loss function to calculate feature similarity, the similarity can be calculated for feature information such as fine-grainedness, sharpness, and texture in the pedestrian sample image and the pedestrian label image, and the calculation results are weighted and summed to obtain the feature similarity.

[0078] For another example, the loss function can be used to calculate feature similarity. The feature similarity can be obtained by respectively obtaining corresponding comprehensive features from feature information such as fine-grainedness, sharpness, and texture in the pedestrian sample image and the pedestrian label image, and performing calculations based on the corresponding comprehensive features.

[0079] Step 140: Based on the feature similarity, update the network parameters of the pedestrian image quality assessment model to obtain a trained pedestrian image quality assessment model.

[0080] Among them, network parameters refer to parameters that can independently reflect the characteristics of the backbone network.

[0081] For example, based on feature similarity, the weights of a pedestrian image quality assessment model can be updated, where the weights are parameters that represent the relationship between input and output samples.

[0082] In this embodiment, according to the feature similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image, the network parameters such as the weight of the pedestrian image quality assessment model are updated until the training is completed to obtain a trained pedestrian image quality assessment model.

[0083] In actual implementation, the network parameters of the pedestrian image quality assessment model can be updated through back propagation of the network.

[0084] In this embodiment, when training the pedestrian image quality assessment model, a target threshold or number of iterations may be set as a condition for completing the training.

[0085] For example, when the obtained feature similarity is greater than the target threshold, the network parameters are stopped from being updated to obtain a trained pedestrian image quality evaluation model.

[0086] For another example, when the number of iterations of the pedestrian image quality assessment model training reaches a preset number of iterations, the training is stopped to obtain a trained pedestrian image quality assessment model.

[0087] In the related art, when evaluating the quality of pedestrian images, the pedestrian's head, upper body, lower body, shoes and other areas are usually extracted separately, and the images of each local area are scored to finally obtain the pedestrian quality evaluation. The global characteristics of the pedestrian image are not taken into account, and the reliability of the pedestrian image quality evaluation is poor.

[0088] In the embodiment of the present application, the backbone network of the pedestrian image quality evaluation model obtains the fine-grained, sharpness, texture and other features of the pedestrian sample images and the pedestrian label images through convolution processing, fully mines the global semantic features and local information features of the pedestrian images, and performs model training. It can evaluate the quality of pedestrian images more comprehensively and objectively, and improve the accuracy of the pedestrian image quality evaluation model in evaluating the quality of pedestrian images.

[0089] According to the training method of the pedestrian image quality assessment model provided in the embodiment of the present application, the acquired pedestrian image data set is input into the pedestrian image quality assessment model to be trained, and convolution processing is performed through the backbone network of the pedestrian image quality assessment model to mine the global semantic features and local information features of the pedestrian image, and the feature information such as fine-grainedness, sharpness and texture of the pedestrian sample image and the pedestrian label image is obtained, so as to evaluate the quality of the pedestrian image more comprehensively and objectively, and based on the feature similarity, the network parameters of the pedestrian image quality assessment model are trained and updated to improve the accuracy and credibility of the model in evaluating the quality of pedestrian images.

[0090] In some embodiments, convolution processing is performed through the backbone network of the pedestrian image quality assessment model, including:

[0091] Multi-order central difference convolutions are performed on the images of the pedestrian image dataset through the backbone network.

[0092] It should be noted that the multi-order central difference convolution can be a multi-order calculation method such as a second-order central difference convolution, a third-order central difference convolution, and a fourth-order central difference convolution. The embodiment of the present application performs convolution processing on the image through multi-order central difference convolution, which can not only detect the edge of the image and extract feature information, but also ensure the accuracy of the extracted image features.

[0093] In the related art, the feature information of the images in the pedestrian image dataset is extracted by using the center difference convolution, which can only extract the fine-grained features of the image, and easily lead to inaccurate pedestrian image quality evaluation.

[0094] In an embodiment of the present application, a multi-order central difference convolution is used to extract feature information of images of a pedestrian image dataset, which can extract fine-grained features of the images of the pedestrian image dataset, and can also extract sharpness features and texture features of the images of the pedestrian image dataset. By extracting more effective image features, the accuracy and reliability of pedestrian image quality evaluation are improved.

[0095] In this embodiment, the backbone network of the pedestrian image quality assessment model performs second-order central difference convolution on the images of the pedestrian image dataset to obtain fine-grained features, sharpness features, and texture features of the images of the pedestrian image dataset.

[0096] In an embodiment of the present application, by performing multi-order central difference convolution on the pedestrian sample images and pedestrian label images in the pedestrian image dataset in the backbone network of the pedestrian image quality assessment model, the fine-grained features, sharpness features and texture features of the images of the pedestrian image dataset are obtained. This can solve the problem that the existing central difference convolution can only extract the fine-grained features of the pedestrian image dataset, and improve the accuracy and credibility of the pedestrian image quality assessment by extracting more effective image features.

[0097] In some embodiments, performing multi-order center difference convolution on the images of the pedestrian image dataset through the backbone network may include:

[0098] Apply the formula

[0099]

[0100] Among them, p0 is the center position of the convolution, θ and λ are hyperparameters, and ω n is the convolution kernel at position p n The weight of is the feature layer at position p n The value of, R is the range of the convolution kernel, is the feature output at position p0 after the central difference convolution, for The corresponding multi-order central differences.

[0101] In this embodiment, the second-order central difference convolution is taken as an example, where: for Second order central difference, The application formula is:

[0102]

[0103] in, is the feature layer at position p n The value of .

[0104] In this embodiment, different from the existing central difference convolution method, the images of the pedestrian image dataset are subjected to multi-order central difference convolution, so that the fine-grained features, sharpness features and texture features of the images of the pedestrian image dataset can be obtained.

[0105] In some embodiments, the backbone network is ResNet18, and the backbone network includes a convolution layer, a central difference convolution layer, a batch normalization layer, an activation function layer, a global pooling layer and an average pooling layer, wherein the central difference convolution layer is used to perform a second-order central difference convolution.

[0106] Among them, the ResNet18 network uses ResNet as the basic framework and has a depth of 18 layers. The ResNet18 network realizes functions such as pedestrian image recognition and feature information extraction.

[0107] The convolution layer is used to extract the image features of pedestrian images; the center difference convolution layer is used to perform multi-order center difference convolution; the batch normalization layer is used to standardize the data; the activation function layer is used to add nonlinear factors to solve problems that cannot be solved by linear models; the global pooling layer is used to replace the fully connected layer and can accept images of any size; the average pooling layer is used to replace the fully connected layer to reduce the pressure in the learning process.

[0108] In this embodiment, ResNet18 is used as the backbone network, and the central difference convolution layer of the ResNet18 network is used to perform multi-order central difference convolution, and cooperates with the convolution layer, batch normalization layer, activation function layer, global pooling layer and average pooling layer to realize the collection of feature information such as fine-grainedness, sharpness and texture of pedestrian sample images and pedestrian label images.

[0109] The loss function of the pedestrian image quality evaluation model is introduced below.

[0110] In some embodiments, obtaining feature similarity between feature information of a pedestrian sample image and feature information of a pedestrian label image includes:

[0111] The feature similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image is calculated through the bulldozer distance loss function.

[0112] The loss function uses Wasserstein Distance, through which the image features are subjected to a unified feature space, thereby improving the training efficiency of the model.

[0113] In actual implementation, the bulldozer loss function is used to calculate the feature similarity between the feature information of the pedestrian sample image and the pedestrian label image, and the quality of the pedestrian image quality evaluation model is judged by the feature similarity.

[0114] The following introduces the training strategy of the pedestrian image quality assessment model.

[0115] In some embodiments, the training strategy of the pedestrian image quality assessment model is a cosine annealing learning rate strategy.

[0116] The training strategy adopts the cosine annealing learning rate strategy (CosineAnnealingLR), and the learning rate is adjusted by the cosine annealing learning rate strategy.

[0117] Among them, the cosine annealing learning rate strategy is a training strategy that uses the cosine function to decrease, which can make the learning rate decrease slowly first and then decrease quickly.

[0118] In this embodiment, the training strategy of the pedestrian image quality assessment model adopts a cosine annealing learning rate strategy, and the learning rate is decreased by a linear function, which helps to maintain the accuracy of the pedestrian image quality assessment model.

[0119] The formula of the cosine annealing learning rate strategy is introduced in detail below.

[0120] The formula of the cosine annealing learning rate strategy is:

[0121]

[0122] Among them, l r Indicates the learning rate of the current iteration, l R represents the initial learning rate, l min represents the minimum learning rate, cos represents the cosine function, epoch is the current iteration number, T max Represents 1 / 2 of the cosine period.

[0123] A specific embodiment is described below.

[0124] like Figure 2 As shown, the video surveillance data is collected and screened, the video of the video surveillance scene is saved by skipping frames, and an image is saved every 10 frames.

[0125] A pedestrian image dataset is constructed, and pedestrian detection models such as YOLOv7, YOLOv5, and SSD are used to extract valid pedestrian images from the collected images. Then, data expansion is performed on the valid pedestrian images by methods such as image rotation and image noise addition.

[0126] Unify the image specifications, set the image size of the pedestrian image dataset to 192x64, set the batch size to 64, normalize, subtract the mean [0, 0, 0], and divide by the standard deviation [255, 255, 255].

[0127] A uniformly distributed sampling method is used for the pedestrian image dataset, ResNet18 is selected as the backbone network, and the convolution method in the pedestrian image quality evaluation model is set to a multi-order central difference convolution method.

[0128] Adopt the cosine annealing learning rate strategy: As a training strategy, where l r Indicates the learning rate of the current iteration, l R represents the initial learning rate, l min represents the minimum learning rate, cos represents the cosine function, epoch is the current iteration number, T max Represents 1 / 2 of the cosine period.

[0129] Set the total number of iterations to 300, T max is 300, the minimum learning rate l min is 0, the initial learning rate is l R is 0.1.

[0130] The pedestrian sample image and the pedestrian label image corresponding to the pedestrian sample image are passed through the convolution layer, central difference convolution layer, batch normalization layer, activation function layer, global pooling layer and average pooling layer of the backbone network to extract the feature information of the pedestrian sample image and the feature information of the pedestrian label image.

[0131] The bulldozer distance loss function is used to calculate the similarity between pedestrian sample images and pedestrian label images.

[0132] Through the back propagation of the network, the network parameters are updated and the pedestrian image quality evaluation model is trained.

[0133] Use the test image to test the trained network and output the test image quality evaluation score.

[0134] In this embodiment, by setting a multi-order central difference convolution method, the fine-grained features, sharpness features and texture features in the pedestrian sample image can be accurately extracted, which can improve the credibility of the pedestrian image quality evaluation and facilitate the subsequent analysis of pedestrian attributes.

[0135] The embodiment of the present application also provides a method for evaluating the quality of pedestrian images.

[0136] like Figure 3 As shown, the pedestrian image quality assessment method includes: step 310 and step 320.

[0137] Step 310: Obtain an image of the pedestrian to be evaluated.

[0138] The image of the pedestrian to be evaluated is a pedestrian image after data preprocessing.

[0139] In this step, the image of the pedestrian under video surveillance is saved by skipping frames, and the pedestrian image is preprocessed to obtain the image of the pedestrian to be evaluated.

[0140] Step 320: input the pedestrian image to be evaluated into the pedestrian image quality evaluation model to obtain the quality evaluation score of the pedestrian image to be evaluated output by the pedestrian image quality evaluation model.

[0141] Among them, the pedestrian image quality assessment model is trained based on the training method of the above-mentioned pedestrian image quality assessment model.

[0142] In this embodiment, the pedestrian image to be evaluated is input into a trained pedestrian image quality assessment model, and the fine-grained features, sharpness features and texture features of the pedestrian image to be evaluated are obtained by using a multi-order central difference convolution method, and the quality evaluation score of the pedestrian image to be evaluated is output.

[0143] According to the pedestrian image quality evaluation method provided in the embodiment of the present application, by inputting the acquired pedestrian image to be evaluated into a trained pedestrian image quality evaluation model, the fine-grained features, sharpness features and texture features of the pedestrian image can be accurately extracted, and the quality evaluation score of the pedestrian image to be evaluated can be output. This can improve the accuracy of the pedestrian image quality evaluation and facilitate the subsequent analysis of pedestrian attributes.

[0144] The training method of the pedestrian image quality assessment model provided in the embodiment of the present application can be executed by a training device for the pedestrian image quality assessment model. In the embodiment of the present application, the training method of the pedestrian image quality assessment model executed by the training device for the pedestrian image quality assessment model is taken as an example to illustrate the training device for the pedestrian image quality assessment model provided in the embodiment of the present application.

[0145] The embodiment of the present application also provides a training device for a pedestrian image quality assessment model.

[0146] like Figure 4 As shown, the training device of the pedestrian image quality assessment model includes:

[0147] A first acquisition module 410 is used to acquire a pedestrian image dataset, where the pedestrian image dataset includes a plurality of pedestrian sample images and a pedestrian label image corresponding to each pedestrian sample image;

[0148] The first processing module 420 is used to input the pedestrian image data set into the pedestrian image quality assessment model to be trained, perform convolution processing through the backbone network of the pedestrian image quality assessment model, and obtain feature information of the pedestrian sample image and feature information of the pedestrian label image output by the backbone network, wherein the feature information of the pedestrian sample image includes sample fine-grained features, sample sharpness features, and sample texture features, and the feature information of the pedestrian label image includes label fine-grained features, label sharpness features, and label texture features;

[0149] The second acquisition module 430 is used to acquire the feature similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image;

[0150] The second processing module 440 is used to update the network parameters of the pedestrian image quality assessment model based on the feature similarity to obtain a trained pedestrian image quality assessment model.

[0151] According to the training device of the pedestrian image quality assessment model provided by the embodiment of the present application, the acquired pedestrian image data set is input into the pedestrian image quality assessment model to be trained, and convolution processing is performed through the backbone network of the pedestrian image quality assessment model to mine the global semantic features and local information features of the pedestrian image, and the feature information such as fine-grainedness, sharpness and texture of the pedestrian sample image and the pedestrian label image is obtained, so as to evaluate the quality of the pedestrian image more comprehensively and objectively, and update the network parameters of the pedestrian image quality assessment model based on the feature similarity, so as to improve the accuracy and credibility of the model in evaluating the quality of the pedestrian image.

[0152] In some embodiments, the first processing module 420 is used to perform convolution processing through the backbone network of the pedestrian image quality assessment model, including:

[0153] Multi-order central difference convolutions are performed on the images of the pedestrian image dataset through the backbone network.

[0154] In some embodiments, the first processing module 420 is used to perform multi-order central difference convolution on the images of the pedestrian image dataset through the backbone network, including:

[0155] Apply the formula

[0156]

[0157] Among them, p0 is the center position of the convolution, θ and λ are hyperparameters, and ω n is the convolution kernel at position p n The weight of is the feature layer at position p n The value of, R is the range of the convolution kernel, is the feature output at position p0 after the central difference convolution, for The corresponding multi-order central differences.

[0158] In some embodiments, the first processing module 420 is used for a backbone network of ResNet18, and the backbone network includes a convolution layer, a central difference convolution layer, a batch normalization layer, an activation function layer, a global pooling layer and an average pooling layer, and the central difference convolution layer is used to perform multi-order central difference convolution.

[0159] In some embodiments, the second acquisition module 430 is used to acquire the feature similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image, including:

[0160] The feature similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image is calculated through the bulldozer distance loss function.

[0161] In some embodiments, the training strategy of the pedestrian image quality assessment model is a cosine annealing learning rate strategy.

[0162] The embodiment of the present application also provides a pedestrian image quality assessment device.

[0163] like Figure 5 As shown, the pedestrian image quality assessment device comprises:

[0164] The third acquisition module 510 is used to acquire the pedestrian image to be evaluated;

[0165] The third processing module 520 is used to input the pedestrian image to be evaluated into the pedestrian image quality evaluation model to obtain the quality evaluation score of the pedestrian image to be evaluated output by the pedestrian image quality evaluation model;

[0166] Among them, the pedestrian image quality assessment model is trained based on the training method of the above-mentioned pedestrian image quality assessment model.

[0167] According to the pedestrian image quality evaluation device provided in the embodiment of the present application, by inputting the acquired pedestrian image to be evaluated into a trained pedestrian image quality evaluation model, the fine-grained features, sharpness features and texture features of the pedestrian image can be accurately extracted, and the quality evaluation score of the pedestrian image to be evaluated can be output. This can improve the accuracy of the pedestrian image quality evaluation and facilitate the subsequent analysis of pedestrian attributes.

[0168] The training device of the pedestrian image quality assessment model in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0169] The training device of the pedestrian image quality assessment model in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0170] The training device of the pedestrian image quality assessment model provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0171] In some embodiments, Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, each process of the training method embodiment of the above-mentioned pedestrian image quality assessment model is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0172] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0173] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the training method embodiment of the above-mentioned pedestrian image quality assessment model are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0174] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0175] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the training method of the above-mentioned pedestrian image quality assessment model.

[0176] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0177] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned training method embodiment of the pedestrian image quality assessment model, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0178] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0179] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0180] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0181] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0182] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0183] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A training method for a pedestrian image quality assessment model, characterized in that: include: Acquire a pedestrian image dataset, wherein the pedestrian image dataset includes a plurality of pedestrian sample images and a pedestrian label image corresponding to each of the pedestrian sample images; The pedestrian image data set is input into the pedestrian image quality assessment model to be trained, and multi-order central difference convolution is performed on the images of the pedestrian image data set through the backbone network of the pedestrian image quality assessment model to obtain feature information of the pedestrian sample images and feature information of the pedestrian label images output by the backbone network, wherein the feature information of the pedestrian sample images includes sample fine-grained features, sample sharpness features, and sample texture features, and the feature information of the pedestrian label images includes label fine-grained features, label sharpness features, and label texture features; Acquire feature similarity between feature information of the pedestrian sample image and feature information of the pedestrian label image; Based on the feature similarity, updating the network parameters of the pedestrian image quality assessment model to obtain the trained pedestrian image quality assessment model; The performing multi-order center difference convolution on the images of the pedestrian image dataset through the backbone network includes: Apply the formula Among them, p0 is the center position of the convolution, θ and λ are hyper parameters, is the convolution kernel at position p n The weight of is the feature layer at position p n The value of, R is the range of the convolution kernel, After the central difference convolution, the position is p 0 The output features, for The corresponding multi-order central differences.

2. The training method of the pedestrian image quality assessment model according to claim 1, characterized in that: The backbone network is ResNet18, which includes a convolution layer, a central difference convolution layer, a batch normalization layer, an activation function layer, a global pooling layer and an average pooling layer. The central difference convolution layer is used to perform multi-order central difference convolution.

3. The training method for a pedestrian image quality assessment model according to any one of claims 1 to 2, characterized in that: The obtaining of feature similarity between feature information of the pedestrian sample image and feature information of the pedestrian label image includes: The feature similarity between the feature information of the pedestrian sample image and the feature information of the pedestrian label image is calculated by using a bulldozer distance loss function.

4. The training method for a pedestrian image quality assessment model according to any one of claims 1 to 2, characterized in that: The training strategy of the pedestrian image quality assessment model is a cosine annealing learning rate strategy.

5. A pedestrian image quality assessment method, characterized in that: include: Obtain the pedestrian image to be evaluated; Inputting the pedestrian image to be evaluated into the pedestrian image quality evaluation model, and obtaining the quality evaluation score of the pedestrian image to be evaluated output by the pedestrian image quality evaluation model; The pedestrian image quality assessment model is trained based on the training method of the pedestrian image quality assessment model described in any one of claims 1 to 4.

6. A training device for a pedestrian image quality assessment model, characterized in that: The device is used to execute the training method of the pedestrian image quality assessment model according to any one of claims 1 to 4, and comprises: A first acquisition module is used to acquire a pedestrian image dataset, wherein the pedestrian image dataset includes a plurality of pedestrian sample images and a pedestrian label image corresponding to each of the pedestrian sample images; a first processing module, configured to input the pedestrian image data set into a pedestrian image quality assessment model to be trained, and perform convolution processing through a backbone network of the pedestrian image quality assessment model to obtain feature information of the pedestrian sample image and feature information of the pedestrian label image output by the backbone network, wherein the feature information of the pedestrian sample image includes sample fine-grained features, sample sharpness features, and sample texture features, and the feature information of the pedestrian label image includes label fine-grained features, label sharpness features, and label texture features; A second acquisition module, used to acquire feature similarity between feature information of the pedestrian sample image and feature information of the pedestrian label image; The second processing module is used to update the network parameters of the pedestrian image quality assessment model based on the feature similarity to obtain the trained pedestrian image quality assessment model.

7. A pedestrian image quality assessment device, characterized in that: include: The third acquisition module is used to acquire the image of the pedestrian to be evaluated; A third processing module is used to input the pedestrian image to be evaluated into the pedestrian image quality evaluation model to obtain the quality evaluation score of the pedestrian image to be evaluated output by the pedestrian image quality evaluation model; The pedestrian image quality assessment model is trained based on the training method of the pedestrian image quality assessment model described in any one of claims 1 to 4.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the training method of the pedestrian image quality assessment model as described in any one of claims 1 to 4 or the pedestrian image quality assessment method as described in claim 6.

Citation Information

Patent Citations

  • Image sample generation capability multi-dimensional evaluation method based on adversarial sample definition

    CN111858345A

  • Eye fundus image recognition and classification method based on deep learning

    CN112966759A

  • Method and device for detecting living body face image, computer equipment and storage medium

    CN113269149A

  • Herpes image recognition method, device, computer equipment and storage medium

    CN113487610A

  • Face image quality evaluation model construction method and device, equipment and medium

    CN113505854A