Pedestrian image quality evaluation method, device and equipment and readable storage medium

Abstract

The application discloses a kind of pedestrian image quality evaluation method, device, equipment and readable storage medium, the method includes: obtaining batch original pedestrian image, adding quality factor label to original pedestrian image, generate training set, wherein quality factor includes pedestrian has or not, shelter, illumination, blur and posture;Training set is input to initial deep learning model and is trained, until the loss value of model tends to convergence, stop training model, obtain pedestrian image quality evaluation model;Obtain the pedestrian image to be detected;Pedestrian image quality evaluation model is based on and is identified to the pedestrian image to be detected, obtains the quality evaluation result of the pedestrian image to be detected.Based on the present application, the technical problem that single dimension deep learning cannot accurately identify pedestrian image in prior art by using template matching or is solved.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and more specifically to a method, apparatus, device, and readable storage medium for evaluating the quality of pedestrian images. Background Technology

[0002] Pedestrian re-identification is a technique that extracts feature information from pedestrian images for identity recognition. It utilizes computer vision technology to determine the presence of a specific pedestrian in an image or video sequence. Given a surveillance pedestrian image, it retrieves images of that pedestrian across different devices, aiming to overcome the visual limitations of fixed cameras and identify pedestrians under surveillance. The quality of the pedestrian image directly determines the usability and accuracy of the target detection algorithm. Although current pedestrian detection algorithms have significantly improved in performance, some false detections (e.g., no pedestrian in the detection box) and incomplete detection results are unavoidable in practical applications, such as the detection box only containing the upper body, lower body, left body, right body, or even multiple people. Furthermore, factors such as lighting and blurring can cause pedestrian images to be overexposed, underexposed, or excessively blurred, severely affecting the reliability and accuracy of pedestrian re-identification algorithms.

[0003] Existing technologies for object detection algorithms generally address false detections and incomplete detections by employing template matching or deep learning methods. Firstly, template matching has inherent limitations, primarily because it can only perform parallel movements. If the target in the original image rotates or changes size, the algorithm fails, making it largely unusable for complex tasks. Secondly, deep learning-based methods offer a simpler solution, typically using classification to further clean up false detections or incomplete targets, filtering out unreasonable images. Patent [CN113076917 A] presents a deep learning-based method that processes pedestrian quality assessment requirements as two parallel tasks: 1. Binary classification of data into occluded and unoccluded categories; 2. Multi-classification of data into five categories: upper body, lower body, left body, right body, and full body. Both tasks share a single backbone network, optimized using weighted loss. However, due to the diverse factors influencing pedestrian quality, this deep learning-based recognition algorithm is too simplistic and cannot accurately identify pedestrian images.

[0004] There is currently no effective solution to the technical problem that existing technologies, such as template matching or single-dimensional deep learning, cannot accurately identify pedestrian images. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, device, and storage medium for evaluating the quality of pedestrian images, which can solve the technical problem that existing technologies using template matching or single-dimensional deep learning cannot accurately identify pedestrian images.

[0006] One aspect of the present invention provides a method for evaluating the quality of pedestrian images. The method includes: acquiring a batch of original pedestrian images; adding quality factor labels to the original pedestrian images to generate a training set, wherein the quality factors include pedestrian presence / absence, occlusion, illumination, blur, and pose; inputting the training set into an initial deep learning model for training until the model's loss value tends to converge, stopping the model training to obtain a pedestrian image quality evaluation model; acquiring pedestrian images to be detected; and performing recognition processing on the pedestrian images to be detected based on the pedestrian image quality evaluation model to obtain a quality evaluation result of the pedestrian images to be detected.

[0007] Optionally, the pedestrian image to be detected is processed based on the pedestrian image quality assessment model to obtain the quality assessment result of the pedestrian image to be detected, including: inputting the pedestrian image to be detected into the convolutional neural network of the pedestrian image quality assessment model for feature recognition to obtain a multi-dimensional feature vector; using multiple sub-task neural networks to perform prediction operations on the multi-dimensional feature vector to obtain the predicted probability value of the pedestrian image to be detected in each sub-task neural network; and integrating, classifying and outputting the predicted probability value based on the softmax layer to obtain the corresponding quality assessment result of the pedestrian image to be detected.

[0008] Optionally, quality factor labels are added to the original pedestrian images, including: classifying the original pedestrian images to obtain a pedestrian image set and a background image set, and adding a "pedestrian present" label to the images in the pedestrian image set and a "no pedestrian" label to the images in the background image set; randomly cropping the pedestrian image set, and adding quality factor labels of different occlusion levels to the original images and the cropped images in the pedestrian image set; determining the feature values ​​of illumination, blur, and pose of the pedestrian image set based on a preset feature recognition algorithm, and adding quality factor labels of different levels corresponding to illumination, blur, and pose to the images in the pedestrian image set according to the determined feature values.

[0009] Optionally, the pedestrian image set is randomly cropped, and quality factor labels with different occlusion levels are added to the original image and the cropped image in the pedestrian image set. This includes: adding an unoccluded label to the original image in the pedestrian image set; randomly cropping the pedestrian image set to extract local images containing pedestrians and adding a first-level occlusion label to the local images; obtaining pedestrian location information from the cropped local images containing pedestrians and the original image of the pedestrian image set; performing coordinate fusion of the pedestrian location information with the local images and the original image of the pedestrian image set to determine the number of people in the fused image; filtering out pedestrian images containing multiple people based on the number of people; and adding a second-level occlusion label to the pedestrian images containing multiple people.

[0010] Optionally, the training set is input into the initial deep learning model for training until the model's loss value tends to converge, at which point the training is stopped, resulting in a pedestrian image quality assessment model. This model includes: initializing the weights of the forward calculation formula using random values; inputting the training set pedestrian images labeled with quality factors into the convolutional neural network layer of the initial deep learning network for supervised learning to obtain multi-layer feature vectors; continuing to input the multi-layer feature vectors into the initial sub-task neural network layer along the forward direction for quality feature calculation, obtaining the predicted values ​​of the training set pedestrian images in each initial sub-task neural network layer, where the initial sub-task neural network layer includes five sub-tasks: pedestrian presence / absence, occlusion, illumination, blur, and pose; obtaining the weight values ​​of each sub-task neural network layer, calculating the error between the predicted value and the true value of the label using the first adjustment function of the softmax layer, and using the normalized sub-task weight values ​​to perform a weighted sum of the errors of each sub-task. If the calculated result exceeds a first preset threshold, the weights of each subtask are updated. The first adjustment function is focal loss, and the weight of the subtask indicating the presence or absence of pedestrians is higher than that of other types of subtasks. A new round of forward propagation is performed using the updated weights. This forward propagation and weight update process is repeated until the error value is less than or equal to the first preset threshold. At this point, training of the initial deep learning model is stopped, resulting in an intermediate model. To improve the generalization performance of the deep learning model and make it applicable to more scenarios, the final error value of the intermediate model is determined. If the final error value exceeds a second preset threshold, the first adjustment function of the intermediate model is replaced with the second adjustment function. The weights of each subtask of the intermediate model are then corrected using the second adjustment function. The intermediate model is repeatedly trained based on the corrected weights until the error value is less than or equal to the second preset threshold. At this point, training stops, resulting in a pedestrian image quality assessment model. The second adjustment function is cosloss.

[0011] Optionally, the initial deep learning model can be any of the network structures ResNet18, MobileNet, or shuffleNet.

[0012] Another aspect of the present invention provides a pedestrian image quality assessment device, comprising: a labeling module for acquiring a batch of original pedestrian images, adding quality factor labels to the original pedestrian images, and generating a training set, wherein the quality factors include pedestrian presence / absence, occlusion, illumination, blur, and pose; a training module for inputting the training set into an initial deep learning model for training until the model's loss value tends to converge, then stopping the model training to obtain a pedestrian image quality assessment model; an acquisition module for acquiring pedestrian images to be detected; and a recognition module for performing recognition processing on the pedestrian images to be detected based on the pedestrian image quality assessment model to obtain a quality assessment result for the pedestrian images to be detected.

[0013] Optionally, the annotation module includes: a first annotation unit, used to classify the original pedestrian images to obtain a pedestrian image set and a background image set, and add a "pedestrian present" label to the images in the pedestrian image set and a "no pedestrian" label to the images in the background image set; a second annotation unit, used to randomly crop the pedestrian image set and add quality factor labels of different occlusion levels to the original images and the cropped images in the pedestrian image set; and a third annotation unit, used to determine the feature values ​​of illumination, blur, and pose of the pedestrian image set based on a preset feature recognition algorithm, and add quality factor labels of different levels corresponding to illumination, blur, and pose to the images in the pedestrian image set according to the determined feature values.

[0014] Another aspect of the present invention provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the pedestrian image quality evaluation method of any of the above embodiments.

[0015] Another aspect of the present invention provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the pedestrian image quality evaluation method of any of the above embodiments. Further, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application program required for a function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.

[0016] This invention proposes a deep learning-based algorithm that utilizes a single-network multi-task training method. This algorithm can effectively filter out false detections (no pedestrians in the detection box) and interference images such as half-body or multiple people generated by pedestrian detection algorithms. At the same time, it can effectively process pedestrian images with poor lighting, high blur, or incorrect poses, thereby significantly improving the usability and recognition accuracy of the pedestrian re-identification system. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0018] Figure 1 A flowchart illustrating an optional method for evaluating pedestrian image quality provided in Embodiment 1 of the present invention is shown.

[0019] Figure 2 A structural block diagram of the pedestrian image quality evaluation device provided in Embodiment 2 of the present invention is shown; and

[0020] Figure 3 A block diagram of a computer device suitable for implementing a method for evaluating pedestrian image quality, as provided in Embodiment 3 of the present invention, is shown. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0022] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0023] Example 1

[0024] In existing technologies, pedestrian images affected by factors such as lighting, blur, and pose are generally assessed using a no-reference method based on several metrics such as mean, standard deviation, average gradient, and entropy. The meanings of each metric are as follows:

[0025] Mean: The average value of all pixels in an image. The higher the mean, the higher the image quality.

[0026] Standard deviation: The degree of dispersion of image pixel gray values ​​relative to the mean. The larger the standard deviation, the more dispersed the gray level distribution in the image, and the better the image quality.

[0027] Average gradient: reflects the contrast and texture changes in an image, and to some extent reflects the image's sharpness.

[0028] Entropy: refers to the average amount of information in an image. From the perspective of information theory, it measures the amount of information in an image. The greater the information entropy, the more information the image contains.

[0029] However, this method is unsatisfactory in terms of versatility and stability, and the algorithm evaluation process is relatively complex, which increases the difficulty of pedestrian image quality recognition.

[0030] This embodiment provides a method for evaluating the quality of pedestrian images. Figure 1 A flowchart illustrating the pedestrian image quality evaluation method is shown, such as... Figure 1 As shown, the method for evaluating the quality of pedestrian images may include steps S101 to S104, wherein:

[0031] Step S101: Obtain a batch of original pedestrian images, add quality factor labels to the original pedestrian images, and generate a training set. The quality factors include pedestrian presence / absence, occlusion, illumination, blur, and pose.

[0032] In this embodiment, pedestrian images can be collected from any monitoring device or server, without limitation. Different application scenarios and source formats will affect the recognition of pedestrian images. This embodiment sets key factors affecting image quality (i.e., quality factors) to characterize image quality. These quality factors correspond to multiple different dimensions of information in the image. By evaluating the quality of pedestrian images through quality factors, the accuracy of pedestrian image quality assessment is improved. The quality factors may include the presence or absence of pedestrians, occlusion, illumination, blur, and pose. Furthermore, depending on different needs, other dimensions of image information can be appropriately added or used for evaluation, without limitation.

[0033] Step S102: Input the training set into the initial deep learning model for training until the model's loss value tends to converge, then stop training the model and obtain the pedestrian image quality assessment model.

[0034] Preferably, the initial deep learning model is any one of ResNet18, MobileNet, or shuffleNet network structures.

[0035] After adding quality factor labels to the original pedestrian images, a training set is obtained. This training set is then applied to the initial deep learning model for repeated training. When the loss value meets the preset rules, the model tends to converge, indicating that the current model training is successful. The training operation ends, and a pedestrian image quality assessment model is generated.

[0036] Step S103: Obtain the image of the pedestrian to be detected;

[0037] Step S104: Based on the pedestrian image quality assessment model, the pedestrian image to be detected is identified and processed to obtain the quality assessment result of the pedestrian image to be detected.

[0038] The quality assessment results can be any combination of quality factors such as pedestrian presence / absence, occlusion, lighting, blur, and pose, and the specific assessment results are determined by the content contained in the actual pedestrian image.

[0039] In this embodiment, the pedestrian image quality assessment model can identify pedestrian images from five different dimensions: presence or absence of pedestrians, occlusion, illumination, blur, and pose, so as to accurately classify the features contained in the pedestrian image to be detected, thereby improving the accuracy of pedestrian image quality assessment.

[0040] Preferably, step S104 may include steps S1041 to S1043, wherein:

[0041] Step S1041: Input the pedestrian image to be detected into the convolutional neural network of the pedestrian image quality assessment model for feature recognition to obtain a multidimensional feature vector;

[0042] The multidimensional feature vector can be an N-dimensional feature vector, where the value of N ranges from [64, 128, 256]. The actual value is adjusted according to the model performance. If fast model computation is required, a smaller value of N, such as 64, is used; if more stable model performance is required, a larger value of N, such as 256, is used. The pedestrian image to be detected is input into a convolutional neural network, and image features are extracted through convolutional layers, pooling layers, edge filling, activation, etc., to obtain an N-dimensional feature vector.

[0043] Step S1042: Use multiple sub-task neural networks to perform prediction operations on the multi-dimensional feature vector to obtain the predicted probability value of the pedestrian image to be detected in each sub-task neural network.

[0044] The sub-task neural network can include five categories: pedestrian presence / absence, occlusion, illumination, blur, and pose, which respectively identify features such as pedestrian presence / absence, occlusion, illumination, blur, and pose in the image to be detected. The sub-task neural network can adopt a conventional convolutional neural network structure or other equivalent neural networks; there are no restrictions here.

[0045] The predicted probability value represents the proportion or similarity of a certain feature contained in the image to be detected to the associated features of the corresponding subtask, wherein the features of the image to be detected are consistent with the associated features of the subtask.

[0046] Step S1043: Based on the softmax layer, the predicted probability values ​​are integrated, classified, and output to obtain the quality assessment result corresponding to the pedestrian image to be detected.

[0047] The predicted probability values ​​may contain meaningless data, which refers to situations where the image to be detected lacks features corresponding to a particular subtask. Directly outputting this data would require an additional manual data filtering process, reducing the efficiency of pedestrian image recognition. The softmax layer integrates and classifies the predicted probability values ​​output by each subtask neural network, identifying the valid data (i.e., features of the pedestrian image to be detected that match the associated features of the subtask neural network). Outputting this valid data makes the assessment of pedestrian image quality more intuitive and improves the efficiency of pedestrian image recognition.

[0048] Preferably, step S101 may include steps S1011 to S1013, wherein:

[0049] Step S1011: Classify the original pedestrian images to obtain a pedestrian image set and a background image set, and add pedestrian labels to the images in the pedestrian image set and non-pedestrian labels to the images in the background image set;

[0050] After acquiring the original pedestrian images, the first step is to label them with a quality factor indicating whether or not pedestrians are present. Based on the presence of human figures, the original pedestrian images are divided into a pedestrian image set and a background image set. The background image set represents the collection of images without human features, while the pedestrian image set is the opposite of the background image set. Using the background images as negative samples and the pedestrian images as positive samples, with a 1:3 ratio of positive to negative samples, the training data for the neural network in this pedestrian presence / absence subtask is constructed.

[0051] Step S1012: Randomly crop the pedestrian image set and add quality factor labels with different occlusion levels to the original images and cropped images in the pedestrian image set.

[0052] Occlusion is commonly used to describe pedestrian features; therefore, the aforementioned segmented pedestrian image set (positive samples) is used as the basis for adding the occlusion quality factor label. Since occlusion corresponds to various scenarios, this embodiment constructs different occlusion levels by randomly cropping images from the pedestrian image set. Different occlusion level quality factor labels are then added to both the original images and the cropped images in the pedestrian image set. This occlusion quality factor is used to further subdivide pedestrian graphics, ensuring the accuracy of subsequent pedestrian image quality assessment.

[0053] Preferably, step S1012 may include steps A1 to A3, wherein:

[0054] Step A1: Add an unoccluded label to the original images in the pedestrian image set;

[0055] Unobstructed labels represent full-body images of a single person;

[0056] Step A2: Randomly crop the pedestrian image set, extract the local images containing pedestrians, and add a first-level occlusion label to the local images;

[0057] Random cropping can be achieved through random horizontal or vertical segmentation; the first-level occlusion label represents a single person's half-body portrait.

[0058] Step A3: Obtain pedestrian location information from the cropped partial image containing pedestrians and the original image of the pedestrian image set. Perform coordinate fusion between the pedestrian location information and the partial image and the original image of the pedestrian image set to determine the number of people in the fused image. Filter out pedestrian images containing multiple people based on the number of people and add secondary occlusion labels to the pedestrian images containing multiple people.

[0059] Pedestrian location information can be obtained through manual annotation or determined by a general location recognition model; no restriction is placed here. Secondary occlusion labels represent multi-person portraits (including full-body and half-body images). After obtaining pedestrian location information from the cropped partial image containing pedestrians and the original image of the pedestrian image set, the pedestrian location information is fused with the coordinates of the partial image and the original image of the pedestrian image set to obtain pedestrian images containing location information. In these images, each pedestrian has corresponding location information; therefore, the number of pedestrians in the image can be determined by the number of location information points. This allows for the selection of pedestrian images containing multiple people, which are then labeled with secondary occlusion tags.

[0060] It should be noted that the terms "none," "level one," and "level two" mentioned above do not imply any degree, size, or order of occlusion; they are merely used to illustrate pedestrian images under different occlusion conditions.

[0061] Step S1013: Based on the preset feature recognition algorithm, determine the feature values ​​of illumination, blur and pose of the pedestrian image set respectively, and add quality factor labels of different levels corresponding to illumination, blur and pose to the images in the pedestrian image set according to the determined feature values.

[0062] The preset feature recognition algorithm can be a conventional image quality algorithm or a pose key point algorithm; there are no restrictions here.

[0063] Regarding the setting of the illumination quality factor label, the brightness of the images in the pedestrian image set is calculated based on the conventional image quality algorithm, thereby dividing the positive samples into four categories: normal light, dark light, bright light, and cloudy / sunny light.

[0064] Regarding the setting of the blur quality factor label, the algorithm calculates the degree of blur based on the standard image quality algorithm according to the pedestrian image set, thereby dividing positive samples into four categories: normal, motion blur, Gaussian blur, and salt and pepper blur.

[0065] Regarding the setting of the quality factor label of posture, the posture of pedestrian images is identified based on the posture key point algorithm, thereby dividing positive samples into two categories: abnormal postures (squatting, sitting, large-angle bending) and other postures. The posture key point algorithm can be OpenPose.

[0066] Preferably, step S102 may include steps S1021 to S1026, wherein:

[0067] Step S1021: Initialize the weights of the forward calculation formula using random values;

[0068] Forward computation formulas can be used to train and adjust certain parameters so that the entire deep learning network can exhibit the behaviors and objects required by the user.

[0069] Step S1022: Input the pedestrian images in the training set labeled with quality factors into the convolutional neural network layer of the initial deep learning network for supervised learning to obtain multi-layer feature vectors.

[0070] The initial deep learning network consists of two stages: a convolutional neural network and a sub-task neural network. The convolutional neural network comes first, followed by the sub-task neural network.

[0071] Step S1023: Continue to input the multi-layer feature vectors into the initial subtask neural network layer along the forward direction to calculate the quality features and obtain the predicted values ​​of the pedestrian images in the training set in each initial subtask neural network layer. The initial subtask neural network layer includes five subtasks: pedestrian presence / absence, occlusion, illumination, blur, and pose.

[0072] Step S1024: Obtain the weight values ​​of each subtask neural network layer, calculate the error between the predicted value and the true value of the label through the first adjustment function of the softmax layer, and use the normalized subtask weight values ​​to perform a weighted summation of the errors of each subtask. If the calculation result is greater than the first preset threshold, update the weight values ​​of each subtask. The first adjustment function is focal loss, and the weight values ​​of the subtasks with and without pedestrians are higher than the weight values ​​of other types of subtasks.

[0073] Due to the different recognition functions, the weights of each subtask are also different. For ease of explanation, the weights of the neural networks for the five subtasks—pedestrian presence / absence, occlusion, illumination, blur, and pose—are represented by α, β, γ, δ, and ε, respectively. Since the importance of each subtask varies, the values ​​of α, β, γ, δ, and ε are also different. Because the pedestrian presence / absence and occlusion subtasks are crucial, it is recommended to adjust the loss weights for α and β subtasks to be higher during the first training iteration and then normalize the weights.

[0074] Step S1025: Perform a new round of forward propagation calculation using the updated weight values, and repeat the forward propagation and weight value update until the error value is less than or equal to the first preset threshold. Then, stop training the initial deep learning model and obtain the intermediate model.

[0075] The algorithm obtains the weight values ​​of each subtask's neural network layer, calculates the error between the predicted value and the true label value using the first adjustment function of the softmax layer, and performs a weighted summation of the errors of each subtask using the normalized subtask weight values. The weighted summation result is compared with a first preset threshold. If the weighted summation result is greater than the first preset threshold, the weight values ​​of each subtask are updated. The first adjustment function is focal loss, and the weight value for the subtask indicating the presence or absence of pedestrians is higher than the weight values ​​for other types of subtasks. The error is backpropagated along the direction of minimum gradient using the focal loss function to correct the weight values ​​of each subtask. A new round of forward propagation is performed using the corrected new weight values. This forward and backward propagation process is repeated until the error value is less than or equal to the first preset threshold. At this point, training of the initial deep learning model is stopped, and an intermediate model is obtained.

[0076] Focal loss aims to clarify the basic prediction results of deep learning models, plays a decisive role in the prediction of pedestrian image quality, and improves the accuracy of pedestrian image quality assessment.

[0077] Step S1026: To improve the generalization performance of the deep learning model and make it applicable to more scenarios, determine the final error value of the intermediate model. If the final error value exceeds the second preset threshold, replace the first adjustment function of the intermediate model with the second adjustment function. Correct the weight values ​​of each subtask of the intermediate model through the second adjustment function, and repeatedly train the intermediate model based on the corrected weight values ​​until the error value is less than or equal to the second preset threshold. Then stop training to obtain the pedestrian image quality assessment model, where the second adjustment function is cosloss.

[0078] To improve the generalization performance of the deep learning model and make it applicable to more scenarios, the final error value of the intermediate model is determined. If the final error value exceeds the second preset threshold, the first adjustment function of the intermediate model is replaced with the second adjustment function. The error is backpropagated along the direction of minimum gradient using the second adjustment function to correct the weight values ​​of each subtask. The second adjustment function is cosloss. A new round of forward propagation calculation is performed using the corrected new weight values. Forward and backward propagation are repeated until the error value is less than or equal to the second preset threshold range. At this point, the training of the intermediate model is stopped, and the pedestrian image quality assessment model is obtained.

[0079] It should be noted that the first and second preset thresholds mentioned above are determined by actual training needs and are not restricted here.

[0080] By using the cosloss function to fine-tune the intermediate model, more granular recognition of individual error data is achieved, thereby improving the accuracy of pedestrian image recognition.

[0081] During the second fine-tuning process, an adaptive weighting method is used to allow the network to automatically learn the weights. To ensure model performance, different data augmentation methods are used for each subtask during both training phases to improve the accuracy of pedestrian image recognition.

[0082] Using the same dataset, the network model performance was stabilized by training with focal loss and fine-tuning with cos loss.

[0083] This embodiment proposes a deep learning-based algorithm that utilizes a single-network multi-task training method. This algorithm can effectively filter out false detections (no pedestrians in the detection box) and interference images such as half-body or multiple people generated by pedestrian detection algorithms. At the same time, it can effectively process pedestrian images with poor lighting, high blur, or incorrect poses, thereby significantly improving the usability and recognition accuracy of the pedestrian re-identification system.

[0084] Example 2

[0085] Embodiment 2 of the present invention also provides a pedestrian image quality evaluation device, which corresponds to the pedestrian image quality evaluation method provided in Embodiment 1 above. The corresponding technical features and effects are not detailed in this embodiment; relevant aspects can be referred to in Embodiment 1 above. Specifically, Figure 2 A structural block diagram of the pedestrian image quality evaluation device is shown. (Example) Figure 2 As shown, the pedestrian image quality evaluation device 200 includes a labeling module 201, a training module 202, an acquisition module 203, and a recognition module 204, wherein:

[0086] The annotation module 201 is used to acquire a batch of original pedestrian images, add quality factor labels to the original pedestrian images, and generate a training set. The quality factors include pedestrian presence / absence, occlusion, illumination, blur, and pose.

[0087] The training module 202, connected to the annotation module 201, is used to input the training set into the initial deep learning model for training until the model's loss value tends to converge, at which point the training of the model is stopped and a pedestrian image quality assessment model is obtained.

[0088] The acquisition module 203, connected to the training module 202, is used to acquire images of pedestrians to be detected.

[0089] The recognition module 204, connected to the acquisition module 203, is used to perform recognition processing on the pedestrian image to be detected based on the pedestrian image quality assessment model, and obtain the quality assessment result of the pedestrian image to be detected.

[0090] Optionally, the recognition module is specifically used to: input the pedestrian image to be detected into the convolutional neural network of the pedestrian image quality assessment model for feature recognition to obtain a multi-dimensional feature vector; use multiple sub-task neural networks to perform prediction operations on the multi-dimensional feature vector to obtain the predicted probability value of the pedestrian image to be detected in each sub-task neural network; and integrate, classify and output the predicted probability value based on the softmax layer to obtain the quality assessment result corresponding to the pedestrian image to be detected.

[0091] Optionally, the annotation module includes: a first annotation unit, used to classify the original pedestrian images to obtain a pedestrian image set and a background image set, and add a "pedestrian present" label to the images in the pedestrian image set and a "no pedestrian" label to the images in the background image set; a second annotation unit, used to randomly crop the pedestrian image set and add quality factor labels of different occlusion levels to the original images and the cropped images in the pedestrian image set; and a third annotation unit, used to determine the feature values ​​of illumination, blur, and pose of the pedestrian image set based on a preset feature recognition algorithm, and add quality factor labels of different levels corresponding to illumination, blur, and pose to the images in the pedestrian image set according to the determined feature values.

[0092] Optionally, the second annotation unit is specifically used for: adding an unoccluded label to the original image in the pedestrian image set; randomly cropping the pedestrian image set to extract local images containing pedestrians and adding a first-level occlusion label to the local images; obtaining pedestrian location information in the cropped local images containing pedestrians and the original images of the pedestrian image set; performing coordinate fusion of the pedestrian location information with the local images and the original images of the pedestrian image set to determine the number of people in the fused image; filtering out pedestrian images containing multiple people based on the number of people; and adding a second-level occlusion label to the pedestrian images containing multiple people.

[0093] Optionally, the training module is specifically used for: initializing the weights of the forward calculation formula with random values; inputting the training set pedestrian images with quality factor labels into the convolutional neural network layer of the initial deep learning network for supervised learning to obtain multi-layer feature vectors; continuing to input the multi-layer feature vectors into the initial sub-task neural network layer along the forward direction for quality feature calculation to obtain the predicted values ​​of the training set pedestrian images in each initial sub-task neural network layer, wherein the initial sub-task neural network layer includes five sub-tasks: pedestrian presence / absence, occlusion, illumination, blur, and pose; obtaining the weight values ​​of each sub-task neural network layer, calculating the error between the predicted value and the true value of the label through the first adjustment function of the softmax layer, using the normalized sub-task weight values ​​to perform a weighted summation of the errors of each sub-task, and updating the weight values ​​of each sub-task if the calculation result is greater than a first preset threshold. The first adjustment function is focal loss, and the weight value of the subtask of pedestrian presence or absence is higher than the weight values ​​of other types of subtasks. A new round of forward propagation calculation is performed using the updated weight values. This forward propagation and weight value update is repeated until the error value is less than or equal to a first preset threshold. Training of the initial deep learning model is then stopped, resulting in an intermediate model. To improve the generalization performance of the deep learning model and make it applicable to more scenarios, the final error value of the intermediate model is determined. If the final error value exceeds a second preset threshold, the first adjustment function of the intermediate model is replaced with a second adjustment function. The weight values ​​of each subtask of the intermediate model are corrected using the second adjustment function, and the intermediate model is repeatedly trained based on the corrected weight values ​​until the error value is less than or equal to the second preset threshold. Training is then stopped, resulting in a pedestrian image quality assessment model, where the second adjustment function is cosloss.

[0094] Optionally, the initial deep learning model can be any of the network structures ResNet18, MobileNet, or shuffleNet.

[0095] Example 3

[0096] Figure 3 A block diagram of a computer device suitable for implementing a pedestrian image quality evaluation method according to Embodiment 3 of the present invention is shown. In this embodiment, the computer device 300 may be a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc., executing a program. Figure 3 As shown, the computer device 300 in this embodiment includes, but is not limited to, a memory 301, a processor 302, and a network interface 303 that are communicatively connected to each other via a system bus. It should be noted that... Figure 3Only a computer device 300 with components 301-303 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0097] In this embodiment, the memory 303 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 301 may be an internal storage unit of the computer device 300, such as the hard disk or memory of the computer device 300. In other embodiments, the memory 301 may also be an external storage device of the computer device 300, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 300. Of course, the memory 301 may also include both the internal storage unit and the external storage device of the computer device 300. In this embodiment, the memory 301 is typically used to store the operating system and various application software installed on the computer device 300, such as the program code for pedestrian image quality evaluation methods.

[0098] In some embodiments, processor 302 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 302 is typically used to control the overall operation of computer device 300. For example, it performs control and processing related to data interaction or communication with computer device 300. In this embodiment, processor 302 is used to run program code containing the steps of a pedestrian image quality evaluation method stored in memory 301.

[0099] In this embodiment, the pedestrian image quality evaluation method stored in memory 301 can also be divided into one or more program modules and executed by one or more processors (processor 302 in this embodiment) to complete the present invention.

[0100] Network interface 303 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 300 and other computer devices. For example, network interface 303 is used to connect computer device 300 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 300 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0101] Example 4

[0102] This embodiment also provides a computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program. When the computer program is executed by a processor, it implements the steps of a method for evaluating pedestrian image quality.

[0103] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0104] It should be noted that the sequence numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0106] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for evaluating the quality of pedestrian images, characterized in that, The method includes: A batch of raw pedestrian images are acquired, and quality factor labels are added to the raw pedestrian images to generate a training set. The quality factors include pedestrian presence / absence, occlusion, illumination, blur, and pose. The training set is input into the initial deep learning model for training until the model's loss value tends to converge. Then, the training of the model is stopped to obtain the pedestrian image quality assessment model. The pedestrian image quality assessment model can identify pedestrian images from five different dimensions: presence or absence of pedestrians, occlusion, illumination, blur, and pose. Acquire images of the pedestrians to be detected; The pedestrian image to be detected is identified and processed based on the pedestrian image quality assessment model to obtain the quality assessment result of the pedestrian image to be detected. Adding quality factor labels to the original pedestrian image includes: The original pedestrian images are classified to obtain a pedestrian image set and a background image set. Images in the pedestrian image set are labeled with "having pedestrians" tags, and images in the background image set are labeled with "not having pedestrians" tags. The pedestrian image set is randomly cropped, and quality factor labels with different occlusion levels are added to the original images and cropped images in the pedestrian image set. Based on a preset feature recognition algorithm, the feature values ​​of illumination, blur and pose of the pedestrian image set are determined respectively. Based on the determined feature values, different levels of quality factor labels corresponding to illumination, blur and pose are added to the images in the pedestrian image set respectively. The step of adding quality factor labels with different occlusion levels to the image obtained by cropping the pedestrian image includes: Add an unobstructed label to the original images in the pedestrian image set; The pedestrian image set is randomly cropped to extract local images containing pedestrians, and a first-level occlusion label is added to the local images. Obtain the pedestrian location information from the cropped partial image containing pedestrians and the original image of the pedestrian image set. Perform coordinate fusion of the pedestrian location information with the partial image and the original image of the pedestrian image set respectively to determine the number of human figures contained in the fused image. Filter out pedestrian images containing multiple people based on the number of human figures and add a secondary occlusion label to the pedestrian images containing multiple people.

2. The method according to claim 1, characterized in that, The process of identifying and processing the pedestrian image to be detected based on the pedestrian image quality assessment model to obtain the quality assessment result of the pedestrian image to be detected includes: The pedestrian image to be detected is input into the convolutional neural network of the pedestrian image quality assessment model for feature recognition to obtain a multidimensional feature vector; Multiple sub-task neural networks are used to perform prediction operations on the multi-dimensional feature vector to obtain the predicted probability value of the pedestrian image to be detected in each sub-task neural network. The predicted probability values ​​are integrated, classified, and output based on the softmax layer to obtain the quality assessment result corresponding to the pedestrian image to be detected.

3. The method according to claim 1, characterized in that, The step of inputting the training set into an initial deep learning model for training, and stopping the training of the model when the model's loss value tends to converge, to obtain a pedestrian image quality assessment model, includes: The weights of the forward computation formula are initialized with random values; Pedestrian images from the training set labeled with quality factors are input into the convolutional neural network layer of the initial deep learning network for supervised learning to obtain multi-layer feature vectors. The multi-layer feature vectors are continued to be input into the initial subtask neural network layer along the forward direction to calculate the quality features and obtain the predicted values ​​of the pedestrian images in the training set in each initial subtask neural network layer. The initial subtask neural network layer includes five subtasks: pedestrian presence / absence, occlusion, illumination, blur, and pose. Obtain the weight values ​​of each subtask neural network layer, calculate the error between the predicted value and the true value of the label using the first adjustment function of the softmax layer, and use the normalized subtask weight values ​​to perform a weighted summation of the errors of each subtask. If the calculation result is greater than a first preset threshold, update the weight values ​​of each subtask. Here, the first adjustment function is focal loss, and the weight value of the subtask of whether or not a pedestrian is present is higher than the weight values ​​of other types of subtasks. A new round of forward propagation calculation is performed using the updated weight values. Forward propagation and weight value updates are repeated until the error value is less than or equal to the first preset threshold. Then, the training of the initial deep learning model is stopped, and an intermediate model is obtained. To improve the generalization performance of the deep learning model and make it applicable to more scenarios, the final error value of the intermediate model is determined. If the final error value exceeds a second preset threshold, the first adjustment function of the intermediate model is replaced with a second adjustment function. The weight values ​​of each subtask of the intermediate model are corrected by the second adjustment function, and the intermediate model is repeatedly trained based on the corrected weight values ​​until the error value is less than or equal to the second preset threshold. Then, training is stopped to obtain a pedestrian image quality assessment model. The second adjustment function is cosloss.

4. The method according to any one of claims 1-3, characterized in that, The initial deep learning model is any one of the network structures ResNet18, MobileNet, or shuffleNet.

5. A device for evaluating the quality of pedestrian images, characterized in that, The device includes: The annotation module is used to acquire a batch of original pedestrian images, add quality factor labels to the original pedestrian images, and generate a training set. The quality factors include pedestrian presence / absence, occlusion, illumination, blur, and pose. The training module is used to input the training set into the initial deep learning model for training until the model's loss value tends to converge, at which point the training of the model is stopped, and a pedestrian image quality assessment model is obtained; wherein, the pedestrian image quality assessment model can identify pedestrian images from five different types of dimensions: presence or absence of pedestrians, occlusion, illumination, blur, and pose. The acquisition module is used to acquire images of pedestrians to be detected. The recognition module is used to perform recognition processing on the pedestrian image to be detected based on the pedestrian image quality assessment model, and obtain the quality assessment result of the pedestrian image to be detected; The annotation module includes: The first annotation unit is used to classify the original pedestrian images to obtain a pedestrian image set and a background image set, and to add pedestrian labels to the images in the pedestrian image set and non-pedestrian labels to the images in the background image set. The second annotation unit is used to randomly crop the pedestrian image set and add quality factor labels with different occlusion levels to the original image and the cropped image in the pedestrian image set. The third annotation unit is used to determine the feature values ​​of illumination, blur and pose of the pedestrian image set based on a preset feature recognition algorithm, and add different levels of quality factor labels corresponding to illumination, blur and pose to the images in the pedestrian image set according to the determined feature values. The second annotation unit is further configured to: add an unoccluded label to the original image in the pedestrian image set; randomly crop the pedestrian image set to extract a local image containing pedestrians, and add a first-level occlusion label to the local image; obtain pedestrian location information in the cropped local image containing pedestrians and the original image of the pedestrian image set, perform coordinate fusion of the pedestrian location information with the local image and the original image of the pedestrian image set respectively, determine the number of people in the fused image, filter out pedestrian images containing multiple people based on the number of people, and add a second-level occlusion label to the pedestrian images containing multiple people.

6. A computer device, the computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 4.

Images