Super-resolution pedestrian re-identification method, device, equipment and program product
By introducing perceptual loss function and adversarial loss function in super-resolution image enhancement, combined with generative adversarial network training, the problem of insufficient image high-frequency detail recovery in the prior art is solved, and the image quality and accuracy of pedestrian re-identification are improved .
Patent Information
- Application Number
- CN202411151034.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-22
AI Technical Summary
Existing super-resolution reconstruction methods are difficult to effectively restore high-frequency details in images, resulting in impaired reconstruction image quality and affecting the accuracy of pedestrian feature extraction and recognition.
Generative adversarial networks are used for super-resolution image enhancement. By introducing perceptual loss function, adversarial loss function and pixel loss function, combined with generator and discriminator for training, we generate more realistic and high-quality super-resolution images.
It improves the ability to generalize image content during super-resolution reconstruction, generates more natural and realistic images, enhances the accuracy of subsequent convolutional neural network feature extraction, and improves the accuracy of pedestrian re-identification.
Smart Images

Figure CN119006286A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer vision, and more specifically, to a super-resolution pedestrian re-identification method, apparatus, device and program product. Background Art
[0002] Person re-identification technology is an important research direction in the field of computer vision. Its purpose is to identify and track the same person from different camera perspectives. This technology has important value in many application scenarios, such as customer behavior analysis in shopping malls, security monitoring in public places, and positioning of individuals in complex environments.
[0003] In practical applications, due to the limitations of shooting conditions (such as shooting distance, lighting conditions, etc.), the pedestrian images obtained are often of low quality, showing problems such as low resolution, blur, and high noise. These low-quality pedestrian images bring great challenges to subsequent target detection and recognition, and reduce the accuracy and reliability of the recognition method.
[0004] At present, super-resolution technology has become an important means to improve image quality and details. Super-resolution technology can not only improve the clarity of images in daily applications, but also have a significant impact in many key areas. Applying super-resolution technology to pedestrian re-identification is expected to improve the recognition accuracy of low-resolution images and achieve better re-identification results.
[0005] Although traditional super-resolution reconstruction methods can improve image resolution to a certain extent, they are often unable to effectively restore high-frequency details in the image, resulting in impaired quality of the reconstructed image and affecting the accuracy of pedestrian feature extraction and recognition. Summary of the invention
[0006] The present application provides a super-resolution pedestrian re-identification method, apparatus, equipment and program product, which solves the problem that in the process of improving image resolution in the prior art, high-frequency details in the image are not effectively restored, resulting in impaired quality of the reconstructed image, improves the generalization ability of the content of different images in the super-resolution reconstruction process, and ensures the quality of the generated super-resolution image.
[0007] In a first aspect of the present application, a method for pedestrian re-identification is provided, the method comprising:
[0008] Obtain an image of a pedestrian to be identified;
[0009] Calling a pre-trained super-resolution image enhancement model to enhance the resolution of the pedestrian image to be identified, and obtaining a super-resolution enhanced image; wherein; taking the first image and the second image as training parameters, combined with a pre-configured loss function, a generative adversarial network composed of a generator and a discriminator is trained, and when the loss function converges, the pre-trained super-resolution image enhancement model is obtained; wherein, the first image is a pedestrian image with a predetermined resolution, and the second image is obtained by enlarging the first image according to a predetermined enlargement factor;
[0010] A pre-trained convolutional neural network is called to extract feature parameters of the super-resolution enhanced image, and the pedestrian to be identified is identified based on the feature parameters.
[0011] In one implementation, before invoking the pre-trained super-resolution image enhancement model to enhance the resolution of the image of the pedestrian to be identified, the method further includes:
[0012] Detecting image size parameters of the image of the pedestrian to be identified to obtain a detection result; wherein the image size parameters include the width and height of the image of the pedestrian to be identified;
[0013] Determine whether the resolution of the image of the pedestrian to be identified meets the predetermined resolution based on the detection result, and adjust the resolution of the image of the pedestrian to be identified to the predetermined resolution when the resolution of the image of the pedestrian to be identified does not meet the predetermined resolution.
[0014] In one implementation, the resolution of the image of the pedestrian to be identified is adjusted to the predetermined resolution, specifically by scaling the image of the pedestrian to be identified using a bicubic interpolation algorithm to adjust the resolution of the image of the pedestrian to be identified to the predetermined resolution.
[0015] In one implementation, before calling a pre-trained convolutional neural network to extract feature parameters of the super-resolution enhanced image, the method further includes:
[0016] arbitrarily selecting a noise pixel of the super-resolution enhanced image, and searching for a pixel fragment similar to the noise pixel in color and texture in the super-resolution enhanced image in combination with a pre-configured local search area, wherein the local search area is a window of the same size as the noise pixel, and the pixel fragment has a plurality of pixels;
[0017] Calculating the Euclidean distance between each pixel of the pixel segment and the noise pixel, and calculating the average value of all the Euclidean distances;
[0018] According to the average value, pixels corresponding to adjacent Euclidean distances are selected to replace the noise pixels. After all the noise pixels in the super-resolution enhanced image are replaced, a super-resolution enhanced image with noise removed is obtained.
[0019] In one implementation, the loss function includes a generator network loss function, a perceptual loss function between a first image and a second image, an adversarial loss function relative to a generative adversarial network, a pixel loss function between the first image and the second image, and a discriminator network loss function.
[0020] In one implementation, the process of configuring the loss function is specifically as follows:
[0021] A weighted sum of the adversarial loss function and the pixel loss function is obtained to obtain a first loss function;
[0022] Summing the first loss function and the perceptual loss function to obtain a second loss function;
[0023] Multiplying the second loss by the generator network loss function to obtain a third loss function;
[0024] The third loss function is summed with the discriminator network loss function to obtain the loss function.
[0025] In one implementation, the loss function is expressed as: Among them, L is the loss function, L PerceptualLoss represents the perceptual loss between the first image and the second image, represents the adversarial loss function of the relative generative adversarial network, L1 is the pixel loss function between the first image and the second image, λ and γ are the weight parameters of the adversarial loss function and the pixel loss function respectively, Represents the discriminator network loss function.
[0026] In a second aspect of the present application, a pedestrian re-identification device is provided, the device comprising:
[0027] An image acquisition module, used to acquire an image of a pedestrian to be identified;
[0028] A resolution enhancement module, used to call a pre-trained super-resolution image enhancement model to enhance the resolution of the pedestrian image to be identified, and obtain a super-resolution enhanced image; wherein; the first image and the second image are used as training parameters, combined with a pre-configured loss function, a generative adversarial network composed of a generator and a discriminator is trained, and when the loss function converges, the pre-trained super-resolution image enhancement model is obtained; wherein, the first image is a pedestrian image with a predetermined resolution, and the second image is obtained by enlarging the first image according to a predetermined enlargement factor;
[0029] The recognition module is used to call a pre-trained convolutional neural network to extract feature parameters of the super-resolution enhanced image, and recognize the pedestrian to be recognized based on the feature parameters.
[0030] According to a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a pedestrian re-identification method as provided in the first aspect of the present application is implemented.
[0031] A fourth aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements a pedestrian re-identification method as provided in the first aspect of the present application.
[0032] Compared with the prior art, this application has the following beneficial effects:
[0033] In a super-resolution pedestrian re-identification method provided in the present application, the method uses the first image and the second image as training parameters, combines the pre-configured loss function, and trains the generative adversarial network composed of a generator and a discriminator. When the loss function converges, a trained super-resolution image enhancement model is obtained, and the first image is the real image; the second image is the image sample. Based on the model, the resolution enhancement of the image of the pedestrian to be identified is completed. In order to solve the problem that the traditional super-resolution reconstruction method improves the image resolution to a certain extent, but it is often difficult to effectively restore the high-frequency details in the image, resulting in the quality of the reconstructed image being impaired, the present application introduces the following loss function in the loss function: Compared with the loss function of the traditional generative adversarial network, the introduced adversarial loss function calculates the relative distance between the image sample and the real image instead of a simple binary classification output, thereby more effectively guiding the generator to produce a more realistic image. The introduced perceptual loss function helps the generator learn richer visual and texture details and improve the perceptual quality of the image by comparing the differences between the image sample and the real image at the feature level, rather than just matching at the pixel level. The pixel loss function ensures the similarity between the image sample and the real image at the pixel level, avoids the problem of over-smoothing, and improves the detail clarity of the image. Finally, the discriminator network loss function is introduced not only to distinguish between real and fake images, but also to enhance the ability to judge the authenticity of the generated image through the calculation of relative distance. Through the loss function, it can be ensured that the super-resolution image enhancement model based on the generative adversarial network generates more natural and realistic images, improves the generalization ability of the content of different images in the super-resolution reconstruction process, ensures the quality of the generated super-resolution image, and finally improves the accuracy of the subsequent convolutional neural network feature extraction, ensuring the accuracy of the pedestrian re-identification task. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The drawings described herein are used to provide a further understanding of the embodiments of the present application, constitute a part of the present application, and do not constitute a limitation on the embodiments of the present application. In the drawings:
[0035] Figure 1 A schematic diagram of a process flow of a super-resolution pedestrian re-identification method provided in an embodiment of the present application;
[0036] Figure 2 A principle block diagram of a super-resolution pedestrian re-identification device provided in an embodiment of the present application;
[0037] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0038] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with examples and drawings. The illustrative implementation scheme of the present application and its description are only used to explain the present application and are not intended to limit the present application.
[0039] It should be noted that the terms "include" or "may include" used in various embodiments of the present application indicate the presence of the function, operation or element applied for, and do not limit the addition of one or more functions, operations or elements. In addition, as used in various embodiments of the present application, the terms "include", "have" and their cognates are only intended to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items.
[0040] In various embodiments of the present application, terms such as "first", "second", and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0041] Pedestrian re-identification technology is an important research direction in the field of computer vision. Its purpose is to identify and track the same pedestrian under different camera perspectives. This technology has important value in many application scenarios, such as shopping mall customer behavior analysis, public security monitoring, and personal positioning in complex environments. However, the existing pedestrian re-identification system faces many shortcomings in practical applications, including: Image quality differences: Due to the different camera resolutions, lighting conditions, and installation locations, the quality of the pedestrian images obtained is uneven, which affects the recognition effect. Perspective and posture changes: The perspective and posture changes of pedestrians under different cameras increase the difficulty of feature extraction and matching. Real-time requirements: In some application scenarios, such as traffic monitoring, the system needs to process a large amount of video data in real time, and has strict requirements on the response time of the algorithm. Privacy protection: When performing pedestrian recognition in public places, it is necessary to consider the issue of privacy protection to avoid infringing personal privacy. In order to solve the above defects, a variety of methods have been proposed, including feature extraction technology based on deep learning, multimodal data fusion methods, and attention mechanisms. Most of these methods are based on the premise that the query image and the gallery image have the same or sufficiently high resolution. In real-world re-ID problems, this assumption may not hold true because the resolution of acquired person images can vary a lot. These variations are caused by natural variations in the distance between a person and the camera, such as the movement of pedestrians relative to the camera. This also causes resolution inconsistency and significant loss of visual information.
[0042] In practical applications, due to the limitations of shooting conditions (such as shooting distance, lighting conditions, etc.), the pedestrian images obtained are often of low quality, showing problems such as low resolution, blur, and high noise. These low-quality pedestrian images bring great challenges to subsequent target detection and recognition, and reduce the accuracy and reliability of the recognition method.
[0043] At present, super-resolution technology has become an important means to improve image quality and details. Super-resolution technology can not only improve the clarity of images in daily applications, but also have a significant impact in many key areas. Applying super-resolution technology to pedestrian re-identification is expected to improve the recognition accuracy of low-resolution images and achieve better re-identification results.
[0044] Although traditional super-resolution reconstruction methods can improve image resolution to a certain extent, they are often unable to effectively restore high-frequency details in the image, resulting in impaired quality of the reconstructed image and affecting the accuracy of pedestrian feature extraction and recognition.
[0045] Therefore, in order to overcome the defects of the prior art, the embodiments of the present application provide a pedestrian re-identification method, device, equipment and program product to improve the accuracy of pedestrian re-identification.
[0046] Prior to this, this embodiment also cites an example of the application of the pedestrian re-identification method, such as the identification and positioning of pedestrian behavior attributes, such as shopping mall customer behavior analysis. In this scenario, the pedestrian attributes of the pedestrian may be the pedestrian's gender, hair length, clothing color, clothing style, hat style, glasses, headphones, backpack, etc. For example, the positioning of the pedestrian's behavior attributes refers to the positioning of the specific position of the pedestrian attribute in the image, such as where the backpack is located on the pedestrian.
[0047] The following will describe in detail the super-resolution person re-identification method provided in the embodiment of the present application in combination with a specific implementation scheme. Figure 1 , Figure 1 A flowchart of a super-resolution person re-identification method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:
[0048] S101, obtaining an image of a pedestrian to be identified.
[0049] In this embodiment, the obtained pedestrian images to be identified may come from images, videos, etc. taken by monitoring systems of various actual identification scenes, such as monitoring systems of shopping malls, traffic intersections, hospitals, etc., and no redundant explanation is given in this embodiment.
[0050] S102, calling a pre-trained super-resolution image enhancement model to enhance the resolution of the image of the pedestrian to be identified, and obtaining a super-resolution enhanced image; wherein; taking the first image and the second image as training parameters, combined with a pre-configured loss function, a generative adversarial network composed of a generator and a discriminator is trained, and when the loss function converges, the pre-trained super-resolution image enhancement model is obtained; wherein, the first image is a pedestrian image with a predetermined resolution, and the second image is obtained by enlarging the first image according to a predetermined magnification factor.
[0051] In this embodiment, the generative adversarial network to be trained is a neural network already available in the prior art. The method provided in this embodiment does not make any improvements to the network structure of the neural network, so the specific structure of the neural network will not be repeated.
[0052] Since the present application is a super-resolution image enhancement model constructed by a generative adversarial network, but under initial conditions, the generative adversarial network does not have the ability to enhance resolution, it is necessary to first perform image training on the network. In this embodiment, the first image and the second image are used as training parameters of the generative adversarial network, where the first image is the real image and the second image is the sample image, or the second image is the image sample, which can be expressed as I z+1During training, the first image is fed into the discriminator of the generative adversarial network, and its resolution is the same as the image after super-resolution enhancement, which can be expressed as
[0053] Among them, the loss function includes the generator network loss function, the perceptual loss function between the first image and the second image, the adversarial loss function of the generative adversarial network, the pixel loss function between the first image and the second image, and the discriminator network loss function.
[0054] Specifically, the loss calculation process of the pre-configured loss function is as follows: weighted sum of the adversarial loss function and the pixel loss function to obtain a first loss function; sum the first loss function and the perceptual loss function to obtain a second loss function; multiply the second loss by the generator network loss function to obtain a third loss function; sum the third loss function and the discriminator network loss function to obtain the loss function.
[0055] The generator then updates its adversarial loss function and creates another sample of the same image, and the whole process continues until the discriminator network cannot distinguish between real and generated images. z To estimate the corresponding high-resolution image given a low-resolution input image The expression of the total loss function is as follows: Among them, L is the loss function, L PerceptualLoss represents the perceptual loss between the first image and the second image, represents the adversarial loss function of the Relativistic GAN (RGAN), L1 is the pixel loss function between the first image and the second image, λ and γ are the weight parameters of the adversarial loss function and the pixel loss function respectively. Represents the discriminator network loss function.
[0056] Calculate the perceptual loss function, which is a feature-level loss used to improve the perceptual quality and texture details of the generated image. z+1 and real images The feature map α is obtained after the jth convolution layer and before the i-th maximum convolution layer using the Euclidean distance. ij The method for calculating the perceptual loss is to calculate the super-resolution image G z (I z ) represents the corresponding real image The perceptual loss is not for the output image I z+1 The pixels and target image Instead of completely matching the pixels of the image, we hope that they can calculate similar feature representations through the loss function. The calculation formula is as follows: Where W and H represent pixel width and height respectively.
[0057] Calculate the pixel loss function. This method uses pixel loss to improve the pixel-level accuracy of the generated image. It forces the high-resolution image I z+1 In terms of pixel values, it is similar to the real image Compared with L2 loss, which usually leads to over-smoothed results, L1 loss can achieve better performance and convergence. The calculation formula is as follows:
[0058] Calculate the adversarial loss function of the relative generative adversarial network, and adopt a realistic discriminator loss in the super-resolution network to ensure that the generated high-resolution images are more natural and realistic than the images in the image library. In GAN, the standard discriminator D st The goal is to classify images. The formula is as follows: D st =σ(f d (I z )))→0, this formula represents the working principle of Standard GAN (SGAN). Here, D st is the output of the discriminator, which is used to classify whether the input image is a real image or a generated image; f d () is the discriminator feature vector; σ represents the sigmoid function. Here, the adversarial loss acts as a binary classifier to check whether the data is real or fake. st Distinguish real images by calculating distance And generate image G z (I z ) probability:
[0059] Compared with SGAN, the pedestrian images output by RGAN have clear edges and contain more visual and frequency details. The following formula explains this: Ra (Real,Fake)=C(Real)-E(C(Fake))→1;D Ra (Fake, Real) = C(Fake) - E(C(Real)) → 0; here, E() is the average of all (fake or real) data in the mini-batch, f d () represents the output value of the discriminator.
[0060] The following formula expresses the expression of the discriminator network loss function:
[0061] In contrast, the following formula represents the adversarial loss function of RGAN:
[0062]
[0063] By reducing the loss of both the discriminator and the generator network simultaneously, And generate image G z (I z ) networks are trained with each other.
[0064] The above-mentioned loss functions are substituted into the total loss function to calculate the loss value between the real image and the image sample. When the loss value reaches the predetermined loss threshold, the training of the generative adversarial network is completed to obtain a super-resolution image enhancement model.
[0065] Therefore, when the image of the pedestrian to be identified is input into the super-resolution image enhancement model, a super-resolution enhanced image can be obtained, which is defined here as a super-resolution enhanced image.
[0066] S103, calling a pre-trained convolutional neural network to extract feature parameters of the super-resolution enhanced image, and identifying the pedestrian to be identified based on the feature parameters.
[0067] In this embodiment, a re-identification network is used to extract pedestrian features from super-resolution enhanced images and perform identity matching. ResNet-50 is used as the backbone network of the convolutional neural network, the Kaiming initialization strategy is adopted, the dropout technology is used to reduce overfitting, and two hyperparameters, random erasing and linear warm-up, are used for adjustment. Finally, the matching result is output.
[0068] Among them, the Kaiming initialization strategy, combined with the pre-trained weights, can provide a good starting point for the convolutional neural network, accelerate convergence and improve network performance. Secondly, the dropout technology is used to reduce overfitting and improve the generalization ability of the network on unknown data. Then, random erasing and linear warm-up are used as hyperparameters, and adjustments are made according to the results of model training and verification to achieve the best recognition effect. After the convolutional neural network training is completed, the feature parameters of the super-resolution enhanced image can be extracted, and then the pedestrian to be identified is identified based on the feature parameters. The recognition process is as follows: The extracted feature parameters are used to compare with the features in the database, and the identity of the pedestrian is identified by calculating the similarity between the features, thereby outputting the matching result, which includes the identity information and confidence of the pedestrian, providing an accurate evaluation for the pedestrian re-identification task.
[0069] Furthermore, random erasing is a data augmentation technique that is commonly used in the training process of image recognition and deep learning models, especially in image classification and person re-identification tasks, to improve the model's generalization ability to different regions in the input image. First, a rectangular area is randomly selected in the image, and the pixel values of the selected area are replaced with any value between 0 and 255. Then, this part of the image is erased, and the model is forced to learn other features of the image instead of relying solely on a specific area in the image. During model training, random erasing is used as part of data augmentation together with standard data augmentation techniques to further improve model performance.
[0070] Further, linear warm-up is a technique used in deep learning to optimize the model training process, especially when training complex neural networks with a large number of parameters. This method is mainly used to adjust the learning rate to promote more stable and efficient learning of the model. It has two strategies: constant warm-up and gradual warm-up. This embodiment adopts a gradual warm-up strategy. In order to avoid large updates at the beginning, the optimal solution may be skipped. At the beginning of training, a lower learning rate is set, which helps the model to explore carefully in the parameter space. As the training progresses, the learning rate gradually increases according to a predetermined linear plan, and the learning rate continues to increase until it reaches a preset maximum value.
[0071] Based on the contents described in the above embodiments, in a pedestrian re-identification method provided in an embodiment of the present application, the method uses the first image and the second image as training parameters, and combines the pre-configured loss function to train the generative adversarial network composed of the generator and the discriminator. When the loss function converges, a trained super-resolution image enhancement model is obtained, in which the first image is the real image; the second image is the image sample. Based on the model, the resolution enhancement of the image of the pedestrian to be identified is completed. In order to solve the problem that the traditional super-resolution reconstruction method improves the image resolution to a certain extent, but it is often difficult to effectively restore the high-frequency details in the image, resulting in the quality of the reconstructed image being impaired, the present application introduces the following loss function in the loss function: Compared with the loss function of the traditional generative adversarial network, the introduced adversarial loss function calculates the relative distance between the image sample and the real image instead of a simple binary classification output, thereby more effectively guiding the generator to produce a more realistic image. The introduced perceptual loss function helps the generator learn richer visual and texture details and improve the perceptual quality of the image by comparing the differences between the image sample and the real image at the feature level, rather than just matching at the pixel level. The pixel loss function ensures the similarity between the image sample and the real image at the pixel level, avoids the problem of over-smoothing, and improves the detail clarity of the image. Finally, the discriminator network loss function is introduced not only to distinguish between real and fake images, but also to enhance the ability to judge the authenticity of the generated image through the calculation of relative distance. Through the loss function, it can be ensured that the super-resolution image enhancement model based on the generative adversarial network generates more natural and realistic images, improves the generalization ability of the content of different images in the super-resolution reconstruction process, ensures the quality of the generated super-resolution image, and finally improves the accuracy of the subsequent convolutional neural network feature extraction, ensuring the accuracy of the pedestrian re-identification task.
[0072] In some embodiments, before calling the pre-trained super-resolution image enhancement model in step S102 to enhance the resolution of the image of the pedestrian to be identified, the method provided in this embodiment further includes:
[0073] Detecting image size parameters of the image of the pedestrian to be identified and obtaining a detection result; wherein the image size parameters include the width and height of the image of the pedestrian to be identified;
[0074] It is determined whether the resolution of the image of the pedestrian to be identified meets the predetermined resolution according to the detection result. When the resolution of the image of the pedestrian to be identified does not meet the predetermined resolution, the resolution of the image of the pedestrian to be identified is adjusted to the predetermined resolution.
[0075] In this embodiment, the graphic size parameters of each input pedestrian image to be identified are automatically detected, including the width and height of the image, and the resolution of the image is evaluated to see whether it meets the predefined standard of the predetermined resolution. If the image resolution is higher or lower than the predefined standard, it will be adjusted accordingly. In this embodiment, the resolution is adjusted to avoid the appearance of too low resolution images, which will affect the feature extraction of the subsequent neural network.
[0076] In some embodiments, the resolution of the image of the pedestrian to be identified is adjusted to a predetermined resolution, specifically by scaling the image of the pedestrian to be identified using a bicubic interpolation algorithm to adjust the resolution of the image of the pedestrian to be identified to the predetermined resolution.
[0077] Specifically, this embodiment uses a bicubic interpolation algorithm to better maintain image quality during the scaling process and reduce blur and pixelation effects. After the images of pedestrians to be identified are scaled, all images will be adjusted to a uniform size. After this step, regardless of the size of the original images of pedestrians to be identified, the output images will have the same resolution and size.
[0078] In some embodiments, before calling the pre-trained convolutional neural network to extract the feature parameters of the super-resolution enhanced image in step S103, the noise generated by the generative adversarial network during the image resolution enhancement process due to the decompression characteristics of the Joint Photographic Group (.jpg) image format will affect the subsequent extraction of feature parameters. Therefore, this embodiment uses the local average value of the pixel color to replace the noise pixel color to reduce the random noise in the image, and the denoised image will be used for re-identification. Specifically, the process of denoising the super-resolution enhanced image is as follows:
[0079] A noise pixel of the super-resolution enhanced image is arbitrarily selected, and a pixel fragment similar to the noise pixel in color and texture is searched in the super-resolution enhanced image in combination with a pre-configured local search area, wherein the local search area is a window of the same size as the noise pixel, and the pixel fragment has a plurality of pixels; the Euclidean distance between each pixel of the pixel fragment and the noise pixel is calculated, and the average value of all the Euclidean distances is calculated; and the pixels corresponding to the adjacent Euclidean distances are selected according to the average value to replace the noise pixel, and after all the noise pixels in the super-resolution enhanced image are replaced, a super-resolution enhanced image with noise removed is obtained.
[0080] Among them, Euclidean distance is used as a measure of pixel similarity in color space to quantify the difference between color values. It should be noted that when calculating the average value of all Euclidean distances, a robust truncated average statistical method is used to reduce the impact of outliers. According to the average value, the pixels corresponding to the adjacent Euclidean distance are selected to replace the noise pixels. After all the noise pixels of the super-resolution enhanced image are replaced, the super-resolution enhanced image with noise removed is obtained. The replaced image shows a smoother visual effect, while retaining important pedestrian feature details, which is convenient for the subsequent feature extraction of the pre-trained convolutional neural network.
[0081] The present application also provides a super-resolution person re-identification device, which can be used to perform each step of a super-resolution person re-identification method described in the above embodiment. Figure 2 , Figure 2 A principle block diagram of a super-resolution pedestrian re-identification device provided in an embodiment of the present application, the device includes:
[0082] An image acquisition module 210 is used to acquire an image of a pedestrian to be identified;
[0083] The resolution enhancement module 220 is used to call a pre-trained super-resolution image enhancement model to enhance the resolution of the pedestrian image to be identified, so as to obtain a super-resolution enhanced image; wherein: the first image and the second image are used as training parameters, and a pre-configured loss function is combined to train a generative adversarial network composed of a generator and a discriminator, and when the loss function converges, the pre-trained super-resolution image enhancement model is obtained; wherein, the first image is a pedestrian image with a predetermined resolution, and the second image is obtained by enlarging the first image according to a predetermined enlargement factor;
[0084] The recognition module 230 is used to call a pre-trained convolutional neural network to extract feature parameters of the super-resolution enhanced image, and recognize the pedestrian to be recognized based on the feature parameters.
[0085] A testing device for an embedded device in an embodiment of the present application is similar to the above Figure 1 The pedestrian re-identification method shown is an invention based on the same concept. Through the detailed description of the pedestrian re-identification method provided in the above embodiment, those skilled in the art can clearly understand the implementation process of a pedestrian re-identification device in this embodiment, so for the sake of brevity of the specification, it will not be repeated here.
[0086] It can be seen that in a super-resolution pedestrian re-identification device provided by the present application, the device uses the first image and the second image as training parameters, and combines the pre-configured loss function to train the generative adversarial network composed of the generator and the discriminator. When the loss function converges, a trained super-resolution image enhancement model is obtained, and the first image is the real image; the second image is the image sample. Based on this model, the resolution enhancement of the image of the pedestrian to be identified is completed. In order to solve the problem that the traditional super-resolution reconstruction method improves the image resolution to a certain extent, but it is often difficult to effectively restore the high-frequency details in the image, resulting in the quality of the reconstructed image being impaired, the present application introduces the following loss function in the loss function: Compared with the loss function of the traditional generative adversarial network, the introduced adversarial loss function calculates the relative distance between the image sample and the real image instead of a simple binary classification output, thereby more effectively guiding the generator to produce a more realistic image. The introduced perceptual loss function helps the generator learn richer visual and texture details and improve the perceptual quality of the image by comparing the differences between the image sample and the real image at the feature level, rather than just matching at the pixel level. The pixel loss function ensures the similarity between the image sample and the real image at the pixel level, avoids the problem of over-smoothing, and improves the detail clarity of the image. Finally, the discriminator network loss function is introduced not only to distinguish between real and fake images, but also to enhance the ability to judge the authenticity of the generated image through the calculation of relative distance. Through the loss function, it can be ensured that the super-resolution image enhancement model based on the generative adversarial network generates more natural and realistic images, improves the generalization ability of the content of different images in the super-resolution reconstruction process, ensures the quality of the generated super-resolution image, and finally improves the accuracy of the subsequent convolutional neural network feature extraction, ensuring the accuracy of the pedestrian re-identification task.
[0087] Please refer to Figure 3 In an embodiment of the present application, a structural diagram of an electronic device is also provided, such as Figure 3As shown, the electronic device may include: a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302 and the memory 303 communicate with each other through the communication bus 304. The processor 301 may call the logic instructions in the memory 303 to execute a super-resolution pedestrian re-identification method, the method comprising: obtaining a pedestrian image to be identified; calling a pre-trained super-resolution image enhancement model to enhance the resolution of the pedestrian image to be identified, and obtaining a super-resolution enhanced image; wherein; using the first image and the second image as training parameters, combined with a pre-configured loss function, a generative adversarial network composed of a generator and a discriminator is trained, and when the loss function converges, the pre-trained super-resolution image enhancement model is obtained; wherein, the first image is a pedestrian image with a predetermined resolution, and the second image is obtained by enlarging the first image according to a predetermined magnification factor; calling the pre-trained convolutional neural network to extract feature parameters of the super-resolution enhanced image, and identifying the pedestrian to be identified based on the feature parameters.
[0088] In addition, the logic instructions in the above-mentioned memory 303 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0089] In an embodiment of the present application, a computer program product is also provided, which includes a computer program, which can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a super-resolution pedestrian re-identification method provided by the above methods, the method including: obtaining a pedestrian image to be identified; calling a pre-trained super-resolution image enhancement model to enhance the resolution of the pedestrian image to be identified, and obtaining a super-resolution enhanced image; wherein; using the first image and the second image as training parameters, combined with a pre-configured loss function, a generative adversarial network composed of a generator and a discriminator is trained, and when the loss function converges, the pre-trained super-resolution image enhancement model is obtained; wherein, the first image is a pedestrian image with a predetermined resolution, and the second image is obtained by enlarging the first image according to a predetermined magnification factor; calling a pre-trained convolutional neural network to extract feature parameters of the super-resolution enhanced image, and identifying the pedestrian to be identified based on the feature parameters.
[0090] In an embodiment of the present application, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute a super-resolution pedestrian re-identification method provided by the above methods. The computer-readable storage medium is a memory device in a computer device, which is used to store programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory (non volatile memory), such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the super-resolution pedestrian re-identification method in the above embodiments. Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD ROM, optical storage, etc.) containing computer-usable program codes.
[0091] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.
Claims
1. A pedestrian re-identification method, characterized in that: Methods include: Obtain an image of a pedestrian to be identified; Invoking a pre-trained super-resolution image enhancement model to enhance the resolution of the image of the pedestrian to be identified, to obtain a super-resolution enhanced image; in; The first image and the second image are used as training parameters, and a pre-configured loss function is combined to train a generative adversarial network composed of a generator and a discriminator, and when the loss function converges, the pre-trained super-resolution image enhancement model is obtained; wherein the first image is a pedestrian image with a predetermined resolution, and the second image is obtained by magnifying the first image according to a predetermined magnification factor; A pre-trained convolutional neural network is called to extract feature parameters of the super-resolution enhanced image, and the pedestrian to be identified is identified based on the feature parameters.
2. A pedestrian re-identification method according to claim 1, characterized in that: Before calling the pre-trained super-resolution image enhancement model to enhance the resolution of the image of the pedestrian to be identified, the method further includes: Detecting image size parameters of the image of the pedestrian to be identified to obtain a detection result; wherein the image size parameters include the width and height of the image of the pedestrian to be identified; Determine whether the resolution of the image of the pedestrian to be identified meets the predetermined resolution based on the detection result, and adjust the resolution of the image of the pedestrian to be identified to the predetermined resolution when the resolution of the image of the pedestrian to be identified does not meet the predetermined resolution.
3. A pedestrian re-identification method according to claim 2, characterized in that: The resolution of the image of the pedestrian to be identified is adjusted to the predetermined resolution, specifically: the image of the pedestrian to be identified is scaled by using a bicubic interpolation algorithm to adjust the resolution of the image of the pedestrian to be identified to the predetermined resolution.
4. The pedestrian re-identification method according to claim 1, characterized in that: Before calling the pre-trained convolutional neural network to extract the feature parameters of the super-resolution enhanced image, the method further includes: arbitrarily selecting a noise pixel of the super-resolution enhanced image, and searching for a pixel fragment similar to the noise pixel in color and texture in the super-resolution enhanced image in combination with a pre-configured local search area, wherein the local search area is a window of the same size as the noise pixel, and the pixel fragment has a plurality of pixels; Calculating the Euclidean distance between each pixel of the pixel segment and the noise pixel, and calculating the average value of all the Euclidean distances; According to the average value, pixels corresponding to adjacent Euclidean distances are selected to replace the noise pixels. After all the noise pixels in the super-resolution enhanced image are replaced, a super-resolution enhanced image with noise removed is obtained.
5. The pedestrian re-identification method according to claim 1, characterized in that: The loss function includes a generator network loss function, a perceptual loss function between the first image and the second image, an adversarial loss function relative to the generative adversarial network, a pixel loss function between the first image and the second image, and a discriminator network loss function.
6. A pedestrian re-identification method according to claim 5, characterized in that: The process of configuring the loss function is specifically as follows: A weighted sum of the adversarial loss function and the pixel loss function is obtained to obtain a first loss function; Summing the first loss function and the perceptual loss function to obtain a second loss function; Multiplying the second loss by the generator network loss function to obtain a third loss function; The third loss function is summed with the discriminator network loss function to obtain the loss function.
7. The method for pedestrian re-identification according to claim 6, characterized in that: The expression of the loss function is: Among them, L is the loss function, L PerceptualLoss represents the perceptual loss between the first image and the second image, represents the adversarial loss function of the relative generative adversarial network, L1 is the pixel loss function between the first image and the second image, λ and γ are the weight parameters of the adversarial loss function and the pixel loss function respectively, Represents the discriminator network loss function.
8. A pedestrian re-identification device, characterized in that: The device includes: An image acquisition module, used to acquire an image of a pedestrian to be identified; A resolution enhancement module, used to call a pre-trained super-resolution image enhancement model to enhance the resolution of the image of the pedestrian to be identified, so as to obtain a super-resolution enhanced image; in; The first image and the second image are used as training parameters, and a pre-configured loss function is combined to train a generative adversarial network composed of a generator and a discriminator, and when the loss function converges, the pre-trained super-resolution image enhancement model is obtained; wherein the first image is a pedestrian image with a predetermined resolution, and the second image is obtained by magnifying the first image according to a predetermined magnification factor; The recognition module is used to call a pre-trained convolutional neural network to extract feature parameters of the super-resolution enhanced image, and recognize the pedestrian to be recognized based on the feature parameters.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, a pedestrian re-identification method as described in any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, a pedestrian re-identification method as described in any one of claims 1 to 7 is implemented.
