Self-supervised pedestrian target detection method based on color and texture features

Abstract

This invention discloses a self-supervised pedestrian target detection method based on color and texture features, consisting of three stages: In the self-supervised pre-training stage, firstly, a large number of unlabeled pedestrian images are processed by aggregating pixels with similar color or texture features to generate pseudo-labels for target localization. Then, a pedestrian target detection model based on an improved SSD is constructed, and self-supervised pre-training is performed using unlabeled pedestrian images and their pseudo-labels to obtain a pre-trained model. In the supervised fine-tuning stage, the pre-trained model is fine-tuned using a small number of labeled pedestrian images to obtain a target model. In the prediction stage, the pedestrian image to be detected is input into the target model, and the detection result of that image is output. This invention uses pseudo-labels for target localization in self-supervised pre-training, making the pre-training process closely related to the subsequent pedestrian target detection task, and utilizes spatial attention to enhance features, thereby improving the detection accuracy of the model.

Description

Technical Field

[0001] This invention belongs to the field of pedestrian target detection technology, specifically relating to a self-supervised pedestrian target detection method based on color and texture features. Background Technology

[0002] In recent years, video surveillance has played a vital role in maintaining social security and combating crime. Among the applications of video surveillance, the detection of specific pedestrian targets is a crucial task. Currently, deep learning technology is widely used in pedestrian target detection, driving the rapid development of this field.

[0003] While previous research has yielded significant results, deep network learning typically requires a large number of labeled image samples. The high cost of manual annotation makes this requirement difficult to meet in practical applications. To address this issue, self-supervised learning techniques have gained widespread attention.

[0004] Current self-supervised learning methods can be divided into two main categories: image reconstruction-based methods and contrastive learning-based methods. However, during self-supervised pre-training, image reconstruction-based methods focus more on the similarity between the image output by the pre-trained model and the original image; contrastive learning-based methods focus more on the similarity between features extracted by the pre-trained model after the original image has undergone different data augmentation methods. This results in a low correlation between the self-supervised pre-training process and the subsequent pedestrian object detection task, thus affecting the performance of pedestrian object detection. Summary of the Invention

[0005] The problem to be solved by this invention is to provide a self-supervised pedestrian target detection method based on color and texture features to address the shortcomings of existing technologies. By generating pseudo-labels for target localization for self-supervised pre-training, the pre-training process is closely related to the subsequent pedestrian target detection task. Furthermore, spatial attention is used to enhance features and improve the detection accuracy of the model.

[0006] The present invention adopts the following technical solution: a self-supervised pedestrian target detection method based on color and texture features, comprising six steps in three stages:

[0007] Phase 1: Self-supervised pre-training phase

[0008] Step 1: Obtain an unlabeled pedestrian image dataset and perform data preprocessing to use it as a self-supervised pre-training dataset;

[0009] Step 2: Construct a pedestrian target detection model based on the improved SSD;

[0010] Step 3: Use the self-supervised pre-training dataset obtained in Step 1 to train the pedestrian target detection model based on the improved SSD, and obtain the pre-trained model;

[0011] Phase 2: Supervised Fine-tuning Phase

[0012] Step 4: Obtain the labeled pedestrian image dataset and perform data preprocessing, dividing the dataset into training set, validation set and test set according to the proportions;

[0013] Step 5: Fine-tune the pre-trained model using the training and validation sets obtained in Step 4 to obtain the target model;

[0014] Phase 3, Prediction Phase

[0015] Step 6: Input the test set image obtained in Step 4 as the pedestrian image to be detected into the target model obtained in Step 5 to obtain the pedestrian target detection result of the image.

[0016] Further, in step 1, acquiring the unlabeled pedestrian image dataset and performing data preprocessing includes the following sub-steps:

[0017] Step 1-1: Download or create your own unlabeled pedestrian image dataset from the internet;

[0018] Steps 1-2: Maintain the aspect ratio of each pedestrian image in the dataset, scale the larger side to a specified number of pixels, and then center the scaled image within a black background image of a specified size.

[0019] Steps 1-3: For each unlabeled pedestrian image in the dataset, generate pseudo-labels for target localization.

[0020] Further, in steps 1-3, for each unlabeled pedestrian image in the dataset, generating pseudo-labels for target localization includes the following sub-steps:

[0021] Step 1-3-1: Let img be a 3D pixel value matrix of an unmarked pedestrian image, with a size of... , Indicates the height of the image. The width of the image is represented by 3, and the number 3 represents the three color channels: red, green, and blue. Pixel values ​​from 0 to 255 are divided into c intervals. Take the median value of each interval as the representative pixel value for that interval, and define the pixel value interval as follows: The representative pixel value is ,in, It is a preset integer. ;

[0022] Step 1-3-2: Generate a size of A three-dimensional matrix A, any element in matrix A Corresponding pixel values ​​on img ,like The range of pixel values ​​is ,make ,in, , , ;

[0023] Step 1-3-3: Generate two sizes. Given two-dimensional matrices B and C, any element in matrices B and C and Corresponding pixels on the img ,in, , Initialize all elements in matrix B to False, and any element in matrix C... ;

[0024] Steps 1-3-4: Select an element in matrix B whose value is False. ,make , Represented by pixels Centered Local Binary Pattern (LBP) features of the region , , ,in, Calculated based on matrix C, It is a pre-set odd number. and This represents the minimum and maximum values ​​of the pixel's x-axis index. and This represents the minimum and maximum values ​​of the pixel's y-axis index; d is a temporary variable.

[0025] Steps 1-3-5: If pixel points The pixel value ranges for the red, green, and blue color channels are respectively Centered on this pixel The LBP characteristics of the region are ,from Surrounding distance equals Choose two types of pixels from the available pixels:

[0026] One type is pixels with similar colors, referring to pixels whose red, green, and blue color channels are located in the same range. The pixels;

[0027] Another type is pixels with similar textures, requiring that the pixel be centered on the desired pixel. LBP characteristics of the region and The correlation is greater than the preset threshold;

[0028] Change the element value of matrix B corresponding to the selected pixel to True, and update it according to the x-axis and y-axis indices of the selected pixel. , , and ,in, , , LBP features are calculated based on matrix C;

[0029] Step 1-3-6: If no pixel was selected in step 1-3-5, proceed to step 1-3-7; otherwise, let... Return to steps 1-3-5;

[0030] Step 1-3-7: If , , and The target detection rectangular positioning box meets the preset requirements and is saved. , , and If matrix B contains no elements with a value of False, then the several groups that have been saved... , , and The rectangular bounding box data is the target location pseudo-marker for the image (img); otherwise, return to steps 1-3-4.

[0031] Furthermore, in step 2, constructing a pedestrian target detection model based on the improved SSD includes the following sub-steps:

[0032] Step 2-1: Replace the backbone network VGG-16 in the original SSD512 network with a backbone network consisting of five sequentially connected attention downsampling convolutional modules;

[0033] Step 2-2: The first multi-scale output is replaced by the output of the VGG-16 convolutional layer Conv4_3 + ReLU + L2Normalization in the original SSD512 with the output feature map F1 of the fourth attention downsampling convolutional module;

[0034] Steps 2-3: Add a batch normalization layer between each convolutional layer and activation layer in the Conv6 and Conv7 modules of the original SSD512 network;

[0035] Steps 2-4: Replace the five convolutional modules of the multi-scale network in the original SSD512 network with two attention downsampling convolutional modules, two downsampling convolutional modules and one normalization convolutional module connected in sequence.

[0036] Steps 2-5: In the self-supervised pre-training stage, the last fully connected layer of the classification and recognition branch in the original SSD512 network is removed, and a batch normalization layer is added after each convolutional layer as a feature extraction branch to extract features of the localization prediction box labeled region.

[0037] Further, in step 3, the pedestrian target detection model based on the improved SSD is trained using the self-supervised pre-training dataset obtained in step 1 to obtain a pre-trained model, specifically including the following steps:

[0038] Step 3-1: Set the initial parameters for model training;

[0039] Step 3-2: During training, calculate the loss using a loss function composed of target localization loss and target similarity loss; calculate the gradient based on the loss and update the network model parameters using the Adam optimizer;

[0040] Step 3-3: Evaluate the model; the smaller the loss value, the more accurate the detection results obtained by the model.

[0041] Steps 3-4: Repeat the training process from steps 3-2 to 3-3. Evaluate the network model after each training round until the preset total number of training rounds is completed. Save the optimal network model as the pre-trained model.

[0042] Furthermore, in step 4, the labeled pedestrian image dataset is obtained and preprocessed. The dataset is then divided into a training set, a validation set, and a test set according to a set ratio. This includes the following sub-steps:

[0043] Step 4-1: Download or create your own labeled pedestrian image dataset from an open-source dataset website;

[0044] Step 4-2: Maintain the aspect ratio of each pedestrian image in the dataset, scale the larger side to a specified number of pixels, and then center the scaled image within a black background image of a specified size.

[0045] Step 4-3: Divide the pedestrian images and their labels in the dataset into training set, validation set and test set according to the proportions.

[0046] Furthermore, in step 5, the pre-trained model is fine-tuned using the training set and validation set obtained in step 4 to obtain the target model, including the following sub-steps:

[0047] Step 5-1: Add a fully connected layer to the end of the feature extraction branch in the pre-trained model as a classification and recognition branch to identify pedestrian targets in the area marked by the prediction bounding box; set the initial parameters for model training.

[0048] Step 5-2: During training, calculate the loss using a loss function composed of target localization loss and target recognition loss; calculate the gradient based on the loss and update the network model parameters using the Adam optimizer;

[0049] Step 5-3: Use the average precision of the entire validation set ( The model is evaluated. A higher value indicates that the model obtains more accurate detection results;

[0050] Step 5-4: Repeat the training process from Step 5-2 to Step 5-3. After each training round, use the validation set to evaluate the network model until the preset total number of training rounds is completed. Save the best network model as the target model.

[0051] Further, in step 6, the test set image obtained in step 4 is input as the pedestrian image to be detected into the target model obtained in step 5 to obtain the pedestrian target detection result of the image, including the following sub-steps:

[0052] Step 6-1: Import the target model saved in Step 5, input the test set images obtained in Step 4 as pedestrian images to be detected into the target model, and output the final pedestrian target detection results;

[0053] Step 6-2: Calculation Value, save the test results.

[0054] The present invention also provides: an electronic device, comprising:

[0055] One or more processors;

[0056] A storage device on which one or more programs are stored;

[0057] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the self-supervised pedestrian target detection methods described above.

[0058] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0059] 1. For a large number of unlabeled pedestrian images, this invention generates pseudo-labels for target localization by aggregating pixels with similar color or texture features. These pseudo-labels are used for self-supervised pre-training of a pedestrian target detection model based on an improved SSD, making the self-supervised pre-training process closely related to the subsequent pedestrian target detection task and improving the detection accuracy of the model.

[0060] 2. When extracting features, this invention utilizes spatial attention to enhance features, better distinguish between foreground and background, and further improve the detection accuracy of the model. Attached Figure Description

[0061] Figure 1 This is a flowchart of the self-supervised pedestrian target detection method based on color and texture features of the present invention;

[0062] Figure 2 Pixels in embodiments of the present invention Surrounding distance equals A diagram illustrating the pixels;

[0063] Figure 3 This is a schematic diagram of the pedestrian target detection model based on the improved SSD of the present invention;

[0064] Figure 4 This is a schematic diagram of the attention downsampling convolution module structure of the present invention;

[0065] Figure 5 This is a schematic diagram of the downsampling convolution module structure of the present invention;

[0066] Figure 6 This is a schematic diagram of the structure of the dilated convolution module of the present invention;

[0067] Figure 7 This is a schematic diagram of the normalized convolution module structure of the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0069] In one embodiment of the present invention, a self-supervised pedestrian target detection method based on color and texture features is provided, the specific process of which is as follows: Figure 1 As shown, the steps include:

[0070] Step 1: Obtain an unlabeled pedestrian image dataset and perform data preprocessing to use it as a self-supervised pre-training dataset.

[0071] Furthermore, step 1 specifically includes the following steps:

[0072] Step 1-1: Download or create an unlabeled pedestrian image dataset from the internet; the unlabeled pedestrian image dataset used in this embodiment contains 20,000 unlabeled pedestrian images collected from the internet;

[0073] Steps 1-2: Maintain the aspect ratio of each pedestrian image in the dataset, scale the larger side to 513 pixels, then center the scaled image in a 513×513 black background image, and then perform data augmentation on the processed image using random cropping, scaling, horizontal flipping, random changes in image brightness, random changes in image contrast, random changes in image chroma, and random changes in image saturation.

[0074] Steps 1-3: For each unlabeled pedestrian image in the dataset, generate pseudo-labels for target localization;

[0075] Furthermore, steps 1-3 specifically include the following steps:

[0076] Step 1-3-1: Let img be a 3D pixel value matrix of an unmarked pedestrian image, with a size of... , Indicates the height of the image. The width of the image is represented by 3, and the number 3 represents the three color channels: red, green, and blue. Pixel values ​​from 0 to 255 are divided into c intervals. Take the median value of each interval as the representative pixel value for that interval, and define the pixel value interval as follows: The representative pixel value is ,in, It is a preset integer. ;

[0077] Preferably, this embodiment is set as follows: .

[0078] Step 1-3-2: Generate a size of A three-dimensional matrix A, any element in matrix A Corresponding pixel values ​​on img ,like The range of pixel values ​​is ,make ,in, , , ;

[0079] Step 1-3-3: Generate two sizes. Given two-dimensional matrices B and C, any element in matrices B and C and Corresponding pixels on the img ,in, , Initialize all elements in matrix B to False, and any element in matrix C... ;

[0080] Steps 1-3-4: Select an element in matrix B whose value is False. ,make , Represented by pixels Centered Local Binary Pattern (LBP) features of the region , , ,in, Calculated based on matrix C, It is a pre-set odd number. and This represents the minimum and maximum values ​​of the pixel's x-axis index. and This represents the minimum and maximum values ​​of the pixel's y-axis index; d is a temporary variable.

[0081] Preferably, this embodiment is set as follows: .

[0082] Steps 1-3-5: If pixel points The pixel value ranges for the red, green, and blue color channels are respectively Centered on this pixel The LBP characteristics of the region are ,from Surrounding distance equals Choose two types of pixels from the available pixels:

[0083] One type is pixels with similar colors, referring to pixels whose red, green, and blue color channels are located in the same range. The pixels;

[0084] Another type is pixels with similar textures, requiring that the pixel be centered on the desired pixel. LBP characteristics of the region and The correlation is greater than the preset threshold;

[0085] Change the element value of matrix B corresponding to the selected pixel to True, and update it according to the x-axis and y-axis indices of the selected pixel. , , and ,in, , , The LBP features are calculated based on matrix C.

[0086] In this embodiment, pixels Surrounding distance equals pixels such as Figure 2 As shown, the dark gray area is Surrounding distance equals The pixels; the light gray area is Surrounding distance equals The pixels; The same principle applies to other situations.

[0087] Step 1-3-6: If no pixel was selected in step 1-3-5, proceed to step 1-3-7; otherwise, let... Return to steps 1-3-5;

[0088] Step 1-3-7: If , , and The target detection rectangular positioning box meets the preset requirements and is saved. , , and If matrix B contains no elements with a value of False, then the several groups that have been saved... , , and The rectangular bounding box data is the target location pseudo-marker for the image (img); otherwise, return to steps 1-3-4.

[0089] Step 2: Construct a pedestrian target detection model based on an improved SSD.

[0090] Furthermore, step 2 specifically includes the following steps:

[0091] Step 2-1: As Figure 3 As shown, the backbone network VGG-16 in the original SSD512 network is replaced with a backbone network consisting of five sequentially connected attention downsampling convolutional modules.

[0092] The structure of the attention downsampling convolutional module is as follows: Figure 4 As shown, the input feature map is first processed through two sets of " The sequence is "convolutional layer + batch normalization layer + ReLU activation function layer". Then, the output feature map of the second ReLU activation function layer is input into the spatial attention submodule, producing the output of the attention downsampling convolutional module. The padding of the convolutional layer is set to 1, the first one. The stride of the convolutional layer is set to 2. In the first, second, third, fourth, and fifth attention downsampling convolutional modules... The number of channels in the convolutional layers were set to 64, 128, 256, 512, and 512, respectively.

[0093] In the spatial attention submodule, the input feature map is fed into a channel average pooling layer and a channel max pooling layer, respectively. The output feature maps of these two pooling layers are added together, and then passed through a... A convolutional layer and a sigmoid activation function layer output a spatial attention feature map. Finally, the input feature map is multiplied by the spatial attention feature map and output. The channel average pooling layer and the channel max pooling layer pool all channels. The padding of the convolutional layer is set to 3, and the number of channels is set to 1.

[0094] Step 2-2: The output of the first scale is replaced by the output of the VGG-16 convolutional layer Conv4_3 + ReLU + L2Normalization in the original SSD512 with the output feature map F1 of the fourth attention downsampling convolutional module.

[0095] Steps 2-3: Add a batch normalization layer between each convolutional layer and activation layer in the Conv6 and Conv7 modules of the original SSD512 network.

[0096] Steps 2-4: Replace the five convolutional modules of the multi-scale network in the original SSD512 network with two attention downsampling convolutional modules, two downsampling convolutional modules and one normalization convolutional module connected in sequence.

[0097] In the sixth attention downsampling convolution module and the seventh attention downsampling convolution module The number of channels in the convolutional layers are set to 512 and 256 respectively; the downsampling convolutional module is as follows: Figure 5 As shown, it contains two sets of " "Convolutional layer + batch normalization layer + ReLU activation function layer", where the padding of the convolutional layer is set to 1, the stride of the first convolutional layer is set to 2, and the two downsampling convolutional modules... The number of channels in the convolutional layers is set to 256.

[0098] Normalized convolution module such as Figure 6 As shown, it contains two sets of " The structure consists of "convolutional layer + batch normalization layer + ReLU activation function layer", where the padding of the first and second convolutional layers is set to 1 and 0 respectively, and the number of channels in the convolutional layers is set to 256.

[0099] Steps 2-5: In the self-supervised pre-training stage, the last fully connected layer of the classification and recognition branch in the original SSD512 network is removed, and a batch normalization layer is added after each convolutional layer as a feature extraction branch to extract features of the localization prediction box labeled region.

[0100] Step 3: Use the self-supervised pre-training dataset obtained in Step 1 to train the pedestrian target detection model based on the improved SSD, and obtain the pre-trained model.

[0101] Furthermore, step 3 specifically includes the following steps:

[0102] Step 3-1: Set the initial parameters for model training as follows:

[0103] Initial learning rate: 0.01;

[0104] The learning rate was changed to 0.001 after the 100th round of training.

[0105] The learning rate was changed to 0.0001 after 150 rounds of training.

[0106] Optimizer: Adam optimizer;

[0107] Batch size: 8;

[0108] Training epochs: 200.

[0109] Step 3-2: During training, a loss function composed of target localization loss and target similarity loss is used to calculate the loss. The loss function is defined as follows:

[0110] ;

[0111] in, This represents the total number of predicted localization boxes that match the target localization marker box. Indicates target location loss. This represents the target similarity loss.

[0112] The target localization loss is calculated using the smooth L1 loss function, as shown below:

[0113] ;

[0114] in,

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] Pos represents the set of all predicted localization boxes that match the target localization box; cx represents the x-coordinate of the center point of the predicted localization box and the target localization box; cy represents the y-coordinate of the center point of the predicted localization box and the target localization box; w represents the width of the predicted localization box and the target localization box; h represents the height of the predicted localization box and the target localization box. Indicates an indicator function, when Time indicates the first The first localization prediction box and the second The target location bounding boxes match, and the category of the target location bounding boxes is... ; Represents the smooth L1 loss function; This represents the value of the i-th localization prediction box at cx; This represents the value of the i-th localization prediction box at cy; This represents the value of the i-th localization prediction box at position w; This represents the value of the i-th location prediction box at time h; This represents the offset of the j-th target localization bounding box relative to the ith localization predicted box in cx. , This represents the value of the i-th default box in cx. This represents the value of cx for locating the j-th target. This represents the value of the i-th default box in w; This represents the offset of the j-th target localization bounding box relative to the i-th predicted localization box at cy. , This represents the value of the j-th target location bounding box in cy. This represents the value of the i-th default box in cy. This represents the value of h for the i-th default box; This represents the offset of the j-th target localization bounding box relative to the i-th predicted localization box in w. , This represents the value of the j-th target location bounding box in w; This represents the offset of the j-th target localization bounding box relative to the i-th predicted localization box at position h. , This represents the value of h for the j-th target location bounding box.

[0120] The target similarity loss is calculated as follows:

[0121] ;

[0122] Where La represents the set of target location bounding boxes; This represents the set of all predicted localization boxes that match the j-th target localization bounding box; The feature representing the i-th localization prediction box that matches the j-th target localization bounding box. The feature representing the k-th localization prediction box that matches the j-th target localization bounding box; Represents the L2 norm;

[0123] The gradient is calculated based on the loss, and the Adam optimizer is used to update the network model parameters.

[0124] Step 3-3: Evaluate the model; the smaller the loss value, the more accurate the detection results obtained by the model.

[0125] Steps 3-4: Repeat the training process from steps 3-2 to 3-3. Evaluate the network model after each training round until the preset total number of training rounds of 200 rounds is completed. Save the best network model as the pre-trained model.

[0126] Step 4: Obtain the labeled pedestrian image dataset and perform data preprocessing, dividing the dataset into training, validation, and test sets according to the proportions.

[0127] Furthermore, step 4 specifically includes the following steps:

[0128] Step 4-1: Download the WiderPerson dataset from the official website of the WiderPerson dataset. The WiderPerson dataset contains 13,382 pedestrian images, including 8,000 training images, 1,000 validation images, and 4,382 test images.

[0129] Step 4-2: Maintain the aspect ratio of each pedestrian image in the dataset, scale the larger side to 513 pixels, then center the scaled image in a 513×513 black background image, and then perform data augmentation on the processed image using random cropping, scaling, horizontal flipping, random changes in image brightness, random changes in image contrast, random changes in image chroma, and random changes in image saturation.

[0130] Step 4-3: Put the processed training images and their labels into the training set, put the processed validation images and their labels into the validation set, and put the processed test images and their labels into the test set.

[0131] Step 5: Use the training set and validation set obtained in Step 4 to fine-tune the pre-trained model to obtain the target model.

[0132] Furthermore, step 5 specifically includes the following steps:

[0133] Step 5-1: Add a fully connected layer to the end of the feature extraction branch in the pre-trained model as a classification and recognition branch to identify pedestrian targets in the area marked by the localization prediction box;

[0134] The initial parameters for model training are set as follows:

[0135] Initial learning rate: 0.01;

[0136] The learning rate was changed to 0.001 after the 100th round of training.

[0137] The learning rate was changed to 0.0001 after 150 rounds of training.

[0138] Optimizer: Adam optimizer;

[0139] Batch size: 8;

[0140] Training epochs: 200.

[0141] Step 5-2: During training, the loss is calculated using a loss function composed of the target localization loss and the target recognition loss. The loss function is defined as follows:

[0142] ;

[0143] in, This represents the total number of predicted localization boxes that match the target localization marker box. Indicates target location loss. This represents the target recognition loss;

[0144] The target recognition loss is calculated as follows:

[0145] ;

[0146] Where Neg represents the set of all predicted localization boxes that are not associated with the target localization bounding box; This represents the confidence score that the normalized i-th localization prediction box in Pos belongs to the p-th class. , This represents the confidence that the i-th localization prediction box in Pos belongs to the p-th class; This represents the confidence that the r-th localized predicted bounding box in Neg belongs to the background class; Represents a logarithmic function;

[0147] The gradient is calculated based on the loss, and the Adam optimizer is used to update the network model parameters.

[0148] Step 5-3: Evaluate the model using the average precision (mAP) of the validation set. A higher mAP value indicates more accurate detection results. The formula for calculating mAP is as follows:

[0149] ;

[0150] ;

[0151] in, This indicates the number of pedestrian targets to be identified. This represents the total number of pedestrians with the same identity as the i-th pedestrian target in the detection results. It refers to the exact location index of the r-th pedestrian with the same identity in the detection sorting list.

[0152] Step 5-4: Repeat the training process from Step 5-2 to Step 5-3. After each training round, use the mAP value of the validation set to evaluate the network model until the preset total number of training rounds of 200 rounds is completed. Save the best network model as the target model.

[0153] Step 6: Input the test set image obtained in Step 4 as the pedestrian image to be detected into the target model obtained in Step 5 to obtain the pedestrian target detection result of the image.

[0154] Furthermore, step 6 specifically includes the following steps:

[0155] Step 6-1: Import the target model saved in Step 5, input the test set images obtained in Step 4 as pedestrian images to be detected into the target model, and output the final pedestrian target detection results;

[0156] Step 6-2: Calculate the mAP value and save the test results.

[0157] In this embodiment of the invention, an electronic device is also provided, including: one or more processors; a storage device storing one or more programs thereon; when the one or more programs are executed by the one or more processors, the one or more processors implement the self-supervised pedestrian target detection method of the foregoing embodiments.

[0158] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A self-supervised pedestrian target detection method based on color and texture features, characterized in that, It includes the following three stages: Phase 1: Self-supervised pre-training phase Step 1: Obtain an unlabeled pedestrian image dataset and perform data preprocessing to use it as a self-supervised pre-training dataset; Step 2: Construct a pedestrian target detection model based on the improved SSD; Step 3: Use the self-supervised pre-training dataset obtained in Step 1 to train the pedestrian target detection model based on the improved SSD to obtain the pre-trained model; Phase 2: Supervised Fine-tuning Phase Step 4: Obtain the labeled pedestrian image dataset and perform data preprocessing. Divide the labeled pedestrian image dataset into training set, validation set and test set according to the proportions. Step 5: Fine-tune the pre-trained model using the training and validation sets obtained in Step 4 to obtain the target model; Phase 3, Prediction Phase Step 6: Input the test set image obtained in Step 4 as the pedestrian image to be detected into the target model obtained in Step 5 to obtain the pedestrian target detection result of the image.

2. The self-supervised pedestrian target detection method according to claim 1, characterized in that, Step 1 involves acquiring an unlabeled pedestrian image dataset and performing data preprocessing, including the following sub-steps: Step 1-1: Download or create your own unlabeled pedestrian image dataset from the internet; Steps 1-2: Maintain the aspect ratio of each pedestrian image in the dataset, scale the larger side to a specified number of pixels, and center the scaled image within a black background image of a specified size. Steps 1-3: For each unlabeled pedestrian image in the dataset, generate pseudo-labels for target localization.

3. The self-supervised pedestrian target detection method according to claim 2, characterized in that, In steps 1-3, for each unlabeled pedestrian image in the dataset, a pseudo-label for target localization is generated, specifically including the following steps: Step 1-3-1: Let img be a 3D pixel value matrix of an unmarked pedestrian image, with a size of... , Indicates the height of the image. The width of the image is represented by 3, and the number 3 represents the three color channels: red, green, and blue. Pixel values ​​from 0 to 255 are divided into c intervals. Take the median value of each interval as the representative pixel value for that interval, and define the pixel value interval as follows: The representative pixel value is ,in, It is a preset integer. ; Step 1-3-2: Generate a size of A three-dimensional matrix A, any element in matrix A Corresponding pixel values ​​on img ,like The range of pixel values ​​is ,make ,in, , , ; Step 1-3-3: Generate two sizes. Given two-dimensional matrices B and C, any element in matrices B and C and Corresponding pixels on the img ,in, , Initialize all elements in matrix B to False, and any element in matrix C... ; Steps 1-3-4: Select an element in matrix B whose value is False. ,make , Represented by pixels Centered Local binary pattern characteristics of the region , , ,in, Calculated based on matrix C, It is a pre-set odd number. and This represents the minimum and maximum values ​​of the pixel's x-axis index. and This represents the minimum and maximum values ​​of the pixel's y-axis index; d is a temporary variable. Steps 1-3-5: If pixel points The pixel value ranges for the red, green, and blue color channels are respectively Centered on this pixel The local binary pattern characteristics of the region are ,from Surrounding distance equals Select pixels with similar color and texture from the selected pixels, change the element values ​​of matrix B corresponding to the selected pixels to True, and update the matrix B based on the x-axis and y-axis indices of the selected pixels. , , and ,in, , , The local binary pattern features are calculated based on matrix C; Step 1-3-6: If no pixel was selected in step 1-3-5, proceed to step 1-3-7; otherwise, let... Return to steps 1-3-5; Step 1-3-7: If , , and The target detection rectangular positioning box meets the preset requirements and is saved. , , and If matrix B contains no elements with a value of False, then the previously saved sets of data will be stored in the matrix. , , and Use the rectangular bounding box data as a pseudo-marker for the target location of the image (img); otherwise, return to steps 1-3-4.

4. The self-supervised pedestrian target detection method according to claim 3, characterized in that, The pixels with similar colors refer to pixels whose pixel values ​​in the red, green, and blue color channels are respectively located in the following ranges: The pixels; The pixels with similar textures refer to those centered around that pixel. Local binary pattern characteristics of the region and The pixels whose correlation is greater than a preset threshold.

5. The self-supervised pedestrian target detection method according to claim 3, characterized in that, Step 2 involves constructing a pedestrian target detection model based on the improved SSD, which includes the following sub-steps: Step 2-1: Replace the backbone network VGG-16 in the original SSD512 network with a backbone network consisting of five sequentially connected attention downsampling convolutional modules; Step 2-2: Replace the first multi-scale output from the output of the VGG-16 convolutional layer Conv4_3 + ReLU + L2Normalization in the original SSD512 network with the output feature map F1 of the fourth attention downsampling convolutional module; Steps 2-3: Add a batch normalization layer between each convolutional layer and activation layer in the Conv6 and Conv7 modules of the original SSD512 network; Steps 2-4: Replace the five convolutional modules of the multi-scale network in the original SSD512 network with two attention downsampling convolutional modules, two downsampling convolutional modules and one normalization convolutional module connected in sequence. Steps 2-5: In the self-supervised pre-training stage, the last fully connected layer of the classification and recognition branch in the original SSD512 network is removed, and a batch normalization layer is added after each convolutional layer as a feature extraction branch to extract features of the localization prediction box labeled region.

6. The self-supervised pedestrian target detection method according to claim 5, characterized in that, In step 3, the pedestrian target detection model based on the improved SSD is trained using the self-supervised pre-training dataset obtained in step 1 to obtain the pre-trained model, including the following sub-steps: Step 3-1: Set the initial parameters for model training; Step 3-2: During training, the loss is calculated using a loss function composed of target localization loss and target similarity loss; the gradient is calculated based on the loss, and the network model parameters are updated using the Adam optimizer; Step 3-3: Evaluate the model; the smaller the loss value, the more accurate the detection results obtained by the model. Step 3-4: Repeat the training process from Step 3-2 to Step 3-3. Evaluate the network model after each training round until the preset total number of training rounds is completed. Save the best network model as the pre-trained model.

7. The self-supervised pedestrian target detection method according to claim 1, characterized in that, In step 4, the labeled pedestrian image dataset is obtained and preprocessed. The dataset is then divided into training, validation, and test sets according to a set ratio. This includes the following sub-steps: Step 4-1: Download or create your own labeled pedestrian image dataset from an open-source dataset website; Step 4-2: Maintain the aspect ratio of each pedestrian image in the dataset, scale the larger side to a specified number of pixels, and center the scaled image within a black background image of a specified size. Step 4-3: Divide the pedestrian images and their labels in the dataset into training set, validation set and test set according to the proportions.

8. The self-supervised pedestrian target detection method according to claim 1, characterized in that, In step 5, the pre-trained model is fine-tuned using the training set and validation set obtained in step 4 to obtain the target model, including the following sub-steps: Step 5-1: Add a fully connected layer to the end of the feature extraction branch in the pre-trained model as a classification and recognition branch to identify pedestrian targets in the area marked by the prediction bounding box; set the initial parameters for model training. Step 5-2: During training, calculate the loss using a loss function composed of target localization loss and target recognition loss; calculate the gradient based on the loss and update the network model parameters using the Adam optimizer; Step 5-3: Evaluate the model using the average precision of the validation set. The higher the average precision of the validation set, the more accurate the detection results obtained by the model. Step 5-4: Repeat the training process from Step 5-2 to Step 5-3. After each training round, use the validation set to evaluate the network model until the preset total number of training rounds is completed. Save the best network model as the target model.

9. The self-supervised pedestrian target detection method according to claim 1, characterized in that, In step 6, the test set image obtained in step 4 is input as the pedestrian image to be detected into the target model obtained in step 5 to obtain the pedestrian target detection result of the image, including the following sub-steps: Step 6-1: Import the target model saved in Step 5, input the test set images obtained in Step 4 as pedestrian images to be detected into the target model, and output the final pedestrian target detection results; Step 6-2: Calculate the average precision value for all classes and save the test results.

10. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the self-supervised pedestrian target detection method as described in any one of claims 1 to 9.

Images