Multi-task pedestrian re-identification method, system and electronic device based on fisheye probe

Abstract

The application provides a multi-task pedestrian re-identification method and system based on a fisheye probe and an electronic device, comprising the following steps: extracting an identity feature of a pedestrian based on a pedestrian image dataset, wherein the pedestrian image dataset is labeled with pedestrian attribute information; establishing a classification learning task and a metric learning task of a feature extraction network model based on the identity feature of the pedestrian and the pedestrian attribute information; optimizing the feature extraction network model by using a classification loss function; optimizing the feature extraction network model by using a triple loss function; obtaining a pedestrian image identified by the fisheye probe, and performing pedestrian re-identification on the pedestrian image identified by the fisheye probe by using the optimized feature extraction network model. The application solves the problem that due to the imaging interference of the fisheye probe, the pedestrian re-identification system has a high missed detection and misfit condition, and the accuracy and robustness of the system matching are reduced, and the multi-task learning is complementary to each other, and the overall identification effect and matching accuracy of the system are improved.

Description

Technical Field

[0001] This application belongs to the field of visual surveillance technology, and in particular relates to a multi-task pedestrian re-identification method, system and electronic device based on a fisheye probe. Background Technology

[0002] Fisheye cameras are panoramic video cameras that provide independent, wide-area, and blind-spot-free monitoring. They achieve the seamless monitoring capabilities of multiple standard surveillance cameras with a smaller number of units, significantly reducing wiring and installation costs and providing economical and effective surveillance. Due to these characteristics, fisheye cameras are widely used in retail stores, shopping malls, elevators, and other similar settings. Various customer flow statistics solutions developed in conjunction with fisheye cameras have become a popular and preferred solution for addressing customer flow issues based on computer vision.

[0003] The introduction of fisheye sensors has also brought about imaging problems and challenges to such pedestrian re-identification systems:

[0004] (1) Image distortion and distortion: When a fisheye probe forms an image at the edge of the frame, it is more likely to cause image distortion or distortion than ordinary camera equipment, thus affecting the image quality.

[0005] (2) Changes in image semantic information: Since fisheye probes are usually installed perpendicular to the ground, this installation method can effectively improve the coverage of the fisheye probe, but at the cost of sacrificing a part of the imaging area. In particular, the quality of pedestrian images obtained at the edge and center of the frame is often very poor. At the same time, as pedestrians move, their semantic information changes constantly, which increases the difficulty of system recognition.

[0006] Therefore, due to the imaging interference of the fisheye probe, the pedestrian re-identification system has a high rate of missed detections and mismatches, resulting in a decrease in the accuracy and robustness of the system's matching. Summary of the Invention

[0007] The purpose of this application is to provide a multi-task pedestrian re-identification method, system, and electronic device based on a fisheye probe, in order to solve the problem that the pedestrian re-identification system in the prior art has a high rate of missed detections and mismatches due to imaging interference from the fisheye probe, which leads to a decrease in the accuracy and robustness of the system matching.

[0008] In a first aspect, this application provides a multi-task pedestrian re-identification method based on a fisheye camera, comprising the following steps: extracting pedestrian identity features from a pedestrian image dataset, wherein the pedestrian image dataset is labeled with pedestrian attribute information; establishing a classification learning task and a metric learning task for a feature extraction network model based on the pedestrian identity features and the pedestrian attribute information; optimizing the feature extraction network model using a classification loss function to achieve the classification learning task of the feature extraction network model; optimizing the feature extraction network model using a triplet loss function to achieve the metric learning task of the feature extraction network model; acquiring pedestrian images identified by the fisheye camera, and performing pedestrian re-identification on the pedestrian images identified by the fisheye camera using the optimized feature extraction network model.

[0009] In this application, multiple tasks complement each other through learning, which improves the overall recognition effect and matching accuracy of the system, reduces the impact of pedestrian image distortion and semantic information difference, improves the robustness of the system, and solves the problem in the prior art that pedestrian re-identification systems have a high rate of missed detections and mismatches due to imaging interference from fisheye probes, which leads to a decrease in the accuracy and robustness of the system matching.

[0010] In one implementation of the first aspect, the pedestrian attribute information includes the shooting angle of the fisheye camera and the pedestrian's walking direction.

[0011] In this implementation, the imaging characteristics of a vertical fisheye probe are utilized to restore the true relative position of pedestrians by capturing images from the angle of the fisheye probe. At the same time, combined with the pedestrian's walking direction, parallel training of multiple attribute tasks is achieved, which effectively improves the system's ability to recognize and perceive pedestrians from different angles and directions.

[0012] In one implementation of the first aspect, extracting pedestrian identity features based on a pedestrian image dataset, wherein the pedestrian image dataset is labeled with pedestrian attribute information, includes the following steps:

[0013] Preprocess pedestrian images;

[0014] The pedestrian's identity features are extracted from the preprocessed pedestrian image using a feature extraction network model.

[0015] In one implementation of the first aspect, preprocessing the pedestrian image includes:

[0016] During the training phase of the feature extraction network model, the pedestrian images are scaled and enhanced.

[0017] During the inference stage of the feature extraction network model, the pedestrian image is scaled.

[0018] In one implementation of the first aspect, enhancing the pedestrian image includes jointly performing random horizontal flipping, random erasure, regularization, motion blur, and adding noise on the pedestrian image.

[0019] In one implementation of the first aspect, optimizing the feature extraction network model using a classification loss function to achieve the classification learning task of the feature extraction network model includes:

[0020] The feature extraction network model is trained in parallel to perform a classification learning task to obtain pedestrian identity labels and attribute labels;

[0021] The identity loss of the identity tag and the attribute loss of the attribute tag are calculated using the classification loss function, respectively.

[0022] Minimize the identity loss and the attribute loss.

[0023] In this implementation, parallel training of multiple tasks is performed by sharing parameters, which improves data efficiency, reduces the risk of overfitting, and provides faster learning speed for related or downstream tasks by utilizing auxiliary attribute information.

[0024] In one implementation of the first aspect, optimizing the feature extraction network model using a triplet loss function to achieve the metric learning task of the feature extraction network model includes:

[0025] The joint loss of the pedestrian's identity features and attribute information is calculated using the triplet loss function;

[0026] Minimize the joint loss.

[0027] In one implementation of the first aspect, the formula for calculating the triplet loss function is:

[0028]

[0029] Where N represents the total number of batches in the pedestrian image dataset; X ai X represents an anchor sample randomly selected from the i-th batch of the pedestrian image dataset; pi X represents a Positive sample in the i-th batch that belongs to the same class as the Anchor; ni This represents the Negative samples in the i-th batch that belong to a different class than the Anchor; m total This represents the threshold value, used to control the distance between Positive and Negative samples in the i-th batch.

[0030] In this implementation, classification learning tasks and metric learning tasks are established to mine pedestrian information from different levels. Multi-task triplet loss and multi-task classification loss are combined and used together. The training optimization problem of triplet loss is compensated by classification loss, thereby improving the pedestrian matching performance of the system.

[0031] Secondly, this application provides a pedestrian re-identification system that combines the angle of a fisheye probe and the direction of the pedestrian, comprising:

[0032] The feature extraction module is used to extract the identity features of pedestrians based on a pedestrian image dataset, wherein the pedestrian image dataset is labeled with pedestrian attribute information;

[0033] The task construction module is used to build a classification learning task and a metric learning task for a feature extraction network model based on the pedestrian's identity features and pedestrian attribute information.

[0034] A multi-classification module is used to optimize the feature extraction network model using a classification loss function to achieve the classification learning task of the feature extraction network model;

[0035] The metric learning module is used to optimize the feature extraction network model using the triplet loss function to achieve the metric learning task of the feature extraction network model.

[0036] The pedestrian re-identification module is used to acquire pedestrian images identified by the fisheye probe and to perform pedestrian re-identification on the pedestrian images identified by the fisheye probe using the optimized feature extraction network model.

[0037] Thirdly, this application provides an electronic device, characterized in that it includes: a processor and a memory;

[0038] The memory is used to store computer programs;

[0039] The processor is used to execute the computer program stored in the memory to cause the electronic device to perform the multi-task pedestrian re-identification method based on a fisheye probe as described in any one of claims 1 to 8.

[0040] As described above, the multi-task pedestrian re-identification method, system, and electronic device based on fisheye probes described in this application have the following beneficial effects:

[0041] (1) The mutual learning between multiple tasks improves the overall recognition effect and matching accuracy of the system, reduces the impact of pedestrian image distortion and semantic information difference, improves the robustness of the system, and solves the problem that the pedestrian re-identification system has a high rate of missed detection and mismatch due to the imaging interference of fisheye probe, which leads to a decrease in the accuracy and robustness of the system matching.

[0042] (2) The imaging characteristics of the vertical fisheye probe are utilized to restore the real relative position of pedestrians by shooting from the angle of the fisheye probe. At the same time, combined with the walking direction of pedestrians, parallel training of multi-attribute tasks is realized, which effectively improves the system's ability to recognize and perceive pedestrians from different angles and directions.

[0043] (3) Parallel training of multiple tasks by sharing parameters improves data efficiency, reduces the risk of overfitting, and provides faster learning speed for related or downstream tasks by utilizing auxiliary attribute information.

[0044] (4) Classification learning task and metric learning task were established to mine pedestrian information from different levels. Multi-task triplet loss and multi-task classification loss were combined and used together. The training optimization problem of triplet loss was compensated by classification loss, which improved the pedestrian matching performance of the system. Attached Figure Description

[0045] Figure 1 The flowchart shown is a multi-task pedestrian re-identification method based on a fisheye probe as described in an embodiment of this application.

[0046] Figure 2 The diagram shown is a spherical projection model of the multi-task pedestrian re-identification method based on a fisheye probe as described in the embodiments of this application.

[0047] Figure 3 The diagram shown is a schematic of the feature extraction network model of the multi-task pedestrian re-identification method based on a fisheye probe as described in the embodiments of this application.

[0048] Figure 4 The diagram shown is a schematic of the multi-task pedestrian re-identification system based on a fisheye probe as described in an embodiment of this application.

[0049] Figure 5 The diagram shown is a schematic diagram of the electronic device structure described in an embodiment of this application.

[0050] Component designation explanation

[0051] 41 Feature Extraction Module

[0052] 42 Task Building Module

[0053] 43 Multi-classification module

[0054] 44. Metric Learning Module

[0055] 45 Pedestrian Re-identification Module

[0056] 51 processor

[0057] 52 Memory

[0058] Steps S1 to S5

[0059] Steps S31 to S33

[0060] Steps S41 to S42 Detailed Implementation

[0061] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0062] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0063] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0064] The following embodiments of this application provide a multi-task pedestrian re-identification method, system, and electronic device based on a fisheye camera. The hardware environment corresponding to the technical solutions in these embodiments includes, but is not limited to, a passenger flow statistics and analysis system and a passenger flow counter. The technical solutions in these embodiments will be described in detail below with reference to the accompanying drawings.

[0065] like Figure 1 As shown, this embodiment provides a multi-task pedestrian re-identification method based on a fisheye camera, including the following steps:

[0066] S1. Extract the identity features of pedestrians based on the pedestrian image dataset, which is labeled with pedestrian attribute information.

[0067] It should be noted that the pedestrian image dataset used in the embodiments of this application is a publicly available dataset that is commercially viable.

[0068] In one embodiment, the pedestrian attribute information includes the shooting angle of the fisheye camera and the pedestrian's walking direction.

[0069] Figure 2 The diagram illustrates the imaging process where an object point P0 in three-dimensional space is first mapped onto a sphere, and then nonlinearly projected onto the imaging plane of a fisheye probe. Specifically, the field of view of a conventional fisheye probe is 180 to 270 degrees, approximately a hemisphere. In the spherical projection model, OZ is the optical axis, XOY is the imaging plane of the fisheye probe, and P0 is any point in the spatial scene. Connecting OP0 intersects the hemisphere at point P1. A line parallel to the optical axis OZ is drawn through point P1, intersecting the imaging plane XOY at P2. Then P2 is the image of the spatial point P0 on the imaging plane XOY. In the embodiments of this application, the shooting angle of the fisheye probe refers to the angle between the optical axis OZ and the extension of the incident ray OP0 or P0P1.

[0070] Because pedestrians face different directions, the torso information displayed in pedestrian images will vary significantly, resulting in semantic information discrepancies. Therefore, in pedestrian re-identification tasks, the pedestrian's walking direction is a key attribute that directly affects the matching effect. This application defines the pedestrian's walking direction as eight different direction categories, including front, left front, right front, left side, right side, left back, right back, and back.

[0071] It should be noted that the shooting angle of the fisheye probe and the walking direction of the pedestrian can be information pre-annotated in the pedestrian image dataset, or it can be obtained through manual calibration later.

[0072] In this implementation, the imaging characteristics of a vertical fisheye probe are utilized to restore the true relative position of pedestrians by capturing images from the angle of the fisheye probe. At the same time, combined with the pedestrian's walking direction, parallel training of multiple attribute tasks is achieved, which effectively improves the system's ability to recognize and perceive pedestrians from different angles and directions.

[0073] In one embodiment, extracting pedestrian identity features based on the pedestrian image dataset, wherein the pedestrian image dataset is labeled with pedestrian attribute information, includes the following steps: preprocessing the pedestrian images; and using a feature extraction network model to extract the pedestrian identity features from the preprocessed pedestrian images.

[0074] In one embodiment, preprocessing the pedestrian image includes:

[0075] During the training phase of the feature extraction network model, the pedestrian image is scaled up and enhanced.

[0076] During the inference stage of the feature extraction network model, the pedestrian image is scaled.

[0077] Specifically, scaling the pedestrian image includes changing its size by a fixed ratio and enlarging or reducing it according to random ratio requirements; enhancing the pedestrian image includes jointly applying random horizontal flipping, random erasing, regularization, motion blur, and noise to increase the diversity of the input data. Preprocessing the pedestrian image allows the feature extraction network model to achieve greater versatility and robustness.

[0078] It should be noted that the feature extraction network model used in the embodiments of this application can be any of the popular and readily available large-scale open-source neural network models, such as ResNet50 and VGG16, which are used as the backbone of the entire network structure to extract high-dimensional pedestrian features. For ease of description, the following embodiments of this application will use ResNet50 as an example, where the structure of ResNet50 is shown in Table 1.

[0079] Table 1. Structure of the feature extraction network ResNet50

[0080] adaptiveAvgPool(kennel_size=1×1) Linear(in_features=2048, out_features=4096)

[0081] Where in_features represents the input dimension and out_features represents the output dimension. The input is a 2048-dimensional preprocessed pedestrian image, which, after being processed by the adaptiveAvgPool layer, outputs 4096-dimensional pedestrian features.

[0082] S2. Based on the pedestrian's identity features and attribute information, establish a feature extraction network model for classification and metric learning tasks.

[0083] Specifically, the classification learning task of the feature extraction network model includes pedestrian identity feature classification, fisheye camera shooting angle classification, and pedestrian walking direction classification, and each classification task corresponds to a separate classification loss function; the metric learning task of the feature extraction network model is used to measure the distance between features and optimize the model parameters through distance loss.

[0084] S3. Optimize the feature extraction network model using the classification loss function to achieve the classification learning task of the feature extraction network model.

[0085] In one embodiment, optimizing the feature extraction network model using a classification loss function to achieve the classification learning task of the feature extraction network model includes:

[0086] S31. The feature extraction network model is trained in parallel for classification learning tasks to obtain pedestrian identity labels and attribute labels.

[0087] Specifically, the classification learning task of training the feature extraction network model in parallel includes: establishing a multi-task classification layer; and using the classification layer to extract the pedestrian's identity label and attribute label respectively.

[0088] like Figure 3 As shown, the multi-task classification layer includes a pedestrian category classification layer, a camera angle classification layer, and a pedestrian direction classification layer. The pedestrian category classification layer can be used to obtain pedestrian category features, the camera angle classification layer can be used to obtain camera angle features, and the pedestrian direction classification layer can be used to obtain pedestrian direction features. The pedestrian category features are pedestrian identity labels, while the camera angle features and pedestrian direction features are pedestrian attribute labels.

[0089] The structures of the pedestrian category classification layer, probe angle classification layer, and pedestrian direction classification layer used in the embodiments of this application are shown in Tables 2 to 4.

[0090] Table 2. Structure of Pedestrian Category Classification Layers

[0091] Linear(in_features=4096, out_features=number of people)

[0092] Here, in_features represents the input dimension, and out_features represents the output dimension. Theoretically, each pedestrian represents a different classification target; therefore, the number of pedestrian classifications equals the total number of pedestrians in the training dataset.

[0093] Table 3. Structure of the probe angle classification layer

[0094] Linear(in_features=4096, out_features=9)

[0095] Similarly, in_features is the input dimension, and out_features is the output dimension. Since the probe angle ranges from 0 to 90 degrees, if we use a step size of 10 degrees, the angles within the range can be divided into 9 equal parts, each corresponding to a classification label for a probe angle.

[0096] Table 4. Structure of Pedestrian Direction Classification Layer

[0097] Linear(in_features=4096, out_features=8)

[0098] Similarly, in_features represents the input dimensions, and out_features represents the output dimensions. The eight output dimensions represent eight different orientation categories for pedestrians: front, left front, right front, left side, right side, back left, back right, and back.

[0099] S32. Calculate the identity loss of the identity tag and the attribute loss of the attribute tag using the classification loss function.

[0100] Specifically, the three classification tasks correspond to three loss functions, all of which are cross-entropy loss functions. These functions calculate the final loss by comparing the pedestrian attribute recognition results with the true attributes. For example, the pedestrian category classification layer uses a pedestrian category classification loss function, calculating the loss based on pedestrian category features to precisely measure the pedestrian category classification effect; the camera angle classification layer uses a camera angle classification loss function, calculating the loss based on camera angle features to measure the camera angle classification effect; and the pedestrian direction classification layer uses a pedestrian direction classification loss function, calculating the loss based on pedestrian direction features to measure the pedestrian direction classification effect.

[0101] S33. Minimize the identity loss and the attribute loss.

[0102] Specifically, common algorithms for minimizing loss functions are employed to minimize the pedestrian category classification loss function, the camera angle classification loss function, and the pedestrian direction classification loss function until convergence. This optimizes the feature extraction network model and updates its parameters, reducing errors in model predictions. Common algorithms for minimizing loss functions include gradient descent, stochastic gradient descent, or batch gradient descent.

[0103] In this implementation, parallel training of multiple tasks is performed by sharing parameters, which improves data efficiency, reduces the risk of overfitting, and provides faster learning speed for related or downstream tasks by utilizing auxiliary attribute information.

[0104] S4. Optimize the feature extraction network model using the triplet loss function to achieve the metric learning task of the feature extraction network model.

[0105] This application introduces probe angle and pedestrian direction into the multi-task classification layer, enabling the neural network model used for pedestrian re-identification to perceive more than two pedestrian attributes. In order to enhance the recognition ability of pedestrian features used in the inference stage and improve the learning effect of the neural network model on difficult samples, this application also adds the constraints of these two attributes to the triplet loss function to optimize the distance distribution of feature vectors.

[0106] In one embodiment, optimizing the feature extraction network model using a triplet loss function includes:

[0107] S41. Calculate the joint loss of the pedestrian's identity features and the pedestrian's attribute information using the triplet loss function.

[0108] In one embodiment, the formula for calculating the triplet loss function is:

[0109]

[0110] Where N represents the total number of batches in the pedestrian image dataset; X ai X represents an anchor sample randomly selected from the i-th batch of the pedestrian image dataset; pi X represents a Positive sample in the i-th batch that belongs to the same class as the Anchor; ni This represents the Negative samples in the i-th batch that belong to a different class than the Anchor; m total This represents the threshold value, used to control the distance between Positive and Negative samples in the i-th batch.

[0111] Specifically, m total The calculation formula is:

[0112] m total =m dist +m dire

[0113] in, A n This is the probe angle corresponding to the negative sample, A. p Here, p is the probe angle corresponding to the positive sample, p is the number of samples in the i-th batch, and m is the number of samples in the i-th batch. d1 It is a preset constant; m dire =b*d an +(1-b)*d ap d an This represents the matching status between Anchor samples and Negative samples regarding the pedestrian's walking direction. dire d ap This represents the matching status between Anchor samples and Positive samples regarding the pedestrian's walking direction. dire b is a preset parameter, and the value of b is 0.7 (it needs to be greater than 0.5).

[0114] a dire The calculation formula is as follows:

[0115]

[0116] S42. Minimize the joint loss.

[0117] Specifically, a commonly used algorithm for minimizing the loss function is employed to minimize the joint loss function until convergence, thereby optimizing the feature extraction network model and updating its parameters, and reducing the error in the model's predictions. Commonly used algorithms for minimizing the loss function include gradient descent, stochastic gradient descent, or batch gradient descent.

[0118] S5. Obtain pedestrian images identified by the fisheye probe, and use the optimized feature extraction network model to perform pedestrian re-identification on the pedestrian images identified by the fisheye probe.

[0119] It should be noted that the pedestrian image data identified by the fisheye probe in this embodiment can be a single frame pedestrian image identified by the fisheye probe at a preset frequency, or it can be a sequence of pedestrian image frames selected from video stream data.

[0120] In this implementation, classification learning tasks and metric learning tasks are established to mine pedestrian information from different levels. Multi-task triplet loss and multi-task classification loss are combined and used together. The training optimization problem of triplet loss is compensated by classification loss, thereby improving the pedestrian matching performance of the system.

[0121] In this application, multiple tasks complement each other through learning, which improves the overall recognition effect and matching accuracy of the system, reduces the impact of pedestrian image distortion and semantic information difference, improves the robustness of the system, and solves the problem in the prior art that pedestrian re-identification systems have a high rate of missed detections and mismatches due to imaging interference from fisheye probes, which leads to a decrease in the accuracy and robustness of the system matching.

[0122] It should be noted that the protection scope of the multi-task pedestrian re-identification method based on fisheye probe described in this application embodiment is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principle of this application is included within the protection scope of this application.

[0123] like Figure 4 As shown, this embodiment provides a multi-task pedestrian re-identification system based on a fisheye probe, including:

[0124] Feature extraction module 41 is used to extract the identity features of pedestrians based on a pedestrian image dataset, wherein the pedestrian image dataset is labeled with pedestrian attribute information;

[0125] Task construction module 42 is used for classification learning tasks and metric learning tasks of building a feature extraction network model based on the pedestrian's identity features and pedestrian attribute information;

[0126] The multi-classification module 43 is used to optimize the feature extraction network model using a classification loss function to achieve the classification learning task of the feature extraction network model;

[0127] The metric learning module 44 is used to optimize the feature extraction network model using the triplet loss function to achieve the metric learning task of the feature extraction network model.

[0128] The pedestrian re-identification module 45 is used to acquire pedestrian images identified by the fisheye probe and to perform pedestrian re-identification on the pedestrian images identified by the fisheye probe using the optimized feature extraction network model.

[0129] It should be noted that the structure and principle of the feature extraction module 41, task construction module 42, multi-classification module 43, metric learning module 44 and pedestrian re-identification module 45 correspond one-to-one with the steps and embodiments in the above-mentioned multi-task pedestrian re-identification method based on fisheye probe, so they will not be repeated here.

[0130] The multi-task pedestrian re-identification system based on a fisheye probe provided in this application can realize the multi-task pedestrian re-identification method based on a fisheye probe described in this application. However, the implementation device of the multi-task pedestrian re-identification method based on a fisheye probe described in this application includes, but is not limited to, the structure of the multi-task pedestrian re-identification system based on a fisheye probe listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this application are included within the protection scope of this application.

[0131] like Figure 5 As shown, this embodiment also provides an electronic device, characterized in that it includes: a processor 51 and a memory 52;

[0132] The memory 52 is used to store computer programs;

[0133] The processor 51 is used to execute the computer program stored in the memory 52, so that the electronic device performs the multi-task pedestrian re-identification method based on the fisheye probe described above.

[0134] Preferably, the processor 51 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The memory 52 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0135] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0136] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0137] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] This application also provides a computer-readable storage medium. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0139] This application embodiment may also provide a computer program product comprising one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in this application embodiment are generated. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0140] When the computer program product is executed by a computer, the computer performs the method described in the foregoing method embodiments. The computer program product can be a software installation package; when the foregoing method is required, the computer program product can be downloaded and executed on the computer.

[0141] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0142] In summary, the multi-task pedestrian re-identification method, system, and electronic device based on fisheye probes described in this application improve the overall recognition effect and matching accuracy of the system through mutual learning among multiple tasks, reduce the impact of pedestrian image distortion and semantic information differences, and enhance the robustness of the system. It solves the problem in existing technologies where pedestrian re-identification systems suffer from high rates of missed detections and mismatches due to imaging interference from fisheye probes, leading to reduced system matching accuracy and robustness. Utilizing the imaging characteristics of vertical fisheye probes, the system reconstructs the true relative position of pedestrians through the angle of the fisheye probe's image, and combines this with the pedestrian's walking direction to achieve parallel training of multiple attribute tasks, effectively improving the system's ability to recognize and perceive pedestrians from different angles and directions. Parallel training of multiple tasks using parameter sharing improves data efficiency, reduces the risk of overfitting, and provides faster learning speeds for related or downstream tasks by utilizing auxiliary attribute information. Classification learning tasks and metric learning tasks are established to mine pedestrian information from different levels. The system combines multi-task triplet loss and multi-task classification loss, using classification loss to compensate for the training optimization problem of triplet loss, thereby improving the system's pedestrian matching performance.

[0143] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A multi-task pedestrian re-identification method based on a fisheye camera, characterized in that, Includes the following steps: Pedestrian identity features are extracted from a pedestrian image dataset, which is labeled with pedestrian attribute information; The pedestrian attribute information includes the shooting angle of the fisheye camera and the pedestrian's walking direction; The classification and metric learning tasks are carried out by establishing a feature extraction network model based on the pedestrian's identity characteristics and attribute information. The feature extraction network model is optimized using a classification loss function to achieve the classification learning task of the feature extraction network model; The feature extraction network model is optimized using a triplet loss function to achieve the metric learning task of the feature extraction network model; Obtain pedestrian images identified by a fisheye camera, and use the optimized feature extraction network model to perform pedestrian re-identification on the pedestrian images identified by the fisheye camera; Optimizing the feature extraction network model using a classification loss function to achieve the classification learning task of the feature extraction network model includes: The feature extraction network model is trained in parallel using parameter sharing to perform a classification learning task in order to obtain the pedestrian's identity label and attribute label; The identity loss of the identity tag and the attribute loss of the attribute tag are calculated using the classification loss function, respectively. Minimize the identity loss and the attribute loss; The classification learning task of training the feature extraction network model in parallel using parameter sharing includes: establishing a multi-task classification layer; extracting the identity label and attribute label of the pedestrian using the classification layer; the multi-task classification layer includes a pedestrian category classification layer, a camera angle classification layer, and a pedestrian direction classification layer; the pedestrian category classification layer uses a pedestrian category classification loss function, the camera angle classification layer uses a camera angle classification loss function, and the pedestrian direction classification layer uses a pedestrian direction classification loss function. The optimization of the feature extraction network model using the triplet loss function to achieve the metric learning task of the feature extraction network model includes: The joint loss of the pedestrian's identity features and attribute information is calculated using the triplet loss function; Minimize the joint loss; The formula for calculating the triplet loss function is as follows: ， Where N represents the total number of batches in the pedestrian image dataset; This represents an Anchor sample randomly selected from the i-th batch of the pedestrian image dataset; This represents a Positive sample in the i-th batch that belongs to the same class as the Anchor. This represents a Negative sample in the i-th batch that belongs to a different class than the Anchor. This represents the threshold used to control the distance between Positive and Negative samples in the i-th batch. The calculation formula is: ； in, , This is the probe angle corresponding to the negative sample. is the probe angle corresponding to the positive sample, and p is the number of samples in the i-th batch. It is a preset constant; , This indicates the matching status between Anchor samples and Negative samples regarding the pedestrian's walking direction. , This indicates the matching status between Anchor samples and Positive samples regarding the pedestrian's walking direction. b is a preset parameter, with a value of 0.7; The calculation formula is as follows: 。 2. The multi-task pedestrian re-identification method based on a fisheye probe according to claim 1, characterized in that, Extracting pedestrian identity features from a pedestrian image dataset, wherein the pedestrian image dataset is labeled with pedestrian attribute information, includes the following steps: Preprocess pedestrian images; The pedestrian's identity features are extracted from the preprocessed pedestrian image using a feature extraction network model.

3. The multi-task pedestrian re-identification method based on a fisheye probe according to claim 2, characterized in that, Preprocessing pedestrian images includes: During the training phase of the feature extraction network model, the pedestrian image is scaled up and enhanced. During the inference stage of the feature extraction network model, the pedestrian image is scaled up.

4. The multi-task pedestrian re-identification method based on a fisheye probe according to claim 3, characterized in that, Enhancing the pedestrian image includes performing random horizontal flipping, random erasure, regularization, motion blur, and noise addition on the pedestrian image.

5. A multi-task pedestrian re-identification system based on a fisheye probe, characterized in that, include: The feature extraction module is used to extract the identity features of pedestrians based on a pedestrian image dataset, wherein the pedestrian image dataset is labeled with pedestrian attribute information; The pedestrian attribute information includes the shooting angle of the fisheye camera and the pedestrian's walking direction; The task construction module is used to build a classification learning task and a metric learning task for a feature extraction network model based on the pedestrian's identity features and pedestrian attribute information. A multi-classification module is used to optimize the feature extraction network model using a classification loss function to achieve the classification learning task of the feature extraction network model; The metric learning module is used to optimize the feature extraction network model using the triplet loss function to achieve the metric learning task of the feature extraction network model. The pedestrian re-identification module is used to acquire pedestrian images identified by the fisheye probe and to perform pedestrian re-identification on the pedestrian images identified by the fisheye probe using the optimized feature extraction network model. Optimizing the feature extraction network model using a classification loss function to achieve the classification learning task of the feature extraction network model includes: The feature extraction network model is trained in parallel using parameter sharing to perform a classification learning task in order to obtain the pedestrian's identity label and attribute label; The identity loss of the identity tag and the attribute loss of the attribute tag are calculated using the classification loss function, respectively. Minimize the identity loss and the attribute loss; The classification learning task of training the feature extraction network model in parallel using parameter sharing includes: establishing a multi-task classification layer; extracting the identity label and attribute label of the pedestrian using the classification layer; the multi-task classification layer includes a pedestrian category classification layer, a camera angle classification layer, and a pedestrian direction classification layer; the pedestrian category classification layer uses a pedestrian category classification loss function, the camera angle classification layer uses a camera angle classification loss function, and the pedestrian direction classification layer uses a pedestrian direction classification loss function. The metric learning module optimizes the feature extraction network model using a triplet loss function to achieve the metric learning task of the feature extraction network model, including: The joint loss of the pedestrian's identity features and attribute information is calculated using the triplet loss function; Minimize the joint loss; The formula for calculating the triplet loss function is as follows: ， Where N represents the total number of batches in the pedestrian image dataset; This represents an Anchor sample randomly selected from the i-th batch of the pedestrian image dataset; This represents a Positive sample in the i-th batch that belongs to the same class as the Anchor. This represents a Negative sample in the i-th batch that belongs to a different class than the Anchor. This represents the threshold used to control the distance between Positive and Negative samples in the i-th batch. The calculation formula is: ； in, , This is the probe angle corresponding to the negative sample. is the probe angle corresponding to the positive sample, and p is the number of samples in the i-th batch. It is a preset constant; , This indicates the matching status between Anchor samples and Negative samples regarding the pedestrian's walking direction. , This indicates the matching status between Anchor samples and Positive samples regarding the pedestrian's walking direction. b is a preset parameter, with a value of 0.7; The calculation formula is as follows: 。 6. An electronic device, characterized in that, include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory to cause the electronic device to perform the multi-task pedestrian re-identification method based on a fisheye probe as described in any one of claims 1 to 4.

Images