Traffic re-identification model training method and traffic re-identification method

Abstract

The invention relates to a traffic re-identification model training method and a traffic re-identification method, and the training method comprises the steps: carrying out the independent pre-training of a sharing layer; based on the pre-trained sharing layer, independently pre-training a pedestrian branch layer, a motor vehicle branch layer and a non-motor vehicle branch layer through an open-source pedestrian data set, an open-source motor vehicle data set and an open-source non-motor vehicle data set; after pre-training, the open source data set and the target local data set are used as a target training set to carry out first joint training of iteration on the traffic re-identification model, image pairs with feature similarity smaller than a threshold value in the target training set are gradually increased in the iteration process, and the target local data set comprises local non-motor vehicle images. The problem that an existing traffic re-recognition model training method is very dependent on the number of samples of non-motor vehicles, and the recognition performance of a re-recognition model is poor due to the fact that the samples are insufficient is solved.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a training method for a traffic re-identification model and a traffic re-identification method. Background Art

[0002] The core of pedestrian and vehicle re-identification (ReID) technology is to track and identify specific objects in large-scale image or video data through computer vision technology. ReID technology is widely used in urban monitoring systems and intelligent transportation systems. Especially in urban safety and public security management, ReID can help identify and track specific targets, such as pedestrians, vehicles and various types of non-motor vehicles. Traditional re-identification model training usually relies on large-scale supervised datasets and sophisticated feature learning. However, in complex urban environments, the recognition demand for non-motor vehicles such as tricycles, four-wheeled non-motor vehicles (such as small electric vehicles), and bicycles is increasing, and existing datasets and methods are difficult to meet these needs.

[0003] Current ReID technology faces some problems in traffic re-identification:

[0004] 1. Insufficient data sets: There are too few data sets for non-motor vehicles such as tricycles, four-wheel non-motor vehicles, bicycles, and electric vehicles, and it is difficult to collect and label these data. Compared with motor vehicles, these non-motor vehicles have a greater problem of insufficient samples in data. The current training method of traffic re-identification model is very dependent on the number of non-motor vehicle samples. Insufficient samples will make it difficult for the re-identification model to have good recognition performance on these object categories.

[0005] 2. Non-motor vehicles of different models are very similar in appearance. Especially when they do not have obvious features, the model may generate a large number of "duplicate" candidates, that is, different non-motor vehicles that are similar are identified as the same motor vehicle. Especially in the application scenario of large-scale camera networks, it is difficult for the re-identification model to effectively screen out unreasonable candidate results.

[0006] The current training method of the traffic re-identification model is highly dependent on the number of non-motor vehicle samples. Insufficient samples will lead to poor recognition performance of the re-identification model. No effective solution has been proposed so far. Summary of the invention

[0007] The present invention provides a training method for a traffic re-identification model and a traffic re-identification method to solve the problem that the current training method for the traffic re-identification model is highly dependent on the number of non-motor vehicle samples, and insufficient samples will lead to poor recognition performance of the re-identification model.

[0008] In a first aspect, the present invention provides a training method for a traffic re-identification model, wherein the traffic re-identification model includes a shared layer and mutually independent human branch layers, motor vehicle branch layers, non-motor vehicle branch layers, and a discriminator layer, wherein the shared layer is used to extract global features from an input image, the human branch layer, the motor vehicle branch layer, and the non-motor vehicle branch layer are respectively used to extract local features from the global features, the discriminator layer is used to discriminate the type of the global features, and the output of the traffic re-identification model is determined according to the outputs of the human branch layer, the motor vehicle branch layer, the non-motor vehicle branch layer, and the discriminator layer;

[0009] The training method comprises:

[0010] Performing independent pre-training on the shared layer;

[0011] Based on the pre-trained shared layer, the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer are independently pre-trained using an open source pedestrian dataset, a motor vehicle dataset and a non-motor vehicle dataset;

[0012] After pre-training, the open source dataset and the target local dataset are used as target training sets to perform a first joint training on the traffic re-identification model. During the iteration process, image pairs whose feature similarity in the target training set is less than a threshold are added generation by generation. The target local dataset includes local non-motor vehicle images.

[0013] In a second aspect, the present invention provides a traffic re-identification method, comprising:

[0014] The images captured by the traffic camera are recognized by a non-motor vehicle recognition model, and the non-motor vehicle recognition model is obtained by using the training method of the traffic re-recognition model provided in the first aspect.

[0015] Compared with the related art, the training method of the traffic re-identification model provided by the present invention can enable the traffic re-identification model to gradually learn the local non-motor vehicle characteristics, thereby improving the recognition accuracy of local non-motor vehicles, even if the local data set contains fewer local non-motor vehicle images, by continuously screening high-quality sample data in the local data set for iterative training. This solves the problem that the current training method of the traffic re-identification model is highly dependent on the number of non-motor vehicle samples, and insufficient samples will lead to poor recognition performance of the re-identification model.

[0016] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1is a schematic diagram of a traffic re-identification model in some embodiments of the present invention;

[0018] Figure 2 4 is a flow chart of a method for training a traffic re-identification model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0020] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] In an embodiment of the present invention, a method for training a traffic re-identification model is provided. Figure 1 is a schematic diagram of a traffic re-identification model in some embodiments of the present invention. Figure 1 The traffic re-identification model includes a shared layer and independent human branch layers and motor vehicle branch layers ( Figure 1 The shared layer is used to extract global features from the input image. The human branch layer, motor vehicle branch layer and non-motor vehicle branch layer are respectively used to extract local features from global features. The discriminator layer is used to discriminate the type of global features. The output of the traffic re-identification model is determined according to the output of the human branch layer, motor vehicle branch layer, non-motor vehicle branch layer and discriminator layer.

[0022] Through training, the person branch layer should have the ability to extract person features, the motor vehicle branch layer should have the ability to extract motor vehicle features, the non-motor vehicle should have the ability to extract person features, and the discriminator layer should have the ability to classify features. Feature classification refers to determining whether the main feature in the global features is person features, motor vehicle features, or non-motor vehicle features.

[0023] In some of the embodiments, the output of the corresponding branch layer is used as the output of the traffic re-identification model according to the feature type output by the discriminator. For example, if the most probable feature among the outputs of the discriminator is the person feature, the output of the person branch layer is used as the output of the traffic re-identification model. In other embodiments, the outputs of the person branch layer, the motor vehicle branch layer, and the non-motor vehicle branch layer are first fully connected, and then the fully connected features are multiplied with the probability map output by the discriminator to obtain the output of the traffic re-identification model.

[0024] Figure 2 is a flow chart of a training method for a traffic re-identification model provided in an embodiment of the present invention. Figure 2 As shown, the process includes step S110, step S120 and step S130.

[0025] Step S110, performing independent pre-training on the shared layer.

[0026] The shared layer is designed to extract common underlying features of the input image. The shared layer can be pre-trained with a large-scale dataset to obtain a good representation of common visual features. Exemplarily, the shared layer can use ResNet-50 as the backbone network, and the large-scale dataset can be ImageNet. Among them, the shared layer includes an input layer, a backbone layer, and an output layer. The input layer accepts an RGB image of size 224×224×3 with a size of 224×224×3. The backbone layer uses the first four stages of ResNet-50 as convolutional layers and pooling layers to extract low-level and mid-level features. The output layer is used to generate a feature map of size 14×14×1024.

[0027] Since the goal of the shared layer is to extract common underlying features, preferably, self-supervised contrastive learning loss can be used for unsupervised training. The loss function formula is:

[0028]

[0029] Among them, N is the batch size of this batch training, and the subscript i represents the batch count. is the feature vector of the anchor point sample, is the feature vector of the positive sample, is the feature vector of negative samples, is the label of the sample pair, the positive sample pair is 1, the negative sample pair is 0, and m is the distance threshold, which is usually set to 1.

[0030] Step S120, based on the pre-trained shared layer, the human branch layer, the motor vehicle branch layer, and the non-motor vehicle branch layer are independently pre-trained using open source pedestrian datasets, motor vehicle datasets, and non-motor vehicle datasets.

[0031] Among them, each branch layer contains an independent network structure to further extract high-level features. Each branch layer can adopt the same network structure, and the network structure includes a convolution layer, a global average pooling layer and a fully connected layer. The convolution layer further extracts high-level features based on the output of the shared layer, the global average pooling layer converts the feature map into a feature vector of a fixed length, and the fully connected layer maps the feature vector to the embedding space. Exemplarily, in an embodiment in which the backbone layer of the shared layer adopts the first four stages of ResNet-50, the convolution layer in the network structure adopts the fifth stage of ResNet-50.

[0032] At this stage, the parameters of the shared layer after pre-training can be frozen, and the human branch layer, motor vehicle branch layer, and non-motor vehicle branch layer can be independently pre-trained in conjunction with different open source data sets. Specifically, the pedestrian data set is used to train the human branch layer's ability to extract personnel features. First, the common underlying features (global features) of the images in the pedestrian data set are extracted through the shared layer, and then features (high-level features) are extracted from the common underlying features through the human branch layer and the loss is calculated for back propagation, thereby training the human branch layer's ability to extract personnel features. Similarly, the motor vehicle branch layer's ability to extract motor vehicle features is trained through the motor vehicle data set, and the non-motor vehicle branch layer's ability to extract non-motor vehicles is trained through the non-motor vehicle data set.

[0033] Exemplarily, the loss function of the person branch layer, the motor vehicle branch layer, and the non-motor vehicle branch layer during training may adopt the triplet loss commonly used in the ReID field.

[0034] The advantage of pre-training each branch is that the traffic re-identification model can first obtain good preliminary feature expression capabilities on rich multi-category open source data, thereby avoiding the cold start problem during joint training of all data. This method enables the model to have faster convergence speed in subsequent full data training and enhances the robustness of the network.

[0035] In this step, independent pre-training of the discriminator layer is also included. The discriminator layer can be trained with open source datasets to enable it to identify the input features of the model, ensuring that the features of different categories are clearly distinguished in the embedding space. For example, the common underlying features (global features) of the images in the open source dataset are first extracted through the shared layer, and then the features (high-level features) are extracted from the common underlying features through the discriminator layer and the category loss is calculated for back propagation, thereby training the discriminator layer to identify the model input features.

[0036] Exemplarily, the discriminator layer structure includes a fully connected layer and a Softmax layer, the fully connected layer accepts the output features of the shared layer and outputs the category score, and the Softmax layer converts the score into a category probability. The loss function of the discriminator uses the common cross entropy loss.

[0037] Step S130, after pre-training, the open source dataset and the target local dataset are used as target training sets to perform a first joint training on the traffic re-identification model iteratively, and in the iterative process, image pairs whose feature recognition degree is less than a threshold in the target training set are added generation by generation, and the target local dataset includes local non-motor vehicle images.

[0038] In the ReID task, the goal of the joint training strategy is to improve the model's generalization, robustness, and adaptability to the local environment through a multi-dataset and multi-stage training process.

[0039] Specifically, the open source dataset and the target local dataset are integrated together to perform end-to-end joint training on the shared layer, each branch layer, and the discriminator layer in the traffic re-identification model. The independence of each branch is maintained so that each branch is optimized specifically on its own category. At the same time, the shared layer learns common features through the input of all category data.

[0040] During the first joint training, the total loss of the traffic re-identification model is:

[0041]

[0042] Among them, L contrastive represents the loss of the shared layer, The sum of the losses of the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer, L classification represents the loss of the discriminator, λ1, λ2 and λ3 represent three loss weights respectively, and the superscript c represents the branch number, and different numbers represent the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer respectively.

[0043] Furthermore, in joint training, in order to prevent the model from being overly biased towards a certain category of data, the weight of the loss term can be adjusted dynamically. For categories with small amounts of data (such as non-motor vehicles in local data), their loss weights can be appropriately increased to make the model's learning effect on small categories more balanced.

[0044] The first joint training is iterative, that is, the traffic re-identification model is repeatedly trained through the target training set. In the iterative process, in order to improve the model training effect, the target training set is continuously optimized based on the training results of the previous generation, so that the training effect of the next generation is better. Specifically, the proportion of positive and negative sample pairs with large differences in the target training set is continuously increased. Positive and negative sample pairs usually represent objects with strong discrimination and can better provide iconic feature information for the model.

[0045] Generally, it is difficult to introduce additional positive and negative sample pairs. Therefore, we can determine the positive and negative sample pairs that meet the conditions in the target local data set and repeatedly add them to the target training set as high-quality samples. These data contain significant category distinction information and are high-quality data for the model to learn local features. Add the screened high-quality data to the target training set and retrain the model. In each iteration, gradually add new high-quality local data so that the model can continuously make adaptive adjustments in the local environment. After each round of training, perform new feature distance screening, select updated high-quality samples, and add them to the training data again for the next round of training, gradually enhancing the model's localized feature extraction capabilities.

[0046] Accordingly, in some of the embodiments, increasing the image pairs whose feature similarity is greater than a threshold in the target training set generation by generation during the iteration process includes: in the previous first joint training, determining the image pairs whose feature similarity is greater than a threshold in the local data set according to the output of the traffic re-identification model, adding the image pairs to the target training set, and the updated target training set is used to implement the next joint training.

[0047] The process of high-quality data screening and model iterative training can be performed several times until the model converges on the target training set and achieves the expected recognition accuracy. Usually, the selection criteria for high-quality data can be gradually relaxed in each round of iteration to ensure that the model gradually adapts to the local environment.

[0048] The advantage of the above training is that through the screening and iterative training of high-quality local data, the model can gradually enhance the recognition ability of objects in the local environment while retaining the ability to extract general features. For example, non-motor vehicles in a specific city may have unique appearance features (such as color, body structure, etc.). Through iterative training, the model can gradually learn these local features, thereby improving recognition accuracy. Moreover, it does not require a large local non-motor vehicle dataset, but can be obtained through screening and iterative training on a smaller local non-motor vehicle dataset.

[0049] It can be seen from the above description that the training method of the traffic re-identification model provided by the present invention can enable the traffic re-identification model to gradually learn the local non-motor vehicle characteristics, thereby improving the recognition accuracy of local non-motor vehicles, even if the local data set contains fewer local non-motor vehicle images, by continuously screening high-quality sample data in the local data set for iterative training. This solves the problem that the current training method of the traffic re-identification model is highly dependent on the number of non-motor vehicle samples, and insufficient samples will lead to poor recognition performance of the re-identification model.

[0050] In order to further improve the recognition accuracy of the traffic re-identification model, the training method provided by the present invention, in some embodiments, further utilizes spatiotemporal information to assist in object recognition, so as to reduce the redundancy and error rate of the re-identification results. Accordingly, in some embodiments, the training method further includes step S140 after the first joint training.

[0051] Step S140, in the case of introducing the spatiotemporal features of the images in the target training set, the traffic re-identification model is subjected to a second joint training through the target training set. In the second joint training, a spatiotemporal constraint coefficient is introduced into the total loss of the traffic re-identification model, and the spatiotemporal constraint coefficient is used to reduce the loss caused by sample pairs that violate the spatiotemporal constraint, and the spatiotemporal constraint includes that the moving speed of the identified object meets the target condition.

[0052] For example, for a sample pair, the two images are taken at a first location and a second location respectively, and there is the same object in the two images. The moving speed of the object can be calculated based on the distance difference between the first location and the second location and the shooting time difference of the two images. If the moving speed of the object is too fast or too slow, which is not in line with the normal situation, it means that the above sample pair is unreasonable, and the impact of the unreasonable sample pair should be reduced. In the loss function, the loss weight caused by such sample pairs should be reduced.

[0053] In some of the embodiments, the images in the target training set are captured by traffic cameras, and the moving speed of the identified object meets the target conditions, including that the time difference when the identified object appears in the target traffic camera pair when it is in the target time period is greater than the corresponding time threshold; different traffic camera pairs in different time periods have different time thresholds. Considering that the movement of objects has different rules in different time periods and different regions (for example, the movement speed of the identified objects in rush hours in the morning and evening is faster), different target conditions can be set for different time periods and different camera pairs. Specifically, different time thresholds can be set for different camera pairs in different time periods, so that the time difference when the identified object appears in a specific camera pair in a specific time period is greater than the corresponding time threshold.

[0054] During the second joint training, the total loss of the traffic re-identification model is:

[0055]

[0056]

[0057]

[0058] Among them, L contrastive represents the loss of the shared layer, The sum of the losses of the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer, L classification represents the loss of the discriminator, λ1, λ2 and λ3 represent three loss weights respectively, N represents the batch size, and the subscript i represents the batch count. represents the feature vector of the plotted sample, represents the feature vector of the positive sample, Represents the feature vector of negative samples, γ i represents the space-time constraint coefficient, △t i represents the time difference between the traffic cameras of the identified object, T min represents the time threshold, the superscript c represents the branch number, and different numbers represent the human branch layer, motor vehicle branch layer, and non-motor vehicle branch layer respectively.

[0059] In some of the embodiments, the step of determining the time threshold includes: based on the image recognition results of the traffic re-identification model in the first joint training, counting the time differences that appear in different traffic camera pairs when different identified objects are in different time periods; determining a time difference histogram for each traffic camera pair in each time period based on the statistical results, the horizontal axis and vertical axis of the time difference histogram respectively represent the time difference and the proportion of the corresponding time difference, converting the time difference histogram into a time difference integral graph, and determining the target time difference corresponding to the target integral area in the time difference integral graph; and using the target time difference corresponding to each traffic camera pair in each time period as its own time threshold.

[0060] The process of determining the time threshold is described below using an example.

[0061] The first step is to collect data statistics.

[0062] 1. Collect re-identification results: After the first joint training of the traffic re-identification model, the model is used to re-identify historical data to obtain the recognition results of the images taken by each camera.

[0063] 2. Time period division: Divide a day into several time periods, such as early morning (0-7 am), morning rush hour (7-9 am), morning (9-12 pm), afternoon (12-5 pm), evening rush hour (17-19 pm), and night (19-24 pm); at the same time, record different data performances from Monday to Sunday.

[0064] 3. Commuting time statistics: For each pair of cameras A and B, the time difference Δt from the appearance of the identified object in camera A to the appearance of camera B is counted in each time period. Since pedestrians may take public transportation, people, motor vehicles, and non-motor vehicles are no longer classified when calculating commuting time.

[0065] The second step is to calculate the minimum commuting time.

[0066] 1. Draw a histogram: For each camera pair and time period, draw a histogram of the commute time (the time difference between the appearance of the identified object in the camera pair).

[0067] 2. Determine the minimum commuting time: Calculate the integral graph of each histogram, and select the commuting time when the integral area is 5% of the total area (target integral area) from left to right as the minimum commuting time (target time difference), so as to obtain the time threshold of different camera pairs in different time periods.

[0068] 3. Exception handling: For camera pairs with too few samples, a default minimum commuting time (e.g., 0) can be set, or an estimate can be made based on the average commuting time of all camera pairs.

[0069] The third step is to design the minimum commuting time function.

[0070] According to the statistical results in the second step, a minimum commuting time function is constructed to map different camera pairs in different time periods with different minimum commuting times.

[0071] For example, the function input is: the week number, which ranges from 1 (Monday) to 7 (Sunday); the time period, which is divided into multiple fixed time intervals, specifically, the time period in which the average value of the timestamps of a group of re-identified images lies; and the camera number.

[0072] Then, the function output is: minimum commuting time; unit is seconds.

[0073] As follows, based on the above example, the training process of the non-motor vehicle model is further explained.

[0074] In the second joint training, in order to enable the model to automatically learn spatiotemporal features during training and avoid relying too much on manual screening, this example further improves the network architecture and training strategy, making the model more sensitive to the spatiotemporal laws implicit in the data, thereby achieving automatic filtering of recognition results. After completing the following data collation, model network structure, and loss function modification, you can use all the data for training again.

[0075] 1. Improvement of network architecture.

[0076] In order to enable the traffic re-identification model to receive the spatiotemporal information of the image, a spatiotemporal feature embedding module can be added between the shared layer and the branch layer of the model. The spatiotemporal feature embedding module embeds (encodes) the spatiotemporal information of the image input to the model and fuses it with the corresponding image features. The specific process includes the following:

[0077] Camera encoding: Map the camera number to a fixed-length embedding vector e cam .

[0078] Day of the week encoding: Map the day of the week to an embedding vector e day .

[0079] Time segment encoding: Mapping time segments into embedding vectors e time .

[0080] Feature fusion: The image features are fused with the spatiotemporal embedding vectors. For example, the features can be spliced ​​after the image features and then sent to the branch network for a second joint training. The transformer network structure can also be used to increase the weight of the spatiotemporal features to avoid the influence of a large amount of image feature information on the subsequent temporal feature learning.

[0081] 2. Loss function.

[0082] In the first joint training, a time-space penalty term is added to the total loss function to impose a penalty on sample pairs that violate the minimum commuting time constraint. Then, the total loss function used in the second joint training shown above can be obtained.

[0083] The above is an explanation of the second joint training of the traffic re-identification model. In the second joint training, by reducing the impact of unreasonable sample pairs, the traffic re-identification model is trained more accurately, and the non-motor vehicle recognition ability of the traffic re-identification model is improved.

[0084] After the training of the traffic re-identification model is completed, the trained traffic re-identification model can be applied to infer and identify the actual image. In the inference stage of the traffic re-identification model, the inference recognition results can still be screened by spatiotemporal constraints to remove the inference results that do not meet the spatiotemporal constraints.

[0085] Correspondingly, a non-motor vehicle recognition method is also provided in an embodiment of the present invention, which includes: recognizing images taken by a traffic camera through a non-motor vehicle recognition model, and the non-motor vehicle recognition model is obtained using the training method of the traffic re-recognition model provided by the present invention.

[0086] In some embodiments, the non-motor vehicle recognition method further includes: filtering out the inference recognition results of the traffic re-recognition model that do not meet the spatiotemporal constraints. The spatiotemporal constraints are as described above and will not be described in detail here.

[0087] In an embodiment of the present invention, a training device for a traffic re-identification model is also provided. The traffic re-identification model includes a shared layer and mutually independent human branch layers, motor vehicle branch layers, non-motor vehicle branch layers and discriminator layers. The shared layer is used to extract global features from the input image. The human branch layer, motor vehicle branch layer and non-motor vehicle branch layer are respectively used to extract local features from global features. The discriminator layer is used to discriminate the type of global features. The output of the traffic re-identification model is determined according to the output of the human branch layer, motor vehicle branch layer, non-motor vehicle branch layer and discriminator layer. The device is used to implement the above-mentioned embodiments and preferred implementations, and those that have been explained will not be repeated. The terms "module", "unit", "subunit" and the like used below can implement a combination of software and / or hardware for predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0088] The training setup for the traffic re-identification model includes:

[0089] The first pre-training module is used to independently pre-train the shared layer.

[0090] The second pre-training module is used to independently pre-train the human branch layer, the motor vehicle branch layer, and the non-motor vehicle branch layer based on the pre-trained shared layer through open source pedestrian datasets, motor vehicle datasets, and non-motor vehicle datasets.

[0091] The joint training module is used to perform the first joint training of the traffic re-identification model iteratively using the open source dataset and the target local dataset as the target training sets after pre-training, and to increase the image pairs whose feature recognition degree in the target training set is less than the threshold generation by generation during the iteration process, and the target local dataset includes local non-motor vehicle images.

[0092] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0093] In an embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the training method of the traffic re-identification model provided by the present invention.

[0094] In an embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the training method of the traffic re-identification model provided by the present invention are implemented.

[0095] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0096] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

Claims

1. A training method for a traffic re-identification model, characterized in that: The traffic re-identification model includes a shared layer and mutually independent human branch layers, motor vehicle branch layers, non-motor vehicle branch layers and a discriminator layer, wherein the shared layer is used to extract global features from the input image, the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer are respectively used to extract local features from the global features, the discriminator layer is used to discriminate the type of the global features, and the output of the traffic re-identification model is determined according to the outputs of the human branch layer, the motor vehicle branch layer, the non-motor vehicle branch layer and the discriminator layer; The training method comprises: Performing independent pre-training on the shared layer; Based on the pre-trained shared layer, the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer are independently pre-trained using an open source pedestrian dataset, a motor vehicle dataset and a non-motor vehicle dataset; After pre-training, the open source dataset and the target local dataset are used as target training sets to perform a first joint training on the traffic re-identification model. During the iteration process, image pairs whose feature similarity in the target training set is less than a threshold are added generation by generation. The target local dataset includes local non-motor vehicle images.

2. The training method of the traffic re-identification model according to claim 1, characterized in that: The shared layer includes the first four stages of ResNet-50, and the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer all include the fifth stage of ResNet-50.

3. The training method of the traffic re-identification model according to claim 1, characterized in that: The loss of the shared layer is a self-supervised contrastive learning loss; And / or, the losses of the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer are all triplet losses; And / or, the loss of the discriminator is a cross entropy loss; During the first joint training, the total loss of the traffic re-identification model is: Among them, L contrastive represents the loss of the shared layer, represents the sum of the losses of the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer, L classification represents the loss of the discriminator, λ1, λ2 and λ3 represent three loss weights respectively, and the superscript c represents the branch number.

4. The method for training a traffic re-identification model according to claim 1, characterized in that: In the iterative process, the image pairs whose feature recognition degree is greater than the threshold in the target training set are added generation by generation, including: In the previous first joint training, image pairs with feature similarity greater than a threshold are determined in the local data set according to the output of the traffic re-identification model, and the image pairs are added to the target training set. The updated target training set is used to implement the next joint training.

5. The training method of the traffic re-identification model according to claim 1, characterized in that: The training method further comprises, after the first joint training: In the case of introducing the spatiotemporal features of the images in the target training set, performing a second joint training on the traffic re-identification model through the target training set; Among them, during the second joint training, a spatiotemporal constraint coefficient is introduced into the total loss of the traffic re-identification model, and the spatiotemporal constraint coefficient is used to reduce the loss caused by sample pairs that violate the spatiotemporal constraints. The spatiotemporal constraints include that the moving speed of the identified object meets the target conditions.

6. The method for training a traffic re-identification model according to claim 5, characterized in that: The images in the target training set are obtained by taking pictures of traffic cameras, and the moving speed of the identified object meets the target condition, including that the time difference of the identified object appearing in the target traffic camera pair when it is in the target time period is greater than the corresponding time threshold, and different traffic camera pairs in different time periods have different time thresholds; During the second joint training, the total loss of the traffic re-identification model is: Among them, L contrastive represents the loss of the shared layer, represents the sum of the losses of the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer, L classification represents the loss of the discriminator, λ1, λ2 and λ3 represent three loss weights respectively, N represents the batch size, and the subscript i represents the batch count. represents the feature vector of the plotted sample, represents the feature vector of the positive sample, Represents the feature vector of negative samples, γ i represents the space-time constraint coefficient, △t i represents the time difference between the traffic cameras of the identified object, T min represents the time threshold, and the superscript c represents the branch number.

7. The method for training a traffic re-identification model according to claim 6, characterized in that: The step of determining the time threshold comprises: According to the image recognition results of the traffic re-recognition model in the first joint training, counting the time differences when different recognized objects appear in different traffic camera pairs in different time periods; Determine the statistical results to determine a time difference histogram of each traffic camera pair in each time period, wherein the horizontal axis and the vertical axis of the time difference histogram represent the time difference and the proportion of the corresponding time difference respectively, convert the time difference histogram into a time difference integral graph, and determine the target time difference corresponding to the target integral area in the time difference integral graph; The target time difference corresponding to each pair of traffic cameras in each time period is used as its own time threshold.

8. A training device for a traffic re-identification model, characterized in that: The traffic re-identification model includes a shared layer and mutually independent human branch layers, motor vehicle branch layers, non-motor vehicle branch layers and a discriminator layer, wherein the shared layer is used to extract global features from the input image, the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer are respectively used to extract local features from the global features, the discriminator layer is used to discriminate the type of the global features, and the output of the traffic re-identification model is determined according to the outputs of the human branch layer, the motor vehicle branch layer, the non-motor vehicle branch layer and the discriminator layer; The training device comprises: A first pre-training module, used for independently pre-training the shared layer; A second pre-training module is used to independently pre-train the human branch layer, the motor vehicle branch layer and the non-motor vehicle branch layer respectively based on the pre-trained shared layer through an open source pedestrian dataset, a motor vehicle dataset and a non-motor vehicle dataset; The joint training module is used for, after pre-training, iteratively performing the first joint training of the traffic re-identification model using the open source dataset and the target local dataset as the target training sets, and gradually increasing the image pairs whose feature similarity is less than a threshold in the target training set during the iteration process, wherein the target local dataset includes local non-motor vehicle images.

9. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the training method for the traffic re-identification model according to any one of claims 1 to 7.

10. A traffic re-identification method, characterized in that: include: The images captured by the traffic camera are recognized by a non-motor vehicle recognition model, wherein the non-motor vehicle recognition model is obtained by using the training method of the traffic re-recognition model described in any one of claims 1 to 7.

Images