Vehicle re-identification model construction method based on multi-image learning

Abstract

The embodiment of the application discloses a vehicle re-identification model construction method based on multi-image learning, comprising: acquiring a vehicle image set, each vehicle image comprising a single vehicle and being labeled with a vehicle category; according to the space-time information of each vehicle image, combining multiple vehicle images taken at the same space-time position into an image package, and taking the vehicle category with the most labels in the image package as the labeled category of the image package; and training a vehicle re-identification model based on deep learning by using each image package. The embodiment improves the accuracy of vehicle re-identification.

Description

Technical Field

[0001] This invention relates to the fields of transportation IoT and image detection technology, and in particular to a method for constructing a vehicle re-identification model based on multi-image learning. Background Technology

[0002] In existing technologies, most deep learning-based vehicle re-identification methods train deep learning models using labeled image data, enabling the models to learn the information carried by the images and thus correctly identify the vehicle type.

[0003] However, in real-world traffic scenarios, due to insufficient resolution of the capturing equipment, or in poorly lit environments such as at night or in the rain, it is difficult to distinguish between vehicles that look similar. Therefore, the possibility of errors in manually labeled data is relatively high, thus affecting the accuracy of vehicle re-identification. Furthermore, when certain capturing equipment malfunctions, vehicle re-identification methods using images captured by that equipment will not function or will experience a decrease in accuracy. Summary of the Invention

[0004] This invention provides a method for constructing a vehicle re-identification model based on multi-image learning, in order to solve at least one of the above-mentioned problems.

[0005] In a first aspect, embodiments of the present invention provide a method for constructing a vehicle re-identification model based on multi-image learning, comprising: Acquire a set of vehicle images, each image consisting of a single vehicle labeled with its vehicle category; Based on the spatiotemporal information of each vehicle image, multiple vehicle images taken at the same spatiotemporal location are combined into an image package, and the vehicle category with the most annotations in the image package is taken as the annotation category of the image package. The deep learning-based vehicle re-identification model is trained using various image packets. The training process includes: The feature extraction layer is used to extract the depth features of each vehicle image in the same image package; Using an image classification layer, the confidence level of each vehicle image belonging to each vehicle category is predicted based on each depth feature; Using a feature fusion layer, the depth features of each vehicle image are fused according to each confidence level to obtain the depth features of the same image package; Using an image packet classification layer, the confidence level of the same image packet belonging to each vehicle category is predicted based on the depth features of the same image packet; The parameters of each layer are updated based on the prediction results and labeling categories of each vehicle image and the same image package.

[0006] Secondly, embodiments of the present invention provide a vehicle re-identification method based on multi-image learning, comprising: Acquire an image packet consisting of multiple vehicle images taken at the same spatiotemporal location; The image packet is input into the vehicle re-identification model obtained by the construction method described in the above embodiments to obtain the confidence level of the image packet belonging to each vehicle category; The vehicle category with the highest confidence level is taken as the vehicle category at the same spatiotemporal location.

[0007] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the vehicle re-identification model construction method based on multi-image learning, or the vehicle re-identification method based on multi-image learning, as described in any embodiment.

[0008] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle re-identification model construction method based on multi-image learning or the vehicle re-identification method based on multi-image learning as described in any embodiment.

[0009] In summary, this invention provides a method for constructing a vehicle re-identification model based on multi-image learning. It utilizes multiple images to construct image packets and replaces single-image classification prediction with the classification prediction of these image packets. This reduces recognition errors caused by single-image quality deviations or annotation biases, improving the stability and accuracy of vehicle re-identification. Specifically, during image packet training, both image confidence and image packet confidence are constrained to ensure high accuracy of intermediate image features and the final image packet prediction results. Attached Figure Description

[0010] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1 This is a flowchart of a vehicle re-identification model construction method based on multi-image learning provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a vehicle re-identification model provided in an embodiment of the present invention; Figure 3This is a schematic diagram of another vehicle re-identification model provided in an embodiment of the present invention; Figure 4 This is a flowchart of a vehicle re-identification method based on multi-image learning provided by an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0013] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0014] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0015] Figure 1 This is a flowchart illustrating a method for constructing a vehicle re-identification model based on multi-image learning, provided in an embodiment of the present invention. This method is executed by an electronic device. Figure 1 As shown, the method specifically includes: S110. Obtain a set of vehicle images, each image containing a single vehicle and labeled with the vehicle category.

[0016] This embodiment first acquires a set of vehicle images as the training set for the vehicle re-identification model. Each image in the vehicle image set contains exactly one vehicle and is labeled with the vehicle category to be identified, such as vehicle ID or model.

[0017] Optionally, vehicle images can be collected from multiple cameras (such as all cameras) on the same road segment to form a vehicle image set; these cameras can cover different angles and heights, and can capture images of vehicles passing through the road segment from different angles and heights.

[0018] Optionally, the vehicle images collected can also come from different road segments. There are multiple cameras around each road segment participating in the image set collection, and the multiple cameras in each road segment can capture vehicles passing through that road segment from different angles and heights.

[0019] Optionally, by cropping the image within the target frame of each vehicle from the overall image captured by each camera, a vehicle image containing only a single target can be obtained.

[0020] S120. Based on the spatiotemporal information of each vehicle image, combine multiple vehicle images taken at the same spatiotemporal location into an image package, and use the vehicle category with the most labels in the image package as the label category of the image package.

[0021] This embodiment will provide a method for vehicle re-identification based on image packets. This method focuses more on the comprehensive recognition results of the image packet, rather than being limited to the recognition results of a single image, in order to reduce the impact of poor quality or incorrect annotation of a single image. Therefore, this step first constructs an image packet including multiple vehicle images.

[0022] Optionally, multiple vehicle images captured by various cameras at the same time and at the same spatial coordinates can be combined into an image package. In one specific embodiment, the vehicle image carries a timestamp and camera number automatically labeled by the camera, as well as the coordinate position of the vehicle target frame within the camera's field of view. Based on this information, the time and space at which the vehicle was captured can be determined, thus enabling the combination of images captured by different cameras on the same road segment at the same time and at the same spatial location to form an image package. Using the same method, multiple image packages can be obtained. All images within the same image package are assumed to capture the same vehicle. The vehicle category with the most images labeled within the package is the vehicle category of the image package. For example, if there are 6 images in the same image package, of which 3 are labeled as vehicle category 1, 2 are labeled as vehicle category 2, and 1 is labeled as vehicle category 3, then the labeled category of the image package is vehicle category 1. Optionally, the same time and same spatial coordinates can also refer to the same time range (less than a set time difference) and the same spatial range (less than a set spatial difference), that is, roughly considered to be at the same spatiotemporal location, allowing for a certain range of fluctuation.

[0023] S130. Train the deep learning-based vehicle re-identification model using each image package.

[0024] First, let's introduce the basic structure of the vehicle re-identification model. For example... Figure 2 As shown, the vehicle re-identification model includes a feature extraction layer, an image classification layer, a feature fusion layer (image not shown), and an image packet classification layer. After an image packet is input into the model, it first enters the feature extraction layer, which extracts the depth features of each vehicle image in the image packet. The depth features of each vehicle image then enter the image classification layer, which outputs the confidence score of each vehicle image belonging to each vehicle category based on its depth features. Next, the depth features and confidence scores of all vehicle images are input into the feature fusion layer, which fuses the depth features of each vehicle image based on these confidence scores to obtain the depth features of the image packet. Finally, the depth features of the image packet enter the image packet classification layer, which outputs the confidence score of the image packet belonging to each vehicle category.

[0025] Optionally, the feature extraction layer can adopt a ResNet structure, where all vehicle images can share a single feature extraction layer, or each vehicle image can have its own dedicated feature extraction layer; the image classification layer and image packet classification layer can adopt an MLP structure, where all vehicle images can share a single image classification layer, or each vehicle image can have its own dedicated image classification layer.

[0026] Optionally, the feature fusion layer may include the following operations: (1) in, Representing the features of an image packet, This indicates the vehicle image index within the image packet. This indicates the number of vehicle images in the image packet. Represents vehicle image depth features, Represents vehicle image Vehicle image The confidence level of the labeled vehicle category; Index representing vehicle category, Indicates the number of vehicle categories. Represents vehicle image Belongs to vehicle category The confidence level.

[0027] Based on the above basic structure, the processing of each image packet during model training includes the following steps: S1. Using the feature extraction layer, extract the depth features of each vehicle image in the same image package.

[0028] S2. Using the image classification layer, predict the confidence level of each vehicle image belonging to each vehicle category based on each depth feature. And based on each vehicle image...j Based on the labeled vehicle categories, the confidence score of each vehicle image belonging to the labeled vehicle category is extracted, thus yielding the following image-level loss function. : (2) S3. Using a feature fusion layer, the depth features of each vehicle image are fused according to each confidence level to obtain the depth features of the same image package. Optionally, the fused features are obtained according to formula (1).

[0029] S4. Using the image packet classification layer, predict which vehicle category the same image packet belongs to based on its depth features. confidence level .

[0030] S5. Update the parameters of each layer based on the prediction results and labeled categories of each vehicle image and the same image packet. Optionally, first determine whether the image packet is a positive or negative packet for each vehicle category based on the labeled category of the same image packet. Specifically, for each vehicle category, if the labeled category of the image packet matches that vehicle category, then the image packet is a positive packet for that vehicle category; if the labeled category of the image packet does not match that vehicle category, then the image packet is a negative packet for that vehicle category.

[0031] Then, for each vehicle category Calculate the image packet-level loss as follows: : (3) Finally, based on the following loss function Update the vehicle re-identification model parameters:

[0032] in, and This represents the weighting parameter.

[0033] For ease of illustration, the figure only shows the parameters for updating the feature extraction layer; in reality, all parameters in the model are being updated. Only constraints Try to preserve the information of the image in other categories; while The confidence level for each category is constrained to improve classification accuracy.

[0034] Furthermore, after the model is trained, if the samples used in training are all image packets captured by multiple fixed cameras on the same road segment, then the trained vehicle re-identification model can be used as a dedicated identification model for that road segment, capable of comprehensively capturing image information provided by the multiple fixed cameras from different heights and angles. However, during model use, if one or more of the multiple fixed cameras malfunction, resulting in the loss of some images, it may affect the model's recognition accuracy. To address this situation, this embodiment interpolates a feature supplementation layer between the feature fusion layer and the image packet classification layer of the trained model, such as... Figure 3 As shown, this layer transforms the depth features of missing image packets into depth features that are as close as possible to those of complete image packets, thereby improving the accuracy of image packet category prediction.

[0035] Optionally, the feature augmentation layer can be a fully connected layer, comprising two parts. The first part increases in dimensionality layer by layer to expand the data information; the second part decreases in dimensionality layer by layer to remap useful data information. The output dimension of the first part is the same as the input dimension of the second part to connect the two parts; the output dimension of the second part is the same as the input dimension of the first part to maintain dimensionality consistency with the feature fusion layer and the image packet prediction layer. (See reference...) Figure 3 After the image packet depth features enter the feature augmentation layer, they first enter the first part of the multi-layer fully connected layer to expand the dimension and obtain intermediate features. The intermediate features then enter the second part of the multi-layer fully connected layer to be remapped and the dimension reduced. Finally, the augmented depth features with the same dimension as the input features are output.

[0036] In one specific implementation, the parameters of the vehicle augmentation layer can be trained in the following manner: Step 1: The image packet containing vehicle images from all cameras used in the training phase is called the standard image packet. Masking the image from at least one camera in the standard image packet yields multiple new image packets, called missing image packets. Each missing image packet is then labeled with the same annotation category as the standard image packet. Optionally, a new standard image packet containing all cameras can be acquired, and images from one or more cameras can be randomly masked. The remaining image after each masking is treated as a missing image packet, resulting in multiple missing image packets for each standard image packet. Simultaneously, this standard image packet is input into the trained original model (the original model refers to a vehicle re-identification model without an added vehicle supplementary layer) to obtain the vehicle category of the standard image packet. This category is then used as the annotation category for the standard image packet and all its generated missing image packets. Optionally, the standard image packet containing all cameras from the training phase can also be reused to construct missing image packets, and the annotation categories from the training phase can be used as the annotation categories for the missing image packets.

[0037] Step 2: From the union of the original image package and the multiple new image packages, extract two image packages to form superior and inferior sample pairs. For ease of distinction and description, in this embodiment, the image package with fewer vehicle images is called the first image package, and the image package with more vehicle images is called the second image package. Inputting the first image package into the trained original model yields the confidence score that the first image package belongs to the labeled category; inputting the second image package into the trained original model yields the confidence score that the second image package belongs to the labeled category; in this embodiment, these two confidence scores are referred to as the first confidence score and the second confidence score, respectively. The superior and inferior sample pairs constructed in this step must all meet the following conditions: the first image package is a subset of the second image package, and the first confidence score is less than or equal to the second confidence score. Performing the above operation on each standard image can construct a large number of superior and inferior sample pairs, which together constitute a superior and inferior sample pair set.

[0038] Step 3: Train the parameters of the feature supplementation layer using each superior and inferior sample. The trained feature supplementation layer takes the depth features of the first image in the superior and inferior sample pair contained in the trained original model as input, and takes the depth features of the second image in the same superior and inferior sample pair contained in the trained original model as output.

[0039] Optionally, the parameters of the original model can be frozen using the following loss function. The parameters of the feature augmentation layer are trained:

[0040] in, and These represent the depth features of the first image packet and the second image packet in the same superior and inferior sample pair, respectively. express Features transformed by the image augmentation layer Indicates feature similarity.

[0041] Furthermore, the training of the feature augmentation layer can be divided into two stages. The first stage freezes the parameters of the original model and then... Update the parameters of the image augmentation layer; in the second stage, unfreeze the parameters of the image packet classification layer using the following loss function. Fine-tuning the parameters of the image augmentation layer and the image packet classification layer:

[0042] in, In It is The confidence score of the first image packet obtained after the input image packet classification layer belongs to each vehicle category. and This represents the weights of the two loss terms.

[0043] Regardless of the training method, the goal is to enable the model to learn missing image information (such as uncaptured angles or heights) from the captured images through image supplementation layers. This allows the depth features of the image packet to continuously improve, supplementing features that are beneficial for vehicle category recognition.

[0044] Furthermore, in another specific implementation, during the training process, it's possible that A is first trained using both superior and inferior samples, enabling the model to acquire a certain ability to learn better features. Then, a next superior and inferior sample B appears. The first image packet in B is the same as the first image packet in A, but the second image packet in B is worse than the second image packet in A. If the model is then trained again using B, it's equivalent to causing the features learned by the image supplementation layer to regress from a superior state to a inferior state. This not only affects the superiority of the features learned by the model but also distorts the model's learning direction, hindering model convergence. To avoid the above situation, this implementation proposes the following training method: If the first image packet in the current training pair of superior and inferior samples (hereinafter referred to as the current superior-inferior sample pair) is the same as the first image packet in another previously trained pair of superior and inferior samples (hereinafter referred to as the historical superior-inferior sample pair) (and the labeling categories are also the same), then the confidence score (hereinafter referred to as the current confidence score) of the second image in the current superior-inferior sample pair predicted by the trained original model to belong to the labeled category, and the confidence score (hereinafter referred to as the historical confidence score) of the second image in the historical superior-inferior sample pair predicted by the trained original model to belong to the labeled category are extracted, and the current confidence score and the historical confidence score are compared. For example, superior-inferior sample pair A includes image packet 1 (small) and image packet 2 (large), and superior-inferior sample pair B includes image packet 1 (small) and image packet 3 (large), and the labeling categories of A and B are the same. The model has now been trained using A, which is able to convert the depth features of image pack 1 into the depth features of image pack 2. Now we encounter B. If we continue to train the model using B, the depth features of image pack 1 will be converted into the depth features of image pack 3. At this point, we can compare which image pack 2 or image pack 3 is better, instead of blindly using B for training.

[0045] If the current confidence level is higher than the historical confidence level, then the parameters of the feature supplementation layer are trained using the current superior and inferior samples. That is, if image pack 3 is better than image pack 2, then training continues using B, causing the existing model to evolve towards better features.

[0046] If the current confidence level is lower than the historical confidence level, then training using the current superior and inferior sample pairs is discarded. That is, if image pack 3 is not as good as image 2, then B is discarded, and the model parameters trained by A are retained to avoid the existing model regressing to a worse level.

[0047] Through training using the methods described above, the final image augmentation layer can transform deep features into superior features. These superior features include information about vehicles with missing cameras, mined from existing camera images, which can help improve the classification accuracy of image packets to some extent. The trained image augmentation layer, together with the original model (which may have been fine-tuned), constitutes the final vehicle re-recognition model.

[0048] By using a feature supplementation layer, this embodiment can supplement the features of missing images with information from existing images when certain shooting devices malfunction, thus avoiding the inability or decreased accuracy of vehicle re-identification methods based on faulty devices and ensuring the accuracy of vehicle re-identification.

[0049] In summary, this embodiment provides a vehicle re-identification model construction method based on multi-image learning. It utilizes multiple images to construct image packets and replaces single-image classification prediction with image packet-based classification prediction, reducing recognition errors caused by single-image quality deviations or annotation biases, and improving the stability and accuracy of vehicle re-identification. Specifically, during image packet training, both image confidence and image packet confidence are constrained to ensure high accuracy of intermediate image features and the final image packet prediction results. Only constraints Preserve as much of the original image information as possible; The confidence level for each category is constrained to further improve classification accuracy. Furthermore, in cases of camera malfunction, this embodiment uses a feature augmentation layer to transform the features of missing image packets towards a better direction, compensating for feature deficiencies and enabling smooth vehicle re-identification while maintaining accuracy and stability.

[0050] Figure 4 This is a flowchart of a vehicle re-identification method based on multi-image learning provided by an embodiment of the present invention. For example... Figure 4 As shown, the method includes: S210: Acquire an image package consisting of multiple vehicle images taken at the same spatiotemporal location.

[0051] S220. Input the image package into the vehicle re-identification model obtained by the construction method described in any of the above embodiments to obtain the confidence level of the image package belonging to each vehicle category.

[0052] Optionally, the vehicle re-identification model here can be Figure 2 The vehicle re-identification model shown can also be used for Figure 4 The final vehicle re-identification model is shown. During the model usage phase, the image classification layer can be omitted; only the classification results of the image packets are retained.

[0053] When using Figure 4 The final vehicle re-identification model shown can be trained by using image packets composed of vehicle images captured by multiple fixed cameras on the same road segment at the same spatiotemporal location. During model usage, if some cameras on the road segment malfunction, images from the remaining cameras and feature augmentation layers can be used to achieve a recognition effect similar to that of all cameras, ensuring that vehicle re-identification can still proceed smoothly while maintaining accuracy and stability.

[0054] S230. The vehicle category with the highest confidence level is taken as the vehicle category at the same spatiotemporal location.

[0055] This embodiment is based on the same inventive concept as any of the above-described model building methods. Any limitation in the above-described model building methods is applicable to this embodiment and can achieve the same technical effect as the above-described model building methods after being applied to this embodiment.

[0056] It should be noted that all data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0057] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5 As shown, the device includes a processor 60, a memory 61, an input device 62, and an output device 63; the number of processors 60 in the device can be one or more. Figure 5 Taking a processor 60 as an example; the processor 60, memory 61, input device 62, and output device 63 in the device can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0058] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the vehicle re-identification method based on multi-image learning in this embodiment of the invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, thereby implementing the aforementioned vehicle re-identification model construction method based on multi-image learning, or the vehicle re-identification method based on multi-image learning.

[0059] The memory 61 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 61 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include memory remotely located relative to the processor 60, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0060] Input device 62 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 63 may include display devices such as a display screen.

[0061] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle re-identification method based on multi-image learning of any embodiment.

[0062] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0063] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0064] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0065] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a vehicle re-identification model based on multi-image learning, characterized in that, include: Acquire a set of vehicle images, each image consisting of a single vehicle labeled with its vehicle category; Based on the spatiotemporal information of each vehicle image, multiple vehicle images taken at the same spatiotemporal location are combined into an image package, and the vehicle category with the most annotations in the image package is taken as the annotation category of the image package. The deep learning-based vehicle re-identification model is trained using various image packets. The training process includes: The feature extraction layer is used to extract the depth features of each vehicle image in the same image package; Using an image classification layer, the confidence level of each vehicle image belonging to each vehicle category is predicted based on each depth feature; Using a feature fusion layer, the depth features of each vehicle image are fused according to each confidence level to obtain the depth features of the same image package; Using an image packet classification layer, the confidence level of the same image packet belonging to each vehicle category is predicted based on the depth features of the same image packet; The parameters of each layer are updated based on the prediction results and labeling categories of each vehicle image and the same image package.

2. The method according to claim 1, characterized in that, The acquisition of the vehicle image set includes: acquiring vehicle images captured by multiple cameras on the same road segment to form a vehicle image set; Accordingly, combining multiple vehicle images taken at the same time and space location into an image package includes: combining multiple vehicle images taken by each camera at the same time and space location into an image package.

3. The method according to claim 1, characterized in that, The process of fusing the depth features of each vehicle image based on each confidence level to obtain the depth features of the same image packet includes: The features of the same image package are generated according to the following formula. : in, This indicates the vehicle image index within the image packet. This indicates the number of vehicle images in the image packet. Represents vehicle image depth features, Represents vehicle image Vehicle image The confidence level of the labeled vehicle category; Index representing vehicle category, Indicates the number of vehicle categories. Represents vehicle image Belongs to vehicle category The confidence level.

4. The method according to claim 1, characterized in that, The step of updating the parameters of each layer based on the prediction results and annotation categories of each vehicle image and the same image package includes: Based on the labeling category of the same image packet, determine whether the same image packet is a positive packet or a negative packet for each vehicle category; Based on the following image-level loss Image packet level loss Update the parameters of the vehicle re-identification model: in, This indicates the confidence level that each image belongs to the labeled vehicle category. This indicates that the image package belongs to the vehicle category. The confidence level.

5. The method according to claim 2, characterized in that, Also includes: From a standard image package containing vehicle images from the multiple cameras, vehicle images from at least one camera are masked to obtain multiple missing image packages, and each missing image package is assigned the same label category as the original image package; From the union of the standard image package and the multiple missing image packages, two image packages are extracted to form a superior and inferior sample pair. The first image package with fewer vehicle images is a subset of the second image package with more vehicle images. The confidence of the first image package in the labeled category predicted by the trained vehicle re-identification model is less than or equal to that of the second image package. The parameters of the feature supplementation layer are trained using each superior and inferior sample. The trained feature supplementation layer takes the depth features of the first image in the superior and inferior sample pair contained in the trained vehicle re-identification model as input and takes the depth features of the second image in the same superior and inferior sample pair contained in the trained vehicle re-identification model as output. The trained feature augmentation layer is added between the feature fusion layer and the image packet classification layer in the trained vehicle re-identification model to obtain the final vehicle re-identification model.

6. The method according to claim 5, characterized in that, The step of training the parameters of the feature enhancement layer using various superior and inferior samples includes: If the first image packet in the current superior-inferior sample pair is the same as the first image packet in the trained historical superior-inferior sample pair, extract the current confidence level of the second image in the current superior-inferior sample pair, which is predicted by the trained vehicle re-identification model to belong to the labeled category, and the historical confidence level of the second image in the historical superior-inferior sample pair, which is predicted by the trained vehicle re-identification model to belong to the labeled category. If the current confidence level is higher than the historical confidence level, the parameters of the feature supplementation layer are trained using the current superior and inferior samples; If the current confidence level is lower than or equal to the historical confidence level, training using the current superior and inferior sample pairs is discarded.

7. A vehicle re-identification method based on multi-image learning, characterized in that, include: Acquire an image packet consisting of multiple vehicle images taken at the same spatiotemporal location; The image packet is input into the vehicle re-identification model obtained by the construction method according to any one of claims 1-6 to obtain the confidence level of the image packet belonging to each vehicle category; The vehicle category with the highest confidence level is taken as the vehicle category at the same spatiotemporal location.

8. The vehicle re-identification method according to claim 7, characterized in that, The acquisition of an image package consisting of multiple vehicle images taken at the same spatiotemporal location includes: acquiring an image package consisting of multiple vehicle images taken at the same spatiotemporal location by some cameras on the same road segment. Accordingly, inputting the image package into the vehicle re-identification model obtained by the construction method according to any one of claims 1-6 includes: inputting the image package into the final vehicle re-identification model obtained by the construction method according to claim 5 or 6, wherein the standard image package used by the final vehicle re-identification model during training is composed of vehicle images taken by all cameras on the same road segment at the same spatiotemporal location.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the vehicle re-identification model construction method based on multi-image learning as described in any one of claims 1-6, or the vehicle re-identification method based on multi-image learning as described in claim 7 or 8.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the vehicle re-identification model construction method based on multi-image learning as described in any one of claims 1-6, or the vehicle re-identification method based on multi-image learning as described in claim 7 or 8.

Images