Unsupervised vehicle re-identification method and device based on hierarchical matching

Abstract

The present invention discloses an unsupervised vehicle re-identification method and device based on hierarchical matching, the method comprising: obtaining a vehicle image to be detected from a video source; extracting vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the vehicle image to be detected; performing a first process on the vehicle overall features to obtain vehicle overall similarity; performing a second process on the vehicle local component information to obtain a vehicle local feature distance value; performing a third process on the vehicle body color information and vehicle model information to obtain a color feature vector and a vehicle model feature vector; performing feature fusion on the vehicle overall similarity, vehicle local feature distance value, color feature vector and vehicle model feature vector to obtain vehicle similarity; and determining a target vehicle corresponding to the vehicle to be detected from all vehicles in a candidate vehicle set according to the vehicle similarity. The present invention improves the accuracy of vehicle re-identification and can be widely used in the field of vehicle identification technology.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle identification, and in particular to an unsupervised vehicle re-identification method and device based on hierarchical matching. Background Art

[0002] Vehicle re-identification refers to the problem of determining whether vehicles in images taken by cameras installed in different areas belong to the same vehicle in a specific traffic monitoring scenario. Vehicle re-identification algorithms can be applied to many difficult problems such as vehicle identification, tracking and positioning, further improving the security and reliability of the monitoring system.

[0003] Due to the constraints of complex traffic scenes, limited camera viewpoints, and changes in illumination, vehicle re-identification remains a very challenging problem. At present, license plate recognition technology is the simplest and most direct way to determine the identity of a vehicle, but relying solely on this method still has obvious drawbacks. For example, due to the limitations of the camera position, the shooting angle of traffic monitoring is relatively fixed. In complex scenes such as congested queues, large vehicles blocking, or drivers deliberately blocking forged license plates, the difficulty of license plate detection increases or is no longer applicable. In order to make up for the shortcomings of license plate recognition, more and more researchers have begun to use visual information other than license plates to perform vehicle retrieval tasks. In past studies in the field of vision, vehicle retrieval mainly involves the retrieval and recognition of content information such as vehicle color or model, but this often fails to accurately retrieve a specific vehicle, but instead obtains vehicles with the same content (such as color, model) as the target vehicle. In recent years, the research on vehicle re-identification, a subfield of vehicle retrieval, has gradually become a research focus. The shortcomings of existing vehicle re-identification technologies are mainly reflected in two aspects: first, the features presented by the same vehicle under different camera perspectives are quite different, resulting in large intra-class differences in the data of vehicle re-identification tasks; second, the differences between different vehicles with similar global features (such as color, model, etc.) under multiple perspectives are further reduced, resulting in smaller inter-class differences in vehicle re-identification tasks. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a high-accuracy unsupervised vehicle re-identification method and device based on hierarchical matching.

[0005] A first aspect of the present invention provides an unsupervised vehicle re-identification method based on hierarchical matching, comprising:

[0006] Get the image of the vehicle to be detected from the video source;

[0007] Extracting vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected;

[0008] Performing a first processing on the overall features of the vehicle to obtain an overall similarity of the vehicle;

[0009] Performing a second process on the vehicle local component information to obtain a vehicle local feature distance value;

[0010] Performing a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector;

[0011] Performing feature fusion on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain vehicle similarity;

[0012] According to the vehicle similarity, a target vehicle corresponding to the vehicle to be detected is determined from all vehicles in the candidate vehicle set.

[0013] Optionally, obtaining a picture of a vehicle to be detected from a video source includes:

[0014] Acquiring image frames from the video source at preset time intervals;

[0015] Performing target detection on the extracted image frame to find the vehicle target in the image frame;

[0016] Image frames with vehicle targets are captured to obtain a vehicle set, and the vehicle set is divided into a search picture set and a candidate picture set.

[0017] Optionally, the step of acquiring the image of the vehicle to be detected from the video source further includes:

[0018] Calculate the intersection-over-union ratio between the detection frame of any current image frame and the detection frame of the corresponding previous image frame;

[0019] When the intersection-over-union ratio is greater than a preset threshold, the detection result corresponding to the current image frame is eliminated;

[0020] The calculation formula of the intersection-over-intersection ratio is:

[0021]

[0022] Among them, IoU stands for intersection over union; R1 is the detection box of the previous image frame; R2 is the detection box of the current image frame; area(R1)∩area(R2) finds the area intersection between the two detection boxes; area(R1)∪area(R2) finds the area union of the two detection boxes.

[0023] Optionally, extracting vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected includes:

[0024] The vehicle is divided into overall vehicle features and local vehicle component information through the ResNet50 feature extractor;

[0025] The image of the vehicle to be detected is processed by global average pooling to obtain the overall characteristics of the vehicle;

[0026] Output the overall features of the vehicle through a CNN network;

[0027] Compare the features of each view of the vehicle to obtain the view visibility score of each view of the vehicle;

[0028] After calculating the weight of the view visibility score, the local feature distance value is calculated in combination with the Euclidean distance calculation method, and the local component information of the vehicle is determined;

[0029] The color feature vectors and vehicle model feature vectors of all vehicles are obtained by feature extraction through the Resnet50 network, and the vehicle body color information and the vehicle model information are determined.

[0030] Optionally, the extracting of vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected further includes:

[0031] The vehicle in the to-be-detected vehicle image is divided into four directions: front, top, back and side;

[0032] The area values ​​in four directions of the vehicle are calculated as the perspective visibility scores, and the perspective visibility scores are used as the confidence values ​​of the local features corresponding to the vehicle. The confidence values ​​are the confidence values ​​of the four local features of the vehicle, which are used to characterize the weight of the feature distance of each component between vehicles in the subsequent calculation of the local feature distance.

[0033] Optionally, the calculation formula for calculating the weight of the viewing angle visibility score is:

[0034]

[0035] in, A weight representing the view visibility score; Represents the visibility score of the vehicle to be detected; Represents the visibility score of the corresponding vehicle in the image library; m, n represent the vehicle to be detected and the corresponding vehicle in the image library; i represents the area corresponding to the visibility score and weight; N represents the number of vehicles in the image library to be searched;

[0036] The calculation formula of the local feature distance value is:

[0037]

[0038] Among them, D m,nrepresents the local feature distance value; D represents the Euclidean distance of the vehicle features in all directions; f i m Represents the characteristic information of the vehicle to be detected; f i n Represents the feature information of the corresponding vehicle in the image library.

[0039] Optionally, the performing feature fusion on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain the vehicle similarity includes:

[0040] Performing L2 norm normalization processing on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector, and the vehicle model feature vector to obtain normalized features;

[0041] Calculating an initial re-identification distance value between the vehicle to be detected and the corresponding vehicle in the image library according to the normalized features;

[0042] Performing weighted sum calculation on the initial re-identification distance value of the vehicle overall similarity, the initial re-identification distance value of the vehicle local feature distance value, the initial re-identification distance value of the color feature vector, and the initial re-identification distance value of the vehicle model feature vector to obtain a total re-identification distance value;

[0043] The vehicle similarity is determined according to the total weight recognition distance value.

[0044] Another aspect of the embodiment of the present invention further provides an unsupervised vehicle re-identification device based on hierarchical matching, comprising:

[0045] The first module is used to obtain the image of the vehicle to be detected from the video source;

[0046] The second module is used to extract the overall characteristics of the vehicle, the information of the local components of the vehicle, the body color information and the model information from the image of the vehicle to be detected;

[0047] The third module is used to perform a first processing on the overall features of the vehicle to obtain the overall similarity of the vehicle;

[0048] A fourth module is used to perform a second processing on the vehicle local component information to obtain a vehicle local feature distance value;

[0049] A fifth module is used to perform a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector;

[0050] A sixth module is used to perform feature fusion on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain vehicle similarity;

[0051] The seventh module is used to determine a target vehicle corresponding to the vehicle to be detected from all vehicles in the candidate vehicle set according to the vehicle similarity.

[0052] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0053] The memory is used to store programs;

[0054] The processor executes the program to implement the method described above.

[0055] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0056] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0057] The embodiment of the present invention obtains a picture of a vehicle to be detected from a video source; extracts the overall features of the vehicle, the local component information of the vehicle, the body color information and the vehicle model information from the picture of the vehicle to be detected; performs a first process on the overall features of the vehicle to obtain the overall similarity of the vehicle; performs a second process on the local component information of the vehicle to obtain the local feature distance value of the vehicle; performs a third process on the body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector; performs feature fusion on the overall similarity of the vehicle, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain vehicle similarity; and determines a target vehicle corresponding to the vehicle to be detected from all vehicles in a candidate vehicle set according to the vehicle similarity. The present invention improves the accuracy of vehicle re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 The present invention provides an overall step flow chart of the embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] In view of the problems existing in the prior art, the embodiments of the present invention provide an unsupervised vehicle re-identification method based on hierarchical matching, such as Figure 1 As shown, the method of the present invention comprises:

[0062] Get the image of the vehicle to be detected from the video source;

[0063] Extracting vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected;

[0064] Performing a first processing on the overall features of the vehicle to obtain an overall similarity of the vehicle;

[0065] Performing a second process on the vehicle local component information to obtain a vehicle local feature distance value;

[0066] Performing a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector;

[0067] Performing feature fusion on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain vehicle similarity;

[0068] According to the vehicle similarity, a target vehicle corresponding to the vehicle to be detected is determined from all vehicles in the candidate vehicle set.

[0069] Optionally, obtaining a picture of a vehicle to be detected from a video source includes:

[0070] Acquiring image frames from the video source at preset time intervals;

[0071] Performing target detection on the extracted image frame to find the vehicle target in the image frame;

[0072] Image frames with vehicle targets are captured to obtain a vehicle set, and the vehicle set is divided into a search picture set and a candidate picture set.

[0073] Optionally, the step of acquiring the image of the vehicle to be detected from the video source further includes:

[0074] Calculate the intersection-over-union ratio between the detection frame of any current image frame and the detection frame of the corresponding previous image frame;

[0075] When the intersection-over-union ratio is greater than a preset threshold, the detection result corresponding to the current image frame is eliminated;

[0076] The calculation formula of the intersection-over-intersection ratio is:

[0077]

[0078] Among them, IoU stands for intersection over union; R1 is the detection box of the previous image frame; R2 is the detection box of the current image frame; area(R1)∩area(R2) finds the area intersection between the two detection boxes; area(R1)∪area(R2) finds the area union of the two detection boxes.

[0079] Optionally, extracting vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected includes:

[0080] The vehicle is divided into overall vehicle features and local vehicle component information through the ResNet50 feature extractor;

[0081] The image of the vehicle to be detected is processed by global average pooling to obtain the overall characteristics of the vehicle;

[0082] Output the overall features of the vehicle through a CNN network;

[0083] Compare the features of each view of the vehicle to obtain the view visibility score of each view of the vehicle;

[0084] After calculating the weight of the view visibility score, the local feature distance value is calculated in combination with the Euclidean distance calculation method, and the local component information of the vehicle is determined;

[0085] The color feature vectors and vehicle model feature vectors of all vehicles are obtained by feature extraction through the Resnet50 network, and the vehicle body color information and the vehicle model information are determined.

[0086] Optionally, the extracting of vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected further includes:

[0087] The vehicle in the to-be-detected vehicle image is divided into four directions: front, top, back and side;

[0088] The area values ​​in four directions of the vehicle are calculated as the perspective visibility scores, and the perspective visibility scores are used as the confidence values ​​of the local features corresponding to the vehicle. The confidence values ​​are the confidence values ​​of the four local features of the vehicle, which are used to characterize the weight of the feature distance of each component between vehicles in the subsequent calculation of the local feature distance.

[0089] Optionally, the calculation formula for calculating the weight of the viewing angle visibility score is:

[0090]

[0091] in, A weight representing the view visibility score; Represents the visibility score of the vehicle to be detected; Represents the visibility score of the corresponding vehicle in the image library; m, n represent the vehicle to be detected and the corresponding vehicle in the image library; i represents the area corresponding to the visibility score and weight; N represents the number of vehicles in the image library to be searched;

[0092] The calculation formula of the local feature distance value is:

[0093]

[0094] Among them, D m,n represents the local feature distance value; D represents the Euclidean distance of the vehicle features in all directions; f i m Represents the characteristic information of the vehicle to be detected; f i n Represents the feature information of the corresponding vehicle in the image library.

[0095] Optionally, the performing feature fusion on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain the vehicle similarity includes:

[0096] Performing L2 norm normalization processing on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector, and the vehicle model feature vector to obtain normalized features;

[0097] Calculating an initial re-identification distance value between the vehicle to be detected and the corresponding vehicle in the image library according to the normalized features;

[0098] Performing weighted sum calculation on the initial re-identification distance value of the vehicle overall similarity, the initial re-identification distance value of the vehicle local feature distance value, the initial re-identification distance value of the color feature vector, and the initial re-identification distance value of the vehicle model feature vector to obtain a total re-identification distance value;

[0099] The vehicle similarity is determined according to the total weight recognition distance value.

[0100] Another aspect of the embodiment of the present invention further provides an unsupervised vehicle re-identification device based on hierarchical matching, comprising:

[0101] The first module is used to obtain the image of the vehicle to be detected from the video source;

[0102] The second module is used to extract the overall characteristics of the vehicle, the information of the local components of the vehicle, the body color information and the model information from the image of the vehicle to be detected;

[0103] The third module is used to perform a first processing on the overall features of the vehicle to obtain the overall similarity of the vehicle;

[0104] A fourth module is used to perform a second processing on the vehicle local component information to obtain a vehicle local feature distance value;

[0105] A fifth module is used to perform a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector;

[0106] A sixth module is used to perform feature fusion on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain vehicle similarity;

[0107] The seventh module is used to determine a target vehicle corresponding to the vehicle to be detected from all vehicles in the candidate vehicle set according to the vehicle similarity.

[0108] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0109] The memory is used to store programs;

[0110] The processor executes the program to implement the method described above.

[0111] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0112] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0113] The specific implementation principle of the present invention is described in detail below in conjunction with the accompanying drawings:

[0114] In view of the problems existing in the prior art, the present invention provides a vehicle re-identification method based on multi-model fusion, including a complete process of detecting, extracting and comparing vehicles from videos. First, the vehicle is detected, extracted and saved from the video by the YOLO detection algorithm. When the vehicle license plate is clear and can be identified, the license plate comparison will be used first, and the remaining vehicle images with low license plate clarity will undergo a subsequent re-identification process. The re-identification part divides the vehicle into four parts: overall vehicle features, local vehicle components, body color, and vehicle model for comparison. After the overall vehicle features are input into the network, CNN is used to output the overall similarity; the local vehicle component features are compared with vehicle features under different perspectives by introducing the perspective visibility score. The perspective visibility score divides the vehicle into four directions: front, back, side, and top. The area of ​​the vehicle in these four directions is counted and used as a weight to weight the Euclidean distance value of each component to obtain the local feature distance value of the vehicle. The body color and vehicle model are classified by the ResNet50 network to obtain the color feature vector and vehicle model feature vector S of the vehicle. Finally, the final vehicle similarity is obtained by fusing the above four vehicle features. The goal is to improve the accuracy of vehicle re-identification by integrating global features with features after vehicle analysis, expanding the inter-class differences of data and reducing the intra-class differences. Finally, the similarity between the target vehicle and all vehicles in the candidate vehicle set is sorted, and the top 5 (or 10) vehicles with the highest similarity are selected as output using the camera space and vehicle appearance time information.

[0115] The present invention realizes a vehicle re-identification method based on multi-feature fusion, which detects and extracts vehicles in videos, classifies vehicles by color and model, compares vehicle component features from different viewing angles, and compares vehicle appearance positions. Then, the weighted feature values ​​are used to sort the similarity values ​​of the entire vehicle, and the vehicle with the highest similarity is selected as the output vehicle.

[0116] Vehicle detection and license plate recognition part:

[0117] The vehicle detection part first takes frames of the given video at a certain interval, and performs target detection on the frame image to determine whether there are vehicles (including cars, buses and trucks) in the scene. The detection result is related to the set frame interval, video frame rate and the speed of the moving object. The target detection adopts the YOLO v4 algorithm, which converts target detection into a regression problem. Classification and bounding box regression are performed simultaneously in the model. The input image is extracted through the backbone network (Backbone) CSPDarknet-53, neck (Neck) pyramid pooling (SPP) and path aggregation network (PAN) and head (Head) YOLO v3 to extract effective features, and then mapped to a tensor to obtain the target detection result. The method inputs the video frame image and then detects whether there is a vehicle target in the image.

[0118] For the problem of repeated detection of stationary vehicles or low-speed vehicles that may exist in the input video, the intersection over union (IoU) of the current detection frame and the previous detection frame is calculated, and the detection results with IoU greater than 0.5 are eliminated to improve the problem of repeated detection.

[0119]

[0120] Among them, R1 is the detection box of the previous video frame, and R2 is the detection box of the current video frame; area(R1)∩area(R2) calculates the area intersection of the two detection boxes, and area(R1)∪area(R2) calculates the area union of the two detection boxes.

[0121] After completing vehicle detection, the vehicle detection part in the frame image is captured and saved as a vehicle set, which can be subsequently divided into a query set and a gallery set according to user needs for vehicle re-identification.

[0122] The vehicle set obtained by vehicle detection is not only used directly for vehicle re-identification, but also for license plate recognition as auxiliary reference information for vehicle re-identification. For the recognition of license plates, the license plate recognition network (License Plate Recognition Net, LPRNet) is used. The vehicle image is input and the license plate part is recognized by the CNN model built by the SqueezeNet Fire module and the Inception module. The license plate recognition results are then saved as a csv result file for license plate information comparison during the re-identification process.

[0123] In the vehicle re-identification comparison part: first, the vehicle is divided into three main parts for prediction. In the vehicle component feature comparison part, the view classification network is used to divide the vehicle into four directions: front, top, back, and side. The area values ​​of the vehicle in these four directions are counted as the view visibility score (vis_scores), which is regarded as the confidence value of the corresponding local features of the vehicle.

[0124] First, the ResNet50 feature extractor is used to divide the vehicle into two parts: global features and vehicle component (local) features. The global features of the vehicle are obtained by using global average pooling for feature maps, and the global features are output through the CNN network. This global feature can bring higher detection accuracy when matching vehicles with the same viewpoint.

[0125] For the feature comparison of each vehicle perspective, after obtaining the view visibility scores of each vehicle perspective, the vehicle feature information of different perspectives can be decoupled into corresponding local features to achieve feature comparison under each perspective. Since the visibility score is the area at this time, it needs to be weighted:

[0126]

[0127] is the weight, and is the visibility score of the two vehicles, i is the area corresponding to the visibility score and weight, m, n are the corresponding vehicles to be searched and the corresponding comparison vehicles in the image library. Then the Euclidean distance D of the vehicle features in all directions is calculated and the visibility score weight is multiplied by it to obtain the final local feature distance value:

[0128]

[0129] f i m and f i n For the corresponding features of the two vehicles for comparison, is the weight obtained by the above calculation.

[0130] The above two parts calculate the similarity of the overall and local features of the vehicle. The following further improves the accuracy of re-identification by classifying the vehicle model and color features. The model and color are extracted through the Resnet50 network to obtain the color feature vector and model feature vector S of all vehicles.

[0131] At this point, this embodiment has obtained four vehicle feature values, namely, the vehicle global feature vector, the vehicle local feature vector, the vehicle model feature vector, and the body color feature vector. Each feature vector is normalized by the L2 norm and then the distance between the query vehicle and the gallery vehicle is calculated. The calculation method of the local distance and the global distance is as shown in the above formulas (1) and (2). The color feature distance and the vehicle model feature distance are obtained by calculating the Euclidean distance between the features. Finally, they are weighted by linear combination to obtain the overall vehicle re-identification distance value D.

[0132]

[0133] Among them, F′ i represents the normalized features, F i is the feature vector output by the network.

[0134]

[0135] Among them, F m ,Fn Represents the feature vectors of different vehicles, and N represents the dimension of the feature vector.

[0136] D=mD global +βD local +γD color +λD type #(5)

[0137] In summary, the present invention first uses vehicle recognition technology to select vehicles in photos or video streams, and then predicts the selected vehicle photos in three parts. In the vehicle component features, the perspective classification network is used to score the four perspectives of the vehicle as the confidence value of the corresponding local features of the vehicle; then the vehicle is divided into two parts, global features and local features, by the feature extractor, and the global features of the vehicle are obtained by global average pooling for the feature map; finally, the vehicle model and color features are used for identification. Finally, the feature distance of these four features is calculated, and the linear combination is weighted to obtain the final total distance value, and then the highest score is output as the result of vehicle re-identification, realizing a complete re-identification process from video to vehicle comparison, which has high practicality and feasibility, and the final detection accuracy is high.

[0138] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0139] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0140] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0141] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0142] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0143] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0144] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0145] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0146] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. Unsupervised vehicle re-identification method based on hierarchical matching, characterized in that: include: Get the image of the vehicle to be detected from the video source; Extracting vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected; Performing a first processing on the overall features of the vehicle to obtain an overall similarity of the vehicle; Performing a second process on the vehicle local component information to obtain a vehicle local feature distance value; Performing a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector; Performing feature fusion on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain vehicle similarity; According to the vehicle similarity, determining a target vehicle corresponding to the vehicle to be detected from all vehicles in the candidate vehicle set; The extracting of vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected includes: The vehicle is divided into overall vehicle features and local vehicle component information through the ResNet50 feature extractor; The image of the vehicle to be detected is processed by global average pooling to obtain the overall characteristics of the vehicle; Output the overall features of the vehicle through a CNN network; Compare the features of each view of the vehicle to obtain the view visibility score of each view of the vehicle; After calculating the weight of the view visibility score, the local feature distance value is calculated in combination with the Euclidean distance calculation method, and the local component information of the vehicle is determined; The color feature vectors and vehicle model feature vectors of all vehicles are obtained by feature extraction through the Resnet50 network, and the vehicle body color information and vehicle model information are determined; The extracting of vehicle overall features, vehicle local component information, vehicle body color information and vehicle model information from the image of the vehicle to be detected also includes: The vehicle in the to-be-detected vehicle image is divided into four directions: front, top, back and side; Calculate the area values ​​of the vehicle in four directions as the visual angle visibility score, and use the visual angle visibility score as the confidence value of the local feature corresponding to the vehicle. The confidence value is the confidence value of the four local features of the vehicle, which is used to characterize the weight of the feature distance of each component between vehicles in the subsequent calculation of the local feature distance; The calculation formula for calculating the weight of the viewing angle visibility score is: in, A weight representing the view visibility score; Represents the visibility score of the vehicle to be detected; Represents the visibility score of the corresponding vehicle in the image library; m, n represent the vehicle to be detected and the corresponding vehicle in the image library; i represents the area corresponding to the visibility score and weight; N represents the number of vehicles in the image library to be searched; The calculation formula of the local feature distance value is: Among them, D m,n represents the local feature distance value; D represents the Euclidean distance of the vehicle features in all directions; f i m Represents the characteristic information of the vehicle to be detected; f i n Represents the feature information of the corresponding vehicle in the image library.

2. The unsupervised vehicle re-identification method based on hierarchical matching according to claim 1 is characterized in that: The step of obtaining a picture of a vehicle to be detected from a video source includes: Acquiring image frames from the video source at preset time intervals; Performing target detection on the extracted image frame to find the vehicle target in the image frame; Image frames with vehicle targets are captured to obtain a vehicle set, and the vehicle set is divided into a search picture set and a candidate picture set.

3. The unsupervised vehicle re-identification method based on hierarchical matching according to claim 2 is characterized in that: The step of obtaining the image of the vehicle to be detected from the video source also includes: Calculate the intersection-over-union ratio between the detection frame of any current image frame and the detection frame of the corresponding previous image frame; When the intersection-over-union ratio is greater than a preset threshold, the detection result corresponding to the current image frame is eliminated; The calculation formula of the intersection-over-intersection ratio is: Among them, IoU stands for intersection over union; R1 is the detection box of the previous image frame; R2 is the detection box of the current image frame; area(R1)∩area(R2) finds the area intersection between the two detection boxes; area(R1)∪area(R2) finds the area union of the two detection boxes.

4. The unsupervised vehicle re-identification method based on hierarchical matching according to claim 1, characterized in that: The step of fusing the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector, and the vehicle model feature vector to obtain the vehicle similarity includes: Performing L2 norm normalization processing on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector, and the vehicle model feature vector to obtain normalized features; Calculating an initial re-identification distance value between the vehicle to be detected and the corresponding vehicle in the image library according to the normalized features; Performing weighted sum calculation on the initial re-identification distance value of the vehicle overall similarity, the initial re-identification distance value of the vehicle local feature distance value, the initial re-identification distance value of the color feature vector, and the initial re-identification distance value of the vehicle model feature vector to obtain a total re-identification distance value; The vehicle similarity is determined according to the total weight recognition distance value.

5. A device for implementing the unsupervised vehicle re-identification method based on hierarchical matching as described in any one of claims 1 to 4, characterized in that: include: The first module is used to obtain the image of the vehicle to be detected from the video source; The second module is used to extract the overall characteristics of the vehicle, the information of the local components of the vehicle, the body color information and the model information from the image of the vehicle to be detected; The third module is used to perform a first processing on the overall features of the vehicle to obtain the overall similarity of the vehicle; A fourth module is used to perform a second processing on the vehicle local component information to obtain a vehicle local feature distance value; A fifth module is used to perform a third process on the vehicle body color information and the vehicle model information to obtain a color feature vector and a vehicle model feature vector; A sixth module is used to perform feature fusion on the overall vehicle similarity, the local feature distance value of the vehicle, the color feature vector and the vehicle model feature vector to obtain vehicle similarity; The seventh module is used to determine a target vehicle corresponding to the vehicle to be detected from all vehicles in the candidate vehicle set according to the vehicle similarity.

6. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 4.

Images