Vehicle Re-identification Method, System, Device and Medium

Abstract

A vehicle re-identification method, system, device and medium proposed by the present invention. The method obtains visible light images and infrared images of the vehicle to be identified, performs a first comparison on the visible light images to obtain candidate images and a first comparison result, then compares the infrared images with the candidate images to obtain a second comparison result, and combines the first comparison result and the second comparison result to determine the candidate rankings of the candidate images, so as to achieve the re-identification of the vehicle to be identified. By simultaneously using the multi-spectrum information of visible light images and infrared images and combining the characteristics of infrared images passing through glass, the feature information used in vehicle re-identification technology is enriched, the accuracy of vehicle re-identification is improved, and higher-precision vehicle re-identification can be completed with the assistance of vehicle internal information in the case of unable to identify the license plate, and it has strong environmental adaptability.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a vehicle re-identification method, system, device and medium. Background Art

[0002] Vehicle re-identification technology has been widely applied in fields such as traffic monitoring and public security, such as vehicle trajectory tracking and vehicle positioning. In related technologies, vehicle re-identification technology mainly relies on visible light imaging technology. Through deep learning technology, the features of the human body to be identified and the features of the target to be retrieved are obtained, and their similarity is matched to complete re-identification. If the imaging conditions are good, license plate recognition technology can also be added to complete accurate vehicle re-identification by comparing license plate numbers.

[0003] However, based on the actual application scenarios, the current vehicle re-identification technology has the following problems and difficulties: 1. The traffic monitoring environment is complex and changeable, and there are a large number of scenarios where license plates cannot be recognized, making it difficult to apply license plate recognition; 2. Vehicle appearances are relatively similar, especially for some popular vehicles of the same brand, model, and color, with a high similarity. It is difficult to distinguish them except by comparing license plates, which may cause mis-matching; 3. The front and rear features of vehicles are quite different, making the matching difficult. Due to the existence of the above problems, the current vehicle re-identification accuracy is low, and it is difficult to perform re-identification in difficult scenarios. Summary of the Invention

[0004] In view of the above-mentioned disadvantages of the prior art, the present invention provides a vehicle re-identification method, system, device and medium to solve the above technical problems.

[0005] The vehicle re-identification method provided by the present invention is characterized in that the method includes:

[0006] Obtain a vehicle image of the vehicle to be identified, where the vehicle image includes a visible light image and an infrared image;

[0007] Compare the visible light image with a preset image in a preset vehicle dataset to obtain one or more candidate images and the first comparison result of each candidate image;

[0008] Compare the infrared image with each candidate image to obtain the second comparison result of each candidate image;

[0009] Determine the candidate ranking of each candidate image according to the first comparison result and the second comparison result to achieve the re-identification of the vehicle to be identified.

[0010] Optionally, comparing the infrared image with each candidate image includes:

[0011] Detect the visible light image to obtain the vehicle position information of the vehicle to be identified;

[0012] Determine the target area position information from the infrared image according to the vehicle position information and the preset depth information, and determine the target image;

[0013] Detect the target image to obtain the internal features of the vehicle to be recognized of the vehicle to be recognized;

[0014] Compare the internal features of the vehicle to be recognized with the candidate vehicle features of the candidate images.

[0015] Optionally, the determination method of the target image includes any one of the following:

[0016] Use the image of the target area where the target area position information is located as the target image;

[0017] Expand the target area to obtain an expanded area, and use the image of the expanded area as the target image.

[0018] Optionally, before comparing the infrared image with each of the candidate images, the method further includes:

[0019] Obtain the transformation matrix between the visible light image and the infrared image;

[0020] According to the transformation matrix, represent the pixel position information of the visible light image and the pixel position information of the infrared image in a preset coordinate system.

[0021] Optionally, after determining the candidate rankings of each of the candidate images according to the first comparison result and the second comparison result, the method further includes:

[0022] Use the candidate images with candidate rankings higher than the preset ranking threshold as the re-identification result of the vehicle to be recognized.

[0023] Optionally, comparing the visible light image with the preset images in the preset image dataset to obtain one or more candidate images, and the first comparison results of each of the candidate images include respectively inputting the visible light image and each of the preset images into a preset vehicle detection model to obtain one or more candidate images and the first confidence levels of each of the candidate images;

[0024] And / or,

[0025] Comparing the infrared image with each of the candidate images to obtain the second comparison results of each of the candidate images includes respectively inputting the infrared image and each of the preset images into a preset target area detection model to obtain the second confidence levels of each of the candidate images.

[0026] Optionally, determining the candidate rankings of the candidate images according to the first comparison result and the second comparison result includes any one of the following:

[0027] Determine a first score for the candidate image according to the first confidence level, determine a second score for the candidate image according to the second confidence level, determine a comprehensive score for the candidate image according to the first score and the second score, and sort according to the comprehensive scores of the candidate images to obtain the candidate rankings;

[0028] Determine a first ranking for the candidate image according to the first confidence level, determine a second ranking for the candidate image according to the second confidence level, and determine the candidate ranking of the candidate image according to the first ranking and the second ranking.

[0029] The present invention also provides a vehicle re-identification system, and the system includes:

[0030] An acquisition module, configured to acquire a vehicle image of a vehicle to be identified, where the vehicle image includes a visible light image and an infrared image;

[0031] A first comparison module, configured to compare the visible light image with a preset image in a preset image dataset to obtain one or more candidate images and a first comparison result of each candidate image;

[0032] A second comparison module, configured to compare the infrared image with each candidate image to obtain a second comparison result of each candidate image;

[0033] A determination module, configured to determine the candidate rankings of the candidate images according to the first comparison result and the second comparison result, so as to realize the re-identification of the vehicle to be identified.

[0034] The present invention also provides an electronic device, including a processor, a memory, and a communication bus;

[0035] The communication bus is used to connect the processor and the memory;

[0036] The processor is configured to execute a computer program stored in the memory to implement the method described in any one of the above embodiments.

[0037] The present invention also provides a computer-readable storage medium, on which a computer program is stored,

[0038] The computer program is used to make a computer execute the method described in any one of the above embodiments.

[0039] Advantages of the present invention: A vehicle re-identification method, system, device and medium proposed by the present invention. The method obtains visible light images and infrared images of the vehicle to be identified, performs a first comparison on the visible light images to obtain candidate images and a first comparison result, then compares the infrared images with the candidate images to obtain a second comparison result, and combines the first comparison result and the second comparison result to determine the candidate rankings of the candidate images, so as to realize the re-identification of the vehicle to be identified. By simultaneously using multiple spectral information of visible light images and infrared images and combining the glass-penetrating characteristics of infrared images, the feature information used in vehicle re-identification technology is enriched, the vehicle re-identification accuracy is improved, and higher-precision vehicle re-identification can be completed with the assistance of vehicle internal information in the case of unrecognizable license plates, and it has strong environmental adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic flowchart of a vehicle re-identification method provided in an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of a visible light image taken by vehicle M under visible light conditions provided in an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of an infrared image taken by vehicle M using short-wave infrared imaging technology provided in an embodiment of the present invention;

[0043] Figure 4 is a schematic diagram of the imaging principle of a visible light-short wave binocular imaging device provided in an embodiment of the present invention;

[0044] Figure 5 is a schematic flowchart of a specific vehicle re-identification method provided in an embodiment of the present invention;

[0045] Figure 6 is a schematic diagram of a vehicle image and a preset image in a preset vehicle dataset provided in an embodiment of the present invention;

[0046] Figure 7 is a schematic flowchart of a specific vehicle re-identification method provided in an embodiment of the present invention;

[0047] Figure 8 is a schematic structural diagram of a vehicle re-identification system provided in an embodiment of the present invention;

[0048] Figure 9 is a schematic structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0050] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and proportion of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0051] In the following description, a large number of details are explored to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0052] See Figure 1 , this embodiment provides a vehicle re-identification method, which includes:

[0053] Step S101: Obtain a vehicle image of the vehicle to be identified. Among them, the vehicle image includes a visible light image and an infrared image.

[0054] Step S102: Compare the visible light image with a preset image in a preset vehicle dataset to obtain one or more candidate images and the first comparison result of each candidate image.

[0055] Step S103: Compare the infrared image with each candidate image to obtain the second comparison result of each candidate image.

[0056] Step S104: Determine the candidate ranking of each candidate image according to the first comparison result and the second comparison result to achieve the re-identification of the vehicle to be identified.

[0057] In the related art, vehicle re-identification often focuses on visible light images captured under visible light conditions. By identifying the visible light images, the external features of the vehicle can be obtained. However, when the light is poor, the vehicle models are the same, and the license plates cannot be accurately recognized, due to the complex and changeable traffic monitoring environment, there are a large number of scenarios where license plates cannot be recognized, and license plate recognition is difficult to apply; the vehicle appearances are relatively similar, especially for some popular vehicles of the same brand, the same model, and the same color, with a high similarity. Except for license plate comparison, it is difficult to distinguish them, which may cause mis-matching; the front and rear features of the vehicle are quite different, and the matching difficulty is relatively large. For these reasons, it is difficult to accurately identify the external features of the vehicle. When encountering the above difficult scenarios, it is very difficult to distinguish similar targets only by the external features of the vehicle. After analyzing the above solutions, the inventor found that the related technologies only use the external features of the vehicle for vehicle feature matching. When encountering the above difficult scenarios, it is very difficult to distinguish similar targets only by the external features of the vehicle. Therefore, the inventor thought of introducing short-wave infrared imaging technology to enhance the richness of vehicle features and improve the accuracy of distinguishing similar targets.

[0058] The wavelength range of the short-wave infrared imaging technology is generally between 900 - 1700 nm (generally extendable to 2200 nm). It can image under illumination far lower than that of visible light imaging devices. At the same time, short-wave infrared imaging can receive both the short-wave infrared radiation emitted by the target itself and the short-wave infrared radiation reflected by the object. Therefore, it has strong glass penetration ability. The short-wave infrared imaging technology provides a good technical basis for in-vehicle imaging and extraction of vehicle internal features.

[0059] See Figure 2 and Figure 3 , Figure 2 is the visible light image of vehicle M captured under visible light conditions, Figure 3 is the infrared image of vehicle M captured by using the short-wave infrared imaging technology. By observing Figure 2 and Figure 3 , it can be known that the visible light image captures the relevant features of the vehicle's external contour clearly, but for the internal situation of the vehicle, the imaging effect is poor. In the infrared image, the internal situation of the vehicle can be observed more clearly, supplementing the feature details of the vehicle interior.

[0060] In one embodiment, the visible light image and the infrared image in the vehicle image can be collected by a visible light - short-wave binocular imaging device or other devices known to those skilled in the art. See Figure 4 , Figure 4It is a schematic diagram of the imaging principle of a visible light-short wave binocular imaging device. Optionally, the visible light-short wave binocular imaging device can be composed of a visible light camera and a short wave infrared camera, and the imaging band range is 400-1700nm, where the visible light camera is 400-900nm, and the short wave infrared camera is 900-1700nm, which can be extended to 2200nm, providing multi-spectral images for vehicle re-identification at the back end. The two cameras are placed relatively fixed on the same horizontal plane. At the same time, it is necessary to use the conventional binocular camera calibration method to calibrate the internal and external parameters of the cameras using the spatial parallax principle to obtain the accurate spatial position information between the two cameras for subsequent use.

[0061] In one embodiment, the preset image in the preset vehicle dataset can be an image taken by other devices, or an image taken by the same shooting device as the vehicle image at other times. Vehicle target recognition can be pre-performed in the preset image to ensure that the preset image includes vehicle targets.

[0062] In one embodiment, the method of comparing the visible light image with the preset image in the preset vehicle dataset to obtain one or more candidate images and the first comparison result of each candidate image can be: respectively input the visible light image and each preset image into the trained preset vehicle detection model to obtain the similarity between each preset image and the visible light image, and use the preset image with a similarity higher than the preset first similarity threshold as the candidate image, and obtain the first confidence level of each candidate image. Among them, the preset vehicle detection model can adopt a CNN model, with CSPNet as the backbone and FPN as the feature fusion part, and the YOLOV 5 head to complete human target detection. By performing transfer learning on the user's specific data, a preset vehicle detection model that meets the requirements of the dataset is constructed. The preset vehicle detection model can also be trained in other ways known to those skilled in the art. The main function of the preset vehicle detection model is to complete the vehicle detection of the visible light image, so as to obtain candidate targets (vehicles to be identified). The preset vehicle detection model also includes a feature matching module, and the main function of the feature matching module is to complete the matching of multiple candidate targets (vehicles to be identified) with the target to be searched (the vehicle in the preset image) and complete the similarity determination. Specifically, the twin neural network algorithm (Siamese) can be used to achieve feature matching.

[0063] In one embodiment, comparing the infrared image with each candidate image to obtain the second comparison result of each candidate image includes respectively inputting the infrared image and each preset image into the preset target area detection model to obtain the second confidence level of each candidate image. Among them, the preset target area detection model is mainly used for feature recognition of the vehicle internal target photographed through the vehicle windshield. The model training method of the preset target area detection model can be implemented in a manner known to those skilled in the art.

[0064] In one embodiment, to accelerate the re-identification speed and efficiency, image detection can be first performed on each preset image to obtain the external vehicle features and internal vehicle features of the preset image, and the external vehicle features and internal vehicle features are labeled on the preset image. In this way, when obtaining the re-identification task of a new vehicle to be identified, only the vehicle image of the vehicle to be identified needs to be detected to obtain the external vehicle features and internal vehicle features of the vehicle to be identified, and then compared to determine the candidate image and the candidate ranking, thereby realizing the re-identification of the vehicle to be identified.

[0065] In one embodiment, before comparing the infrared image with each candidate image, the method further includes:

[0066] Obtaining a transformation matrix between the visible light image and the infrared image;

[0067] According to the transformation matrix, the pixel position information of the visible light image and the pixel position information of the infrared image are converted into a preset coordinate system for representation.

[0068] Optionally, the transformation matrix can be obtained based on the relevant parameters of the infrared image and visible light image acquisition devices to determine the transformation matrix, for example, the transformation matrix is determined by using binocular calibration.

[0069] Optionally, the transformation matrix can also be obtained by performing image recognition on the visible light image and the infrared image respectively, and then determining a calibration feature, which is a feature uniquely existing in both the visible light image and the infrared image. Based on the position information of the calibration feature in the two images, the transformation matrix is determined.

[0070] Through the transformation matrix, the feature points in the visible light image and the infrared image can be converted to the same coordinate system, realizing the stereo correction and row alignment of the images.

[0071] In one embodiment, comparing the infrared image with each candidate image includes:

[0072] Detecting the visible light image to obtain the vehicle position information of the vehicle to be identified;

[0073] Determining the target area position information from the infrared image according to the vehicle position information and the preset depth information, and determining the target image;

[0074] Detecting the target image to obtain the internal features of the vehicle to be identified of the vehicle to be identified. Specifically, detecting the target image includes identifying the target object (such as the windshield, etc.) in the target image, and then extracting features from the target object area where the target object is located to obtain the internal features of the vehicle to be identified of the vehicle to be identified;

[0075] Compare the internal features of the vehicle to be recognized with the candidate vehicle features of the candidate images.

[0076] Among them, the preset depth information can be set by those skilled in the art according to needs. Based on the binocular parallax principle, by setting the depth, the mapping of the vehicle position from the visible light image to the short-wave image is completed, and then the position information of the target area is determined. Specifically, the method of completing the mapping of the vehicle position from the visible light image to the short-wave image by setting the depth can be achieved by the methods known to those skilled in the art, and will not be elaborated here.

[0077] Optionally, the determination method of the target image includes any one of the following:

[0078] Use the image of the target area where the target area position information is located as the target image;

[0079] Expand the target area to obtain an expanded area, and use the image of the expanded area as the target image.

[0080] Since the preset depth information may be inaccurate, to ensure the accuracy of in-vehicle feature detection, the target area can be expanded, and the image of the expanded area can be used as the target image to detect the windshield and extract the in-vehicle features.

[0081] Based on the binocular parallax principle, the detection area of the windshield is the vehicle range, which has higher accuracy and efficiency than detecting the windshield on the original input image (infrared image).

[0082] Optionally, the vehicle position information can be the position information of the vehicle to be recognized in the visible light image. Based on this vehicle position information and the preset depth information, the area where the vehicle to be recognized is located in the infrared image can be obtained, and then the area where the vehicle to be recognized is located, or the expanded area of this area, can be used as the target area to detect the windshield, and then extract the features of the image of the area where the windshield is located.

[0083] In one embodiment, after determining the candidate rankings of the candidate images according to the first comparison result and the second comparison result, the method further includes:

[0084] Use the candidate images with candidate rankings higher than the preset ranking threshold as the re-identification results of the vehicle to be recognized.

[0085] Among them, the preset ranking threshold can be a fixed value, such as ranking first, or a non-fixed threshold. The preset ranking threshold is determined according to the number of rankings of the candidate rankings. For example, if there are 20 candidate rankings, the preset ranking threshold is 10, that is, the top ten candidate images in the candidate rankings are the re-identification results. When the candidate rankings of multiple candidate images are the same, if the candidate ranking of the candidate image is higher than the preset ranking threshold, each candidate image is used as the re-identification result.

[0086] In one embodiment, the first comparison result can be the first score or the first confidence level, and the second comparison result can be the second score or the second confidence level. Determining the candidate rankings of the candidate images according to the first comparison result and the second comparison result includes any one of the following:

[0087] Determine the first score of the candidate image according to the first confidence level, determine the second score of the candidate image according to the second confidence level, determine the comprehensive score of the candidate image according to the first score and the second score, and sort according to the comprehensive scores of the candidate images to obtain the candidate rankings;

[0088] Determine the first ranking of the candidate image according to the first confidence level, determine the second ranking of the candidate image according to the second confidence level, and determine the candidate ranking of the candidate image according to the first ranking and the second ranking.

[0089] Among them, the first score and the second score can be gradient scores. For example, if the confidence level is a - b, the score is 10, and if the confidence level is c - d, the score is 8, etc.

[0090] The vehicle re-identification method provided in this embodiment obtains the visible light image and the infrared image of the vehicle to be identified, performs the first comparison on the visible light image to obtain the candidate images and the first comparison result, then compares the infrared image with the candidate images to obtain the second comparison result, and combines the first comparison result and the second comparison result to determine the candidate rankings of the candidate images, so as to realize the re-identification of the vehicle to be identified. At the same time, it uses the multiple spectral information of the visible light image and the infrared image, combines the characteristics of the infrared image passing through the glass, enriches the feature information used in the vehicle re-identification technology, and improves the vehicle re-identification accuracy. It can complete higher-precision vehicle re-identification through the assistance of vehicle internal information in the case of unable to recognize the license plate; it can realize vehicle re-identification in different dimensions including single-spectrum and multi-spectrum, and has strong environmental adaptability. It can provide more channels of light information, provide more methods and rich image (spectral) information for the backend detection algorithm, improve the environmental adaptability of the backend algorithm, and enhance the algorithm detection ability.

[0091] To achieve accurate vehicle re-identification in situations where there are many vehicles, the environment is complex, and license plates cannot be recognized, this embodiment proposes a vehicle re-identification algorithm based on a deep neural network and multi-dimensional features inside and outside the vehicle. By combining multi-spectral features inside and outside the vehicle, vehicle re-identification in different scenarios and with different information dimensions is realized. Refer to Figure 5 , Figure 5 which is a schematic flowchart of a specific vehicle re-identification method. As Figure 5 shown, camera A captures an image or video of vehicle 1. Through vehicle detection, feature extraction, and expression, the internal and external features of vehicle 1 are obtained. Vehicle detection is performed on the preset images ( Figure 5 all vehicle images or videos in it) in the preset vehicle dataset, and feature extraction and expression are carried out to obtain the internal and external features of the vehicles in the preset images. A similarity metric calculation is performed on both, and candidate images are determined from all vehicle images or videos based on the similarity. All vehicle images or videos can be collected by cameras B, C, D, etc., or can also include other images or videos collected by camera A. The candidate images form the retrieval result, and the retrieval result can include images collected by cameras C, B, D, B of vehicle 1, that is, the retrieval result can include images collected by the same image acquisition device or images collected by different image acquisition devices. Refer to Figure 6 , Figure 6 which are examples of vehicle images and preset images in the preset vehicle dataset. As Figure 6 shown, vehicle 1 and vehicle 776 can be the query set, and images of vehicle 1 and vehicle 776 from other angles and images of other vehicles are used as the gallery set for vehicle re-identification.

[0092] Refer to Figure 7 , Figure 7 which is a schematic flowchart of another specific vehicle re-identification method. As Figure 7 shown, this method includes:

[0093] (1) Stereo rectification: By calibrating visible light images and short-wave infrared images (infrared images), the images input by the two devices (visible light images and short-wave infrared images) are transformed into the same coordinate system using the rotation and translation matrix (calibration information) between the cameras, achieving stereo rectification and row alignment of the images.

[0094] (2) Vehicle detection and candidate target acquisition: Based on visible light images and a deep convolutional neural network, vehicle detection in the target scene is completed, multiple candidate targets (candidate images) are obtained, and the positions (vehicle position information) are marked.

[0095] (3) Windshield Detection: Based on the binocular disparity principle, by setting the depth, the mapping of the candidate target position from the visible light image to the short-wave image is completed.

[0096] Optionally, the depth can be the empirical depth. When the depth is the empirical depth, accurate positioning cannot be completed. Therefore, when mapping, the target is expanded, and the vehicle windshield is detected in the expanded area based on the short-wave image and the depth convolutional neural network, and the internal features of the candidate target vehicle are extracted. At the same time, based on the binocular disparity principle, the detection area of the windshield is the vehicle range, which has higher accuracy and efficiency than detecting the windshield on the original input image.

[0097] (4) Feature Matching: Based on the visible light vehicle image, the short-wave in-vehicle image, and the similarity image determination neural network respectively, the matching between the candidate target and the object to be searched is completed, and the similarity confidence levels are output respectively.

[0098] (5) Fusion of Similarity Ranking and Judgment: Based on the candidate target id and confidence score obtained in step (4), all the obtained information is fused and scored to obtain the final candidate ranking. Vehicle re-identification is completed.

[0099] It can be seen from the above process that the vehicle re-identification method proposed in this embodiment can simultaneously obtain different spectral features inside and outside the vehicle, enrich the information used for feature matching, and improve the matching accuracy.

[0100] Optionally, the detection method in this embodiment can be implemented by using an object detection algorithm. The object detection algorithm module mainly includes vehicle detection and windshield detection. Among them, the main function of vehicle detection is to complete the vehicle detection of the target scene (detecting the visible light image) to obtain the candidate target. There are relatively mature solutions for image vehicle detection algorithms, and those skilled in the art can select according to needs. For example, a conventional CNN model can be used, with CSPNet as the backbone, FPN as the feature fusion part, and YOLOV5 head to complete human target detection. Transfer learning is performed on the user's specific data to construct a new model that conforms to its own dataset.

[0101] Detecting the target image includes detecting the windshield and the internal features of the vehicle. Among them, the training method of the windshield detection model can be based on the short-wave infrared imaging characteristics, and the same solution as vehicle detection can be used for processing. The windshield detection model is obtained by preparing a relatively rich short-wave windshield image data set for training.

[0102] To improve the accuracy and efficiency of windshield detection, the binocular disparity principle can be used to complete the mapping from the vehicle position in visible light to the short-wave image, and the windshield detection is only performed in a partial area (target area).

[0103] In this embodiment, the main function of the feature matching module is to complete the matching of multiple candidate targets (vehicles and internal features of vehicles in candidate images) with the target to be searched (the vehicle to be recognized and the internal features of the vehicle to be recognized), and complete the similarity determination. The Siamese neural network algorithm can be used.

[0104] 3. Fusion Similarity Ranking and Decision-making Module:

[0105] Based on the above method, the external similarity confidence (the first confidence) and the internal similarity confidence (the second confidence) of different candidate targets can be obtained. It can be carried out by voting and scoring. Ranking is carried out according to the external similarity confidence and the internal confidence of the vehicle respectively, and scoring is carried out according to the ranking. For example, the highest confidence is scored 10 points, the lowest is scored 1 point, and so on. Then the two scores are weighted, score = α * score_visible + β * score_swir, and finally the fusion similarity ranking is output to achieve vehicle re-identification. Optionally, it is default set that α = 0.7 and β = 0.3. Among them, score is the candidate score, α and β are preset coefficients, which can be the weights of visible light images and infrared images, score_visible is the first score obtained according to the first confidence of visible light images, and score_swir is the second score obtained according to the second confidence of infrared images.

[0106] By simultaneously using the multi-spectrum information of visible light images and infrared images, combined with the glass-penetrating characteristics of infrared images, the feature information used in vehicle re-identification technology is enriched, and the accuracy of vehicle re-identification is improved. In the case where the license plate cannot be recognized, higher-precision vehicle re-identification can be completed with the assistance of vehicle internal information; vehicle re-identification in different dimensions including single-spectrum and multi-spectrum can be realized, and it has strong environmental adaptability. It can provide more channels of light information, provide more methods and rich image (spectrum) information for the backend detection algorithm, improve the environmental adaptability of the backend algorithm, and enhance the detection ability of the algorithm.

[0107] Current mainstream vehicle re-identification devices are all based on visible light single-spectrum imaging technology, which has certain limitations. There are many situations in actual application scenarios where license plates cannot be recognized, and vehicles with similar or even identical appearances exist, making it difficult to meet the real vehicle re-identification requirements. This embodiment proposes a multi-dimensional feature vehicle re-identification device based on multi-spectrum imaging technology. This device is developed based on visible light-short wave imaging equipment. At the same time, the present invention also proposes a vehicle re-identification model based on a deep neural network and multi-dimensional features inside and outside the vehicle. By combining the visible light features of the vehicle appearance and the short wave features inside the vehicle, it provides richer vehicle feature information, completes higher-precision target matching, improves the performance and robustness of the vehicle re-identification algorithm, and provides a new vehicle re-identification solution for the fields of transportation and security monitoring. The following uses another specific embodiment to exemplarily illustrate the vehicle re-identification method, which includes:

[0108] 1. Collect and label visible light and short wave data of the target scene.

[0109] For vehicle monitoring, vehicle re-identification is often applied in scenarios such as streets, roads, and highways. Therefore, it is necessary to collect a large amount of visible light and short wave data of the corresponding scenarios. That is, collect a large number of visible light images and infrared images as original image pairs, and store and mark the collected visible light images and infrared images in pairs.

[0110] 2. Build an image database

[0111] Use a large number of original image pairs, calibration image pairs, and corresponding annotation information (such as vehicle position) to build a vehicle analysis database, and annotation information can be added according to actual needs;

[0112] 3. Training of the vehicle re-identification model

[0113] The vehicle re-identification model includes but is not limited to a vehicle detection model and a windshield detection model. Based on the existing mature detection model and similarity matching model, transfer training of the model can be completed for the data in the image database and the data self-built by users.

[0114] 4. Vehicle re-identification based on visible light-short wave imaging

[0115] Deploy the vehicle re-identification model constructed in this embodiment to the corresponding processing platform, connect it to the visible light-short wave imaging equipment, and complete real-time vehicle re-identification.

[0116] This embodiment also provides a vehicle re-identification device, and this system includes:

[0117] Visible light-short wave binocular imaging device;

[0118] A re-identification device equipped with a vehicle re-identification model (the vehicle re-identification system in the following embodiments) based on a deep neural network and multi-dimensional features inside and outside the vehicle. This model uses the vehicle re-identification method provided in the above embodiments to achieve the re-identification of the vehicle to be identified. It can extract and fuse the appearance features and internal information of the target vehicle, and complete the accurate vehicle re-identification in different environments.

[0119] Please refer to Figure 8 , this embodiment also provides a vehicle re-identification system 800, which includes:

[0120] An acquisition module 801, configured to acquire a vehicle image of the vehicle to be identified, where the vehicle image includes a visible light image and an infrared image;

[0121] A first comparison module 802, configured to compare the visible light image with a preset image in a preset image dataset to obtain one or more candidate images and the first comparison results of each candidate image;

[0122] A second comparison module 803, configured to compare the infrared image with each candidate image to obtain the second comparison results of each candidate image;

[0123] A determination module 804, configured to determine the candidate rankings of each candidate image according to the first comparison results and the second comparison results, so as to achieve the re-identification of the vehicle to be identified.

[0124] In this embodiment, the system essentially sets up multiple modules to execute the methods in the above embodiments. For the specific functions and technical effects, please refer to the above method embodiments, and details will not be described here.

[0125] See Figure 9 , this embodiment of the present invention also provides an electronic device 1000, including a processor 1001, a memory 1002, and a communication bus 1003;

[0126] The communication bus 1003 is used to connect the processor 1001 and the memory 1002;

[0127] The processor 1001 is configured to execute a computer program stored in the memory 1002 to implement one or more of the methods as described in the first embodiment above.

[0128] This embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored,

[0129] The computer program is used to cause a computer to execute any one of the methods described in the first embodiment above.

[0130] The embodiments of the present application also provide a non-volatile readable storage medium, in which one or more modules (programs) are stored. When the one or more modules are applied to a device, the device can be caused to execute the instructions included in the first embodiment of the embodiments of the present application.

[0131] It should be noted that the computer-readable medium described above in the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0132] The above computer-readable medium may be included in the above electronic device; or it may exist separately and not be assembled into the electronic device.

[0133] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0135] The above embodiments are only illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A vehicle re-identification method, characterized in that, The method includes: Obtaining a vehicle image of a vehicle to be recognized, where the vehicle image includes a visible light image and an infrared image; Comparing the visible light image with a preset image in a preset vehicle dataset to obtain one or more candidate images and a first comparison result of each candidate image; Comparing the infrared image with each candidate image to obtain a second comparison result of each candidate image; comparing the infrared image with each candidate image includes: detecting the visible light image to obtain vehicle position information of the vehicle to be recognized; determining target region position information from the infrared image according to the vehicle position information and preset depth information, and determining a target image; detecting the target image to obtain internal features of the vehicle to be recognized of the vehicle to be recognized; comparing the internal features of the vehicle to be recognized with candidate vehicle features of the candidate image; Determining a candidate ranking of each candidate image according to the first comparison result and the second comparison result to achieve re-identification of the vehicle to be recognized.

2. The vehicle re-identification method according to claim 1, characterized in that, The determination method of the target image includes any one of the following: Taking the image of the target region where the target region position information is located as the target image; Expanding the target region to obtain an expanded region, and taking the image of the expanded region as the target image.

3. The vehicle re-identification method according to claim 1, characterized in that Before comparing the infrared image with each candidate image, the method further includes: Obtaining a transformation matrix between the visible light image and the infrared image; Converting the pixel point position information of the visible light image and the pixel point position information of the infrared image into a preset coordinate system for representation according to the transformation matrix.

4. The vehicle re-identification method according to claim 1, wherein After determining the candidate ranking of each candidate image according to the first comparison result and the second comparison result, the method further includes: Taking the candidate image with a candidate ranking higher than a preset ranking threshold as the re-identification result of the vehicle to be recognized.

5. The vehicle re-identification method according to any one of claims 1-4, characterized in that Comparing the visible light image with a preset image in a preset image dataset to obtain one or more candidate images and a first comparison result of each candidate image includes respectively inputting the visible light image and each preset image into a preset vehicle detection model to obtain one or more candidate images and a first confidence level of each candidate image; And / or Comparing the infrared image with each candidate image to obtain a second comparison result of each candidate image includes respectively inputting the infrared image and each preset image into a preset target region detection model to obtain a second confidence level of each candidate image.

6. The vehicle re-identification method according to claim 5, wherein, Determining the candidate ranking of each candidate image according to the first comparison result and the second comparison result includes any one of the following: Determining a first score of the candidate image according to the first confidence level, determining a second score of the candidate image according to the second confidence level, determining a comprehensive score of the candidate image according to the first score and the second score, and sorting according to the comprehensive scores of each candidate image to obtain a candidate ranking; Determine the first ranking of the candidate image according to the first confidence level, determine the second ranking of the candidate image according to the second confidence level, and determine the candidate ranking of the candidate image according to the first ranking and the second ranking.

7. A vehicle re-identification system, characterized in that The system includes: An acquisition module, configured to acquire a vehicle image of a vehicle to be recognized, where the vehicle image includes a visible light image and an infrared image; A first comparison module, configured to compare the visible light image with a preset image in a preset image dataset to obtain one or more candidate images and a first comparison result of each candidate image; A second comparison module, configured to compare the infrared image with each candidate image to obtain a second comparison result of each candidate image; comparing the infrared image with each candidate image includes: detecting the visible light image to obtain vehicle position information of the vehicle to be recognized; determining target region position information from the infrared image according to the vehicle position information and preset depth information, and determining a target image; detecting the target image to obtain internal features of the vehicle to be recognized of the vehicle to be recognized; comparing the internal features of the vehicle to be recognized with candidate vehicle features of the candidate image; A determination module, configured to determine the candidate ranking of each candidate image according to the first comparison result and the second comparison result, so as to implement re-identification of the vehicle to be recognized.

8. An electronic device, characterized in that, It includes a processor, a memory, and a communication bus; The communication bus is used to connect the processor and the memory; The processor is configured to execute a computer program stored in the memory to implement the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, The computer program is used to cause a computer to execute the method according to any one of claims 1-6.

Images