A modified vehicle identification method based on attribute fusion and fuzzy product quantization

Abstract

This invention is applicable to the fields of artificial intelligence and intelligent transportation, providing a modified vehicle identification method based on attribute fusion and fuzzy product quantization. In the comprehensive vector feature extraction, this invention employs vehicle and license plate detection, parallel feature extraction, and attribute fusion concatenation to obtain comprehensive features containing high-level license plate semantic information, as well as low-level vehicle model and color attributes. This eliminates background noise interference, enhances the response strength of three easily modified vehicle features, and helps compensate for the limitations of single features. Furthermore, it applies fuzzy clustering and cosine similarity distance calculation methods, which can accommodate the uncertainty of input information during modified vehicle retrieval. Compared to traditional Euclidean distance, cosine similarity is not affected by vector length, and can better handle differences in dimensions when retrieving modified vehicles. Compared to the original product vector method, this invention's method has higher retrieval accuracy and robustness in modified vehicle scenarios.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and intelligent transportation, and in particular relates to a method for identifying modified vehicles based on attribute fusion and fuzzy product quantization. Background Technology

[0002] In recent years, with the development of road traffic safety management systems and intelligent transportation systems, modified vehicles have attracted much attention. This includes modifications to vehicle models, colors, and license plate appearance attributes. At the same time, vehicle attribute recognition and image retrieval algorithms have also been widely used in the field of intelligent transportation.

[0003] Vehicle attribute recognition includes coarse-grained recognition and fine-grained vehicle type recognition. Coarse-grained recognition is a basic vehicle classification task that categorizes vehicles into only a few basic types, such as cars, trucks, and buses. Fine-grained vehicle type recognition aims to distinguish specific vehicle attributes, such as model, color, and license plate. With the increasing demands of intelligent transportation, fine-grained vehicle type recognition will be a key area of ​​future research. When performing vehicle detection on images obtained from traffic monitoring, factors such as lighting and weather conditions can significantly affect the accuracy of vehicle features and backgrounds, leading to a decrease in recognition precision. Therefore, solving the problem of accurately searching for target images within large amounts of traffic monitoring video data will have significant practical application value.

[0004] Content-based image retrieval technologies mainly fall into two categories: product quantization and hashing. Image retrieval tasks often require higher accuracy, so product quantization is more widely used. Existing product quantization subspace partitioning can utilize the K-Means clustering algorithm. However, K-Means clustering is sensitive to noise and outliers, which can significantly impact the clustering results. K-Means is also highly sensitive to the choice of initial centroids; different initial points can lead to different clustering results, reducing retrieval accuracy.

[0005] Current vehicle attribute recognition and image retrieval methods mainly face two fundamental problems: first, how to perform fast and accurate searches when faced with large amounts of video and image data; and second, how to reduce the influence of background factors for accurate searches when some images have very complex backgrounds and the modified vehicles themselves. Existing methods all suffer from these problems. Summary of the Invention

[0006] In view of the above problems, the purpose of this invention is to provide a modified vehicle identification method based on attribute fusion and fuzzy product quantization, which aims to solve the above technical problems.

[0007] The present invention adopts the following technical solution:

[0008] The modified vehicle identification method based on attribute fusion and fuzzy product quantization includes the following steps:

[0009] Step S1: For the collected vehicle driving video, extract the video frames containing vehicle targets and save them. Collect the extracted images with different timestamps to form a library. Use the vehicle license plate detection algorithm to detect the images in the library and crop out the vehicles and corresponding license plates in the images.

[0010] Step S2: Use the corresponding feature extraction algorithm to extract the original features of vehicle type, license plate and color of the cropped image in parallel. After concatenating the three feature vectors, the vehicle's n-dimensional comprehensive feature vector is obtained. Then, the vector is segmented to obtain s sub-vectors.

[0011] Step S3: Perform fuzzy clustering on all sub-vectors to obtain k cluster centers and corresponding membership matrices. Sort the corresponding membership degrees from largest to smallest and take the largest membership degrees to classify the sub-vectors. For the vehicle image to be queried, use the same method as in step S2 to obtain s sub-vectors. Calculate the cosine distance from each sub-vector to the k cluster centers to obtain a distance table.

[0012] Step S4: Calculate the weighted distance of each sub-vector of the vehicle to be queried according to the distance table and sort them. Calculate the distance sorting index of each sub-vector and sort the index in ascending order. The samples corresponding to the first K values ​​are the K nearest neighbor samples of the vehicle to be queried.

[0013] Furthermore, in step S1, the PaddleDetection algorithm is selected for vehicle detection, and the PaddleOCRv3 algorithm is selected for license plate detection.

[0014] Furthermore, the specific process of step S2 is as follows:

[0015] The ResNet50 feature extraction algorithm was used to extract vehicle and license plate features from the cropped vehicle and license plate photos.

[0016] Statistical information based on color histograms was used to extract features of vehicle colors;

[0017] The three extracted feature vectors are fused and concatenated together to obtain the vehicle's n-dimensional comprehensive feature vector, and finally, a comprehensive feature vector set is obtained.

[0018] For each of the h n-dimensional comprehensive feature vectors in the comprehensive feature vector set, each is divided into s segments, and each segment is a sub-vector with a dimension of n / s.

[0019] Furthermore, the specific process of step S3 is as follows:

[0020] For each subvector, perform fuzzy clustering, and the clusters are divided into k classes, resulting in k cluster centers. Set a fuzzy clustering objective function J.

[0021] At the start of clustering, a membership matrix that satisfies the constraints is randomly generated. Through iteration of the membership matrix and cluster centers, the iteration stops when the value of the objective function J tends to stabilize, resulting in the final membership matrix. Each row of the membership matrix represents a sample, with a total of h samples. Each value in each column of the membership matrix represents the membership degree of the corresponding class, with a total of k columns. The corresponding membership degrees are sorted from largest to smallest, and the largest membership degrees are used for sub-vector classification.

[0022] For the vehicle image to be queried, after the same feature extraction, vector concatenation, and vector segmentation, s sub-vectors are obtained. For each sub-vector, the cosine distance to k cluster centers is calculated to obtain a k*s distance table.

[0023] Furthermore, the specific process of step S4 is as follows:

[0024] The weighted distance of each sub-vector of the vehicle to be queried is calculated based on the distance table to obtain a distance set of h distances. The distances are then sorted from smallest to largest, and the sub-vectors corresponding to the distances are labeled with the sorting index to obtain a set of h codes encoded with the sorting index.

[0025] For each encoding set, calculate the sum of the sequence numbers, and arrange the results in ascending order to form a new set;

[0026] The samples corresponding to the first K values ​​in the new set are the K nearest neighbor samples of the vehicle to be queried.

[0027] The beneficial effects of this invention are:

[0028] First, this invention uses different algorithms to extract features from vehicle type, license plate, and color, and then concatenates the vectors to obtain a comprehensive vehicle feature vector. This feature vector not only has high-level license plate semantic information, but also contains low-level vehicle type and color attributes. Furthermore, the corresponding feature extraction algorithm can fully leverage the advantages of deep features and texture and color features, which helps to improve the accuracy and robustness of modified vehicle retrieval without reducing retrieval speed.

[0029] Secondly, this invention employs fuzzy clustering, which reduces sensitivity to noise and outliers by introducing membership degrees. Furthermore, fuzzy clustering uses random initialization to select initial membership degrees, thereby reducing dependence on initial points. Additionally, this method also meets the needs of modified vehicle retrieval, where the uncertainty of input information necessitates displaying multiple suspect vehicles similar to the target modified vehicle.

[0030] Third, this invention uses cosine similarity for similarity measurement. Cosine distance is not affected by vector length, and it can better handle differences between dimensions when searching for modified vehicles. Furthermore, vector directionality is important among vehicle features, and cosine distance can effectively capture this directional information between vectors.

[0031] Finally, existing product quantization methods still do not provide high enough retrieval accuracy for nearest neighbor samples when dealing with modified car scenes and images with complex backgrounds. This invention combines the above-mentioned improved methods to improve the retrieval accuracy of images in modified car scenes. Attached Figure Description

[0032] Figure 1 This is a flowchart of the modified vehicle identification method based on attribute fusion and fuzzy product quantization provided in the embodiments of the present invention;

[0033] Figure 2 This is a flowchart of vehicle and license plate detection provided in an embodiment of the present invention;

[0034] Figure 3 This is a flowchart of vehicle feature extraction and attribute fusion provided in an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of sub-vector classification provided in an embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of the weighted asymmetric cosine distance calculation provided in an embodiment of the present invention;

[0037] Figure 6 This is a distance table and a distance calculation diagram provided in an embodiment of the present invention;

[0038] Figure 7 This is a flowchart illustrating the implementation of a modified vehicle identification system provided in an embodiment of the present invention.

[0039] Figure 8 This is an interface diagram of the modified vehicle identification system provided in an embodiment of the present invention;

[0040] Figure 9 This is a schematic diagram comparing the method provided in the embodiments of the present invention with the original product quantization retrieval results. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] To illustrate the technical solution described in this invention, specific embodiments are described below.

[0043] like Figure 1 As shown, the modified vehicle identification method based on attribute fusion and fuzzy product quantization provided in this embodiment includes the following steps:

[0044] Step S1: For the collected vehicle driving video, extract the video frames containing vehicle targets and save them. Collect the extracted images with different timestamps to form a library. Use a vehicle license plate detection algorithm to detect the images in the library and crop out the vehicles and corresponding license plates in the images.

[0045] Road cameras are typically installed on municipal roads or highways to capture video of vehicles in motion. This step first collects image samples. Videos of vehicles on the road are captured by the road cameras, and frames containing vehicles are extracted and saved. Images of vehicles at different timestamps are collected to form a library. Then, a vehicle license plate detection algorithm is used to detect and crop out the vehicles and their corresponding license plates from the images in the library.

[0046] For example, if a database contains 779 vehicle images, the PaddleDetection algorithm is used to detect vehicles in the images, and the PaddleOCRv3 algorithm is used to detect license plates. Both algorithms have high detection accuracy. After cropping the images, the vehicles and their corresponding license plates are extracted, resulting in 779 vehicle images and their corresponding license plate images. For instance, a vehicle image with dimensions (1600, 1200) is cropped to obtain a vehicle image with dimensions (1138, 731) and a license plate image with dimensions (131, 39). Figure 2 This example illustrates the vehicle and license plate detection process. The numbers in parentheses represent the image's dimensions in pixels, and the same applies below.

[0047] Step S2: Use the corresponding feature extraction algorithm to extract the original features of vehicle type, license plate and color of the cropped image in parallel. After concatenating the three feature vectors, obtain the n-dimensional comprehensive feature vector of the vehicle. Then, perform vector segmentation to obtain s sub-vectors.

[0048] This step involves vehicle feature extraction and attribute fusion. Different feature extraction algorithms are used to extract the original features of vehicle type, license plate, and color from the cropped images in parallel. Then, these three feature vectors are concatenated to obtain an n-dimensional comprehensive feature vector for the vehicle. This n-dimensional comprehensive feature vector set is then segmented. Combined with... Figure 3 As shown, the specific process of this step is as follows:

[0049] S21. The ResNet50 feature extraction algorithm is used to extract the features of vehicle type and license plate from the cropped vehicle photos and license plate photos.

[0050] After extracting the original features, we obtain vehicle model features (Model_features) and license plate features (Plate_features) with dimensions (779, 2048). Examples of vehicle model features and license plate features are as follows:

[0051] Vehicle characteristics with dimensions (779, 2048):

[0052]

[0053] License plate features of size (779, 2048):

[0054]

[0055] S22. Use statistical information based on color histograms to extract features of vehicle colors.

[0056] This step uses the color histogram algorithm to extract the original features from the cropped vehicle image, obtaining color features of size (779, 4096), as follows:

[0057]

[0058] S23. The three extracted feature vectors are fused and spliced ​​together to obtain the n-dimensional comprehensive feature vector of the vehicle.

[0059] For example, concatenating the three feature vectors together yields an 8192-dimensional composite feature vector, Composite_features, which is represented as follows:

[0060] Composite_features=Model_features+Plate_features+Color_features

[0061] After attribute fusion and concatenation, we obtain the vehicle composite feature vector Composite_features with a size of (779, 8192), as follows:

[0062]

[0063] S24. For the h images in the image library, the h n-dimensional comprehensive feature vectors are all divided into s segments, each segment being a sub-vector with a dimension of n / s.

[0064] After processing the h images in the image library according to the above steps, h n-dimensional comprehensive feature vectors are obtained. For ease of subsequent description, the h n-dimensional comprehensive feature vectors in the image library are referred to as library vectors.

[0065] Taking a single library vector as an example, it is averaged into s segments, with each sub-vector having a dimension of n / s. For ease of description, let n = 2048 and s = 16, then n / s = 128. Each sub-vector of all comprehensive feature vectors can be represented as:

[0066]

[0067] For steps S1 and S2 above, existing global feature extraction methods for vehicle photos have certain limitations in the case of modified vehicle retrieval. This invention first uses PaddleDetection and PaddleOCRv3, two algorithms with high detection accuracy, to detect vehicles and license plates. Then, the vehicles and corresponding license plates are cropped out. Parallel feature extraction is then used, selecting the color histogram algorithm (excellent in color feature extraction) and the ResNet50 algorithm (good at extracting vehicle and license plate features) for color feature extraction and vehicle / license plate feature extraction, respectively. Finally, the vehicle / license plate and color feature vectors are fused and concatenated to obtain a comprehensive vehicle feature vector. This feature vector not only contains high-level license plate semantic information but also low-level vehicle / model and color attributes. This method can remove background noise and interference from cluttered pixels, enhance the feature response strength of the target vehicle, and fully utilize the advantages of deep features, texture, and color features. All of these contribute to improving the accuracy and robustness of modified vehicle retrieval without reducing retrieval speed.

[0068] Step S3: Perform fuzzy clustering on all sub-vectors to obtain k cluster centers and corresponding membership matrices. Sort the corresponding membership degrees from largest to smallest and take the largest membership degrees to classify the sub-vectors. For the vehicle image to be queried, use the same method as in step S2 to obtain s sub-vectors. Calculate the cosine distance from each sub-vector to the k cluster centers to obtain a distance table.

[0069] The specific process for this step is as follows:

[0070] S31. Perform fuzzy clustering on each sub-vector, and divide the cluster into k classes to obtain k cluster centers. Set a fuzzy clustering objective function J.

[0071] For example, if we perform fuzzy clustering on the first segment of vectors and divide it into k classes (for ease of understanding, let's take k = 256), then each segment will have 256 cluster centers e. i Each subvector V i Belongs to a certain category e i And the corresponding membership matrix is ​​u ij .

[0072] S32. At the start of clustering, a membership matrix that satisfies the constraints is randomly generated. Through iteration of the membership matrix and cluster centers, the iteration stops when the value of the objective function J tends to stabilize, and the final membership matrix is ​​obtained. Each row of the membership matrix represents a sample, and there are a total of h samples. Each value in each column of the membership matrix represents the membership degree of the corresponding class, and there are a total of k columns. The corresponding membership degrees are sorted from largest to smallest, and the largest membership degrees are used for sub-vector classification.

[0073] At the start of clustering, a cluster u that satisfies the constraints is randomly generated. ij e is obtained through iterative clustering of cluster centers. i Repeat the membership matrix iteration and cluster center iteration. When the value of the objective function J tends to stabilize, stop the iteration. The condition for stopping the iteration is:

[0074]

[0075] Where ε is taken as a very small constant as the error threshold, and t is the number of iterations. When u ij When the conditions are met, the final clustering result can be obtained.

[0076] Taking the membership matrix of the first sub-vector clustering result as an example, each row of the matrix represents a sample, with a total of h samples. Each value in each column represents the membership degree of each cluster, with a total of 256 columns. Below is the membership matrix u output after fuzzy clustering iteration. ij The result is:

[0077]

[0078] Based on the membership matrix, the top 3 membership degrees of each sub-vector are used for sub-vector classification. In the modified vehicle retrieval scenario based on vehicle attributes, as described in step S1, the comprehensive feature vector is formed by combining the three feature vectors of vehicle model, color, and license plate. Here, the top 3 membership degrees of each sub-vector are selected. The sub-vector classification diagram is shown below. Figure 4 As shown.

[0079] For example, in the first sub-vector ω 1,1 ω 1，2 and ω 1，3 We can classify the subvectors. Each subvector can be represented by three numeric IDs, and the entire subvector can be represented by three 16-bit numeric IDs.

[0080] Following the above representation method, each class can be represented by a numeric ID, so the 256 clusters can be represented by numbers from 1 to 256. For example... Figure 4 Vehicle subvector 1 can be used The three IDs are used to represent the vehicle subvector 2. Let h be the vehicle subvector. We'll use three IDs to represent this. Here, we'll set an ID like this as... To express.

[0081] Each vehicle subvector is divided into 16 segments, each represented by a numeric ID, and then digitally quantized and encoded. The complete vehicle subvector can then be represented as:

[0082]

[0083] To facilitate understanding, a specific calculation example is provided. For instance, let V1 be:

[0084]

[0085] S33. For the vehicle image to be queried, after the same feature extraction, vector concatenation, and vector segmentation, s sub-vectors are obtained. For each sub-vector, the cosine distance to the k cluster centers is calculated to obtain a k*s distance table.

[0086] To determine whether a vehicle photo is a modified vehicle, the vehicle image is extracted and concatenated using the same features as described above to obtain an n-dimensional comprehensive feature vector, denoted as P, which is then divided into 16 segments on average.

[0087] Calculate the cosine distance from each subvector of Z to the 256 cluster centers. The result of the approximate nearest neighbor search method depends on the distance between the vector of the queried vehicle and the library vector; here, a weighted summation asymmetric distance cosine calculation method is used.

[0088] For ease of understanding, let's assume a specific modified car vector P to be queried and a library vector V. The library vector V is represented by its three fuzzy cluster centers e1(v), e2(v), and e3(v). We then extract the top three largest membership degrees w1, w2, and w3 of all clusters to which the library vector V belongs, as follows: Figure 5 As shown, the approximate distance between the modified car vector P to be queried and the library vector V is expressed as:

[0089] D(P,V)=w1·D[P,e1(v)]+w2·D[P,e2(v)]+w3·D[P,e3(v)]

[0090]

[0091] The value of D[P,e(v)] ranges from [0,2]. The closer the value is to 2, the less similar the two vectors are; the closer the value is to 0, the more similar the two vectors are.

[0092] The distance from the first segment of the modified car vector P to the first segment of the vector in this database can be calculated as: D 1 1,1 =w 1，1 Dis 1，1 +w 1，2 Dis 2，1 +w 1，3 Dis 3，1

[0093] Dis t,i =||P i ,e r (v)||2,1≤r≤3,1≤i≤h

[0094] The set of distances from each segment of the library vector to the corresponding segment of the modified car vector to be queried is:

[0095] D = {D i j,j |1≤i≤h,1≤j≤16}

[0096] in

[0097] Where i is the index of the h library vectors, and j represents the index of which segment of the library vector. This indicates a query for the j-th segment of the modified car vector to U. i,j The distance to the center of the class they represent. A distance table and a diagram illustrating distance calculation are shown below. Figure 6 As shown.

[0098] Step S4: Calculate the weighted distance of each sub-vector of the vehicle to be queried according to the distance table and sort them. Calculate the distance sorting index of each sub-vector and sort the index in ascending order. The samples corresponding to the first K values ​​are the K nearest neighbor samples of the vehicle to be queried.

[0099] The specific process for this step is as follows:

[0100] S41. Calculate the weighted distance of each sub-vector of the vehicle to be queried according to the distance table, obtain a distance set of h distances, sort the distances from smallest to largest, and label the sub-vectors corresponding to the distances according to the sorting number, to obtain a set of h codes encoded by the sorting number.

[0101] Taking the first sub-vector as an example, the distance from the first sub-vector of the h vectors in the image library to the first sub-vector of the vehicle to be queried is the set of h distances:

[0102] D = {D i 1,1 |1≤i≤h}=D 1 1,1 D 2 1,1D 3 1,1 ,...,D h 1,1 .

[0103] S42. For each encoding set, calculate the sum of the sequence numbers, arrange the results in ascending order, and form a new set.

[0104] Label the remaining 15 subvectors of the h vectors using the method described above. This will result in a set T of codes for the h sorted index codes, represented as:

[0105] T = {index1, index2, ..., index...} h}

[0106] Where index is a 16-bit numeric code, for example, index1 = (1,2,2,3,3,3,4,4,4,4,5,10,11,13,15,16).

[0107] In the encoded set, calculate the sum of the 16-bit numbers encoded for each index, for example, sum(index1) = 100 to form a new set, and sort it in ascending order:

[0108] New set T sum =Rank{sum(index1),sum(index2),...,sum(index h )}.

[0109] S43. The samples corresponding to the first K values ​​in the new set are the K nearest neighbor samples of the vehicle to be queried.

[0110] Then the new set T sum The samples in the image library corresponding to the top K values ​​in the sorted sequence are the K nearest neighbors of the modified car. Assume there are two samples whose 16-bit numeric codes for the sorted ranking of all their subvectors are index1 and index2, respectively:

[0111] index1=(1,1,2,2,2,3,3,5,6,8,9,9,11,13,15,16), index2=(2,2,3,3,3,4,4,6,7,9,10,10,12,14,14,15).

[0112] Summing the rankings of the two sample vectors, we get sum(index1) = 106 and sum(index2) = 118, where 106 < 118. Therefore, the image vehicle represented by index1 is more similar to the vector of the vehicle to be queried.

[0113] In the method of this invention, existing product quantization subspace partitioning uses K-Means clustering, which is sensitive to noise and outliers. These outliers can significantly impact the clustering results. K-Means is also highly sensitive to the selection of initial centroids; different initial centroids can lead to different clustering results. This invention employs fuzzy clustering, which reduces sensitivity to noise and outliers by introducing the concept of membership. Furthermore, fuzzy clustering uses random initialization to select initial memberships, thus reducing dependence on initial centroids. Additionally, this method aligns with the need to display multiple suspect vehicles similar to the target modified vehicle during modified vehicle retrieval due to the uncertainty of input information.

[0114] Furthermore, existing product quantization methods use Euclidean distance for vector similarity measurement. Euclidean distance is sensitive to vector dimension; when dealing with high-dimensional vectors, the difference between any two dimensions becomes negligible in the overall distance because differences in other dimensions may be more significant. This easily leads to distortion in the distance measurement. This invention uses cosine similarity for similarity measurement. Cosine distance is calculated based on the cosine of the angle between vectors and is unaffected by vector length. When searching for modified vehicles, it can better handle differences in dimension. Additionally, vector directionality is important among vehicle features, and cosine distance can effectively capture this directional information between vectors.

[0115] The above method can be further developed into a modified vehicle identification system, which can more intuitively monitor vehicle data and obtain relevant information about modified vehicles.

[0116] like Figure 7 As shown, the system acquires video data from road cameras, extracts frames containing vehicle images to form a vehicle image database. The server reads sample images from the graph database and performs a series of processing steps on the acquired images, including preprocessing, vehicle and license plate detection, feature extraction, vector concatenation, vector partitioning, and fuzzy clustering. Appropriate processing is applied to both types of images to obtain the K nearest similar images for the query image, which are then sent to a message queue. The front-end webpage (Chrome) accesses the server via HTTP through an Nginx reverse proxy, receives *a* similar images, and the server queries the corresponding metadata in the MySQL database based on the ID and returns it to the front-end webpage, where the relevant information can then be viewed.

[0117] like Figure 8As shown, the modified vehicle recognition system displays vehicle information for a certain road segment. The system interface displays surveillance video from cameras deployed across a road network within a certain range, along with the detection results, vehicle type, color, and license plate recognition results for the vehicles in the video. The backend server performs a series of processing steps based on the recognition results and a given query image, including feature extraction, vector concatenation, vector partitioning, and fuzzy clustering, returning the k closest similar images to the query image. Furthermore, this system can be integrated with other traffic systems, such as abnormal behavior analysis systems and traffic control systems, to provide a comprehensive view of traffic in a given area.

[0118] In summary, this invention concatenates vehicle model, license plate, and color feature vectors together in vehicle feature extraction, fully utilizing the information from these features to obtain more comprehensive and integrated vehicle characteristics. In the subspace partitioning of product quantization, fuzzy clustering is employed, introducing the concept of membership to reduce sensitivity to noise and outliers. Furthermore, fuzzy clustering uses random initialization to select initial membership, thereby reducing dependence on initial points. Compared to existing product vector methods, this invention's method achieves higher accuracy in retrieving multiple neighboring similar images in modified vehicle scenarios, such as... Figure 9 In the original product quantization retrieval results, the second and third images are incorrect and significantly different; while the first three images obtained by this method are highly similar, suggesting a high probability of a modified car. In summary, it can be preliminarily concluded that this invention offers improved performance compared to the original product quantization in the scenario of modified car retrieval.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A retrofit vehicle identification method based on attribute fusion and fuzzy product quantization, characterized in that, The modified vehicle identification method comprises the following steps: Step S1, for the collected vehicle driving video, the video frame containing the vehicle target is intercepted and saved, the intercepted pictures of different time stamps are collected to form a gallery, and a vehicle license plate detection algorithm is used to detect the gallery pictures and crop the vehicle and the corresponding license plate in the pictures; Step S2, the original features of the vehicle type, license plate and color of the cropped pictures are extracted respectively by using the corresponding feature extraction algorithm, the three kinds of feature vectors extracted are spliced to obtain an n-dimensional comprehensive feature vector of the vehicle, and then the vector is segmented to obtain s sub-vectors; Step S3, fuzzy clustering is performed on all the sub-vectors to obtain k cluster centers and a corresponding membership matrix, the corresponding membership is sorted from large to small, and the maximum membership is taken for the classification of the sub-vectors, for the vehicle picture to be queried, s sub-vectors are obtained in the same way as step S2, the cosine distance of each sub-vector to the k cluster centers is calculated, and a distance table is obtained; Step S4, the weighted distance of each sub-vector of the vehicle to be queried is calculated according to the distance table and sorted, the distance order numbers of each sub-vector are counted and summed, and the sum is sorted from small to large, and the first K values correspond to the K nearest neighbor samples of the vehicle to be queried; In step S1, PaddleDetection algorithm is used for vehicle detection, and PaddleOCRv3 algorithm is used for license plate detection; The specific process of step S2 is as follows: ResNet50 feature extraction algorithm is used to extract the features of the vehicle type and license plate of the cropped vehicle and license plate pictures; The color histogram-based statistical information is used to extract the features of the vehicle color; The three kinds of feature vectors extracted are spliced together to obtain an n-dimensional comprehensive feature vector of the vehicle; The h n-dimensional comprehensive feature vectors obtained from the h pictures in the gallery are each divided into s segments, and each segment is a sub-vector, and the dimension of the sub-vector is n / s; The specific process of step S3 is as follows: Fuzzy clustering is performed on each segment of the sub-vector, the clustering is divided into k classes, k cluster centers are obtained, and a fuzzy clustering objective function J is set; At the beginning of clustering, a membership matrix satisfying the constraint condition is randomly generated, the membership matrix is iterated and the cluster centers are iterated, when the value of the objective function J tends to be stable, the iteration is stopped, and the final membership matrix is obtained, wherein each row of the membership matrix represents a sample, there are h samples, each column of the membership matrix represents the membership of the corresponding class, and there are k columns; the corresponding membership is sorted from large to small, and the maximum membership is taken for the classification of the sub-vectors; For the vehicle picture to be queried, s sub-vectors are obtained after the same feature extraction, vector splicing and vector segmentation, the cosine distance of each segment of the sub-vector to the k cluster centers is calculated, and a distance table of k*s is obtained; The specific process of step S4 is as follows: According to the distance table, a weighted distance of each sub-vector of the vehicle to be queried is calculated, a distance set of h distances is obtained, and the distances are sorted from small to large, so that the sub-vectors corresponding to the distances are labeled with the sorting serial numbers to obtain an encoding set of h encodings coded with the sorting serial numbers; For each encoding set, the sum of the serial numbers is calculated, and the results are arranged from small to large to form a new set; The samples corresponding to the first K values in the new set are K nearest neighbor samples of the vehicle to be queried.

Images