Cross-camera vehicle re-identification method and system and electronic equipment
Through the vehicle feature extraction method of Resnet50 and the target dense feature pyramid network combined with the complementary door attention module, the problem of low recognition accuracy caused by weather, angle and occlusion in vehicle re-identification is solved, and the accuracy and accuracy of vehicle recognition are improved.
Patent Information
- Application Number
- CN202411277908.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-12-27
AI Technical Summary
The existing vehicle re-identification technology causes blurred license plates under factors such as weather, shooting angle, lighting changes or low image resolution, and the characteristics of the same vehicle vary greatly under different viewing angles and occlusions, which increases the possibility of mismatch, resulting in The recognition accuracy is not high.
The target dense feature pyramid network with Resnet50 as the backbone of the network is used for vehicle feature extraction. Combined with the complementary door attention module, query vehicle feature vectors are generated and the most matching feature vector is selected from the pre-established vehicle feature database to enhance the significance of the vehicle. Constraints and expressions of characteristics.
The accuracy of the vehicle re-identification algorithm is improved, the problem of misidentification of vehicles with small differences in monitoring scenarios is alleviated, the ability to express local and global prominent features is enhanced, and the risk of mismatch is reduced.
Smart Images
Figure CN119206641A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of deep learning technology, and in particular to a cross-camera vehicle re-identification method, system and electronic device. Background Art
[0002] Vehicle re-identification technology matches vehicle images with captured images through a network of cameras. This technology not only solves illegal behaviors such as license plate forgery and hit-and-run, but also provides decision support for urban traffic planning. It is an important tool for people's livelihood services and maintaining public safety. Users only need one or several vehicle pictures to quickly find the same or similar vehicles within a selected range, thereby achieving comprehensive vehicle management, tracking and automatic positioning.
[0003] However, when performing vehicle re-identification, the license plate may be blurred due to factors such as weather, shooting angle, lighting changes or low image resolution, making it difficult to accurately identify the vehicle by license plate recognition alone; many vehicles look very similar, especially some common brands, models and vehicles of the same color, making them difficult to distinguish, increasing the possibility of mismatching; the same vehicle will have significant differences in pictures taken under different viewing angles, lighting conditions, and in the presence of interference factors such as occlusion. For example, the features obtained when the vehicle is occluded and when it is at a normal viewing angle will be significantly different, which will increase the difficulty of matching.
[0004] Therefore, the current vehicle re-identification method still has the risk of low recognition accuracy and a high possibility of mismatching. Summary of the invention
[0005] The purpose of this application is to provide a cross-camera vehicle re-identification method, system and electronic device to alleviate the above-mentioned technical problems existing in the prior art.
[0006] In a first aspect, the present invention provides a cross-camera vehicle re-identification method, the method comprising:
[0007] In response to the vehicle image to be queried provided by the user, feature extraction is performed on the vehicle image to be queried through a pre-built vehicle feature extraction module to generate a query vehicle feature vector; wherein the pre-built vehicle feature extraction module is a network structure with Resnet50 as the network backbone, and is used to extract features from the vehicle image to be queried through a target dense feature pyramid network;
[0008] Based on the query vehicle feature vector, selecting the best matching one or more matching vehicle feature vectors in a pre-established vehicle feature library;
[0009] The target vehicle matching the query vehicle image is determined based on the matching vehicle features.
[0010] In an optional implementation, a pre-built vehicle feature extraction module is used to extract features from a query vehicle image to generate a query vehicle feature vector, including:
[0011] Inputting the vehicle image to be queried into a target dense feature pyramid network, upsampling a first feature vector extracted by a first preset layer of the target dense feature pyramid network, and merging the first feature vector extracted by a second preset layer of the target dense feature pyramid network with the second feature vector extracted by a second preset layer of the target dense feature pyramid network to obtain a merged feature vector;
[0012] Performing fusion processing on the merged feature vector to obtain a fused feature vector;
[0013] Based on the fused feature vector, feature extraction processing is performed through the complementary gate attention module to generate the query vehicle feature vector.
[0014] In an optional embodiment, upsampling a first feature vector extracted by a first preset layer of a target dense feature pyramid network and merging the first feature vector extracted by a second preset layer of the target dense feature pyramid network to obtain a merged feature vector includes:
[0015] The feature vector is merged by the following formula:
[0016]
[0017] in, The first feature vector extracted from the first preset layer of the target dense feature pyramid network, is the second feature vector extracted from the second preset layer of the target dense feature pyramid network, λ is the preset weight coefficient; f 1 is the merged feature vector.
[0018] In an optional implementation, the merged feature vector is fused to obtain a fused feature vector, including:
[0019] The feature vector is fused by the following formula:
[0020] F 1 =sigmoid(f 1 (Leaky ReLU(c 1×c / r ((h())))))
[0021] Among them, F 1 is the fused feature vector.
[0022] In an optional embodiment, based on the fused feature vector, feature extraction processing is performed through a complementary gate attention module to generate a query vehicle feature vector, including:
[0023] The first feature vector extracted by the first preset layer of the target dense feature pyramid network and the second feature vector extracted by the second preset layer of the target dense feature pyramid network are respectively subjected to global average pooling processing, and then input into the gated structure for OR processing, and the complementary feature vector is obtained as follows:
[0024] Feature extraction is performed using the following formula:
[0025] g 1 =OR(GAP(p),AP(q))
[0026] The feature extraction process is performed through the complementary gate attention module to generate the query vehicle feature vector as follows:
[0027] H 1 = 1 +oftmax(g 1 *GAP(F 1 ))*GAP(F 1 )
[0028] in, 1 To query the vehicle feature vector, g 1 are complementary eigenvectors.
[0029] In an optional embodiment, the construction of the pre-established vehicle feature library includes:
[0030] Obtain vehicle video streams collected by multiple cameras, decode the vehicle video streams, extract frames and perform preprocessing operations, perform vehicle detection on the images obtained by the preprocessing operations and cut out the detected vehicles to obtain cutout vehicle images; wherein the preprocessing operations at least include filtering, denoising and cropping operations;
[0031] The pre-built vehicle feature extraction module is used to extract features from the cutout vehicle image and generate a matching vehicle feature vector;
[0032] The matching vehicle feature vector is stored in a pre-established vehicle feature library.
[0033] In an optional embodiment, based on the query vehicle feature vector, selecting the most matching one or more matching vehicle feature vectors in a pre-established vehicle feature library includes:
[0034] Based on the query vehicle feature vector, calculating a matching similarity between the query vehicle feature vector and one or more matching vehicle features in a pre-established vehicle feature library;
[0035] The best matching song or multiple matching vehicle feature vectors are selected from a pre-established vehicle feature library based on a pre-set similarity threshold and matching similarity.
[0036] In a second aspect, the present invention provides a cross-camera vehicle re-identification system, the system comprising:
[0037] A query feature extraction module, which is used to extract features of the query vehicle image provided by the user through a pre-built vehicle feature extraction module to generate a query vehicle feature vector; wherein the pre-built vehicle feature extraction module is a network structure with Resnet50 as the network backbone, and is used to extract features of the query vehicle image through a target dense feature pyramid network;
[0038] A feature matching module, for selecting one or more best matching vehicle feature vectors in a pre-established vehicle feature library based on the query vehicle feature vector;
[0039] The vehicle feature matching module is used to determine the target vehicle that matches the vehicle image to be queried based on the matching vehicle features.
[0040] In a third aspect, the present invention provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the cross-camera vehicle re-identification method of any one of the aforementioned implementations.
[0041] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the cross-camera vehicle re-identification method of any one of the aforementioned implementations.
[0042] The cross-camera vehicle re-identification method, system and electronic device provided by the present application have the following beneficial effects:
[0043] By selecting multiple high-level feature maps in the feature pyramid, using dense connections, the rich semantic information in the high-level feature maps is recursively passed down, and fused with the lower-level feature maps in a specific way. The fused feature maps take into account both global semantic information and local detail information, which can enhance the constraints and expression of vehicle salient features, improve the accuracy of vehicle re-identification algorithms, and thus alleviate the problem of vehicles with small differences in monitoring scenes being easily misidentified. The complementary gate attention module allows the network to learn local and global significant vehicle features to further enhance the expression ability and recognition accuracy of vehicle features, targeting the differences within vehicle classes and the similarities between classes caused by interference factors such as occlusion and perspective changes in cross-camera scenes. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0045] Figure 1 A flowchart of a cross-camera vehicle re-identification method provided in an embodiment of the present application;
[0046] Figure 2 A flowchart of a specific cross-camera vehicle re-identification method provided in an embodiment of the present application;
[0047] Figure 3 A structure of an adaptive multi-scale subtle feature extraction network provided in an embodiment of the present application;
[0048] Figure 4 A schematic diagram of the structure of a complementary gate attention module provided in an embodiment of the present application;
[0049] Figure 5 A structural diagram of a cross-camera vehicle re-identification system provided in an embodiment of the present application;
[0050] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.
[0052] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0053] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0054] With the acceleration of urban modernization, traffic safety issues have become increasingly prominent, which not only restricts the rapid development of cities, but also becomes a difficult problem that urban managers need to solve urgently. Therefore, improving the efficiency of urban transportation services and ensuring the safety and efficient travel of citizens have become important issues. The development of smart cities has made vehicle re-identification technology more important. This technology can help management departments better monitor road traffic conditions, improve traffic fluidity and safety, reduce traffic congestion, and improve the urban traffic environment. Therefore, the construction of "smart transportation" is one of the urgent tasks of today's urban development. As an important part of the intelligent transportation system, vehicle re-identification can accurately identify specific vehicles from images or videos in various monitoring scenarios based on computer vision technology.
[0055] In recent years, as cars have become the main mode of transportation, the increase in the number of vehicles has brought about management and safety issues. For example, in a complex road traffic environment, the increase in illegal driving, license plate duplication, vehicle theft, traffic accidents and hit-and-run has made manual identification and case follow-up difficult. However, a large number of cameras installed in modern public transportation systems can record a large amount of vehicle driving information through video monitoring and snapshots. Using computers to automatically analyze this information will greatly improve traffic control and safety in smart cities. Therefore, vehicle re-identification plays a key role in public safety, traffic management and smart city construction.
[0056] Vehicle re-identification technology matches vehicle images with captured images through a network of cameras. This technology can not only solve illegal behaviors such as license plate forgery and hit-and-run, but also provide decision support for urban traffic planning. It is an important tool for people's livelihood services and maintaining public safety. Users can quickly find the same or similar vehicles within a selected range through one or several vehicle pictures, thereby realizing comprehensive vehicle management, tracking and automatic positioning. This not only improves the efficiency of vehicle monitoring, but also provides a trajectory basis for traffic accidents. Although license plate recognition is the most direct method of vehicle re-identification, in many cases, it is difficult to complete the task by license plate recognition alone. There are two main reasons: first, factors such as weather, shooting angle, lighting changes or low image resolution may cause the license plate to be blurred, thereby reducing the recognition accuracy; second, the license plate is often blocked, unlicensed or duplicated in illegal behaviors, which increases the difficulty of identifying the target vehicle.
[0057] However, vehicle re-identification technology still has the following problems and difficulties:
[0058] 1. Many vehicles look very similar, especially some common makes, models, and vehicles of the same color, which makes them difficult to distinguish, increasing the possibility of mismatching.
[0059] 2. The pictures of the same car taken at different viewing angles, lighting conditions, and in the presence of interference factors such as occlusion will have significant differences. For example, the features obtained when the car is occluded and when it is viewed from a normal perspective will be significantly different, which will increase the difficulty of matching.
[0060] Based on this, the embodiments of the present application provide a cross-camera vehicle re-identification method, system and electronic device. By selecting multiple high-level feature maps in the feature pyramid, the rich semantic information in the high-level feature maps is recursively passed downward using a dense connection method, and is fused with the lower-level feature maps in a specific manner. The fused feature maps take into account both global semantic information and local detail information, which can enhance the constraints and expression of vehicle significant features, improve the accuracy of the vehicle re-identification algorithm, and thereby alleviate the problem of vehicles with small differences in monitoring scenes being easily misidentified.
[0061] This application embodiment provides a cross-camera vehicle re-identification method, see Figure 1 As shown, the method mainly includes the following steps:
[0062] Step S110, in response to the vehicle image to be queried provided by the user, feature extraction is performed on the vehicle image to be queried through a pre-built vehicle feature extraction module to generate a query vehicle feature vector; wherein the pre-built vehicle feature extraction module is a network structure with Resnet50 as the network backbone, which is used to extract features from the vehicle image to be queried through a target dense feature pyramid network.
[0063] In one embodiment, extracting features from a query vehicle image by using a pre-built vehicle feature extraction module to generate a query vehicle feature vector may include the following steps 1-1 to 1-3:
[0064] Step 1-1, input the vehicle image to be queried into the target dense feature pyramid network, upsample the first feature vector extracted by the first preset layer of the target dense feature pyramid network, and merge it with the second feature vector extracted by the second preset layer of the target dense feature pyramid network to obtain a merged feature vector.
[0065] In one example, the target dense feature pyramid network refers to a network structure after structural improvement of a specific layer in the dense feature pyramid network. In this embodiment, the specific layer is the first preset layer and the second preset layer. In practical applications, the first preset layer may be the F4 layer of the dense feature pyramid network, and the second preset layer may be the F3 layer of the dense feature pyramid network. The first feature vector extracted from the first preset layer is used to characterize local information, and the second feature vector extracted from the second preset layer is used to characterize global information.
[0066] In an optional embodiment, upsampling a first feature vector extracted by a first preset layer of a target dense feature pyramid network and merging the first feature vector extracted by a second preset layer of the target dense feature pyramid network to obtain a merged feature vector includes:
[0067] The feature vector is merged by the following formula:
[0068]
[0069] in, The first feature vector extracted from the first preset layer of the target dense feature pyramid network, is the second feature vector extracted from the second preset layer of the target dense feature pyramid network, λ is the preset weight coefficient; f 1 is the merged feature vector.
[0070] Step 1-2, fusing the merged feature vector to obtain a fused feature vector.
[0071] In an optional implementation, the merged feature vector is fused to obtain a fused feature vector, including:
[0072] The feature vector is fused by the following formula:
[0073] F 1 =sigmoid(f 1 (Leaky ReLU(c 1×C / r ((h())))))
[0074] Among them, F 1 is the fused feature vector.
[0075] Step 1-3, based on the fused feature vector, feature extraction processing is performed through the complementary gate attention module to generate the query vehicle feature vector.
[0076] In an optional embodiment, based on the fused feature vector, feature extraction processing is performed through a complementary gate attention module to generate a query vehicle feature vector, including:
[0077] The first feature vector extracted by the first preset layer of the target dense feature pyramid network and the second feature vector extracted by the second preset layer of the target dense feature pyramid network are respectively subjected to global average pooling processing, and then input into the gated structure for OR processing, and the complementary feature vector is obtained as follows:
[0078] Feature extraction is performed using the following formula:
[0079] g 1 =OR(GAP(p),AP(q))
[0080] The feature extraction process is performed through the complementary gate attention module to generate the query vehicle feature vector as follows:
[0081] H 1 = 1 +oftmax(g 1 *GAP(F 1 ))*GAP(F 1 )
[0082] in, 1 To query the vehicle feature vector, g 1 are complementary eigenvectors.
[0083] Optionally, the feature extraction of the query vehicle image through the target dense feature pyramid network can be performed by selecting multiple high-level feature maps in the feature pyramid, recursively passing the rich semantic information in the high-level feature maps downward by dense connection, and fusing them with the feature maps of the lower layers in a specific manner.
[0084] Step S120 , based on the query vehicle feature vector, selecting one or more best matching vehicle feature vectors from a pre-established vehicle feature library.
[0085] In one embodiment, the construction of the above-mentioned pre-established vehicle feature library can first obtain vehicle video streams captured by multiple cameras, decode the vehicle video streams, extract frames and perform preprocessing operations, perform vehicle detection on the image obtained by the preprocessing operation and cut out the detected vehicle to obtain a cutout vehicle image; wherein the preprocessing operation includes at least filtering, denoising and cropping operations; feature extraction is performed on the cutout vehicle image through a pre-established vehicle feature extraction module to generate a matching vehicle feature vector; and the matching vehicle feature vector is stored in a pre-established vehicle feature library.
[0086] In actual applications, the feature extraction operation used when constructing the vehicle feature library is the same as the feature extraction method in the aforementioned step S110, and a series of processing is also performed using a pre-constructed vehicle feature extraction module, which will not be repeated here.
[0087] Furthermore, based on the query vehicle feature vector, one or more best matching vehicle feature vectors are selected from a pre-established vehicle feature library. Based on the query vehicle feature vector, a matching similarity between the query vehicle feature vector and one or more matching vehicle features in a pre-established vehicle feature library can be calculated; and then based on a pre-set similarity threshold and matching similarity, the best matching one or more matching vehicle feature vectors are selected from the pre-established vehicle feature library.
[0088] Step S130, determining a target vehicle that matches the vehicle image to be queried according to the matching vehicle features.
[0089] In one embodiment, if the matching vehicle feature is a unique feature, it is possible to determine that the target vehicle that matches the vehicle image to be queried is the matching vehicle feature; if the matching vehicle feature includes at least two features, it is possible to determine whether the at least two features belong to the same vehicle. If so, the vehicle is determined to be the target vehicle; if not, it is possible to further sort by similarity and determine that the matching vehicle feature with the greatest similarity is the target vehicle that matches the vehicle image to be queried.
[0090] Figure 2 A flowchart of a specific cross-camera vehicle re-identification method is shown, which mainly includes the following steps S1 to S6:
[0091] Step S1: First, a vehicle image is captured by multiple cameras (camera 1, camera 2, ..., camera n), and then the video stream is decoded and framed, and then pre-processed such as filtering, noise reduction, and cropping are performed, and then transmitted to the vehicle detection module;
[0092] Step S2: In the vehicle detection module, a vehicle detection algorithm is used to perform vehicle detection on the image obtained from the image preprocessing module, and the detected vehicle is cut out, and then transmitted to the vehicle feature extraction module;
[0093] Step S3: In the vehicle feature extraction module, the vehicle feature extraction algorithm proposed in the present invention is used to extract vehicle features and the extracted vehicle features are stored in a vehicle feature library as a reference for subsequent query and matching.
[0094] Step S4: The user provides a vehicle image to be queried, and features are extracted from the query image to generate a query vehicle feature vector.
[0095] Step S5: Match the query vehicle feature vector with the features in the vehicle feature library to find the feature vector with the highest similarity.
[0096] Step S6: Output the query result based on the matching result, and display the vehicle that is most similar to the query image.
[0097] The core function of the embodiment of the present application is mainly in the vehicle feature extraction part, and its specific process is as follows:
[0098] In the vehicle feature extraction module, the present invention uses Resnet50 as the network backbone and extracts features from the input vehicle image through a dense feature pyramid network. The present invention proposes an adaptive multi-scale subtle feature extraction network, which can adaptively extract the most discriminative subtle features in the vehicle, taking into account both global semantic information and local detail information, enhancing the constraints and expressions of the vehicle's significant features, and improving the accuracy of the vehicle re-identification algorithm. The structure of the network is as follows: Figure 3 As shown in Figure 2, the structure can fuse features of two different scales, and the fusion method is as follows:
[0099] (1) The feature maps from the F3 and F4 layers of the dense feature pyramid network are represented as:
[0100] (2) Upsample the features (local) of the F4 layer and then add them to the features (global) of the F3 layer to obtain f 1 ;
[0101]
[0102] (3) The fused feature F 1 It can be expressed as:
[0103] F 1 =sigmoid(f 1 (Leaky ReLU(c 1×C / r ((h())))))
[0104] Further, after obtaining the fused feature F 1 After that, the complementary gate attention module can be used to perform more accurate feature recognition, so that the adaptive multi-scale subtle feature extraction network can better learn local and global significant vehicle features to further enhance the expression ability and recognition accuracy of vehicle features. The structure of the network of the complementary gate attention module is as follows: Figure 4 As shown, the complementary gate attention module is calculated as follows:
[0105] (1) The features from the F3 and F4 layers of the dense feature pyramid network are first passed through GAP and then input into the gated structure. The output can be expressed as:
[0106] g 1 =OR(GAP(p),GAP())
[0107] (2) Output H of the complementary gate attention module 1 It can be expressed as:
[0108] H 1 = 1+softmax(g 1 *GAP(F 1 ))*GAP(F 1 )
[0109] In summary, the embodiment of the present application selects multiple high-level feature maps in the feature pyramid, uses dense connections to recursively pass down the rich semantic information in the high-level feature maps, and fuses them with the feature maps of the lower layers in a specific way. The fused feature maps take into account both global semantic information and local detail information, which can enhance the constraints and expression of vehicle significant features, improve the accuracy of the vehicle re-identification algorithm, and thus alleviate the problem that vehicles with small differences in monitoring scenes are easily misidentified. The complementary gate attention module enables the network to learn local and global significant vehicle features to further enhance the expression ability and recognition accuracy of vehicle features, and address the differences within vehicle classes and similarities between classes caused by interference factors such as occlusion and perspective changes in cross-camera scenes.
[0110] Based on the above method embodiment, the present application embodiment also provides a cross-camera vehicle re-identification system, see Figure 5 As shown, the system mainly includes the following parts:
[0111] The query feature extraction module 510 is used to extract features of the query vehicle image provided by the user through a pre-built vehicle feature extraction module to generate a query vehicle feature vector; wherein the pre-built vehicle feature extraction module is a network structure with Resnet50 as the network backbone, and is used to extract features of the query vehicle image through a target dense feature pyramid network;
[0112] A feature matching module 520, for selecting one or more best matching vehicle feature vectors in a pre-established vehicle feature library based on the query vehicle feature vector;
[0113] The vehicle feature matching module 530 is used to determine the target vehicle that matches the vehicle image to be queried based on the matching vehicle features.
[0114] In a feasible implementation manner, the query feature extraction module 510 is used to:
[0115] Inputting the vehicle image to be queried into a target dense feature pyramid network, upsampling a first feature vector extracted by a first preset layer of the target dense feature pyramid network, and merging the first feature vector extracted by a second preset layer of the target dense feature pyramid network with the second feature vector extracted by a second preset layer of the target dense feature pyramid network to obtain a merged feature vector;
[0116] Performing fusion processing on the merged feature vector to obtain a fused feature vector;
[0117] Based on the fused feature vector, feature extraction processing is performed through the complementary gate attention module to generate the query vehicle feature vector.
[0118] In a feasible implementation manner, the query feature extraction module 510 is used to:
[0119] The feature vector is merged by the following formula:
[0120]
[0121] in, The first feature vector extracted from the first preset layer of the target dense feature pyramid network, is the second feature vector extracted from the second preset layer of the target dense feature pyramid network, λ is the preset weight coefficient; f 1 is the merged feature vector.
[0122] In a feasible implementation manner, the query feature extraction module 510 is used to:
[0123] The feature vector is fused by the following formula:
[0124] F 1 =sigmoid(f 1 (Leaky ReLU(c 1×c / r ((h())))))
[0125] Among them, F 1 is the fused feature vector.
[0126] In a feasible implementation manner, the query feature extraction module 510 is used to:
[0127] The first feature vector extracted by the first preset layer of the target dense feature pyramid network and the second feature vector extracted by the second preset layer of the target dense feature pyramid network are respectively subjected to global average pooling processing, and then input into the gated structure for OR processing, and the complementary feature vector is obtained as follows:
[0128] Feature extraction is performed using the following formula:
[0129] g 1 =OR(GAP(p),AP(q))
[0130] The feature extraction process is performed through the complementary gate attention module to generate the query vehicle feature vector as follows:
[0131] H 1 = 1 +oftmax(g 1 *GAP(F 1))*GAP(F 1 )
[0132] in, 1 To query the vehicle feature vector, g 1 are complementary eigenvectors.
[0133] In a feasible implementation manner, the above system further includes: a vehicle feature library construction module, which is used to:
[0134] Obtain vehicle video streams collected by multiple cameras, decode the vehicle video streams, extract frames and perform preprocessing operations, perform vehicle detection on the images obtained by the preprocessing operations and cut out the detected vehicles to obtain cutout vehicle images; wherein the preprocessing operations at least include filtering, denoising and cropping operations;
[0135] The pre-built vehicle feature extraction module is used to extract features from the cutout vehicle image and generate a matching vehicle feature vector;
[0136] The matching vehicle feature vector is stored in a pre-established vehicle feature library.
[0137] In a feasible implementation manner, the vehicle feature matching module 530:
[0138] Based on the query vehicle feature vector, calculating a matching similarity between the query vehicle feature vector and one or more matching vehicle features in a pre-established vehicle feature library;
[0139] The best matching song or multiple matching vehicle feature vectors are selected from a pre-established vehicle feature library based on a pre-set similarity threshold and matching similarity.
[0140] The cross-camera vehicle re-identification system provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for parts not mentioned in the embodiment of the cross-camera vehicle re-identification system, reference may be made to the corresponding contents in the aforementioned cross-camera vehicle re-identification method embodiment.
[0141] The present application also provides an electronic device, such as Figure 6 As shown, it is a structural diagram of the electronic device, wherein the electronic device 100 includes a processor 61 and a memory 60, the memory 60 stores computer executable instructions that can be executed by the processor 61, and the processor 61 executes the computer executable instructions to implement any of the above-mentioned cross-camera vehicle re-identification methods.
[0142] exist Figure 6 In the illustrated embodiment, the electronic device further includes a bus 62 and a communication interface 63 , wherein the processor 61 , the communication interface 63 and the memory 60 are connected via the bus 62 .
[0143] Among them, the memory 60 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 63 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 62 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 62 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0144] The processor 61 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 61. The above processor 61 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 61 reads the information in the memory and completes the steps of the cross-camera vehicle re-identification method of the aforementioned embodiment in combination with its hardware.
[0145] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned cross-camera vehicle re-identification method. The specific implementation can be found in the aforementioned method embodiment, which will not be repeated here.
[0146] The computer program product of the cross-camera vehicle re-identification method, system and electronic device provided in the embodiments of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.
[0147] Unless otherwise specifically stated, the relative steps, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present application.
[0148] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A cross-camera vehicle re-identification method, characterized in that: The method comprises: In response to a vehicle image to be queried provided by a user, a pre-built vehicle feature extraction module is used to extract features of the vehicle image to be queried, and a query vehicle feature vector is generated; wherein the pre-built vehicle feature extraction module is a network structure with Resnet50 as a network backbone, and is used to extract features of the vehicle image to be queried through a target dense feature pyramid network; Based on the query vehicle feature vector, selecting one or more matching vehicle feature vectors that best match in a pre-established vehicle feature library; A target vehicle matching the to-be-queried vehicle image is determined according to the matching vehicle features.
2. The cross-camera vehicle re-identification method according to claim 1, characterized in that: The vehicle feature extraction module constructed in advance is used to extract features from the vehicle image to be queried, and a query vehicle feature vector is generated, including: Inputting the vehicle image to be queried into the target dense feature pyramid network, upsampling the first feature vector extracted by the first preset layer of the target dense feature pyramid network, and merging it with the second feature vector extracted by the second preset layer of the target dense feature pyramid network to obtain a merged feature vector; Performing fusion processing on the combined feature vector to obtain a fused feature vector; Based on the fused feature vector, feature extraction processing is performed through a complementary gate attention module to generate a query vehicle feature vector.
3. The cross-camera vehicle re-identification method according to claim 2, characterized in that: Upsampling a first feature vector extracted by a first preset layer of the target dense feature pyramid network and merging the first feature vector extracted by a second preset layer of the target dense feature pyramid network to obtain a merged feature vector, including: The feature vector is merged by the following formula: in, a first feature vector extracted from a first preset layer of the target dense feature pyramid network, is the second feature vector extracted from the second preset layer of the target dense feature pyramid network, λ is the preset weight coefficient; f1 is the merged feature vector.
4. The cross-camera vehicle re-identification method according to claim 2, characterized in that: The merged feature vector is fused to obtain a fused feature vector, including: The feature vector fusion is performed by the following formula: F1=sigmoid(f1(Leaky ReLU(fc 1×C / r (g(h(z)))))) Among them, F1 is the fused feature vector.
5. The cross-camera vehicle re-identification method according to claim 2, characterized in that: Based on the fused feature vector, feature extraction processing is performed through a complementary gate attention module to generate a query vehicle feature vector, including: The first feature vector extracted by the first preset layer of the target dense feature pyramid network and the second feature vector extracted by the second preset layer of the target dense feature pyramid network are respectively subjected to global average pooling processing, and then input into the gated structure for OR processing, and the complementary feature vector is obtained as follows: Feature extraction is performed using the following formula: g1=OR(GAP(p),GAP(q)) The feature extraction process is performed through the complementary gate attention module to generate the query vehicle feature vector as follows: H1=g1+softmax(g1*GAP(F1))*GAP(F1) Wherein, H1 is the query vehicle feature vector, and g1 is the complementary feature vector.
6. The cross-camera vehicle re-identification method according to claim 1, characterized in that: The construction of the pre-established vehicle feature library includes: Obtain vehicle video streams collected by multiple cameras, decode and extract frames and perform preprocessing operations on the vehicle video streams, perform vehicle detection on the images obtained by the preprocessing operations, and cut out the detected vehicles to obtain cutout vehicle images; wherein the preprocessing operations at least include filtering, denoising and cropping operations; Extracting features from the cutout vehicle image using a pre-built vehicle feature extraction module to generate a matching vehicle feature vector; The matching vehicle feature vector is stored in the pre-established vehicle feature library.
7. The cross-camera vehicle re-identification method according to claim 6, characterized in that: Based on the query vehicle feature vector, selecting one or more matching vehicle feature vectors that best match the query vehicle feature vector in a pre-established vehicle feature library includes: Based on the query vehicle feature vector, calculating a matching similarity between the query vehicle feature vector and the one or more matching vehicle features in a pre-established vehicle feature library; The best matching song or multiple matching vehicle feature vectors are selected from a pre-established vehicle feature library based on a preset similarity threshold and the matching similarity.
8. A cross-camera vehicle re-identification system, characterized in that: The system comprises: A query feature extraction module, which is used to extract features of the vehicle image to be queried provided by the user through a pre-built vehicle feature extraction module to generate a query vehicle feature vector; wherein the pre-built vehicle feature extraction module is a network structure with Resnet50 as the network backbone, and is used to extract features of the vehicle image to be queried through a target dense feature pyramid network; A feature matching module, configured to select one or more best matching vehicle feature vectors from a pre-established vehicle feature library based on the query vehicle feature vector; The vehicle feature matching module is used to determine a target vehicle that matches the vehicle image to be queried based on the matching vehicle features.
9. An electronic device, characterized in that: It includes a processor and a memory, the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the cross-camera vehicle re-identification method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to implement the cross-camera vehicle re-identification method according to any one of claims 1 to 7.