A new vehicle-road cooperation re-identification model, dataset and model training method in real vehicle scene
By using the DAID-V2XReid dataset and a cross-camera feature aggregation network, the problems of insufficient adaptability of vehicle re-identification datasets and inaccurate feature fusion were solved, achieving more comprehensive vehicle perspective acquisition and efficient re-identification results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-26
- Publication Date
- 2026-04-10
AI Technical Summary
Existing vehicle re-identification datasets are ill-suited for the autonomous driving field. Fixed camera acquisition methods are insufficient, and existing methods neglect details during feature fusion, resulting in cumbersome and inaccurate computations.
The DAID-V2XReid dataset was designed, which uses data collected jointly by vehicle-side and roadside cameras. Vehicle re-identification is performed through a cross-camera feature aggregation network, including a camera embedding module, a cross-stage feature fusion module, and a multi-directional attention module, to solve the problem of large changes in vehicle perspective in cross-camera scenarios.
It achieves more comprehensive vehicle perspective acquisition, increases background diversity, and improves the accuracy and efficiency of vehicle re-identification, reaching 58.47% mAP and 80.67% accuracy.
Smart Images
Figure CN115620245B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of intelligent networked vehicles, and particularly designs a vehicle-road cooperation re-identification model, a data set and a model training method based on a real vehicle scene. BACKGROUND
[0002] Vehicle-road cooperation is a cooperation between vehicles and infrastructure, which uses infrastructure to provide vehicles with a global perspective far beyond the current field of view and information covering blind areas, etc., so as to complete tasks such as target detection, identification and tracking, to ensure correct and safe control decisions. The vehicle-road cooperation perception technology aims to make up for the limited range and frequent occlusion in single vehicle perception through information exchange between vehicles and infrastructure, to achieve global perception of road target information. However, the research on vehicle-road cooperation perception is just starting, and there is a lot of research space. Kim et al. first proposed a multi-mode cooperative perception system, which realized front collision warning, automatic hidden obstacle avoidance and other cooperative driving. Recently, Li et al. proposed a new distillation cooperation graph to build a trainable adaptive cooperation, trying to better improve the performance and bandwidth of multi-device perception.
[0003] Vehicle re-identification (Re-ID) aims to find the same vehicle identity pictures collected by different cameras. With the continuous development of technology, it has a wide application scenario in intelligent transportation systems, and has become an essential technology to realize automatic driving. For the vehicle re-identification task, a comprehensive, reliable and fair data set is helpful to objectively evaluate the performance superiority of a vehicle re-identification algorithm, and is one of the keys to the whole task. For existing vehicle re-identification research, the re-identification problem is mainly solved from two aspects: detail capture and feature fusion. In the aspect of detail capture, a large number of additional labels (such as color, angle, license plate, etc.) are often used to focus on the distinguishing features of the vehicle. Although these methods can better focus on the small differences of the vehicle, they not only require a large amount of manual labeling, but also need to introduce a second calculation network, making the calculation cumbersome and inefficient. In the aspect of feature fusion, some innovative post-processing is often performed on the final high-level semantics. This method has a simple structure and is easy to calculate, but often ignores the importance of details, and inaccurate high-level semantics may cause serious errors in feature fusion.
[0004] With the development of the field of computer vision, a large number of vehicle re-identification data sets have emerged, such as Veri-wild, VehicleID, VeRi776, etc. The existence of these data sets has brought many excellent algorithms for vehicle re-identification tasks. However, the shortcoming is that these data sets are difficult to effectively adapt to the field of automatic driving, and the root cause lies in the fact that existing re-identification data sets are for security purposes, which makes them often use fixed cameras as a way to obtain data. SUMMARY
[0005] To solve the above problems, the application proposes a new vehicle re-identification dataset DAID-V2XReid. This dataset is based on the DAID-V2X dataset proposed by Tsinghua University in real vehicle scenarios. The data collected by the vehicle camera and the road camera are used to complete the vehicle re-identification task. The DAID-V2XReid dataset has the following advantages: (1) The vehicle camera is used as the mobile terminal, and the road camera is used as the fixed terminal. The vehicle camera can make up for the shortcomings of the fixed camera, making the same vehicle perspective more comprehensive. (2) Due to the height inconsistency of the two device cameras, the same vehicle collected even at the same perspective can have slight differences, making the dataset samples diverse. (3) The dataset is obtained in a real scene, and is collected by two devices, which can obtain a variable background and increase the background diversity.
[0006] The dataset is made by the following method:
[0007] The data matching of the road end collection device and the vehicle end collection device is completed, and the vehicle that meets the matching is framed and selected; the vehicle picture is intercepted; the vehicle ID and camera ID are respectively assigned values; the specific process is as follows:
[0008] Four pairs of high-resolution cameras are deployed at the intersection as road end collection devices, and a high-quality forward camera is configured on the vehicle as a vehicle end collection device to jointly complete data collection.
[0009] The pictures with the same vehicle in the data of the two collection devices are manually selected and assigned the same vehicle label, and the data is also labeled with camera ID. The vehicle camera ID is set to 0, and the road camera ID is set to 1. Finally, 205 matched vehicles are obtained, each group has at least 2 photos, and a total of 2556 photos.
[0010] The vehicle samples in the dataset are divided into train and gallery two sub-datasets according to the ratio of 2:1, and one picture under each camera ID in the gallery dataset is randomly selected to generate the query dataset; the train dataset is used for training, and the gallery and query datasets are used for testing.
[0011] Meanwhile, the application also proposes a new network model for vehicle re-identification: cross-lens feature aggregation network, which mainly contains three modules and does not require any additional labeling. First, in order to deal with the problem of different camera angles, the application proposes a camera embedding module to embed camera information into the backbone network, which realizes the simple grouping of different cameras. Then, in order to avoid the leakage of high-order semantic local information, a cross-stage feature fusion module is proposed to fuse high-order semantics and low-order semantics, which realizes the distinction between background and vehicle. Finally, a multi-direction attention module is proposed to obtain attention features in spatial position for further refinement for the final re-identification task. The specific modules are as follows:
[0012] The camera embedding module: assuming that there are N cameras, denoted as ID r ,r∈[1,N] and using a randomly generated sequence for module initialization, the camera embedding is obtained after initialization Where A=HxW, H and W represent the height and width of the picture corresponding to the current V0 channel.
[0013] For a camera ID r , the corresponding camera embedding feature of the photo img i can be expressed as The camera embedding feature is input into the original resnet50 backbone to obtain the expression
[0014]
[0015] Where V0 is a low-order feature in the original resnet50 backbone, and λ is a hyperparameter for balancing the camera embedding module.
[0016] Further, the cross-stage feature fusion module: two features of low-order semantics V0 and high-order semantics V1 are obtained, which are fused to realize the final feature extraction, and the specific fusion process is as follows:
[0017] First, the two features are mapped to the same space, and then they are fused; for low-order semantics, the height and width dimensions are multiplied first, and the dimension after multiplication is changed through linear transformation, denoted as V 01 ; the high-order semantics is reduced in channel by 1x1 convolution, denoted as V 11 , and the same number of channels as the low-order semantics is obtained, and the two vectors are spliced into one whole, defined as
[0018] V′ C1 =C(V 01 ,V 11 ),
[0019] Where C represents the cascade operation;
[0020] After the splicing is completed, V′ C1 Perform another channel transformation using a 1×1 convolution to give it V 11 With the same number of channels, we get V′ C2 Then, V′ C2 With higher-order semantics V 11 Add them together to get V2, thus completing feature fusion:
[0021] V2=C(conv(V′ C1 ),V 11 ).
[0022] Furthermore, the multi-directional attention module decomposes the global pooling layer to obtain positional encodings along the horizontal and vertical directions, as well as global encodings of the overall features. Then, it encodes spatial information in the horizontal, vertical, and offset directions, with the following expression:
[0023]
[0024]
[0025]
[0026] Where V2 is the output feature of the camera embedding module;
[0027] The above three types of encoding are aggregated along the three directions of horizontal, vertical and information deviation to obtain three perceptual feature maps, so that the features can be accurately encoded along their respective attention directions to obtain the position information points of the three-dimensional model.
[0028] After spatial information encoding, an attention mechanism is used to aggregate object features along the channel dimension. First, to reduce model complexity, the features are dimensionality-reduced to obtain more discriminative channels. Specifically,
[0029] First, the spatial encoded features are concatenated to obtain a multi-feature fusion representation vector. Then, a set of 1×1 convolutions is used to reduce the dimensionality of the standard vector. The calculation process is shown below:
[0030]
[0031] In the formula, C is a concatenation operation, representing concatenation along the spatial dimensions h, w, and I respectively. This is an intermediate feature map, and r is the compression ratio that controls the channel size. To reduce computation, a smaller compression ratio r is chosen to reduce the number of channels; therefore, r = 32.
[0032] Subsequently, the module splits f into three separate tensors along the resulting spatial dimension And Using three 1x1 convolutions to transform them into tensors with the same number of input V2 channels, respectively, to correlate the channels of the three features with each other, the formula is as follows:
[0033]
[0034] Where sigma represents the sigmoid function,
[0035] Finally, in order to make full use of the spatial encoding added in the feature, make the final feature have a more correct positioning, obtain the attention feature positioning, fuse the three tensors to obtain the output V, complete the final feature extraction:
[0036] V=(g w xg I ) xg h xV2
[0037] The feature V is linearly transformed and passed into the loss function to calculate the loss.
[0038] The application also proposes a test method for the above-mentioned new vehicle-road cooperative re-identification model in a real vehicle scene, which uses the above-mentioned data set for training and testing, randomly crops the input picture to 256x256, uses random erasing, horizontal flipping and other methods for data enhancement, sets the batch size to 48, uses the cross-entropy loss function and the ternary loss function to train the network for 80 epochs, uses the SGD optimizer, sets the momentum of the optimizer to 0.9 and the weight decay to 1e-4, the learning rate is from 7.7e-5 to 1e-2 in the first 20 epochs, the original resnet50 backbone is frozen during the training stage, the learning rate is kept at 1e-2 until the 60th, and the rest of the time uses the cosine annealing algorithm to slowly reduce the learning rate to 7.7e-5.
[0039] Further, the training and testing are based on the Pytorch1.8 deep learning framework and are tested on an NVIDIA RTX2080Ti GPU, and the training time is 20 minutes.
[0040] Further, it further includes visualization processing of test results.
[0041] The application has the following beneficial effects:
[0042] 1. In order to solve the problem of lack of data set, the application establishes a vehicle-road cooperative vehicle re-identification data set-DAIR-V2XReID in a real vehicle scene.
[0043] 2. In order to solve the vehicle re-identification problem, the cross-lens feature aggregation network is designed, the problems such as vehicle perspective transformation in cross-lens scene are solved, and through the cooperation of the camera embedding module, the cross-stage feature fusion module and the multi-direction attention module, better vehicle feature matching effect is brought.
[0044] 3. In order to prove the performance of the model, the DAIR-V2XReID dataset is put into the cross-lens feature aggregation network for experiment, the performance reaches the maximum, and the mAP is 58.47%. At the same time, in order to verify the generalization of the model, experiments are also carried out on the VeRi776 dataset, and good accuracy is also obtained, reaching the accuracy of 80.67%. BRIEF DESCRIPTION OF DRAWINGS
[0045] Figure 1 . Algorithm flowchart of the present application
[0046] Figure 2 . Cross-stage feature fusion module architecture diagram of the present application
[0047] Figure 3 . Multi-direction attention module architecture diagram of the present application
[0048] Figure 4 . Model making flowchart of the present application
[0049] Figure 5 . Heat map visualization result of the present application on the dataset
[0050] Figure 6 Pre-rank5 comparison chart of the present application
[0051] Figure 7 Matching schematic diagram of the present application DETAILED DESCRIPTION
[0052] The application provides a new vehicle-road cooperation re-identification model in a real vehicle scene, a data set and a model training method, and is based on a vehicle-road cooperation DAID-V2X data set in a real vehicle scene.
[0053] The application will be further described below with reference to the drawings.
[0054] As shown in the overall operation flow of the algorithm of the application. Figure 1 Firstly, the camera embedding module is added to the original resnet50 backbone for feature extraction. Secondly, a low-order feature of the original resnet50 backbone is saved, and the obtained global feature is fused to complete the cross-stage feature fusion. Finally, the obtained feature is subjected to multi-directional attention feature acquisition.
[0055] The application first explains the related terms:
[0056] ResNet-50 is a general deep learning network constructed based on a residual structure, which solves the problem of gradient explosion / disappearance caused by common linear stacking through a skip connection, and improves the feature extraction ability of the network.
[0057] The specific implementation process of the application includes the following:
[0058] Step 1: making a data set
[0059] The DAIR-V2XReid data set making process is as follows: (a) complete data matching of two devices, and frame out the vehicles meeting the matching. (b) perform vehicle picture cutting. (c) respectively assign values to the vehicle ID and camera ID. The orange frame represents the vehicle camera, and the green frame represents the road end camera. The specific steps are shown in the accompanying drawings. Figure 2
[0060] The data set is collected at 28 intersections in the Beijing Advanced Autonomous Driving Demonstration Zone, and 4 pairs of high-resolution cameras are deployed at each intersection as road end devices. A high-quality forward-looking camera is also configured on the vehicle as a vehicle end device to complete the collection together.
[0061] The data set first manually selects pictures with the same vehicle from the data of the two devices, assigns the same vehicle label, and also annotates the camera ID. The vehicle camera ID is set to 0, and the road end camera ID is set to 1. Finally, the model obtains 205 matched vehicles, each with at least 2 photos, totaling 2556. Subsequently, in accordance with the existing vehicle re-identification data set sample distribution convention, the vehicle samples are divided into train and gallery two sub-datasets in a ratio of 2:1, and one picture under each camera ID in the gallery dataset is randomly selected to generate the query dataset. The train dataset is used for training, and the gallery and query datasets are used for testing.
[0062] Before publication, the entire data set has been obscured for all possible privacy-infringing information, including license plates, faces, road signs, etc., to maximize the protection of public privacy.
[0063] Step 2: Design camera embedding module
[0064] The data set has N cameras, denoted as ID r r∈[1,N]. This module uses a randomly generated sequence for module initialization, and after initialization, the camera embedding is where A = H x W, H and W represent the height and width of the picture corresponding to the current V0 channel.
[0065] Therefore, for a camera ID r r, the corresponding camera embedding feature of the photo img i can be represented as Finally, the camera embedding feature is input into the Backbone, and the expression is
[0066]
[0067] where V0 is a low-order feature in the original resnet50 backbone, and λ is a hyperparameter that balances the camera embedding module.
[0068] Step 3: Design cross-stage feature fusion module
[0069] Theoretically, the background feature and the foreground feature are significantly different, so that the model should have good fitting ability in the case of sparse samples. But in actual use, although the high-level semantics has the advantage of clear features, it will lead to the mixing of environmental information and the loss of vehicle information, resulting in inaccurate positioning of the final target features. While the low-level semantics, although the semantic features are not clear enough, contains a large amount of location information and background features, which is just the opposite of the high-level semantics. Therefore, the present application proposes a cross-stage feature fusion module to realize the fusion of low-level semantics and high-level semantics, the obvious distinction between background and target vehicle, solve the possible of high-level semantics into non-key area and improve the anti-interference ability of the model. The specific fusion process is shown in the accompanying Figure 3
[0070] The module first obtains two features of low-level semantics V0 and high-level semantics V1 in the model, but since their sizes are different, they need to be fused to realize the final feature extraction. The existing fusion methods can be divided into Concat and Add two categories, but these two methods often ignore the structural relationship between the two features, which may lead to confusion in feature hierarchy, making it difficult to effectively complete the later re-identification task. In order to solve this problem, the module first maps the two features to the same space, and then fuses them. For low-level semantics, first multiply the height and width dimensions, change the dimension after multiplication through linear transformation, denoted as V 01 . The high-level semantics is reduced in channel dimension by 1x1 convolution, denoted as V 11 , which obtains the same number of channels as the low-level semantics and reduces the amount of calculation. The two vectors are connected into one whole, defined as
[0071] V′ C1 =C(V 01 ,V 11 ),
[0072] Where C represents the concatenation operation.
[0073] From experience, in high-level semantics, although the perception ability of details is not enough, the obtained feature noise is less and the characteristics are more clear, so the module selects high-level features as the main in fusion to ensure the advantage of high-level semantics. After splicing, the module changes the channel of V′ C1 again through 1x1 convolution, so that it has the same number of channels as V 11 , and obtains V′ C2 . Then, V′ C2 is added to high-level semantics V 11 to obtain V2, which fully embodies the advantage of high-level semantics.
[0074] V2=C(conv(V′ C1 ),V11 )
[0075] At this time, the cross-stage feature fusion module is completed, and the current feature not only fuses the background information of low-order semantics, but also does not lose the clear feature of high-order semantics, and combines the background and scene information.
[0076] Step 4: Designing a multi-direction attention module
[0077] In the final feature extraction, the global encoding is usually performed by using the method of global pooling layer to capture the global information, but this will ignore the position information of the feature. For the vehicle re-identification task, the vehicle to be re-identified is a three-dimensional model, and the spatial information plays a key role in obtaining the three-dimensional structure, so the module needs to add the necessary spatial information, so that more accurate vehicle appearance features can be obtained. The specific process of the module is shown in the following figure: Figure 4
[0078] Firstly, the module decomposes the global pooling layer, respectively calculates the horizontal direction, vertical direction and information deviation of the feature map, and obtains the position encoding along the horizontal direction, vertical direction and global encoding of the overall feature. Then, the spatial information is encoded in the horizontal, vertical and deviation directions, and the expression is as follows:
[0079]
[0080]
[0081]
[0082] Wherein, V2 is the output feature of the camera embedding module. W, H respectively represent the dimensions of the horizontal and vertical directions.
[0083] The above three encodings are respectively aggregated along the horizontal, vertical and information deviation directions to obtain three perception feature maps. In this way, the feature can be accurately encoded in the spatial information along the respective attention direction, which is helpful for positioning the more interesting feature and realizing the acquisition of the position information point of the three-dimensional model.
[0084] After completing the spatial information encoding, the attention mechanism is used to realize the aggregation of the object feature in the channel dimension. Firstly, in order to reduce the complexity of the model, the feature is reduced to obtain more distinctive channels. Specifically, the spatial encoding feature is first spliced to obtain a multi-feature fusion representation vector, and then a group of 1x1 convolution is used to realize the dimension reduction of the standard vector. The related calculation process is shown as follows:
[0085]
[0086] Wherein, in the formula, C is a series operation, representing series connection along the horizontal dimension direction h, the vertical dimension direction w and the information bias dimension direction I spatial dimension direction respectively, is the intermediate feature map, and r is a compression ratio for controlling the channel size. In order to reduce the amount of calculation, a smaller compression ratio r is selected to reduce the number of channels, and for this purpose, r = 32.
[0087] Subsequently, the module divides f into three separate tensors along the obtained spatial dimension direction and Using three 1x1 convolutions, it is respectively transformed into a tensor with the same number of input V2 channels, so that the channels of the three features are related to each other, and the encoding g of the h, w and I dimension directions is obtained h , g w , g I , the formula is as follows:
[0088]
[0089] Wherein, sigma represents the sigmoid function, conv is the convolution change, f h , f w , f I respectively represent the tensors of the three directions.
[0090] Finally, in order to make full use of the spatial encoding added in the feature, so that the final feature has a more correct positioning, obtain the attention feature positioning, fuse the three tensors, and the output V can be written as:
[0091] V = (g w x g I ) x g h x V2, # (9)
[0092] At this time, the method completes the final feature extraction, and the feature V is transmitted into the loss function after linear transformation, and the loss is calculated.
[0093] Step 5: DAIR-V2XReid dataset is transmitted into the network model of the application for training and test verification.
[0094] The input picture is randomly cropped to 256x256, and data augmentation is performed using random erasing, horizontal flipping, etc. The batch size is set to 48, and the network is trained for 80 epochs using cross-entropy loss function and triplet loss function. The SGD optimizer is used, and the momentum of the optimizer is set to 0.9 and the weight decay is set to 1e-4. The learning rate is from 7.7e-5 to 1e-2 in the first 20 epochs, and the original resnet50 backbone is frozen during the training stage. The learning rate is kept at 1e-2 until the 60th, and the rest of the time uses the cosine annealing algorithm to slowly decrease the learning rate to 7.7e-5.
[0095] The present application is based on the Pytorch1.8 deep learning framework and is tested on an NVIDIA RTX 2080Ti GPU. The DAIR-V2XReid dataset is trained using 1 GPU, and the training time is 20 minutes.
[0096] Step 6: Visualization processing
[0097] Module performance visualization. In order to intuitively understand the learning of the method, visualization is performed on the VeRi776 and DAIR-V2XReid datasets. As shown in FIG. 1, the results show the original image, the baseline heat map and the heat map of the method, respectively. The results show that the method can better focus on the vehicle, avoid the interference of background information, encourage the module to pay attention to the information of the vehicle that is more discriminative (such as car lights, car windows, etc.), and does not lose the exploration of the overall information. Figure 5 As can be seen, the results show the original image, the baseline heat map and the heat map of the method, respectively. The results show that the method can better focus on the vehicle, avoid the interference of background information, encourage the module to pay attention to the information of the vehicle that is more discriminative (such as car lights, car windows, etc.), and does not lose the exploration of the overall information.
[0098] Retrieval result visualization. In order to verify the effectiveness of the method, the retrieval results of different methods on samples are visualized in FIG. 2. Figure 6 As can be seen from the figure, in the baseline, vehicles with similar appearances are difficult for the model to distinguish. However, since the method of the present application allows it to better focus on local information, it can better extract discriminative information and distinguish vehicles with similar appearances.
[0099] Step 7: Test phase
[0100] In order to verify the effectiveness of the method, it is compared with the baseline. The mAP obtained by putting it into the baseline is 51.44%, and the rank-1 is 48.12%. In the method, the mAP obtained is 58.47%, and the rank-1 is 59.40%, which shows the effectiveness of the present application.
[0101] The above detailed description merely illustrates preferred and possible embodiments of the application, and is not intended to limit the scope of the application. Any equivalent variations or modifications of the above described embodiments are included within the scope of the present application.
Claims
1. A new vehicle-road cooperation re-identification model in a real vehicle scene, characterized in that, Comprise: A camera embedding module, a cross-stage feature fusion module and a multi-direction attention module; The camera embedding module is to embed camera information into the backbone network by using the camera label in the data set to solve the problem of different camera angles, and to realize simple grouping of different cameras; The cross-stage feature fusion module is used to prevent the leakage of high-order semantic local information, fuse high-order semantics and low-order semantics, and realize the distinction between background and vehicles; The multi-direction attention module obtains attention features in spatial positions, further refines them, and is used for the final re-identification task; The camera embedding module: assuming there are N cameras, denoted as , using a randomly generated sequence for module initialization, after initialization, the camera embedding is obtained as , where , H and W represent the height and width of the current channel corresponding image; For a certain camera The photo taken The corresponding camera embedding feature can be expressed as The camera embedding feature Input into the Backbone, and the expression is wherein is a low-order feature in Backbone, is a hyperparameter balancing the camera embedding module; The cross-stage feature fusion module: acquire low-order semantics and high-order semantics Two features, fuse them to realize the final feature extraction, the specific fusion process: First, the two features are mapped into the same space, and then they are fused; for low-order semantics, the height and width dimensions are multiplied, and the dimension after multiplication is changed through linear transformation, denoted as ; high-order semantics uses 1x1 convolution for channel dimension reduction, denoted as , the same number of channels as low-order semantics is obtained, and the two vectors are spliced into one whole, defined as Where C represents a series operation; After splicing is completed, the channel transformation is performed again through 1x1 convolution, so that it has the same number of channels, and , then, the is added to the high-order semantics , to obtain , and the feature fusion is completed: ； The multi-direction attention module: decomposes the global pooling layer to obtain position encoding along the horizontal direction, vertical direction and global encoding of the overall feature; then, the spatial information is encoded in the horizontal, vertical and deviation directions, and the expression is: wherein, is the output characteristic of the CFF; Three kinds of encoding are respectively aggregated along the horizontal, vertical and information deviation directions to obtain three perception feature maps, so that the features can be accurately encoded in the spatial information along the respective attention direction, and the three-dimensional model position information point is obtained; After completing the spatial information encoding, the attention mechanism is used to realize the aggregation of object features in the channel dimension; first, in order to reduce the complexity of the model, the features are reduced to obtain more distinctive channels, specifically, First, the spatial encoded features are concatenated to obtain a multi-feature fusion representation vector, and then a set of... The convolution is used to reduce the dimensionality of the standard vector, and the calculation process is as follows: wherein, in the formula is a series operation, representing series along the h, w, I spatial dimension direction respectively, is an intermediate feature map, r is a compression ratio for controlling the channel size, in order to reduce the amount of calculation, a smaller compression ratio r is selected to reduce the channel number, for this reason, r=32; Subsequently, the module splits the obtained spatial dimension into three separate tensors , , and along the direction of the spatial dimension using three 1x1 convolutions to transform them into tensors with the same number of channels as the input , making the channels of the three features interdependent, as shown in the following equations: wherein, denotes a sigmoid function, Finally, in order to make full use of the space coding added in the feature, make the final feature have more correct positioning, obtain attention feature positioning, fuse the three tensors to obtain the output , complete the final feature extraction:
2. The new vehicle-road cooperation recognition model in a real vehicle scene according to claim 1, characterized in that, Also included is the feature Through a linear transformation, the incoming loss function is passed into, which computes the loss.
3. The method of any one of claims 1-2, wherein, The data set of claim 1 or 2 is used for training and testing, the input picture is randomly cropped to 256x256, random erasing, horizontal flipping and other methods are used for data enhancement, the batch size is set to 48, the cross-entropy loss function and the ternary loss function are used for 80 epochs training network, the SGD optimizer is used, the momentum of the optimizer is set to 0.9, the weight decay is 1e-4, the learning rate is from 7.7e-5 to 1e-2 in the first 20 epochs, the backbone is frozen in the training stage, the learning rate is kept at 1e-2 until the 60th, and the rest of the time uses the cosine annealing algorithm to make the learning rate slowly decrease to 7.7e-5.
4. The method according to claim 3, wherein, The training and testing are based on the Pytorch1.8 deep learning framework and are tested on an NVIDIA RTX 2080Ti GPU, and the training takes 20 minutes.
5. The method of claim 3, wherein the method is a new vehicle-road cooperative recognition model testing method in a real vehicle scene, characterized by, It also includes visualization processing of test results.