Vehicle fine granularity detection method based on three-dimensional grid and YOLOv11 transfer learning

Abstract

The invention discloses a vehicle fine granularity detection method based on a three-dimensional grid and YOLOv11 transfer learning. The method mainly comprises three components: a source domain prediction network structure, a target domain prediction network structure and a transfer learning module. Wherein the source domain prediction network and the target domain prediction network are dual-channel deep networks fusing two-dimensional images and three-dimensional grids, and efficient migration of source domain knowledge in a target domain is realized by aligning feature distribution of the source domain and the target domain. Through deep fusion of three-dimensional grid information and two-dimensional image features, the method can maintain robust detection performance under adverse conditions of vehicle attitude change, illumination interference, shielding and the like, can effectively reduce large-scale data annotation and training cost, improves the rapid adaptation capability of the model in a new scene or a new vehicle type, and improves the robustness of the model. And a high-precision, extensible and rapid-iteration fine-grained identification and detection solution is provided for intelligent traffic monitoring, unmanned driving perception, military equipment identification and digital twin systems.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target recognition and detection, and particularly relates to a vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning. BACKGROUND

[0002] With the rapid development of intelligent transportation, unmanned driving, and military equipment perception, vehicle fine-grained detection technology plays an increasingly important role in urban traffic management, automatic driving navigation, intelligent monitoring, and equipment support. Existing vehicle recognition and detection methods mostly rely on deep learning detection networks for two-dimensional images, such as the YOLO series, Faster R-CNN, etc. Such methods have advantages in terms of simple structure, real-time performance, etc., but they are less sensitive to changes in light, posture, occlusion, and fine-grained differences in vehicles under different scenarios, making it difficult to meet the actual demand for recognition accuracy in complex environments, especially in scenarios where the differences in vehicle categories, models, or structures are relatively subtle.

[0003] In recent years, the continuous development of three-dimensional perception technology enables three-dimensional grid data to more realistically and comprehensively describe the geometric structure and spatial form of vehicles. Three-dimensional grids not only contain the surface texture of vehicles, but also capture the volume information and local structural features of vehicles under different perspectives, providing a new approach to fine-grained vehicle recognition. However, three-dimensional data processing usually requires complex network structures and large computing resources, and there are significant feature space differences between two-dimensional image data. How to effectively integrate three-dimensional geometric information with two-dimensional visual information remains a technical difficulty in current research.

[0004] In addition, for new vehicle models, new environments, or new application scenarios, existing vehicle recognition and detection systems often need a large amount of labeled data for retraining, resulting in insufficient model transferability and high deployment costs, which limits the promotion of recognition and detection technology in cross-scenario applications. Transfer learning, as a technology that enables knowledge sharing between source and target domains, provides an important way to address high labeling costs and large domain distribution differences. However, current research on transfer learning that simultaneously integrates two-dimensional images and three-dimensional grids is still in its infancy, and there is still a lack of effective solutions that consider real-time performance, robustness, and cross-domain adaptability.

[0005] Therefore, there is an urgent need for a fine-grained vehicle recognition transfer learning method that can fully utilize two-dimensional image and three-dimensional grid information, combine efficient target detection networks with graph convolution feature modeling, to improve the accuracy and cross-domain adaptability of vehicle recognition and detection in complex environments, and meet the urgent demand for high-reliability and high-precision vehicle recognition in intelligent transportation, unmanned driving, and military equipment applications.

[0006] However, the prior art still has the following technical difficulties in vehicle recognition detection and transfer learning application: (1) Limitations of two-dimensional image features. Existing vehicle detection methods based on two-dimensional images mainly rely on the texture, contour and other planar features of vehicle appearance. Although high recognition accuracy can be achieved in scenes with stable lighting conditions and small changes in viewing angle, in environments with varying vehicle poses, complex lighting, low illumination at night, partial occlusion, etc., two-dimensional features are easily disturbed, leading to unstable detection results and making it difficult to accurately distinguish vehicle models, local components and details.

[0007] (2) Difficulty in cross-modal fusion of three-dimensional mesh and two-dimensional image features. Three-dimensional mesh can provide complete geometric structure and spatial form information, but the data has high dimension and complex topological structure, and the feature distribution is significantly different from that of two-dimensional images. Existing methods often require additional network design and high computational overhead in feature extraction and fusion, which prevents two-dimensional and three-dimensional information from complementing each other and affects overall detection accuracy.

[0008] (3) Adaptability challenges of transfer learning in cross-domain applications. There are differences in data distribution between different application scenarios, such as changes in vehicle models, differences in shooting devices, environmental noise, etc., which leads to a significant decrease in recognition performance when the source domain training model is directly applied to the target domain. In addition, existing transfer learning methods lack mature feature alignment and distribution adaptation strategies when dealing with multi-modal inputs of two-dimensional images and three-dimensional meshes, limiting their application in practical deployment. To overcome the shortcomings of the prior art, the present application provides a vehicle fine-grained detection method based on three-dimensional mesh and YOLOv11 transfer learning, which mainly includes three components: source domain prediction network structure, target domain prediction network structure, and transfer learning module. The source domain prediction network and the target domain prediction network are both dual-channel deep networks that fuse two-dimensional image and three-dimensional mesh features. By aligning the feature distributions of the source domain and the target domain, the source domain knowledge is efficiently transferred to the target domain. By deeply fusing three-dimensional mesh information and two-dimensional image features, this method can maintain robust detection performance under adverse conditions such as changes in vehicle pose, lighting interference, and occlusion, while effectively reducing the cost of large-scale data labeling and training, and improving the model's ability to quickly adapt to new scenarios or new vehicle models. It provides a high-precision, scalable, and fast-iterative fine-grained recognition and detection solution for intelligent traffic monitoring, autonomous driving perception, military equipment identification, and digital twin systems.

[0009] The technical solution adopted by the application to solve its technical problems is as follows: Step 1: Construction and training of source domain prediction network structure; The source domain prediction network adopts a dual-channel deep network that fuses two-dimensional images and three-dimensional grids. The two-dimensional vehicle images of the source domain are input into a YOLOv11 subnetwork to extract texture and appearance features, and the three-dimensional grids of the source domain are input into a graph convolution subnetwork to extract geometric and topological features. The two types of features are jointly optimized and trained to form a cross-domain recognition base model. Step 2: Establishment of the target domain prediction network structure; The target domain prediction network follows the dual-channel structure of the YOLOv11 plus graph convolution subnetwork of the source domain prediction network. In this network structure, the target domain image and grid data are input separately to extract key features in the target environment. Through weight fine-tuning and hierarchical optimization, the network is used to realize vehicle recognition and fine-grained attribute prediction in the target scene. Step 3: Optimization strategy of the transfer learning module; The source domain training weights are transferred to the target domain, combined with the domain adaptation strategy to reduce the feature distribution difference, and optimized through the joint loss to improve the generalization performance and recognition accuracy of the model in the target domain.

[0010] Preferably, the source domain prediction network structure includes a YOLOv11 subnetwork for processing two-dimensional image data and a graph convolution subnetwork for processing three-dimensional grid data. The target domain prediction network structure is consistent with the source domain structure and also consists of a YOLOv11 subnetwork and a graph convolution subnetwork.

[0011] Preferably, the YOLOv11 subnetwork is responsible for extracting the texture, contour, and fine-grained appearance features of the vehicle image, the graph convolution subnetwork is used to extract the geometric shape and spatial structure features of the three-dimensional grid, and the multi-modal feature interaction is performed in the feature fusion layer. Through joint training on the source domain dataset, the depth fusion of two-dimensional and three-dimensional features and the learning of preliminary recognition ability are realized.

[0012] Preferably, the target domain prediction network loads the weight parameters obtained by training the source domain at the initialization time, and fine-tunes them combined with the target domain dataset to ensure that the network can quickly adapt to the new data distribution and environmental characteristics.

[0013] Preferably, the transfer learning module adopts a joint strategy of parameter sharing, adversarial feature alignment, and adaptive loss function. On the one hand, parameter sharing preserves the recognition ability of the source domain network; on the other hand, adversarial feature alignment reduces the feature distribution difference between the source domain and the target domain; at the same time, the adaptive loss function dynamically adjusts the optimization direction according to the characteristics of the target domain samples, thereby realizing efficient transfer of source domain knowledge and continuous improvement of target domain recognition and detection performance.

[0014] Preferably, the graph convolution subnetwork consists of the following hierarchical structure: Graph Convolutional Network layer: extracts the geometric shape, spatial topology, and local structure features of the vehicle from three-dimensional grid data, providing high-dimensional representation for subsequent identification; LayerNorm layer: normalizes the network output to reduce the difficulty of neural network training, accelerate the convergence process, and improve the training stability; PReLU activation function layer: introduces non-linear mapping capability to improve the network's fitting ability for complex features, thereby enhancing the recognition effect of vehicle fine-grained features.

[0015] Preferably, the YOLOv11 subnetwork and the graph convolutional subnetwork are fused through implicit feature splicing; the two-dimensional image features extracted by the YOLOv11 subnetwork and the three-dimensional grid features extracted by the graph convolutional subnetwork are spliced in the corresponding dimensions in the feature space to realize implicit integration of information.

[0016] Preferably, the transfer learning module training is completed in two steps: Source domain training stage: input the source domain dataset into the source domain prediction network to train the YOLOv11 subnetwork and the graph convolutional subnetwork, respectively, and optimize the network parameters through a joint loss function to obtain initial weights with high precision recognition ability in the source domain; Target domain migration stage: migrate the weights obtained by the source domain training to the target domain prediction network for fine-tuning on the target domain data. Through domain adaptation and joint loss optimization strategies, the feature distribution difference between the source domain and the target domain is reduced, the efficient migration of source domain knowledge to the target domain is realized, and the vehicle recognition detection precision and cross-scene generalization ability of the target domain are improved.

[0017] Preferably, the vehicle fine-grained detection uses the following loss function: The bounding box regression loss function is GIOU loss function The mathematical expression is as follows: (1) Wherein represents the area of the real box, represents the area of the predicted box, represents the minimum circumscribed rectangle area of the union of the two boxes, represents the intersection over union of the real box and the predicted box, represents the area Subtract the area ; The Focal Loss confidence loss function is used to alleviate the training bias caused by the imbalance between positive and negative samples, and the mathematical expression is as follows: (2) In the formula, Indicates the first One sample, Indicates the first The true value of each sample Indicates the first Predicted values ​​for each sample; Using classification loss function The formula is as follows: (3) In the formula, Indicates the first One sample, Indicates the first The true value of each sample Indicates the first Predicted values ​​for each sample; In the feature learning part of the source domain training phase, the cross-channel loss function is used. This is used to constrain the correlation and information complementarity between different feature channels, and its mathematical expression is as follows: (4) In the formula, This represents the discriminative branch loss function. Represents the diversity branch loss function. The gain ratio of the diversity branch loss function; Discriminating branch loss function The formula is expressed as follows: (5) In the formula, Represents the cross-entropy loss function. Indicates a real-world tag. Indicates the number of categories. The formula is expressed as follows: (6) In the formula, This represents cross-class max pooling, used to extract the maximum value from all class responses at each spatial location. This indicates category-level weighted aggregation, used to weight the feature channels of each category using a weight vector. Represents the eigenvector. and These represent the width and height of the feature map, respectively. This represents the number of feature channels corresponding to each category. The formula is expressed as follows: (7) In the formula, a mask consisting of 0 and 1; diversity branch loss function is expressed as: (8) Preferably, the transfer learning module adopts a domain adaptation transfer learning method, and the loss function used is expressed as:

[0018] wherein, represents the encoder output hidden feature of the source domain prediction network, represents the encoder output hidden feature of the target domain prediction network, represents a hyperparameter, represents the training parameter of the source domain prediction network, represents the training parameter of the target domain prediction network, represents the encoder output hidden feature of the target domain prediction network storing the negative sample, is expressed as:

[0019] wherein, represents a trainable convolutional neural network, represents the input of the convolutional neural network, represents the training parameter of the convolutional neural network.

[0020] The beneficial effects of the present application are as follows: (1) The present application improves the recognition detection precision and robustness. By introducing a graph convolution subnetwork to extract features from a three-dimensional grid and implicitly splicing with YOLOv11 two-dimensional image features, the representation ability of vehicle geometric structure and appearance texture is enhanced, thereby realizing higher precision fine-grained recognition in complex environments such as pose changes, light interference and local occlusion.

[0021] (2) The present application improves the cross-scene transfer generalization ability. A two-step transfer learning training strategy is adopted to effectively transfer the source domain model knowledge to the target domain, and a domain adaptation strategy is combined to realize the alignment of source domain and target domain feature distribution, significantly enhancing the generalization ability of the model in different scenes.

[0022] (3) The present application effectively reduces the labeling and training cost. Through the transfer learning and target domain fine-tuning method, the dependence on large-scale labeled data of the target domain is reduced, while maintaining high recognition detection performance, which is conducive to rapid deployment and application in new environments, new types or data scarce scenes. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1A composition schematic diagram of a vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning of the present application.

[0024] Figure 2 An input and output structure schematic diagram of the transfer learning method constructed by the present application.

[0025] Figure 3 A vehicle fine-grained detection network architecture schematic diagram based on three-dimensional grid and YOLOv11 transfer learning of the source domain and the target domain in the present application.

[0026] Figure 4 A transfer learning module structure diagram in the present application.

[0027] Figure 5 The recognition detection results of the embodiment of the present application on the source domain and the target domain.

[0028] Figure 6 The fine-grained detection results of the embodiment of the present application on the source domain and the target domain. DETAILED DESCRIPTION

[0029] The present application will be further described below in combination with the drawings and embodiments.

[0030] In order to overcome the problems in the prior art such as that the two-dimensional image recognition is susceptible to environmental interference, the three-dimensional grid and two-dimensional image feature fusion is difficult, and the transfer learning has insufficient adaptability in cross-domain application, the present application proposes a vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning, so as to realize efficient fusion of two-dimensional image and three-dimensional grid multi-modal features and cross-domain knowledge transfer, and improve vehicle recognition detection precision, robustness and cross-scene adaptability in complex environment.

[0031] In order to realize the above functions, the present application provides a vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning, which specifically comprises: Construction and training of source domain prediction network structure: input the source domain two-dimensional vehicle image into the YOLOv11 subnetwork to extract texture and appearance features, and input the source domain three-dimensional grid into the graph convolution subnetwork to extract geometric and topological features; the two types of features are respectively trained by joint optimization, which is used to form a robust cross-domain recognition base model; Establishment of target domain prediction network structure: input the target domain image and grid data into the YOLOv11 and graph convolution subnetworks with shared weights with the source domain respectively to extract key features in the target environment; through weight fine-tuning and hierarchical optimization, the vehicle recognition and fine-grained attribute prediction in the target scene are realized; Optimization strategy of transfer learning module: migrate source domain training weights to target domain, combine domain adaptation strategy to reduce feature distribution difference, and optimize through joint loss, to improve the generalization performance and recognition accuracy of the model in the target domain.

[0032] The source domain prediction network structure includes a YOLOv11 subnetwork for processing two-dimensional image data and a graph convolution subnetwork for processing three-dimensional grid data. The target domain prediction network structure is consistent with the source domain structure, also composed of YOLOv11 subnetwork and graph convolution subnetwork.

[0033] The YOLOv11 subnetwork is responsible for extracting the texture, contour and fine-grained appearance features of the vehicle image, and the graph convolution subnetwork is used to extract the geometric shape and spatial structure features of the three-dimensional grid, and the multi-modal feature interaction is performed in the feature fusion layer. Through joint training on the source domain dataset, the deep fusion of two-dimensional and three-dimensional features and the learning of preliminary recognition ability are realized.

[0034] The target domain prediction network loads the weight parameters obtained by training the source domain when initialized, and fine-tunes the target domain dataset to ensure that the network can quickly adapt to the new data distribution and environmental characteristics.

[0035] The joint strategy of parameter sharing, adversarial feature alignment and adaptive loss function is adopted in the transfer learning module. On the one hand, the key recognition ability of the source domain network is preserved through parameter sharing; on the other hand, the feature distribution difference between the source domain and the target domain is reduced through adversarial feature alignment; at the same time, the adaptive loss function dynamically adjusts the optimization direction according to the characteristics of the target domain samples, so as to realize the efficient transfer of source domain knowledge and the continuous improvement of target domain recognition and detection performance.

[0036] The graph convolution subnetwork is composed of the following hierarchical structure: Graph Convolutional Network (GCN) layer: aims to extract the geometric shape, spatial topology and local structure features of the vehicle from the three-dimensional grid data, providing high-dimensional representation for subsequent recognition.

[0037] LayerNorm layer: aims to normalize the network output to reduce the difficulty of neural network training, speed up the convergence process and improve the training stability.

[0038] Activation function (PReLU) layer: aims to introduce non-linear mapping ability to improve the network's fitting ability for complex features, thereby enhancing the recognition effect of fine-grained features of vehicles.

[0039] Through the combination and synergistic effect of the above layers, the graph convolution subnetwork can efficiently extract the structural features of the three-dimensional vehicle grid, providing a reliable geometric information basis for the source domain and target domain prediction network.

[0040] The YOLOv11 subnetwork and the graph convolution subnetwork are fused through an implicit feature splicing manner. The two-dimensional image features extracted by the YOLOv11 subnetwork and the three-dimensional grid features extracted by the graph convolution subnetwork are spliced in the corresponding dimensions in the feature space, so that the information is implicitly integrated without explicitly constructing a complex fusion layer. This fusion manner can keep the network lightweight while making the two-dimensional appearance features and the three-dimensional geometric features complement each other, thereby improving the precision and robustness of vehicle fine-grained recognition detection.

[0041] The transfer learning training is completed in two steps: Source domain training stage: input the source domain data set into the source domain prediction network, train the YOLOv11 subnetwork and the graph convolution subnetwork respectively, and optimize the network parameters through a joint loss function, so as to obtain initial weights with high precision recognition ability in the source domain.

[0042] Target domain migration stage: migrate the weights obtained by the source domain training to the target domain prediction network, and fine-tune on the target domain data. Through domain adaptation and joint loss optimization strategies, the feature distribution difference between the source domain and the target domain is reduced, the efficient migration of source domain knowledge to the target domain is realized, and the vehicle recognition detection precision and cross-scene generalization ability of the target domain are improved.

[0043] Embodiment: The present application upgrades the existing vehicle fine-grained recognition detection method by introducing the cooperative design of three-dimensional grid feature extraction and YOLOv11 detection framework, not only improves the expression precision and feature analysis ability of vehicle appearance and structure information, but also enhances the adaptability and cross-scene generalization ability of the network in complex environments. Compared with the traditional recognition method relying only on two-dimensional images, the present application realizes efficient migration and stable recognition of cross-domain knowledge while ensuring the lightweight and efficient operation of the model, significantly improves the precision, robustness and practical application value of fine-grained vehicle recognition.

[0044] As shown in Figure 1 The fine-grained vehicle recognition detection transfer learning method of the vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning proposed by the present application includes three parts: source domain prediction network structure, target domain prediction network structure and transfer learning module.

[0045] As shown in Figure 2As shown, the source domain prediction network structure is mainly used to complete the training task of feature extraction and key part positioning on the source domain dataset. The input includes the two-dimensional RGB image, color map, normal map and corresponding three-dimensional mesh data of the source domain vehicle. The network consists of two parallel substructures: one is a YOLOv11-based image feature extraction and fine-grained identification subnetwork, which is used to capture vehicle appearance and texture information; the other is a three-dimensional mesh feature extraction subnetwork based on graph convolution, which is used to obtain vehicle geometric and structural features. The outputs of the two subnetworks are spliced into a joint representation, and then the precise coordinates of the key positions of the source domain vehicle are output by the key position regression module, providing high-quality source domain knowledge support for the subsequent transfer learning module.

[0046] As shown in Figure 2 The target domain prediction network structure is used to achieve fine-grained vehicle identification and key part positioning on the target domain data. Its input is the two-dimensional RGB image and the corresponding three-dimensional mesh data of the target domain vehicle, and the overall framework of the network is consistent with that of the source domain, including the YOLOv11-based image feature extraction subnetwork and the three-dimensional mesh feature extraction subnetwork based on graph convolution. The source domain parameters obtained through transfer learning are used as the initial weights, and the target domain network is fine-tuned on a small amount of labeled data. The joint features obtained through implicit feature splicing are output, and the precise identification results of the target domain vehicle are generated by the key position regression module, thereby realizing high-precision fine-grained identification across domains.

[0047] As shown in Figure 2 The transfer learning module is used to effectively transfer the learned feature knowledge in the source domain prediction network to the target domain. This module first uses the weight parameters obtained by training the source domain as the initial model of the target domain network. By freezing the low-level general feature extraction layer, only the high-level task-related layer is fine-tuned to maintain the stability of the basic features. Then, combined with the domain adaptation strategy, the differences in data distribution between the source domain and the target domain are reduced, and the generalization ability of the model in the target domain is improved. Finally, through the step-by-step unfreezing strategy, the whole network is optimized, realizing the full transfer of source domain knowledge and the overall improvement of target domain identification performance.

[0048] As shown in Figure 3As shown, the source domain prediction network structure and the target domain prediction network structure both adopt the improved YOLOv11 framework, which is composed of Backbone, Neck and Head. Among them, Backbone is composed of multiple layers of Conv, C3k2, SPPF and C2PSA modules, which is used for deep feature extraction of the input two-dimensional image data, and gradually captures the multi-scale fine-grained features of the vehicle appearance. Neck adopts multiple layers of Upsample, Concat, Conv and C3k2 modules to build a cross-scale feature fusion channel, effectively integrate semantic information at different levels, and enhance the detection and positioning ability of key components. Head uses C3k2 prediction layer to output the coordinates of the key position of the vehicle to realize accurate identification and positioning of fine-grained components.

[0049] As shown in Figure 3 The three-dimensional grid feature branch first constructs the input vehicle three-dimensional grid data into a grid graph, and then extracts local topological and overall structural features through a graph convolution network (GCN). Then, the feature standardization processing is performed through the normalization (LayerNorm) layer, and the nonlinear expression ability is introduced through the parameter linear rectifier function (PReLU) layer to obtain stable and recognizable three-dimensional grid features. The extracted three-dimensional structure features are fused with the multi-scale feature maps output by the two-dimensional image branch of YOLOv11 at the Neck stage through the implicit feature splicing method, so that the three-dimensional morphological information can cooperate with the two-dimensional texture and edge information, thereby improving the precision and cross-domain adaptability of fine-grained vehicle identification.

[0050] The transfer learning module is mainly composed of source domain Backbone, target domain Backbone, similarity loss function, domain label discriminator and implicit feature expression unit, as shown in Figure 4 Among them, the source domain Backbone and the target domain Backbone respectively extract deep features of the image data of the source domain and the target domain, and form a multi-dimensional representation combined with the three-dimensional grid features; the similarity loss function is used to constrain the distribution difference between the source domain and the target domain in the latent space, to ensure the consistency of the key features in the cross-domain migration; the domain label discriminator introduces an adversarial training strategy to weaken the difference between the domains and improve the domain-invariant feature extraction ability of the model; the implicit feature expression unit maps the two-dimensional image features and three-dimensional grid features of the source domain and the target domain to a shared hidden space, and inputs the implicit features to the Neck stage of the YOLOv11 network for subsequent processing, to realize effective transfer of source domain knowledge to the target domain.

[0051] The present application focuses on extracting fine-grained features of the identified vehicle, and under the premise of ensuring the lightweight of the network volume and the high efficiency of the inference speed, a mechanism that can focus on different discrimination regions is introduced to fully exploit the local difference information of the vehicle. The boundary box regression loss function used is a GIOU loss function This method not only considers the overlap between the predicted and ground truth bounding boxes but also integrates constraints such as center point distance and aspect ratio, thereby enhancing the detection and recognition performance of fine-grained vehicle features. Its mathematical expression is as follows: (1) In the formula This represents the area of ​​the actual bounding box. This represents the area of ​​the prediction box. Let represent the area of ​​the smallest bounding rectangle of the union of two boxes. This represents the intersection-union ratio (IUGR) of the areas of the ground truth bounding boxes and the predicted bounding boxes. Representing area Subtract area .

[0052] This invention employs the Focal Loss confidence loss function for confidence prediction. This strategy is used to mitigate training bias caused by an imbalance in the ratio of positive to negative samples. In fine-grained vehicle recognition tasks, this strategy can significantly improve the detection accuracy of vehicles with weak features or small target vehicles, and enhance the overall stability and robustness of the recognition. Its mathematical expression is as follows: (2) In the formula, Indicates the first One sample, Indicates the first The true value of each sample Indicates the first The predicted value of each sample, where lg(∙) represents the logarithmic function operation.

[0053] The classification loss function used in this invention The formula is expressed as follows: (3) In the formula, Indicates the first One sample, Indicates the first The true value of each sample Indicates the first The predicted value of each sample, where lg(∙) represents the logarithmic function operation.

[0054] This invention employs an interchannel loss function in the feature learning stage. , which is used to constrain the correlation and information complementarity between different feature channels. The loss function calculates the difference or correlation coefficient of the feature distribution of each channel to impose a penalty on redundant channels and encourage moderate independence between channels, thereby improving the discriminability of the network in multi-dimensional feature representation. Through this mechanism, the discrimination of vehicle fine-grained features can be strengthened while ensuring the lightweight of the network, effectively improving the recognition accuracy and generalization ability of the model in complex backgrounds. The mathematical expression is as follows: (4) In the formula, denotes the discriminative branch loss function, denotes the diversity branch loss function. denotes the gain ratio of the diversity branch loss function.

[0055] Discriminative branch loss function The formula is as follows: (5) In the formula, denotes the cross-entropy loss function, denotes the real class label, denotes the number of categories, The formula is as follows: (6) In the formula, denotes the feature vector, and denote the width and height of the feature map respectively, denotes the number of feature channels corresponding to each category, The formula is as follows: (7) In the formula, denotes a mask composed of 0 and 1.

[0056] Diversity branch loss function The formula is as follows: (8) The precision rate used in the application The performance evaluation index is expressed by the formula as follows: (9) In the formula, denotes that the real sample is a positive sample and the predicted sample is a positive sample, denotes that the real sample is a negative sample and the predicted sample is a positive sample.

[0057] The recall rate used in the application The performance evaluation index is expressed by the formula as follows: (10) In the formula, True sample is a positive sample, and the predicted sample is a negative sample.

[0058] The application adopts The performance evaluation index is expressed by the following formula: (11) The application adopts the average precision of a single classification The performance evaluation index is expressed by the following formula: (12) The application adopts the average precision of a single classification The performance evaluation index is expressed by the following formula: (13) In the formula, Total number of categories, The value of the category.

[0059] The application adopts a domain adaptation method to efficiently migrate the source domain prediction network to the target domain. Specifically, by statistically aligning the feature distribution of the source domain and the target domain, combining the adversarial training and feature reweighting strategy, the differences between the two domains in data distribution, environmental light, vehicle appearance and collection conditions are reduced, thereby realizing cross-domain migration of knowledge. Without the need to construct a large-scale labeled dataset in the target domain, the method can significantly improve the precision and generalization ability of fine-grained vehicle recognition detection in the target domain under the premise of keeping the model structure lightweight, and meet the rapid deployment needs in different scenarios. The loss function used in the transfer learning is expressed by the following formula: (14) In the formula, The encoder output hidden feature of the source domain prediction network, The encoder output hidden feature of the target domain prediction network, The hyperparameter, The training parameter of the source domain prediction network, The training parameter of the target domain prediction network, The encoder output hidden feature of the target domain prediction network storing the negative sample, The formula can be expressed as follows: (15) In the formula, The trainable convolutional neural network, The input of the convolutional neural network,​ representing training parameters of a convolutional neural network.

[0060] The transfer learning training of the present application is mainly used for effectively transferring the learned feature knowledge in the source domain prediction network to the target domain to realize high precision and high robustness of cross-domain fine-grained vehicle identification. The specific method comprises the following steps: Source domain parameter initialization. The Backbone and Neck network parameters trained on the source domain data are used as the initial weights of the target domain network to fully utilize the existing knowledge of the source domain.

[0061] Low-layer freezing and high-layer fine-tuning. The low-layer general feature extraction layer of the target domain network is frozen, and only the high-layer task-related layer is fine-tuned to ensure the stability of the basic features, and the target domain data is trained for task adaptability.

[0062] Domain adaptation strategy. A similarity loss function is introduced to constrain the distribution consistency of the source domain and the target domain in the latent feature space. A domain label discriminator is used for adversarial training to weaken the difference between the source domain and the target domain and improve the domain-invariant feature extraction capability of the model.

[0063] Implicit feature mapping and joint optimization. The target domain image features and three-dimensional grid features are mapped to a shared hidden space, and are uniformly processed through an implicit feature expression unit. A step-by-step unfreezing strategy is adopted to perform joint optimization on the whole network to realize full transfer of the source domain knowledge and improve the key part positioning and fine-grained recognition accuracy of the target domain.

[0064] Output result. After the transfer learning training is completed, the target domain prediction network can output high-quality representations after joint feature splicing, and generate accurate target domain vehicle key part coordinates through a key position regression module to realize cross-domain fine-grained identification.

[0065] The embodiment of the vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning proposed in the present application has a software running interface as shown in Figure 5 and Figure 6 The experimental results show that the method can effectively transfer the recognition knowledge trained in the source domain (such as small car type) to the target domain (such as truck type) to realize high-precision fine-grained identification across vehicle types. In the target domain, the method not only can accurately predict the key part positions of the vehicle, including the engine, driver and other key points, but also can maintain the comprehensive analysis ability of the appearance, texture and structural features. Further experiments show that the method has good adaptability and robustness under different vehicle types, complex environments and partial occlusion, fully verifying the effectiveness and reliability of the present application in cross-domain knowledge transfer ability, key part positioning accuracy and practical application value.

[0066] The related parameters of the fine-grained vehicle detection method based on three-dimensional grid and YOLOv11 transfer learning are shown in Table 1.

[0067] Table 1

[0068] By running the embodiment of the fine-grained vehicle detection method based on three-dimensional grid and YOLOv11 transfer learning, it can be shown that the method has the following advantages: (1) High-precision fine-grained recognition detection. The method fully fuses two-dimensional image features and three-dimensional structure information, realizes the comprehensive analysis of vehicle appearance, texture and geometric structure, and improves the precision of key part positioning and fine-grained recognition.

[0069] (2) High-efficiency cross-domain migration ability. Through the transfer learning strategy, the source domain knowledge can be efficiently migrated to the target domain, significantly improving the recognition performance across vehicle types and scenes, and completing fine-tuning on a small amount of target domain labeled data, reducing the data labeling cost.

[0070] (3) Stability and practical application value. Combined with multi-scale feature fusion and implicit feature splicing technology, the method can still maintain good recognition stability under complex environment and partial occlusion, has strong cross-domain generalization ability and practical application potential.

Claims

1. A vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning, characterized in that, Comprising the following steps: Step 1: Construction and training of source domain prediction network structure; The source domain prediction network adopts a dual-channel deep network that combines two-dimensional images and three-dimensional grids. The two-dimensional vehicle images of the source domain are input into the YOLOv11 subnetwork to extract texture and appearance features, and the three-dimensional grid is input into the graph convolution subnetwork to extract geometric and topological features. The two types of features are jointly optimized and trained to form a cross-domain recognition base model. Step 2: Establishment of target domain prediction network structure; The target domain prediction network follows the dual-channel structure of the source domain prediction network, which includes YOLOv11 and graph convolution subnetworks. In this network structure, the target domain image and grid data are input separately to extract key features in the target environment. Through weight fine-tuning and hierarchical optimization, vehicle recognition and fine-grained attribute prediction in the target scene are realized. Step 3: Optimization strategy of transfer learning module; The source domain training weights are transferred to the target domain, combined with the domain adaptation strategy to reduce the feature distribution difference, and optimized through the joint loss to improve the generalization performance and recognition accuracy of the model in the target domain.

2. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 1, characterized in that, The source domain prediction network structure includes a YOLOv11 subnetwork for processing two-dimensional image data and a graph convolution subnetwork for processing three-dimensional grid data. The target domain prediction network structure is consistent with the source domain structure and also consists of YOLOv11 and graph convolution subnetworks.

3. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 2, characterized in that, The YOLOv11 subnetwork is responsible for extracting texture, contour, and fine-grained appearance features of vehicle images, and the graph convolution subnetwork is used to extract geometric shape and spatial structure features of three-dimensional grids. The multi-modal feature interaction is performed in the feature fusion layer. Through joint training on the source domain dataset, deep fusion of two-dimensional and three-dimensional features and preliminary recognition ability learning are realized.

4. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 3, characterized in that, The target domain prediction network loads the weight parameters obtained by training the source domain at initialization and fine-tunes them with the target domain dataset to ensure that the network can quickly adapt to new data distribution and environmental characteristics.

5. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 4, characterized in that, The transfer learning module adopts a joint strategy of parameter sharing, adversarial feature alignment, and adaptive loss function. On the one hand, parameter sharing preserves the recognition ability of the source domain network. On the other hand, adversarial feature alignment reduces the feature distribution difference between the source domain and the target domain. At the same time, the adaptive loss function dynamically adjusts the optimization direction according to the characteristics of the target domain samples, thereby realizing efficient transfer of source domain knowledge and continuous improvement of target domain recognition and detection performance.

6. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 5, characterized in that, The graph convolution subnetwork consists of the following hierarchical structure: Graph convolution layer: extracts geometric shape, spatial topology, and local structure features of vehicles from three-dimensional grid data, providing high-dimensional representation for subsequent recognition; Regularization layer: normalizes the network output to reduce the difficulty of neural network training, speed up the convergence process, and improve the stability of training; Activation function layer: introduces non-linear mapping capability to improve the network's fitting ability for complex features, thereby enhancing the recognition effect of fine-grained features of vehicles.

7. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 6, characterized in that, The YOLOv11 subnetwork and the graph convolution subnetwork are fused through an implicit feature splicing manner; two-dimensional image features extracted by the YOLOv11 subnetwork and three-dimensional grid features extracted by the graph convolution subnetwork are spliced in a corresponding dimension in a feature space, so that implicit integration of information is realized.

8. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 7, characterized in that, The training of the transfer learning module is completed in two steps: In the source domain training stage, the source domain data set is input into the source domain prediction network, the YOLOv11 subnetwork and the graph convolution subnetwork are trained respectively, the network parameters are optimized through a joint loss function, and initial weights with high precision recognition ability in the source domain are obtained; In the target domain transfer stage, the weights obtained in the source domain training are transferred to the target domain prediction network, and fine tuning is performed on the target domain data; through a domain adaptation and joint loss optimization strategy, the feature distribution difference between the source domain and the target domain is reduced, efficient transfer of source domain knowledge to the target domain is realized, and the vehicle recognition detection precision and cross-scene generalization ability of the target domain are improved.

9. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 8, characterized in that, The vehicle fine-grained detection adopts the following loss function: The bounding box regression loss function is a GIOU loss function The mathematical expression is as follows: (1) wherein denotes the area of the true box, denotes the area of the predicted box, denotes the area of the minimal enclosing rectangle of the union of the two boxes, denotes the intersection over union of the areas of the true and predicted boxes, denotes the area subtracts the area ; Adopting focal loss confidence loss function , for relieving training deviation brought by positive and negative sample proportion imbalance, and its mathematical expression is as follows: (2) In the formula, denotes the sample, denotes the true value of the sample, denotes the predicted value of the sample; Using a classification loss function which is expressed by the following equation. (3) In the formula, represents the first sample, represents the true value of the first sample, represents the predicted value of the first sample; The inter-channel loss function is used in the feature learning part of the source domain training stage to constrain the correlation and information complementarity between different feature channels, and the mathematical expression is as follows: (4) wherein denotes a discriminative branch loss function, denotes a diversity branch loss function, denotes a gain ratio of the diversity branch loss function; Discriminative branch loss function Is expressed by the formula as follows: (5) In the formula, denotes a cross-entropy loss function, denotes a real class label, denotes the number of classes, is expressed by the following formula: (6) wherein, denotes cross-class max-pooling that takes the maximum value among all class responses at each spatial location, denotes class-wise weighted aggregation that uses a weight vector to weight the feature channels for each class, denotes a feature vector, and denote the width and height of the feature map, respectively, denotes the number of feature channels corresponding to each class, is expressed by the following formula: (7) wherein represents a mask consisting of 0 and 1; Diversity branch loss function In formula, (8)。 10. The vehicle fine-grained detection method based on three-dimensional grid and YOLOv11 transfer learning according to claim 9, characterized in that, The transfer learning module training adopts a domain adaptation transfer learning method, and the loss function adopted is represented by the following formula: wherein denotes the encoder output hidden features of the source domain prediction network, denotes the encoder output hidden features of the target domain prediction network, denotes a hyperparameter, denotes the training parameters of the source domain prediction network, denotes the training parameters of the target domain prediction network, denotes the encoder output hidden features of the target domain prediction network storing the negative samples, is formulated as follows: wherein denotes a trainable convolutional neural network, denotes an input of the convolutional neural network, denotes a training parameter of the convolutional neural network.

Images