A feature and prior-based vehicle component detection method, system, and medium

Abstract

The application discloses a vehicle component detection method and system based on features and priori, and a medium. The method comprises the following steps: constructing a multi-target vehicle component dataset suitable for a vehicle loss assessment scene, completing data labeling and quality checking; configuring a hardware environment and a software framework required for training, and setting training hyperparameters; based on a convolutional neural network, integrating an enhanced feature pyramid network and an embedded component knowledge module to form a detection branch and a segmentation branch; training a detection network framework by using the dataset to obtain a trained detection model, and verifying the model detection and segmentation performance; and inputting a vehicle component image to be detected into the trained detection model to perform vehicle component detection. The application focuses on the guiding effect of knowledge on a deep learning model, enhances feature knowledge of a vehicle image, and introduces priori knowledge among vehicle components to improve the detection and segmentation accuracy of the model for multi-target vehicle components, and provides reliable technical support for vehicle intelligent loss assessment.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent damage assessment, and in particular to a method, system and medium for vehicle component detection based on features and prior knowledge. Background Technology

[0002] With the continuous growth of private vehicle ownership, the frequent occurrence of traffic accidents has led to a surge in demand for vehicle damage assessment. The traditional manual on-site damage assessment model has drawbacks such as high cost, low efficiency, and strong subjectivity of results. It not only increases the human and material resources invested by insurance companies, but also easily leads to claims disputes and delayed payments, while exacerbating urban traffic pressure.

[0003] In response, combining deep learning with the damage assessment industry to achieve intelligent damage assessment has become an important direction for the industry's digital transformation. Specifically, this method uses instance segmentation technology to accurately detect and segment the type of vehicle damage and its corresponding component categories at the pixel level, thereby automatically and efficiently completing damage assessment and location. This technological approach can significantly reduce manual intervention in the damage assessment process, greatly improve the efficiency and objectivity of vehicle damage assessment, bring insurance companies and customers a faster and more transparent claims service experience, and effectively alleviate urban traffic pressure.

[0004] The realization of intelligent vehicle damage assessment relies heavily on high-precision computer vision technology. Currently, mainstream systems typically follow this technical process: users capture and upload images of vehicle damage via mobile devices; the system uses a pre-trained deep learning model to segment vehicle components and damaged areas; then, it spatially associates and maps the damaged areas with the component segmentation results to determine the specific component corresponding to each damage; finally, the insurance company combines component information, damage type, vehicle model, and repair data to conduct loss assessment and pricing.

[0005] As can be seen from the above process, achieving high-precision multi-target vehicle component instance segmentation—that is, accurately detecting and segmenting each vehicle component—is a prerequisite for realizing intelligent vehicle damage assessment. However, multi-target vehicle component instance segmentation faces severe challenges in practical applications: a user-captured image typically contains dozens of component categories, resulting in a large number of targets; the color and shape features of the same target category often differ across different vehicle images, indicating low intra-class similarity; and due to the mirror symmetry of vehicles, features between symmetrical categories (such as left and right doors) are easily confused, resulting in high inter-class similarity. These challenges—a large number of target categories, low intra-class similarity, and high inter-class similarity—make existing models prone to false positives and false negatives, posing a significant challenge to high-precision vehicle component instance segmentation.

[0006] Currently, research on the detection and segmentation of vehicle components is in its developmental stage, and the existing technical solutions are summarized as follows: Technical Solution 1: Vehicle Component Detection and Segmentation Method Based on Traditional Algorithms. Before the widespread application of deep learning technology, vehicle component detection and segmentation mainly relied on traditional computer vision algorithms. This method uses manually designed feature extractors (such as edge detection and corner detection) combined with machine learning classifiers (such as support vector machines and adaptive boosting algorithms) for component localization and classification. For example, some studies extract features through normalized cross-correlation and use sliding windows for multi-scale detection; others construct geometric models by defining spatial relationships between components, limiting the search area, and then using cascaded classifiers for detection. Although these methods do not require large-scale labeled data, they rely on carefully designed features and rules, have weak generalization ability, and limited accuracy in handling complex and variable scenes and numerous component categories, making it difficult to meet the practical requirements of high accuracy and robustness.

[0007] Technical Solution Two: Vehicle Part Detection and Segmentation Method Based on Deep Learning Algorithms. Currently, deep learning-based vehicle part detection and segmentation has become a mainstream technical solution. In vehicle part detection, the core task is to locate and identify vehicle parts in an image, often employing advanced object detection frameworks (such as single-stage YOLO series or two-stage Faster R-CNN) to output the part's category and bounding box. For vehicle part segmentation, semantic segmentation models (such as FCN and U-Net) are used for pixel-level classification, or instance segmentation models (such as Mask R-CNN) are used to generate independent masks for each part instance. Currently, deep learning-based vehicle part detection and segmentation algorithms have achieved good results when applied to a small number of vehicle part categories. For example, some studies have proposed a vehicle part instance segmentation method based on Mask R-CNN to solve the localization error problem caused by part overlap; Chang Qinwei, in "Vehicle Part Recognition Based on Symbiotic Relationships," mentioned utilizing the symbiotic relationships between vehicle parts to construct a part recognition network through semantic feature merging, etc.

[0008] While deep learning-based methods have made some progress, they still have certain shortcomings: they struggle to fully learn the features of multiple targets and cannot cover the diverse component types in real-world damage assessment scenarios, severely limiting the practical application value of intelligent vehicle damage assessment. Furthermore, existing methods fail to effectively address the significant intra-class differences in vehicle components caused by factors such as vehicle model, color, and lighting, easily leading to false positives and false negatives. Simultaneously, due to the high similarity in features between symmetrical components on the left and right sides of a vehicle, existing methods lack effective mechanisms to accurately distinguish them, resulting in misclassification and affecting the reliability of damage assessment. Therefore, there is an urgent need to introduce new technological approaches to overcome these current technological bottlenecks. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing vehicle component detection methods in areas such as multi-target coverage, intra-class difference adaptation, and symmetrical component differentiation. It provides a feature- and prior-based vehicle component detection method, system, and medium, focusing on the guiding role of knowledge in deep learning models. By enhancing the feature knowledge of vehicle images and introducing prior knowledge between vehicle components, the invention improves the model's accuracy in detecting and segmenting multi-target vehicle components, providing reliable technical support for intelligent vehicle damage assessment.

[0010] To achieve the above objectives, the present invention provides the following solution: A feature- and prior-based vehicle component detection method includes the following steps: S1. Dataset and Training Environment Preparation: Construct a multi-target vehicle component dataset that fits the vehicle damage assessment scenario, and complete data annotation and quality verification; Configure the hardware environment and software framework required for training, and set the training hyperparameters; S2, Constructing a detection network framework that integrates feature enhancement and prior knowledge: Based on a convolutional neural network, it integrates an enhanced feature pyramid network and an embedded component knowledge module to form a dual-branch structure with a detection branch and a segmentation branch; the construction methods of the enhanced feature pyramid network and the embedded component knowledge module include: S2.1, Construction of Enhanced Feature Pyramid Network: Introducing detail attention module and key knowledge extraction module, which work together with basic feature extraction structure to achieve semantic enhancement of multi-scale features and extraction of key information; S2.2, Embedded Component Knowledge Module Design: Based on prior knowledge of vehicle components, main sign, front and rear sign and auxiliary sign components are defined, and the distinction of symmetrical components is realized through multi-rule judgment logic; S3, Model Training and Inference: Use the constructed dataset to train a detection network framework that integrates feature enhancement and prior knowledge. By outputting component categories and bounding boxes from the detection branch and pixel-level segmentation masks from the segmentation branch, the trained detection model is obtained. S4, Model Performance Evaluation: Average Precision (AP) and Mean Average Precision (mAP) are used as evaluation metrics to verify the model's detection and segmentation performance. S5, Vehicle Component Detection: Input the image of the vehicle component to be detected into the trained detection model to perform vehicle component detection.

[0011] Furthermore, in S1, the preparation of the dataset and training environment specifically includes: S1.1, Dataset Construction: Collect vehicle images from the daily damage assessment business of insurance companies, covering vehicles of different brands, models, colors and damage conditions, and form the original dataset after anonymization. S1.2, Data annotation: Based on the established annotation standards, the Labelme tool is used to perform mask annotation on multiple types of vehicle parts. The annotation results are saved as JSON files in COCO format, and the annotation quality is determined by sampling verification. S1.3, Dataset partitioning: Divide the dataset into training set and validation set according to a set ratio; S1.4, Environment Configuration: NVIDIA 3090 accelerator card, Ubuntu 24.04.1 LTS operating system, CUDA 11.1 acceleration framework, Python 3.8 programming language and PyTorch network development framework; S1.5, Hyperparameter settings: Use multi-GPU training, set batch size, use SGD optimizer, set initial learning rate and learning rate during training.

[0012] Furthermore, in S2.1, during the construction of the enhanced feature pyramid network, the detail attention module is used to implement the following feature processing scheme: For input features The process is divided into two paths: one path undergoes global average pooling and convolution, while the other path directly undergoes convolution. The outputs of the two paths are summed and then processed using the Sigmoid function to obtain the channel weights. Multiplying by F0 yields the initial corrected feature. ; Apply the Softmax function to feature F1 in both the horizontal and vertical dimensions to obtain the horizontal position weights. and vertical position weight The feature F0 is corrected using weights W1 and W2 to obtain the feature. : (1) in, This is an adjustment parameter, initially set to 0, and gradually increased as the model learns. Using feature F2 as input, we feed it into two convolutional layers to perform dimensionality transformation, changing the feature's dimension to... , Next, matrix multiplication is performed, and the results are fed into the Softmax function to obtain the spatial location weights of the features. The larger the W3 value, the more similar and correlated the spatial location features of the two locations are. Perform matrix multiplication on the transformed result F and the weight W3, then reshape the result and sum it with the feature F2 to refine the feature F2 into the final output feature. : (2) in, These are transformation coefficients, initialized to 0, and gradually increased as training progresses.

[0013] Furthermore, in S2.1, during the construction of the enhanced feature pyramid network, the key knowledge extraction module includes a feature selection module and a hybrid scale module used in combination. The feature selection module learns the importance of feature locations and assigns weights through convolution, ReLU activation, and deformable convolution operations. The implementation process of the feature selection module is as follows: (3) in, This represents the convolution operation. Indicates ReLU operation. This is a deformable convolution operation; the feature selection module selects input features... Convolution operations are performed to obtain the offsets and corresponding offset weights at each position. The original position is then aggregated with the offsets and multiplied by the corresponding weights to obtain the corrected feature values. These corrected feature values ​​are then fed into the ReLU function and the convolution function to obtain the output features of the feature selection module. ; The hybrid scale module obtains multi-scale context information through channel compression, fully connected layers, and average pooling operations; the implementation process of the hybrid scale module is as follows: (4) (5) (6) in, Indicates a fully connected operation. This indicates the average pooling operation. This indicates a channel compression operation. This represents the convolution operation. This represents the average operation along the channel. This indicates a ReLU operation; the hybrid scale module first compresses each channel of the feature, unifying the number of channels to the output channel number, and then calculates the channel coefficients through the operations in equations (4) and (5). and spatial coefficient This assigns importance to different feature channels; at the same time, the two coefficients are multiplied by the original features, and the results are passed into the ReLU function and the convolution function to obtain the output features of the mixed scale module.

[0014] Furthermore, in S2.2, the embedded component knowledge module design includes the vehicle's license plate, tires, rearview mirrors, and fenders; the front and rear signage components are the grille and trunk lid; the auxiliary signage components are the door handles, doors, and headlights; the priority of the main signage components from high to low is: license plate > tires > rearview mirrors > fenders, and the priority of the auxiliary signage components from high to low is: headlights > doors > door handles.

[0015] Furthermore, in S2.2, the multi-rule judgment logic in the embedded component knowledge module design is as follows: If a front logo component exists, the left and right components are classified by comparing the coordinates of the remaining components in the image with the central component, using the license plate or grille as the central component. If a rear marker component exists, the left and right components are classified by comparing the coordinates of the remaining components in the image with the central component, using the license plate or trunk lid as the central component. If auxiliary sign components exist, the network is directly guided to classify the left and right symmetrical components of the vehicle based on the relationship between the main sign components and the auxiliary sign components; If none of the above-mentioned marker components exist, the left and right categories of the component with the highest confidence level in the image are used as a benchmark to help determine the category of the remaining components.

[0016] Furthermore, S3, model training and inference, specifically includes: S3.1, The images in the input dataset are processed by a convolutional neural network to extract single-scale features, which are then fed into an enhanced feature pyramid network for multi-scale semantic knowledge fusion to obtain multi-scale features rich in detailed information. S3.2, Multi-scale features are fed into the region proposal network to generate candidate boxes. The candidate boxes and multi-scale features are respectively used to generate two feature vectors through two regions of interest alignment structures, which are then input into the detection branch and the segmentation branch, respectively. S3.3 In the detection branch, the feature vector is processed by a classifier to obtain the initial bounding box and category. The category judgment is then optimized by the embedded component knowledge module. Duplicate boxes are removed by non-maximum suppression, and the final detection result is output. In S3.4, in the segmentation branch, the feature vector is used to generate a pixel-level segmentation mask and corresponding class score through a fully convolutional network.

[0017] The present invention also provides a feature- and prior-based vehicle component detection system, applied to perform the above-described feature- and prior-based vehicle component detection method, comprising: The data preparation module is used to construct a multi-target vehicle component dataset that fits the vehicle damage assessment scenario, complete data annotation and quality verification; configure the hardware environment and software framework required for training, and set training hyperparameters. The detection network framework construction module is used to integrate an enhanced feature pyramid network and an embedded component knowledge module based on a convolutional neural network, forming a dual-branch structure of detection and segmentation branches. The construction methods for the enhanced feature pyramid network and the embedded component knowledge module include: Enhanced Feature Pyramid Network Construction: Introducing detail attention module and key knowledge extraction module, which work together with basic feature extraction structure to achieve semantic enhancement of multi-scale features and extraction of key information; Embedded component knowledge module design: Based on prior knowledge of vehicle components, main sign, front and rear sign and auxiliary sign components are defined, and the distinction of symmetrical components is realized through multi-rule judgment logic; The model training and inference module is used to train a detection network framework that integrates feature enhancement and prior knowledge using the constructed dataset. It outputs component categories and bounding boxes from the detection branches and outputs pixel-level segmentation masks from the segmentation branches. The model performance evaluation module is used to verify the model's detection and segmentation performance by using mean accuracy (AP) and mean average accuracy (mAP) as evaluation metrics. The vehicle component detection module is used to input images of vehicle components to be detected into a trained detection model to perform vehicle component detection.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the feature- and prior-based vehicle component detection method as described above.

[0019] As can be seen from the above technical solutions, compared with the prior art, the feature-based and prior art vehicle component detection method and system provided by the present invention have the following beneficial effects: This invention leverages and integrates feature knowledge from vehicle images and prior structural knowledge between vehicle components to guide deep learning models in more accurate learning and reasoning. Through the collaborative design of an enhanced feature pyramid network and an embedded component knowledge module, it addresses existing methods' shortcomings in multi-target vehicle component detection, such as insufficient feature extraction, poor adaptation to intra-class differences, and confusion of symmetrical components. The combination of a detail attention module and a key knowledge extraction module effectively enhances the semantic expression of multi-scale features and reduces the loss of detailed information; the embedded component knowledge module significantly improves the distinguishability of symmetrical components through prior knowledge embedding. This invention can accurately cover multiple types of vehicle components, significantly improving the accuracy and robustness of multi-target vehicle component instance segmentation, and significantly reducing false positive and false negative rates. It provides high-precision and reliable technical support for intelligent vehicle damage assessment, greatly improving assessment efficiency and objectivity, and is suitable for the accurate detection and segmentation of multi-target vehicle components in intelligent vehicle damage assessment scenarios. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a diagram of the detection network framework that integrates feature enhancement and prior knowledge in this invention. Figure 2 This is a block diagram of the attention module in a specific embodiment of the present invention; Figure 3 This is a flowchart of the embedded component knowledge module in an embodiment of the present invention; Figure 4 The above is a comparison of the results of vehicle component images in the general model and the model of the present invention in the embodiments of the present invention. Among them, (a) and (b) are the front and side instance segmentation effect images of the basic instance segmentation model Mask R-CNN, respectively, and (c) and (d) are the front and side instance segmentation effect images of the present invention, respectively. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The purpose of this invention is to provide a vehicle component detection method and system based on features and prior knowledge. Addressing the challenge of low intra-class similarity and high inter-class similarity among vehicle components, this invention studies the characteristics of vehicle image feature knowledge and its enhancement methods, and explores the acquisition and application of prior knowledge of vehicle components to achieve knowledge-guided vehicle component instance segmentation, thereby effectively reducing the problems of missed and false detections of components.

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Example 1 like Figures 1-3 As shown, the present invention provides a vehicle component detection method based on features and priors, comprising the following steps: S1, Dataset and Training Environment Preparation: Construct a multi-target vehicle component dataset that fits the vehicle damage assessment scenario, and complete data annotation and quality verification; configure the hardware environment and software framework required for training, and set the training hyperparameters.

[0026] Specifically, a multi-part vehicle dataset was used to evaluate the effectiveness of the method of this invention. The images in this dataset were taken during the routine insurance business of an insurance company. All images underwent rigorous anonymization after acquisition, removing all personally identifiable information (such as license plates and faces) and sensitive information related to the insurance company. The dataset contains images of vehicles of different colors, brands, and models, taken from different angles (front, rear, left, and right). Most importantly, most of the vehicle images in the dataset exhibit varying degrees of damage, including deformation, scratches, tears, and breakage, making the multi-part vehicle dataset perfectly suited for vehicle damage assessment applications.

[0027] During the data annotation phase, annotation standards were jointly developed with professional claims adjusters from insurance companies. Based on these standards, the Labelme tool was used to annotate vehicle components. Labelme uses point-based connections to precisely segment the target and thus annotate the mask information of the vehicle components. Annotations are saved as JSON files in COCO format. Finally, the data is submitted to professional claims adjusters for sampling and verification to ensure the annotation quality of the dataset.

[0028] In this embodiment, the vehicle multi-component dataset contains 45,503 training set images and 11,376 validation set images. To meet the needs of intelligent vehicle damage assessment, the vehicle component categories in the data are divided into 59 classes, with the number of images in each category in the validation set proportional to their number in the training set at a ratio of 3:7.

[0029] The model was trained and tested using an NVIDIA 3090 professional accelerator card. The operating system was Ubuntu 24.04.1 LTS, and CUDA 11.1 was used for training acceleration. Python 3.8 was used as the programming language, and the PyTorch network development framework was employed. A multi-GPU training method was used, with a batch size of 4, and the SGD algorithm was used with a momentum of 0.9. The learning rate was dynamically adjusted, initially set to 0.008, and then reduced to 0.0008 at the 16th and 22nd epochs.

[0030] S2, building a detection network framework that integrates feature enhancement and prior knowledge: Based on convolutional neural networks, an enhanced feature pyramid network and an embedded component knowledge module are integrated to form a dual-branch structure with detection and segmentation branches.

[0031] The enhanced feature pyramid network proposed in this invention provides stronger semantic information support for subsequent tasks through multi-scale semantic fusion. The proposed embedded component knowledge module further embeds prior knowledge of vehicle components based on the features extracted by the enhanced feature pyramid network to enhance the model's identification of left and right components. The two complement each other from the two dimensions of semantic enhancement and prior guidance, respectively, and jointly improve the model's performance.

[0032] The construction methods for the enhanced feature pyramid network and embedded component knowledge modules include: S2.1, Construction of Enhanced Feature Pyramid Network: By introducing a detail attention module and a key knowledge extraction module, which work together with the basic feature extraction structure, we can achieve semantic enhancement of multi-scale features and extraction of key information.

[0033] One reason existing algorithms struggle to fully extract feature knowledge from vehicle components is that the downsampling operations used during model training can lead to information loss. Another reason is that when extracting features, the algorithm doesn't learn the importance of all feature information, potentially causing the model to neglect key feature knowledge and instead focus on learning obfuscated feature knowledge.

[0034] To improve the model's ability to extract component feature knowledge, and to reduce the loss of detailed features during model training and testing while allowing the model to fully learn the key feature knowledge of various vehicle components, this invention proposes a detail attention module and a key knowledge extraction module that can be added to the feature pyramid network structure. Combining these modules with the basic feature extraction structure forms an enhanced feature pyramid network. The structure of the enhanced feature pyramid network is as follows: Figure 1 As shown in the top left corner, the detail attention module, designed specifically for the characteristics of vehicle components, helps the model learn the importance of different channels and positions among features. Simultaneously, the key knowledge extraction module reduces feature information loss and allows the model to learn more key feature knowledge and intra-scale features from the input image. The enhanced feature pyramid network, composed of these two modules, enables the model to achieve more accurate detection and segmentation of vehicle components.

[0035] ① Detail Attention Module (containing 7 convolutional layers and 1 global average pooling layer) The network structure of the detail attention module is as follows: Channel weight branch: 1 layer global average pooling, 2 layers 1×1 convolution (stride 1, padding 1), ReLU activation, added to the feature branch directly through 2 layers of convolution with the same parameters and ReLU activation, and generated channel weight W0 by Sigmoid, realizing adaptive weighting of feature channels.

[0036] Horizontal and vertical position optimization branches: Softmax is executed in the horizontal and vertical dimensions to generate position weights W1 and W2.

[0037] Spatial location optimization branch: After dimensionality reduction by two layers of 1×1 convolution, spatial weights W3 are generated by matrix multiplication and Softmax, and spatial feature association enhancement is achieved by combining one layer of convolution transformation.

[0038] like Figure 2 As shown, in the detail attention module, for a given feature... The algorithm is divided into two paths. One path obtains the scalar value for each channel through a global average pooling layer, and then performs two convolutional operations to obtain the corresponding feature information. The other path directly performs two convolutional operations to obtain another set of feature information. The corresponding positions of the convolutional outputs from the two paths are summed, and the channel weights are obtained after sigmoid activation. .

[0039] Next, the feature F0 is directly multiplied by the weight W0 to obtain the initially corrected feature. To enable the model to better learn the relationships between horizontal and vertical features in the image, softmax operations are performed on the F1 feature in both the horizontal and vertical dimensions to obtain the horizontal position weights. and vertical position weight The larger the values ​​in the corresponding dimensions of W1 and W2, the more similar the horizontal and vertical feature representations of the two locations are, that is, the greater the correlation between the two locations.

[0040] Next, the feature F0 is corrected using weights W1 and W2 to obtain the feature. The implementation process is as follows: (1) in, This is an adjustment parameter, initially set to 0, and gradually increased as the model learns. It is set by... The value of allows the network to gradually shift its focus from horizontal positional features during training to vertical positional features. This setting is because vehicle images contain more shallow information in the horizontal position, and the model should initially focus on learning shallow features. However, as training progresses and the model's learning ability increases, it should place greater emphasis on learning vertical positional features that contain deeper information.

[0041] To enable the model to further learn the relationships between spatial locations in the image, feature F2 is taken as input and fed into two convolutional layers for dimensionality transformation, changing the feature dimensions to... ( Next, matrix multiplication is performed, and the results are fed into the Softmax function to obtain the spatial location weights of the features. The larger the W3 value, the more similar the spatial location features of the two locations are, and the greater the correlation between them.

[0042] Simultaneously, feature F2 is input into another convolutional layer for dimensionality transformation; the transformation result is then processed. Perform matrix multiplication with weight W3, reshape the result, and then sum it with feature F2 to refine feature F2 into the final output feature. : (2) in, These are the transformation coefficients, initialized to 0, and gradually increased as training progresses. Through... This setting allows the model to gradually focus on the spatial location information of the image as training progresses.

[0043] ② Key knowledge extraction module (including 5 convolutional layers and 1 average pooling layer) The network structure of the key knowledge extraction module is as follows: Feature selection module (3 convolutional layers): 1 layer of 3×3 convolution + 1 layer of 3×3 variable convolution (learning position offset and weights) + ReLU activation + 1 layer of 3×3 convolution to achieve adaptive allocation of feature position importance.

[0044] Hybrid Scale Unit (2 convolutional layers + 1 average pooling layer): 1 layer of channel dimension concatenation + dual-branch parallel processing (branch 1: 1 layer of 1×1 average pooling + 1 layer of fully connected layer; branch 2: 1 layer of channel dimension mean calculation + 1 layer of 7×7 convolution), generating channel coefficients and spatial coefficients, which are then weighted and fused and output through 1 layer of ReLU activation + 1 layer of 3×3 convolution.

[0045] Module chaining: Feature selection units and mixed scale units are alternately superimposed three times to form a “selection-mixing” cyclic structure, which strengthens the learning of scale information within deep features.

[0046] To enhance the network's learning of intra-scale information from deep features, the key knowledge extraction module feeds deep features into a superimposed feature selection module and a mixed-scale module. The feature selection module learns the importance of feature information at different locations and assigns corresponding weights. The mixed-scale module can acquire multi-scale contextual information while maintaining resolution. By alternately introducing the feature selection module and the mixed-scale module, the key knowledge extraction module allows the two modules to mutually reinforce each other, thereby enabling the model to fully learn intra-scale features.

[0047] The implementation process of the feature selection module is as follows: (3) in, This represents the convolution operation. Indicates ReLU operation. This is a deformable convolution operation. The feature selection module selects features based on the input features. First, convolution operations are performed to obtain the offsets and corresponding offset weights at each position. The original position is then aggregated with the offsets and multiplied by the corresponding weights to obtain the corrected feature values. Next, the corrected feature values ​​are fed into the ReLU function and the convolution function to obtain the output features of the feature selection module. .

[0048] The implementation process of the hybrid scale module is as follows: (4) (5) (6) in, Indicates a fully connected operation. This indicates the average pooling operation. This indicates a channel compression operation. This represents the convolution operation. This represents the average operation along the channel. This represents the ReLU operation. The hybrid scale module first compresses each channel of the feature, unifying the number of channels to the output channel number, and then calculates the channel coefficients through the operations in equations (4) and (5). and spatial coefficient This assigns importance to different feature channels. Simultaneously, the two coefficients are multiplied by the original features, and the results are passed into the ReLU function and the convolution function to obtain the output features of the mixed-scale module.

[0049] S2.2, Embedded Component Knowledge Module Design: The embedded component knowledge module adopts a discriminative network structure, guided by prior knowledge, and constructs a two-level architecture of "label filtering - multi-rule classification", as follows: Signage component filtering layer: The input is the predicted category label of the component, and the output is the confidence score of 8 categories of signage components (4 categories of main signs: license plate, tire, rearview mirror, fender; 2 categories of front and rear signs: grille, trunk lid; 2 categories of auxiliary signs: headlights, doors, door handles) to filter the valid signage components in the image.

[0050] Multi-rule classification layer: Front-position classification branch: Input component label + center component coordinates, output coordinate offset classification result, and realize left and right classification based on the positional relationship between license plate / grill and other components.

[0051] Rear position classification branch: Consistent with the front position branch structure, it uses the license plate / trunk lid as the central component and determines the left and right categories by coordinate offset.

[0052] Auxiliary classification branch: Input the main label and coordinates + auxiliary label and coordinates, and output the category judgment result directly by comparing the coordinate relationship between the main label and the auxiliary label.

[0053] Confidence baseline branch: Input all component labels and use the component category with the highest confidence level as the baseline for auxiliary classification.

[0054] Based on prior knowledge of vehicle components, main signs, front and rear signs, and auxiliary signs are defined, and symmetrical components are distinguished through multi-rule judgment logic.

[0055] The mirror-symmetry of vehicles results in low similarity among similar samples of vehicle components. Although enhanced feature pyramid networks have improved the network's ability to extract vehicle component features to some extent, the feature knowledge extracted by the network itself is still insufficient to help the model distinguish between mirror-symmetric left and right components. This invention combines prior knowledge of vehicle components with the network's own classification capabilities, and the proposed embedded component knowledge module greatly improves the model's ability to classify left and right symmetrical vehicle components.

[0056] The prior knowledge of vehicle components proposed in this invention is derived from people's accumulated experience in identifying vehicle components. By combining this prior knowledge with the model's own component detection capabilities, the four most distinctive categories—license plate, tires, rearview mirrors, and fenders—are ultimately selected as primary markers, the grille and trunk lid as front and rear markers, and door handles, doors, and headlights as secondary markers. The primary marker components are equivalent to the vehicle components that people first notice during identification; the front and rear marker components are the key components that play a decisive role in determining the vehicle's front and rear orientation in people's classification experience; and the secondary marker components are key components derived from human experience that can assist in determining the vehicle's left and right orientation.

[0057] The reason for designing these three types of marker components is that the auxiliary effect of prior knowledge is limited by the network's own classification ability, and the network's ability to integrate and summarize knowledge does not possess the flexibility of human thinking. When embedding knowledge, the network needs to be given accurate judgment markers. For example, license plates, tires, rearview mirrors, and fenders are selected as the four main categories of markers because, among the vehicle components that humans first focus on in their discriminative thinking, the network has already learned many features from these four categories, and its localization and classification of them are relatively accurate. Similarly, the components for front and rear markers and auxiliary markers are also selected from key components learned from human experience, choosing those with better feature extraction performance from the network.

[0058] The embedded component knowledge module first identifies suitable prior knowledge from people's experience in classifying vehicle components and assigns corresponding priorities to components in two categories: main signs and auxiliary signs. The priorities of main signs, from highest to lowest, are: license plate > tires > rearview mirrors > fenders; the priorities of auxiliary signs, from highest to lowest, are: headlights > doors > door handles. By determining the relationship between main signs and the signs in front and behind, or between main signs and auxiliary signs, the current location of the vehicle (i.e., front, rear, left, and right) can be determined. This achieves the goal of using prior knowledge of the vehicle to assist the model in accurately classifying symmetrical vehicle components.

[0059] The embedded component knowledge module integrates the vehicle prior knowledge into four modules to assist in network classification. Figure 3 This is represented by Operations 1 through 4. Operations 2 and 3 are performed when the model classifies the vehicle image as front or rear, respectively. Operation 2 uses the license plate or grille as the central component, while Operation 3 uses the license plate or trunk lid as the central component. By comparing the coordinates of the remaining components in the image with the central components, the model is assisted in classifying the components as left or right. Operation 4 is performed when the model classifies the vehicle image as left or right. This operation directly assists the network in classifying the left and right components of the vehicle based on the relationship between the main and auxiliary markers. When there is no relationship between the main marker and the front / rear or auxiliary markers in the vehicle image, Operation 1 uses the left / right category of the component with the highest confidence level in the vehicle image to assist in determining the left / right category of the remaining components.

[0060] S3, Model Training and Inference: A detection network framework that integrates feature enhancement and prior knowledge is trained using the constructed dataset. The detection branch outputs component categories and bounding boxes, and the segmentation branch outputs pixel-level segmentation masks to obtain the trained detection model.

[0061] Detection network framework such as Figure 1As shown in the diagram, in the network, the damaged image is first subjected to feature extraction through a convolutional neural network to obtain single-scale features. Subsequently, the innovative enhanced feature pyramid network of this invention performs multi-scale semantic knowledge fusion to obtain multi-scale features containing key detailed information. These features are fed into the classification and regression parts. First, candidate boxes are obtained through a region proposal network. Then, the multi-scale features and candidate boxes are fed into two Region of Interest Align (RoI Align) structures to generate two features, which are then fed into the detection branch and the segmentation branch, respectively. In the detection branch, the input features are passed through a classifier to obtain preliminary predicted bounding boxes and their categories. Then, the embedded component knowledge module proposed in this invention embeds spatial reasoning knowledge to guide the network to make more accurate adjustments to the preliminary predicted categories. Finally, non-maximum suppression is used to remove duplicate bounding boxes after adjustment, thus obtaining the final predicted bounding boxes and corresponding category scores. In the segmentation branch, the input features are passed through a fully convolutional network to obtain the predicted mask and its corresponding category score.

[0062] S4, Model Performance Evaluation: The average precision (AP) and mean average precision (mAP) were used as evaluation metrics to verify the model's detection and segmentation performance.

[0063] To evaluate the superiority of the model of this invention and its feasibility in applying it to an intelligent damage assessment system, two general metrics, Average Precision (AP) and Mean Average Precision (mAP), are specified for model evaluation. These metrics can be defined as follows: (7) (8) (9) (10) In equations (7) and (8), Let n be the precision of category n. Let be the recall rate for category n. This refers to the case where category n is correctly predicted as category n. This refers to the case where the model predicts other categories as category n. This refers to the case where the model predicts category n as another category. In equation (9), It is the average precision of category n, calculated by comparing category n with different recall rates. Precision rate The average value is obtained and can be used to measure the accuracy of the model in detecting and segmenting targets of each category. In equation (10), This refers to the total number of all categories. The mean accuracy is obtained by averaging the AP values ​​across all categories, and it can be used to measure the overall performance of the model.

[0064] S5, Vehicle Component Detection: Input the image of the vehicle component to be detected into the trained detection model to perform vehicle component detection.

[0065] Instance segmentation result analysis: Figure 4 The paper demonstrates a comparison of instance segmentation performance on the same test image using the base instance segmentation model Mask R-CNN and the model of this invention. Figure 4 (a) and Figure 4 (b) shows the results of Mask R-CNN. Figure 4 (c) and Figure 4 (d) shows the results of the model of this invention. It can be seen from the segmentation results of the baseline model that a large number of left and right component categories are falsely detected, such as... Figure 4 (a) The right headlight was detected as the left headlight, and the left rearview mirror was detected as the right rearview mirror. Additionally, some categories experienced missed detections, such as... Figure 4 The rear bumper cover in (b) was not detected. However, the detection model trained using this invention can effectively reduce missed detections and false detections, especially in similar left and right component categories, where false detections are virtually eliminated.

[0066] In summary, to address the problem that existing algorithms struggle to fully capture key information about vehicle components, this invention proposes a detail attention module and a key knowledge extraction module. This invention integrates these two modules to form an enhanced feature pyramid network, aiming to enrich the model's understanding of vehicle components. This reduces the loss of detail information while enabling the model to accurately filter and capture key information from various vehicle components.

[0067] The detail attention module allows the model to automatically learn the importance of different feature channels, enhancing the relevant feature representations of vehicle components while suppressing background features. Based on this, it learns the horizontal and vertical positional information and spatial location information of each component in the image, enriching the model's overall understanding of the vehicle components. The key knowledge extraction module further aggregates semantic knowledge based on the pyramid structure, thereby reducing feature loss and preserving deep features as much as possible. It also enhances the network's learning of internal scale information in deep features by processing deep semantics through the superimposed feature selection module and hybrid scale module.

[0068] To address the problem that the mirror symmetry of vehicles leads to highly similar representations of left and right symmetrical components, making it difficult for models to accurately distinguish between them, this invention proposes an embedded component knowledge module. By combining prior knowledge of vehicle components with the network's feature extraction performance and utilizing the discrimination of positions between different marker components, the model's ability to classify left and right symmetrical components is significantly improved.

[0069] Example 2 The present invention also provides a feature- and prior-based vehicle component detection system, applied to perform the above-described feature- and prior-based vehicle component detection method, comprising: The data preparation module is used to construct a multi-target vehicle component dataset that fits the vehicle damage assessment scenario, complete data annotation and quality verification; configure the hardware environment and software framework required for training, and set training hyperparameters. The detection network framework construction module is used to integrate an enhanced feature pyramid network and an embedded component knowledge module based on a convolutional neural network, forming a dual-branch structure of detection and segmentation branches. The construction methods for the enhanced feature pyramid network and the embedded component knowledge module include: Enhanced Feature Pyramid Network Construction: Introducing detail attention module and key knowledge extraction module, which work together with basic feature extraction structure to achieve semantic enhancement of multi-scale features and extraction of key information; Embedded component knowledge module design: Based on prior knowledge of vehicle components, main sign, front and rear sign and auxiliary sign components are defined, and the distinction of symmetrical components is realized through multi-rule judgment logic; The model training and inference module is used to train a detection network framework that integrates feature enhancement and prior knowledge using the constructed dataset. It outputs component categories and bounding boxes from the detection branches and outputs pixel-level segmentation masks from the segmentation branches. The model performance evaluation module is used to verify the model's detection and segmentation performance by using mean accuracy (AP) and mean average accuracy (mAP) as evaluation metrics. The vehicle component detection module is used to input images of vehicle components to be detected into a trained detection model to perform vehicle component detection.

[0070] Furthermore, embodiments of the present invention provide a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the feature- and prior-based vehicle component detection method described in Embodiment 1.

[0071] Matters not covered in this invention are common knowledge.

[0072] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0073] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting vehicle components based on features and priors, characterized in that, Includes the following steps: S1. Dataset and Training Environment Preparation: Construct a multi-target vehicle component dataset that fits the vehicle damage assessment scenario, and complete data annotation and quality verification; Configure the hardware environment and software framework required for training, and set the training hyperparameters; S2, Constructing a detection network framework that integrates feature enhancement and prior knowledge: Based on a convolutional neural network, it integrates an enhanced feature pyramid network and an embedded component knowledge module to form a dual-branch structure with a detection branch and a segmentation branch; the construction methods of the enhanced feature pyramid network and the embedded component knowledge module include: S2.1, Construction of Enhanced Feature Pyramid Network: Introducing detail attention module and key knowledge extraction module, which work together with basic feature extraction structure to achieve semantic enhancement of multi-scale features and extraction of key information; S2.2, Embedded Component Knowledge Module Design: Based on prior knowledge of vehicle components, main sign, front and rear sign and auxiliary sign components are defined, and the distinction of symmetrical components is realized through multi-rule judgment logic; S3, Model Training and Inference: Use the constructed dataset to train a detection network framework that integrates feature enhancement and prior knowledge. By outputting component categories and bounding boxes from the detection branch and pixel-level segmentation masks from the segmentation branch, the trained detection model is obtained. S4, Model Performance Evaluation: Average Precision (AP) and Mean Average Precision (mAP) are used as evaluation metrics to verify the model's detection and segmentation performance. S5, Vehicle Component Detection: Input the image of the vehicle component to be detected into the trained detection model to perform vehicle component detection.

2. The vehicle component detection method based on features and priors according to claim 1, characterized in that, In S1, the preparation of the dataset and training environment specifically includes: S1.1, Dataset Construction: Collect vehicle images from the daily damage assessment business of insurance companies, covering vehicles of different brands, models, colors and damage conditions, and form the original dataset after anonymization. S1.2, Data annotation: Based on the established annotation standards, the Labelme tool is used to perform mask annotation on multiple types of vehicle parts. The annotation results are saved as JSON files in COCO format, and the annotation quality is determined by sampling verification. S1.3, Dataset partitioning: Divide the dataset into training set and validation set according to a set ratio; S1.4, Environment Configuration: NVIDIA 3090 accelerator card, Ubuntu 24.04.1 LTS operating system, CUDA 11.1 acceleration framework, Python 3.8 programming language and PyTorch network development framework; S1.5, Hyperparameter settings: Use multi-GPU training, set batch size, use SGD optimizer, set initial learning rate and learning rate during training.

3. The vehicle component detection method based on features and priors according to claim 1, characterized in that, In step S2.1, during the construction of the enhanced feature pyramid network, the detail attention module is used to implement the following feature processing schemes: For input features The process is divided into two paths: one path undergoes global average pooling and convolution, while the other path directly undergoes convolution. The outputs of the two paths are summed and then processed using the Sigmoid function to obtain the channel weights. Multiplying by F0 yields the initial corrected feature. ; Apply the Softmax function to feature F1 in both the horizontal and vertical dimensions to obtain the horizontal position weights. and vertical position weight The feature F0 is corrected using weights W1 and W2 to obtain the feature. : (1) in, This is an adjustment parameter, initially set to 0, and gradually increased as the model learns. Using feature F2 as input, we feed it into two convolutional layers to perform dimensionality transformation, changing the feature's dimension to... , Next, matrix multiplication is performed, and the results are fed into the Softmax function to obtain the spatial location weights of the features. The larger the W3 value, the more similar and correlated the spatial location features of the two locations are. Perform matrix multiplication on the transformed result F and the weight W3, then reshape the result and sum it with the feature F2 to refine the feature F2 into the final output feature. : (2) in, These are transformation coefficients, initialized to 0, and gradually increased as training progresses.

4. The vehicle component detection method based on features and priors according to claim 1, characterized in that, In S2.1, during the construction of the enhanced feature pyramid network, the key knowledge extraction module includes a feature selection module and a hybrid scale module used in combination. The feature selection module learns the importance of feature locations and assigns weights through convolution, ReLU activation, and deformable convolution operations. The implementation process of the feature selection module is as follows: (3) in, This represents the convolution operation. Indicates ReLU operation. This is a deformable convolution operation; the feature selection module selects input features... Convolution operations are performed to obtain the offsets and corresponding offset weights at each position. The original position is then aggregated with the offsets and multiplied by the corresponding weights to obtain the corrected feature values. These corrected feature values ​​are then fed into the ReLU function and the convolution function to obtain the output features of the feature selection module. ; The hybrid scale module obtains multi-scale context information through channel compression, fully connected layers, and average pooling operations; the implementation process of the hybrid scale module is as follows: (4) (5) (6) in, This indicates a fully connected operation. This indicates the average pooling operation. This indicates a channel compression operation. This represents the convolution operation. This represents the average operation along the channel. This indicates a ReLU operation; the hybrid scale module first compresses each channel of the feature, unifying the number of channels to the output channel number, and then calculates the channel coefficients through the operations in equations (4) and (5). and spatial coefficient This assigns importance to different feature channels; at the same time, the two coefficients are multiplied by the original features, and the results are passed into the ReLU function and the convolution function to obtain the output features of the mixed scale module.

5. The vehicle component detection method based on features and priors according to claim 1, characterized in that, In S2.2, the embedded component knowledge module design includes the vehicle's license plate, tires, rearview mirrors, and fenders; the front and rear signage components are the grille and trunk lid; and the auxiliary signage components are the door handles, doors, and headlights. The priority of the main signage components, from high to low, is: license plate > tires > rearview mirrors > fenders, and the priority of the auxiliary signage components, from high to low, is: headlights > doors > door handles.

6. The vehicle component detection method based on features and priors according to claim 5, characterized in that, In S2.2, the multi-rule judgment logic in the embedded component knowledge module design is as follows: If a front logo component exists, the left and right components are classified by comparing the coordinates of the remaining components in the image with the central component, using the license plate or grille as the central component. If a rear marker component exists, the left and right components are classified by comparing the coordinates of the remaining components in the image with the central component, using the license plate or trunk lid as the central component. If auxiliary sign components exist, the network is directly guided to classify the left and right symmetrical components of the vehicle based on the relationship between the main sign components and the auxiliary sign components; If none of the above-mentioned marker components exist, the left and right categories of the component with the highest confidence level in the image are used as a benchmark to help determine the category of the remaining components.

7. The vehicle component detection method based on features and priors according to claim 1, characterized in that, S3, model training and inference, specifically includes: S3.1, The images in the input dataset are processed by a convolutional neural network to extract single-scale features, which are then fed into an enhanced feature pyramid network for multi-scale semantic knowledge fusion to obtain multi-scale features rich in detailed information. S3.2, Multi-scale features are fed into the region proposal network to generate candidate boxes. The candidate boxes and multi-scale features are respectively used to generate two feature vectors through two regions of interest alignment structures, which are then input into the detection branch and the segmentation branch, respectively. S3.3 In the detection branch, the feature vector is processed by a classifier to obtain the initial bounding box and category. The category judgment is then optimized by the embedded component knowledge module. Duplicate boxes are removed by non-maximum suppression, and the final detection result is output. In S3.4, in the segmentation branch, the feature vector is used to generate a pixel-level segmentation mask and corresponding class score through a fully convolutional network.

8. A feature- and prior-based vehicle component detection system, applied to the feature- and prior-based vehicle component detection method according to any one of claims 1-7, characterized in that, include: The data preparation module is used to build a multi-target vehicle component dataset that fits the vehicle damage assessment scenario, and to complete data annotation and quality verification. Configure the hardware environment and software framework required for training, and set the training hyperparameters; The detection network framework construction module is used to integrate an enhanced feature pyramid network and an embedded component knowledge module based on a convolutional neural network, forming a dual-branch structure of detection and segmentation branches. The construction methods for the enhanced feature pyramid network and the embedded component knowledge module include: Enhanced Feature Pyramid Network Construction: Introducing detail attention module and key knowledge extraction module, which work together with basic feature extraction structure to achieve semantic enhancement of multi-scale features and extraction of key information; Embedded component knowledge module design: Based on prior knowledge of vehicle components, main sign, front and rear sign and auxiliary sign components are defined, and the distinction of symmetrical components is realized through multi-rule judgment logic; The model training and inference module is used to train a detection network framework that integrates feature enhancement and prior knowledge using the constructed dataset. It outputs component categories and bounding boxes from the detection branches and outputs pixel-level segmentation masks from the segmentation branches. The model performance evaluation module is used to verify the model's detection and segmentation performance by using mean accuracy (AP) and mean average accuracy (mAP) as evaluation metrics. The vehicle component detection module is used to input images of vehicle components to be detected into a trained detection model to perform vehicle component detection.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the feature- and prior-based vehicle component detection method as described in any one of claims 1 to 7.

Images