Method, device and equipment for detecting assembly quality of vehicle exterior trim part and medium

Abstract

The invention discloses a method, a device and equipment for detecting the assembly quality of a vehicle exterior trim part, and a medium. The method comprises the following steps: collecting a sample image of the vehicle exterior trim part; pre-annotating the sample image based on a pre-trained large model to obtain a pre-annotation result, and generating an annotation data set based on the pre-annotation result after manual verification; performing data enhancement processing on the labeled data set, and training the deep learning detection model by using the enhanced data set to obtain an external decoration assembly quality detection model; reasoning the to-be-detected image by using the external decoration assembly quality detection model to obtain an assembly quality detection result; and comparing the assembly quality detection result with the standard configuration information, and when the comparison result is inconsistent, outputting abnormal prompt information. By applying the technical scheme of the invention, the problems of low manual detection efficiency, easy missing detection and false detection and the like can be solved, and the efficiency, the accuracy and the adaptability to complex working conditions of the assembly quality detection of the vehicle exterior decorating part are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a vehicle exterior trim assembly quality detection method, device, equipment and medium. BACKGROUND

[0002] At present, in the automobile manufacturing process, the assembly quality of the exterior trim is crucial, which relates to the appearance, safety and brand image of the vehicle. However, the current exterior trim assembly detection mainly relies on manual visual inspection, which has the problems of low efficiency, easy missed detection or misdiagnosis, high skill requirement of employees, etc., and the manual detection is disturbed by environmental factors such as factory lighting and background complexity, which affects the detection accuracy. Although the existing machine learning algorithms based on OpenCV and other open source libraries have been applied, they have limitations such as dependence on manual parameter adjustment and poor robustness, which are difficult to adapt to complex working conditions.

[0003] Therefore, it is urgent to provide a technical solution to solve the above technical problems. SUMMARY

[0004] In view of the above problems, the present application provides a vehicle exterior trim assembly quality detection method, device, equipment and medium, which is used to solve the problem of low accuracy and efficiency of vehicle exterior trim assembly quality detection in the prior art.

[0005] According to an aspect of an embodiment of the present application, a vehicle exterior trim assembly quality detection method is provided, the method comprising: collecting an original sample image containing a vehicle exterior trim; pre-labeling the original sample image based on a pre-trained large model to obtain a pre-labeling result, and generating a labeling data set based on the pre-labeling result after artificial verification; performing data enhancement processing on the labeling data set, and training a deep learning detection model for vehicle exterior trim assembly quality detection using the enhanced data set to obtain a vehicle exterior trim assembly quality detection model; wherein the deep learning detection model introduces a multi-channel inverse residual module in the Backbone layer of the YOLOv8 architecture, and introduces a convolution attention module in the C2f module of the YOLOv8 architecture; using the vehicle exterior trim assembly quality detection model to infer a to-be-detected image containing a vehicle exterior trim to obtain an assembly quality detection result; comparing the assembly quality detection result with standard configuration information, and outputting an abnormal prompt information when the comparison result is inconsistent.

[0006] In an optional manner, the step of pre-labeling the original sample image based on the pre-trained large model to obtain the pre-labeling result further comprises: The weights of the pre-trained large model are fine-tuned to obtain a fine-tuned large model; wherein the pre-trained large model has a general zero-shot generalization capability; The original sample image is pre-labeled using the fine-tuned large model to obtain the pre-labeling result.

[0007] In an optional manner, the step of performing data enhancement processing on the labeled data set further comprises: Performing data enhancement operations on the labeled sample images in the labeled data set to obtain a first data set; wherein the data enhancement operations include at least one type of operation in image space transformation, image appearance transformation, and data normalization processing; Collecting a negative sample data set corresponding to the original sample image, and collecting the negative sample data set according to a set probability to obtain a second data set; Based on the first data set and the second data set, the enhanced data set is constituted.

[0008] In an optional manner, the multi-channel inverse residual module is used for: The output feature map of the Backbone layer is divided into multiple partial feature maps, and inverse residual operations are respectively performed on the multiple partial feature maps, and the output features of each branch after the inverse residual operations are spliced; The spliced features are subjected to channel number recovery processing, and the processing results are added to the output feature map of the Backbone layer to obtain a first output feature map and output.

[0009] In an optional manner, the convolution attention module is used for: The feature map input to the C2f module is sequentially subjected to channel attention mechanism and spatial attention mechanism, and the channel weighted feature and the spatial attention weight are multiplied to obtain a second output feature map and output; Wherein, the channel attention mechanism is used for: performing global average pooling and global maximum pooling operations on the input feature to generate channel attention weight, and multiplying the channel attention weight and the input feature to obtain the channel weighted feature, and the spatial attention mechanism is used for: performing convolution operation on the channel weighted feature to generate the spatial attention weight.

[0010] In an optional manner, the assembly quality detection result includes detection position information, detection quantity information and detection category information of the vehicle exterior trim part; and when the comparison result is inconsistent, the step of outputting abnormal prompt information further comprises: The detection position information is compared with preset position information in the standard configuration information, the detection quantity information is compared with a preset quantity in the standard configuration information, and the detection category information is compared with a preset category in the standard configuration information. If all comparison results are consistent, qualified prompt information is output; if any comparison result is inconsistent, abnormal prompt information is output.

[0011] In an optional manner, the method further includes: When the abnormal prompt information is output, a control instruction is sent to the production control system to stop the production line from running, and a repair prompt information is sent to an operator through an Internet of Things platform.

[0012] According to another aspect of the embodiments of the present application, a vehicle exterior trim assembly quality detection device is provided, including: The acquisition module is configured to acquire an original sample image containing a vehicle exterior trim assembly. The pre-labeling module is configured to pre-label the original sample image based on a pre-trained large model to obtain a pre-labeling result, and generate a labeling data set based on the pre-labeling result after manual verification. The training module is configured to perform data enhancement processing on the labeling data set, and train a deep learning detection model for vehicle exterior trim assembly quality detection using an enhanced data set to obtain a vehicle exterior trim assembly quality detection model. The deep learning detection model introduces a multi-channel inverse residual module in the Backbone layer of the YOLOv8 architecture, and introduces a convolution attention module in the C2f module of the YOLOv8 architecture. The detection module is configured to use the vehicle exterior trim assembly quality detection model to infer a to-be-detected image containing a vehicle exterior trim assembly to obtain an assembly quality detection result. The comparison module is configured to compare the assembly quality detection result with standard configuration information, and output abnormal prompt information when the comparison result is inconsistent.

[0013] According to another aspect of the embodiments of the present application, a vehicle exterior trim assembly quality detection device is provided, including: A controller; A memory configured to store one or more programs, which, when executed by the controller, cause the controller to implement the vehicle exterior trim assembly quality detection method of the present application.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the storage medium storing at least one executable instruction, which, when executed on a vehicle exterior trim assembly quality inspection device / equipment, causes the vehicle exterior trim assembly quality inspection device / equipment to perform operations as described in the vehicle exterior trim assembly quality inspection method of this application.

[0015] This application embodiment acquires original sample images containing vehicle exterior trim parts; pre-annotates the original sample images based on a pre-trained large model to obtain pre-annotation results, and generates an annotated dataset based on the manually verified pre-annotation results; performs data augmentation processing on the annotated dataset, and uses the augmented dataset to train a deep learning detection model for vehicle exterior trim part assembly quality inspection, obtaining a vehicle exterior trim part assembly quality inspection model; uses the vehicle exterior trim part assembly quality inspection model to infer the image to be tested containing vehicle exterior trim parts to obtain assembly quality inspection results; compares the assembly quality inspection results with standard configuration information, and outputs an abnormal prompt message when the comparison results are inconsistent. This effectively solves the problems of low efficiency, easy omission and false detection in manual inspection, and significantly improves the efficiency, accuracy and adaptability to complex working conditions of vehicle exterior trim part assembly quality inspection.

[0016] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0018] Figure 1 A flowchart illustrating an embodiment of the vehicle exterior trim assembly quality inspection method provided in this application is shown.

[0019] Figure 2 The diagram shows examples of the correct assembly state for three typical vehicle exterior trim pieces.

[0020] Figure 3 A schematic diagram of the trained LiteYOLOv8 model is shown.

[0021] Figure 4The input graph is shown as a data flow trend diagram in the LiteYOLOv8 model.

[0022] Figure 5 A schematic diagram of the overall process is shown.

[0023] Figure 6 A schematic diagram of an embodiment of the vehicle exterior trim assembly quality inspection device provided in this application is shown.

[0024] Figure 7 A schematic diagram of an embodiment of the vehicle exterior trim assembly quality inspection equipment provided in this application is shown. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0026] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0027] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0028] In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0029] Currently, in the automotive manufacturing process, the assembly quality of exterior trim parts is crucial, affecting the vehicle's appearance, safety, and brand image. However, current inspection of exterior trim part assembly largely relies on manual visual inspection, which suffers from low efficiency, susceptibility to missed or false inspections, and high skill requirements for employees. Furthermore, manual inspection is affected by environmental factors such as factory lighting and background complexity, impacting accuracy. While existing machine learning algorithms based on open-source libraries like OpenCV are used, they have limitations such as reliance on manual parameter adjustment and poor robustness, making them unsuitable for complex working conditions. Therefore: Figure 1 A flowchart illustrating an embodiment of the vehicle exterior trim assembly quality inspection method provided in this application is shown. This method is performed by a vehicle exterior trim assembly quality inspection device. Please refer to... Figure 1 As shown, the method includes the following steps: Step S110: Acquire raw sample images containing vehicle exterior trim parts.

[0030] Among them, vehicle exterior trim parts refer to components installed on the exterior of a vehicle for decorative, protective, or functional assistance purposes; for example, lidar, vehicle body lettering trim, and sound insulation clips. Original sample images refer to images of vehicle exterior trim parts directly captured by a camera without any annotation or processing; for example, original photographs containing lidar, vehicle body lettering trim, or sound insulation clips.

[0031] It should be noted that, Figure 2 The images show example images of the correct assembly state for three typical vehicle exterior trim pieces in a real-vehicle inspection scenario. Figure 2 From left to right, the images show the installation effects of the LiDAR, vehicle body emblem trim, and sound insulation clips. Each image was captured using a fixed camera on-site, clearly showing the standard installation position, orientation, and integrity of the trim pieces on the vehicle. These raw sample images serve as the data foundation for model training and validation, specifically demonstrating the installation posture of the LiDAR at the front of the roof, the placement of the vehicle body emblem on specific sheet metal surfaces, and the engagement status of the sound insulation clips in their corresponding holes. This set of images intuitively reflects the diversity and complexity of the objects being inspected, providing a realistic visual basis for subsequent data annotation, model training, and quality comparison.

[0032] Step S120: Pre-label the original sample images based on the pre-trained large model to obtain the pre-labeling results, and generate a labeled dataset based on the pre-labeling results after manual verification.

[0033] The pre-trained large model refers to a detection model that has been pre-trained on a large amount of general data and has zero-shot generalization ability; for example, a detection large model based on the Grounding DINO architecture. Pre-annotation results refer to the preliminary annotation information generated by the pre-trained large model after automatically annotating the original sample images; for example, the location and category labels automatically bounded by the model for LiDAR images. Manually verified pre-annotation results refer to pre-annotation results that have been manually checked and corrected for errors; for example, operators correcting inaccurate boundaries of LiDAR location boxes in pre-annotations. The labeled dataset refers to the dataset used for model training, consisting of manually verified pre-annotation results; for example, a set of correctly labeled images containing LiDAR, vehicle body lettering decorations, and sound insulation clips.

[0034] Step S130: Perform data augmentation processing on the labeled dataset, and use the augmented dataset to train a deep learning detection model for vehicle exterior trim assembly quality inspection, thereby obtaining a vehicle exterior trim assembly quality inspection model.

[0035] The deep learning detection model introduces a multi-channel inverse residual module in the Backbone layer of the YOLOv8 architecture and a convolutional attention module in the C2f module of the YOLOv8 architecture.

[0036] Among these, "augmented dataset" refers to an expanded dataset obtained by augmenting the labeled dataset; for example, a new set of images generated by flipping and rotating labeled LiDAR images. "Deep learning detection model" refers to a target detection model built on a deep learning architecture; for example, a detection model using YOLOv8 as its basic framework. "Vehicle exterior trim assembly quality detection model" refers to a deep learning model specifically designed to detect the assembly quality of vehicle exterior trim parts; for example, a trained LiteYOLOv8 model used to determine whether the LiDAR is installed correctly. "YOLOv8 architecture" refers to the You Only Look Once version 8 target detection algorithm framework; for example, a neural network structure used for real-time LiDAR position detection. "Backbone layer" refers to the backbone feature extraction network layer in a deep learning model; for example, a combination of convolutional layers in YOLOv8 used to extract LiDAR image features. "Multi-channel Inverted Residual (MCIR) module" refers to a neural network module employing multi-channel parallel processing and an inverted residual structure; for example, a module that segments and processes feature maps separately to enhance LiDAR feature extraction. The C2f (Cross Stage Partial network with 2 convolutions) module refers to the feature fusion module in the YOLOv8 architecture; for example, a network component used to fuse features from different scales of LiDAR. The Convolutional Block Attention Module (CBAM) refers to a neural network module that combines channel attention and spatial attention mechanisms; for example, a CBAM module used to improve the model's attention to key areas of the LiDAR.

[0037] It should be noted that, Figure 3 This diagram illustrates the structure of the trained LiteYOLOv8 model (vehicle exterior trim assembly quality inspection model) in this embodiment. Based on the YOLOv8 architecture, the model clearly shows a hierarchical structure: the input layer receives a size-normalized input image; the backbone network is responsible for basic feature extraction, with a key improvement being the introduction of a multi-channel inverse residual module to enhance multi-scale feature capture capabilities; the neck network includes an improved C2f module, which embeds a convolutional attention module to strengthen spatial and channel dimension attention control during feature fusion; finally, the detection head outputs the category, location, and confidence information of the vehicle exterior trim. Figure 3The text clearly indicates the data flow and connection relationships between the modules of the LiteYOLOv8 model, demonstrating the complete forward propagation path from the original image input to the final detection result output, highlighting the structural position and role of the MCIR and CBAM modules in improving feature extraction and fusion efficiency.

[0038] Step S140: Using the vehicle exterior trim assembly quality inspection model, reason about the image to be tested containing the vehicle exterior trim to obtain the assembly quality inspection result.

[0039] The image to be tested refers to a real-time image of the vehicle's exterior trim parts whose assembly quality needs to be inspected; for example, a photo of the LiDAR installation location taken on the production line. The assembly quality inspection result refers to the trim part status information obtained by the model after inferring from the image to be tested; for example, the installation location, quantity, and type of the LiDAR.

[0040] It should be noted that, as Figure 4 As shown, taking a specific image to be tested (e.g., an image with a size of 3×640×640 that has been preprocessed) as an example, the image to be tested goes through the multi-channel inverse residual module in the Backbone for feature extraction and multi-scale fusion, and then enters the C2f module in the Neck part (which integrates the CBAM attention mechanism) for feature enhancement and context information aggregation. Finally, the detection head outputs the predicted bounding box and category. Figure 4 The paper uses arrows and data transformations to visually demonstrate the scale changes, channel number adjustments, and key nodes for feature map enhancement at each network layer. It clarifies how the vehicle exterior trim assembly quality inspection model can accurately locate and identify vehicle exterior trim by gradually optimizing feature representation through the introduction of specific modules.

[0041] Step S150: Compare the assembly quality inspection results with the standard configuration information. If the comparison results are inconsistent, output an abnormal prompt message.

[0042] The standard configuration information refers to the standard installation specifications for vehicle trim components defined in the production control system; for example, the lidar should be installed in a preset position at the front of the roof. The anomaly warning information refers to the alarm signal output when the assembly quality inspection results are inconsistent with the standard configuration information; for example, the "NG" signal output when the lidar is missing.

[0043] The technical solution of this embodiment can effectively solve the problems of low efficiency, easy omission and false detection in manual inspection, and significantly improve the efficiency, accuracy and adaptability to complex working conditions of vehicle exterior trim assembly quality inspection.

[0044] In one alternative approach, the step of pre-labeling the sample images based on a pre-trained large model to obtain pre-labeling results further includes: The weights of the pre-trained large model are fine-tuned to obtain the fine-tuned large model; wherein the pre-trained large model has general zero-shot generalization ability.

[0045] The fine-tuned large model refers to a model adapted to a specific detection task, obtained by adjusting and optimizing the model weights of the pre-trained large model using specific task data. For example, based on labeled images containing the correct assembly positions of vehicle LiDAR, vehicle body lettering trim, and sound insulation cotton clips, the network parameters of the pre-trained Grounding DINO large model are iteratively trained and updated, causing the original model weights to change adaptively, thereby obtaining an optimized model specifically for accurately identifying and locating these types of vehicle exterior trim.

[0046] The original sample image is pre-labeled using the fine-tuned large model to obtain the pre-labeling result.

[0047] Among the above-mentioned optional methods, pre-trained large models with general zero-shot generalization capabilities are further utilized to achieve efficient and accurate pre-labeling, reduce manual intervention, and improve the reliability of results.

[0048] In an alternative approach, the step of performing data augmentation on the labeled dataset further includes: Data augmentation operations are performed on the labeled sample images in the labeled dataset to obtain the first dataset.

[0049] The data augmentation operations include at least one of the following: image spatial transformation, image appearance transformation, and data normalization. Image spatial transformation includes: flipping, rotation, translation, geometric perturbation, and Mosaic enhancement. Image appearance transformation includes: color transformation, illumination enhancement, ColorJitter operation, noise addition, and blurring. Data normalization includes: normalization.

[0050] The first dataset refers to the subset of data obtained after performing data augmentation on the labeled dataset; for example, the image set generated after color transformation of the LiDAR labeled images.

[0051] Collect a negative sample dataset corresponding to the original sample image, and collect the negative sample dataset according to a set probability to obtain a second dataset.

[0052] The negative sample dataset refers to a collection of images that do not contain the target decorative element; for example, vehicle images without LiDAR. The set probability refers to a fixed ratio of negative samples sampled during data loading; for example, randomly selecting negative samples with a probability of 0.3. The second dataset refers to a subset of negative sample data collected according to the set probability; for example, randomly selecting 30% of the images from the negative sample dataset.

[0053] The enhanced dataset is constructed based on the first dataset and the second dataset.

[0054] Among the above-mentioned optional approaches, further enriching the diversity and robustness of the labeled dataset through various data augmentation techniques and negative sample collection can enhance the model's adaptability to complex working conditions.

[0055] In one alternative approach, the multi-channel inverse residual module is used for: The output feature map of the Backbone layer is divided into multiple partial feature maps, and inverse residual operations are performed on each of the multiple partial feature maps. The output features of each branch after the inverse residual operations are then concatenated.

[0056] In this context, the output feature map of the Backbone layer refers to the feature representation generated after the Backbone network processes the input image; for example, the multidimensional feature tensor obtained after a LiDAR image passes through the Backbone layer. A partial feature map refers to the sub-feature map resulting from dividing the Backbone output feature map; for example, dividing a LiDAR feature map into four equal parts. Inverse residual operation refers to the residual calculation process of first expanding the number of channels and then compressing them; for example, performing depthwise convolution and pointwise convolution operations on partial feature maps. The output feature after inverse residual operation refers to the feature obtained by processing the partial feature map using inverse residual processing; for example, the enhanced feature after branching the LiDAR feature map.

[0057] The concatenated features are processed to restore the number of channels. The processing result is added to the output feature map of the Backbone layer to obtain the first output feature map, which is then output.

[0058] Here, the concatenated feature refers to the result of merging the output features of multiple branches along the channel dimension; for example, the fused feature after concatenating four LiDAR feature branches. Channel number restoration processing refers to the operation of adjusting the concatenated features back to the original number of channels using convolution; for example, restoring the number of channels using a 1x1 convolution. The first output feature map refers to the feature map finally output by the multi-channel inverse residual module; for example, the result of processing LiDAR features by the MCIR module.

[0059] like Figure 2As shown, the Multi-Channel Inverse Residual Module (MCIR) employs a multi-channel parallel processing strategy. It divides the input feature map and applies inverse residual structures to each channel for processing. The inverse residual structure decomposes the convolution operation into two steps: depthwise convolution and pointwise convolution, effectively reducing the number of model parameters. Internally, the inverse residual structure uses a sequence structure of "3×3 depthwise convolution - 1×1 ordinary convolution - 1×1 ordinary convolution - 3×3 depthwise convolution" to implement depthwise separable convolution while expanding the number of input channels. Combined with residual connections, it promotes gradient propagation, avoids gradient vanishing, and achieves the fusion of multi-layer features, thereby improving detection accuracy.

[0060] Specifically, the MCIR module first performs channel dimensionality reduction on the input feature map using a CBR convolution with a 1×1 kernel while maintaining the feature map size to reduce subsequent computational complexity. Then, the dimensionality-reduced feature map is divided into four parts. One part is directly passed to retain the original scale information, while the other three parts are processed by independent inverse residual structures. The output of each branch is sequentially added to the output of the previous branch, achieving step-by-step fusion of multi-scale features. This design effectively improves feature processing efficiency and avoids cumulative interference that may occur in sequential feature fusion. Finally, the output features of all branches are concatenated and then passed through a CBR convolution with a 1×1 kernel to restore the channel count. The restored features are then added to the module's input feature map via residual connections, ensuring that the output feature map maintains the same size as the input feature map, thus completing the extraction and fusion of multi-scale feature information.

[0061] Among the above optional methods, the feature extraction structure is further optimized by using a multi-channel inverse residual module to enhance the model's ability to capture multi-scale features and improve the model's detection accuracy.

[0062] In one alternative approach, the convolutional attention module is used for: The feature map input to the C2f module is sequentially subjected to channel attention mechanism and spatial attention mechanism. The channel weighted features are multiplied by the spatial attention weight to obtain the second output feature map and output it.

[0063] The channel attention mechanism is used to: perform global average pooling and global max pooling operations on the input features and generate channel attention weights, and multiply the channel attention weights with the input features to obtain the channel-weighted features. The spatial attention mechanism is used to: perform convolution operations on the channel-weighted features to generate the spatial attention weights.

[0064] The channel attention mechanism refers to a mechanism that weights and focuses on feature channels; for example, using pooling operations to generate importance weights for LiDAR feature channels. The spatial attention mechanism refers to a mechanism that weights and focuses on the spatial location of features; for example, using convolution to generate importance weights for LiDAR feature spatial regions. Input features refer to the original feature data entering the attention module; for example, the LiDAR feature map input to the C2f module. Channel-weighted features refer to the result of weighting the input features through channel attention; for example, the feature obtained by multiplying the LiDAR feature channel weights by the original features. Spatial attention weights refer to the location importance map generated by the spatial attention mechanism; for example, the weight value of each spatial point on the LiDAR feature map. The second output feature map refers to the final feature map output by the convolutional attention module; for example, the result of processing LiDAR features by the CBAM module.

[0065] Global average pooling refers to the operation of averaging all pixels in each channel of the feature map; for example, calculating the global average for each channel of a LiDAR feature map. Global max pooling refers to the operation of maximizing all pixels in each channel of the feature map; for example, calculating the global maximum for each channel of a LiDAR feature map. Channel attention weights refer to the channel importance vectors generated by the channel attention mechanism; for example, the LiDAR feature channel weights generated based on the pooling results.

[0066] like Figure 2 As shown, a Convolutional Attention Module (CBAM) is introduced on top of the C2f module to form the C2f-MS module, enhancing the model's ability to locate dense small targets in complex backgrounds. CBAM effectively improves the network's response to key features by combining channel attention and spatial attention mechanisms, strengthening its focus on important regions in complex backgrounds. Specifically, the channel attention module generates channel attention weights by performing global average pooling and global max pooling operations on the input features, enhancing the model's sensitivity to small target features; the spatial attention module performs convolution operations on the channel-weighted features to generate spatial attention weights, enabling the model to focus on target regions in the image. By applying these two attention mechanisms sequentially to the feature map and adjusting the feature responses in the channel and spatial dimensions through weighted adjustments, the accuracy and robustness of small target detection are significantly improved. Furthermore, this module achieves efficient parallel computation based on global pooling and convolution operations, introducing fewer parameters, allowing C2f-MS to achieve efficient and accurate small target detection performance while maintaining a low computational burden.

[0067] In the above-mentioned optional methods, a convolutional attention module is further introduced to enhance feature representation in both channel and spatial dimensions, thereby improving the model's ability to distinguish target details.

[0068] In one optional approach, the assembly quality inspection results include: inspection location information, inspection quantity information, and inspection category information of the vehicle exterior trim parts; the step of comparing the assembly quality inspection results with the standard configuration information, and outputting an error message when the comparison results are inconsistent, further includes: The detection location information is compared with the preset location information in the standard configuration information, the detection quantity information is compared with the preset quantity in the standard configuration information, and the detection category information is compared with the preset category in the standard configuration information.

[0069] The detection location information refers to the coordinates of the decorative element detected by the model in the image; for example, the bounding box position of the LiDAR in the image. The detection quantity information refers to the number of decorative element instances detected by the model; for example, the number of LiDARs in the image is 1. The detection category information refers to the type identifier of the decorative element detected by the model; for example, the category label for the LiDAR is "lidar". The preset location information refers to the range of installation locations for the decorative element specified in the standard configuration; for example, the LiDAR should be located within the image coordinates (100, 200) to (150, 250). The preset quantity refers to the number of decorative elements that should be installed as specified in the standard configuration; for example, one LiDAR should be installed per vehicle. The preset category refers to the type of decorative element specified in the standard configuration; for example, the preset category for the LiDAR is "radar".

[0070] If all comparison results are consistent, a pass / fail message will be output; if any comparison result is inconsistent, an error message will be output.

[0071] Among them, the qualified indication information refers to the signal output when the assembly quality inspection result is consistent with the standard configuration; for example, the "OK" signal output when the lidar is installed correctly.

[0072] Among the above-mentioned optional methods, the comparison mechanism of assembly quality inspection results can be further refined to conduct precise comparisons from dimensions such as location, quantity, and category, so as to ensure the reliability of the inspection results.

[0073] In an alternative approach, the method further includes: When the abnormal prompt message is output, a control command is sent to the production control system to stop the production line, and a repair prompt message is sent to the operator through the Internet of Things platform.

[0074] In this context, the production control system refers to the Manufacturing Operations Management (MOM) system that manages the production line; for example, an MOM system. Control commands refer to the operational commands sent to the production control system; for example, an instruction to stop the production line. The Internet of Things (IoT) platform refers to the network platform used for communication and data transmission between devices; for example, an IoT system that supports DingTalk message push. Operators refer to the production line workers responsible for handling assembly anomalies; for example, workers at the rework station. Rework notification information refers to rework notifications sent to operators through the IoT platform; for example, a DingTalk message such as "LiDAR installation abnormal, please handle promptly."

[0075] It should be noted that, as Figure 5 As shown, this embodiment uses a fixed camera on-site to capture images of the vehicle's exterior trim parts; the image data is sent to an edge server for automatic analysis and reasoning based on an AI model (vehicle exterior trim part assembly quality inspection model) to obtain assembly quality inspection results; the assembly quality inspection results are automatically compared with the standard configuration information of the production control system (MOM system), and a qualified (OK) or abnormal (NG) signal is output according to the comparison result; if the result is abnormal, a control command is automatically sent to the MOM system to stop the production line operation, and at the same time, the abnormal information is pushed to the relevant operators in a timely manner through the integrated IoT platform; the operators perform on-site rework according to the notification, and the vehicle is released after passing the inspection, and the whole process forms a closed-loop quality control.

[0076] Among the above-mentioned optional methods, further integration of anomaly alerts and production line control can achieve real-time alarms and rework reminders, thereby improving the intelligence and automation level of the production process.

[0077] Figure 6 A schematic diagram of an embodiment of the vehicle exterior trim assembly quality inspection device provided in this application is shown. Please refer to... Figure 6 As shown, the device 300 includes: a data acquisition module 310, a pre-labeling module 320, a training module 330, a detection module 340, and a comparison module 350.

[0078] The acquisition module 310 is used to acquire raw sample images containing vehicle exterior trim parts; The pre-annotation module 320 is used to pre-annotate the original sample images based on a pre-trained large model, obtain pre-annotation results, and generate an annotated dataset based on the pre-annotation results after manual verification. Training module 330 is used to perform data augmentation processing on the labeled dataset and use the augmented dataset to train a deep learning detection model for the assembly quality inspection of vehicle exterior trim parts, thereby obtaining a vehicle exterior trim part assembly quality inspection model; wherein, the deep learning detection model introduces a multi-channel inverse residual module in the Backbone layer of the YOLOv8 architecture and introduces a convolutional attention module in the C2f module of the YOLOv8 architecture. The detection module 340 is used to use the vehicle exterior trim assembly quality detection model to infer the image to be tested, which contains the vehicle exterior trim, and obtain the assembly quality detection result. The comparison module 350 is used to compare the assembly quality inspection results with the standard configuration information. When the comparison results are inconsistent, an abnormal prompt message is output.

[0079] In an alternative embodiment, the pre-annotation module 320 is specifically used for: The weights of the pre-trained large model are fine-tuned to obtain the fine-tuned large model; wherein the pre-trained large model has general zero-shot generalization ability. The original sample image is pre-labeled using the fine-tuned large model to obtain the pre-labeling result.

[0080] In an alternative embodiment, the training module 330 is specifically used for: Data augmentation operations are performed on the labeled sample images in the labeled dataset to obtain a first dataset; wherein the data augmentation operations include at least one of the following: image spatial transformation, image appearance transformation, and data normalization processing. Collect a negative sample dataset corresponding to the original sample image, and collect the negative sample dataset according to a set probability to obtain a second dataset; The enhanced dataset is constructed based on the first dataset and the second dataset.

[0081] In one alternative approach, the multi-channel inverse residual module is used for: The output feature map of the Backbone layer is divided into multiple partial feature maps, and inverse residual operations are performed on each of the multiple partial feature maps. The output features of each branch after the inverse residual operations are then concatenated. The concatenated features are processed to restore the number of channels. The processing result is added to the output feature map of the Backbone layer to obtain the first output feature map, which is then output.

[0082] In one alternative approach, the convolutional attention module is used for: The feature map input to the C2f module is sequentially subjected to channel attention mechanism and spatial attention mechanism. The channel weighted features are multiplied by the spatial attention weights to obtain the second output feature map and output it. The channel attention mechanism is used to: perform global average pooling and global max pooling operations on the input features and generate channel attention weights, and multiply the channel attention weights with the input features to obtain the channel-weighted features. The spatial attention mechanism is used to: perform convolution operations on the channel-weighted features to generate the spatial attention weights.

[0083] In one optional embodiment, the assembly quality inspection results include: inspection location information, inspection quantity information, and inspection category information of the vehicle exterior trim parts; the comparison module 350 is specifically used for: The detection location information is compared with the preset location information in the standard configuration information, the detection quantity information is compared with the preset quantity in the standard configuration information, and the detection category information is compared with the preset category in the standard configuration information. If all comparison results are consistent, a pass / fail message will be output; if any comparison result is inconsistent, an error message will be output.

[0084] In one alternative embodiment, the apparatus further includes: an exception handling module; the exception handling module is used for: When the abnormal prompt message is output, a control command is sent to the production control system to stop the production line, and a repair prompt message is sent to the operator through the Internet of Things platform.

[0085] The technical solution of this embodiment can effectively solve the problems of low efficiency, easy omission and false detection in manual inspection, and significantly improve the efficiency, accuracy and adaptability to complex working conditions of vehicle exterior trim assembly quality inspection.

[0086] It should be noted that the vehicle exterior trim assembly quality inspection device provided in the above embodiments and the vehicle exterior trim assembly quality inspection method provided in the aforementioned embodiments belong to the same concept. The specific way in which each module and unit performs its operation has been described in detail in the method embodiments, and will not be repeated here.

[0087] Figure 7 This document illustrates a structural schematic diagram of an embodiment of the vehicle exterior trim assembly quality inspection equipment provided in this application. It also shows a structural schematic diagram of a computer system suitable for implementing the vehicle exterior trim assembly quality inspection equipment of this application. The specific embodiments of this application do not limit the specific implementation of the vehicle exterior trim assembly quality inspection equipment.

[0088] Please see Figure 7As shown, the vehicle exterior trim assembly quality inspection equipment includes: a controller; and a memory for storing one or more programs, which, when executed by the controller, enable the controller to implement the aforementioned vehicle exterior trim assembly quality inspection method.

[0089] Please continue reading. Figure 7 As shown, the computer system 500 of the vehicle exterior trim assembly quality inspection equipment includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage section 508 into Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0090] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0091] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0092] Another aspect of this application provides a computer-readable storage medium storing at least one executable instruction that, when executed on a vehicle exterior trim assembly quality inspection device / equipment, causes the device / equipment to perform the operation of the vehicle exterior trim assembly quality inspection method as described above. This computer-readable storage medium may be included in the vehicle exterior trim assembly quality inspection device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0093] Another aspect of this application provides a computer program product or computer program that includes at least one executable instruction that, when executed on a vehicle exterior trim assembly quality inspection device / equipment, causes the vehicle exterior trim assembly quality inspection device / equipment to perform the vehicle exterior trim assembly quality inspection method as described above.

[0094] Specifically, the executable instructions can be used to cause the vehicle exterior trim assembly quality inspection equipment / device to perform the following operations: Collect raw sample images containing vehicle exterior trim parts; The original sample images are pre-annotated based on a pre-trained large model to obtain pre-annotation results, and an annotated dataset is generated based on the pre-annotation results after manual verification. The labeled dataset is augmented, and the augmented dataset is used to train a deep learning detection model for the assembly quality inspection of vehicle exterior trim parts, thereby obtaining a vehicle exterior trim part assembly quality inspection model. The deep learning detection model introduces a multi-channel inverse residual module in the Backbone layer of the YOLOv8 architecture and a convolutional attention module in the C2f module of the YOLOv8 architecture. Using the vehicle exterior trim assembly quality inspection model, reasoning is performed on the test image containing the vehicle exterior trim to obtain the assembly quality inspection result; The assembly quality inspection results are compared with the standard configuration information. If the comparison results are inconsistent, an error message is output.

[0095] In one alternative approach, the step of pre-labeling the sample images based on a pre-trained large model to obtain pre-labeling results further includes: The weights of the pre-trained large model are fine-tuned to obtain the fine-tuned large model; wherein the pre-trained large model has general zero-shot generalization ability. The original sample image is pre-labeled using the fine-tuned large model to obtain the pre-labeling result.

[0096] In an alternative approach, the step of performing data augmentation on the labeled dataset further includes: Data augmentation operations are performed on the labeled sample images in the labeled dataset to obtain a first dataset; wherein the data augmentation operations include at least one of the following: image spatial transformation, image appearance transformation, and data normalization processing. Collect a negative sample dataset corresponding to the original sample image, and collect the negative sample dataset according to a set probability to obtain a second dataset; The enhanced dataset is constructed based on the first dataset and the second dataset.

[0097] In one alternative approach, the multi-channel inverse residual module is used for: The output feature map of the Backbone layer is divided into multiple partial feature maps, and inverse residual operations are performed on each of the multiple partial feature maps. The output features of each branch after the inverse residual operations are then concatenated. The concatenated features are processed to restore the number of channels. The processing result is added to the output feature map of the Backbone layer to obtain the first output feature map, which is then output.

[0098] In one alternative approach, the convolutional attention module is used for: The feature map input to the C2f module is sequentially subjected to channel attention mechanism and spatial attention mechanism. The channel weighted features are multiplied by the spatial attention weights to obtain the second output feature map and output it. The channel attention mechanism is used to: perform global average pooling and global max pooling operations on the input features and generate channel attention weights, and multiply the channel attention weights with the input features to obtain the channel-weighted features. The spatial attention mechanism is used to: perform convolution operations on the channel-weighted features to generate the spatial attention weights.

[0099] In one optional approach, the assembly quality inspection results include: inspection location information, inspection quantity information, and inspection category information of the vehicle exterior trim parts; the step of comparing the assembly quality inspection results with the standard configuration information, and outputting an error message when the comparison results are inconsistent, further includes: The detection location information is compared with the preset location information in the standard configuration information, the detection quantity information is compared with the preset quantity in the standard configuration information, and the detection category information is compared with the preset category in the standard configuration information. If all comparison results are consistent, a pass / fail message will be output; if any comparison result is inconsistent, an error message will be output.

[0100] In an alternative approach, the method further includes: When the abnormal prompt message is output, a control command is sent to the production control system to stop the production line, and a repair prompt message is sent to the operator through the Internet of Things platform.

[0101] The technical solution of this embodiment can effectively solve the problems of low efficiency, easy omission and false detection in manual inspection, and significantly improve the efficiency, accuracy and adaptability to complex working conditions of vehicle exterior trim assembly quality inspection.

[0102] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0104] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0105] According to one aspect of the embodiments of this application, a computer system is also provided, including a Central Processing Unit (CPU), which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from storage into random access memory (RAM), such as performing the methods described above. Various programs and data required for system operation are also stored in the RAM. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0106] The following components are connected to the I / O interface: input components including keyboards, mice, etc.; output components including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage components including hard drives; and communication components including network interface cards such as LAN (Local Area Network) cards and modems. The communication components perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage components as required.

[0107] The above content is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A method for inspecting the assembly quality of vehicle exterior trim parts, characterized in that, The method includes: Collect raw sample images containing vehicle exterior trim parts; The original sample images are pre-annotated based on a pre-trained large model to obtain pre-annotation results, and an annotated dataset is generated based on the pre-annotation results after manual verification. The labeled dataset is augmented, and the augmented dataset is used to train a deep learning detection model for the assembly quality inspection of vehicle exterior trim parts, thereby obtaining a vehicle exterior trim part assembly quality inspection model. The deep learning detection model introduces a multi-channel inverse residual module in the Backbone layer of the YOLOv8 architecture and a convolutional attention module in the C2f module of the YOLOv8 architecture. Using the vehicle exterior trim assembly quality inspection model, reasoning is performed on the test image containing the vehicle exterior trim to obtain the assembly quality inspection result; The assembly quality inspection results are compared with the standard configuration information. If the comparison results are inconsistent, an error message is output.

2. The method according to claim 1, characterized in that, The step of pre-annotating the original sample image based on the pre-trained large model to obtain the pre-annotation result further includes: The weights of the pre-trained large model are fine-tuned to obtain the fine-tuned large model; wherein the pre-trained large model has general zero-shot generalization ability. The original sample image is pre-labeled using the fine-tuned large model to obtain the pre-labeling result.

3. The method according to claim 1, characterized in that, The step of performing data augmentation on the labeled dataset further includes: Data augmentation operations are performed on the labeled sample images in the labeled dataset to obtain a first dataset; wherein the data augmentation operations include at least one of the following: image spatial transformation, image appearance transformation, and data normalization processing. Collect a negative sample dataset corresponding to the original sample image, and collect the negative sample dataset according to a set probability to obtain a second dataset; The enhanced dataset is constructed based on the first dataset and the second dataset.

4. The method according to any one of claims 1 to 3, characterized in that, The multi-channel inverse residual module is used for: The output feature map of the Backbone layer is divided into multiple partial feature maps, and inverse residual operations are performed on each of the multiple partial feature maps. The output features of each branch after the inverse residual operations are then concatenated. The concatenated features are processed to restore the number of channels. The processing result is added to the output feature map of the Backbone layer to obtain the first output feature map, which is then output.

5. The method according to any one of claims 1 to 3, characterized in that, The convolutional attention module is used for: The feature map input to the C2f module is sequentially subjected to channel attention mechanism and spatial attention mechanism. The channel weighted features are multiplied by the spatial attention weights to obtain the second output feature map and output it. The channel attention mechanism is used to: perform global average pooling and global max pooling operations on the input features and generate channel attention weights, and multiply the channel attention weights with the input features to obtain the channel-weighted features. The spatial attention mechanism is used to: perform convolution operations on the channel-weighted features to generate the spatial attention weights.

6. The method according to claim 1, characterized in that, The assembly quality inspection results include: inspection location information, inspection quantity information, and inspection category information of the vehicle exterior trim parts; the step of comparing the assembly quality inspection results with the standard configuration information, and outputting an error message when the comparison results are inconsistent, further includes: The detection location information is compared with the preset location information in the standard configuration information, the detection quantity information is compared with the preset quantity in the standard configuration information, and the detection category information is compared with the preset category in the standard configuration information. If all comparison results are consistent, a pass / fail message will be output; if any comparison result is inconsistent, an error message will be output.

7. The method according to claim 6, characterized in that, The method further includes: When the abnormal prompt message is output, a control command is sent to the production control system to stop the production line, and a repair prompt message is sent to the operator through the Internet of Things platform.

8. A device for inspecting the assembly quality of vehicle exterior trim parts, characterized in that, The device includes: The acquisition module is used to acquire raw sample images containing vehicle exterior trim parts; The pre-annotation module is used to pre-annotate the original sample images based on a pre-trained large model, obtain pre-annotation results, and generate an annotated dataset based on the pre-annotation results after manual verification. The training module is used to perform data augmentation processing on the labeled dataset and use the augmented dataset to train a deep learning detection model for the assembly quality inspection of vehicle exterior trim parts, thereby obtaining a vehicle exterior trim part assembly quality inspection model; wherein, the deep learning detection model introduces a multi-channel inverse residual module in the Backbone layer of the YOLOv8 architecture and introduces a convolutional attention module in the C2f module of the YOLOv8 architecture. The detection module is used to use the vehicle exterior trim assembly quality detection model to infer the image to be tested, which contains the vehicle exterior trim, and obtain the assembly quality detection result. The comparison module is used to compare the assembly quality inspection results with the standard configuration information. When the comparison results are inconsistent, an abnormal prompt message is output.

9. A vehicle exterior trim assembly quality inspection device, characterized in that, include: Controller; A memory for storing one or more programs, which, when executed by a controller, cause the controller to implement the vehicle exterior trim assembly quality inspection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on the vehicle exterior trim assembly quality inspection device / equipment, causes the vehicle exterior trim assembly quality inspection device / equipment to perform the operation of the vehicle exterior trim assembly quality inspection method as described in any one of claims 1-7.

Images