A machine vision-based automobile accessory detection method and system

Abstract

The application belongs to the technical field of automobile part detection, and provides a kind of automobile accessory detection method and system based on machine vision, the method includes obtaining three wave band images;Respectively to the three wave band images are pretreated;The three wave band images after pretreatment are input into the detection model trained in advance, and the preliminary detection result containing defect type, position and confidence is obtained;The confidence of the preliminary detection result is compared with the confidence threshold value set in advance, and the final detection result is obtained;Through the construction of improved YOLOv8 model, the detection is carried out in combination with ultraviolet light, visible light and near infrared light three wave band images, wherein the ultraviolet light image is suitable for encapsulation layer surface defect detection, the visible light image clearly presents the bonding wire form, the near infrared light image can penetrate the encapsulation layer to capture internal structure information, through model multi-branch feature extraction, fusion and accurate prediction, the identification of IGBT module weld micro-crack, bonding wire offset and other defects is realized.

Description

Technical Field

[0001] This invention belongs to the field of automotive parts inspection technology, and specifically relates to an automotive parts inspection method and system based on machine vision. Background Technology

[0002] IGBT modules (Insulated Gate Bipolar Transistors) are fully controllable, voltage-driven power semiconductor devices. They are core components of core power units such as automotive inverters and motor controllers. Internally, they integrate multiple layers of precision structures, including chips, solder joints, bonding wires, ceramic substrates, and epoxy resin encapsulation layers. These components work together to achieve efficient conversion and control of electrical energy.

[0003] In the brazing process between chips and ceramic substrates, residual stress is easily formed due to differences in thermal expansion coefficients, which can easily lead to microcracks in the solder joints after manufacturing. During the bonding process of aluminum wires connecting chips and electrodes, slight misalignment or hidden fractures often occur due to fluctuations in welding temperature and mechanical vibration. If residual gas or impurities are present during the encapsulation injection molding stage, tiny bubbles or foreign objects will form inside the encapsulation layer. These defects are initially very small, but they can rapidly expand under the high-frequency vibration, high and low temperature cycling, and high current impact environment of vehicle operation. They can cause module conduction failure within a few thousand kilometers, leading to serious malfunctions such as power interruption and inverter burnout, and even inducing vehicle safety accidents.

[0004] Machine vision technology, with its advantages of being non-contact, highly efficient, and capable of batch inspection, has gradually replaced manual visual inspection and become the mainstream method for quality control of power semiconductor modules.

[0005] However, existing machine vision inspection technologies mostly use a single visible light imaging scheme. Although they can detect visible defects such as scratches and stains on the surface of the encapsulation layer, the epoxy resin encapsulation layer has poor transmittance to visible light, making it impossible to obtain structural information of internal solder joints and bonding lines. This results in insufficient ability to identify defects inside the module, and the detection accuracy is easily affected by factors such as ambient light intensity and light source attenuation in the workshop, making it difficult to meet the inspection requirements. Summary of the Invention

[0006] To address the problems in the background art, this invention proposes a machine vision-based method and system for inspecting automotive parts.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention proposes a machine vision-based method for inspecting automotive parts, comprising:

[0009] Acquire three-band images; wherein the three-band images include ultraviolet light images, visible light images, and near-infrared light images;

[0010] The three-band images are preprocessed respectively;

[0011] The preprocessed three-band image is input into a pre-trained detection model to obtain preliminary detection results including defect type, location, and confidence level. The detection model is built based on the YOLOv8 model and includes a serially connected input layer, backbone network, neck network, head network, and output layer.

[0012] The confidence level of the preliminary detection results is compared with a pre-set confidence threshold to obtain the final detection result.

[0013] Furthermore, the automotive component is an IGBT module.

[0014] Furthermore, the acquisition of the three-band image specifically involves: instructing the ultraviolet light, visible light, and near-infrared light sources to be turned on sequentially, instructing the camera to capture the corresponding band reflection signals, and receiving the image data transmitted by the camera to obtain the three-band image.

[0015] Furthermore, the preprocessing of the three-band images specifically includes:

[0016] Gaussian filtering was used to remove high-frequency noise from each single-band image;

[0017] Adaptive histogram equalization is used to enhance the contrast between defect areas and the background in each single-band image.

[0018] Pixel values ​​of a normalized three-band image.

[0019] Furthermore, the detection model is trained to obtain a well-trained detection model, specifically including:

[0020] Obtain IGBT module samples; wherein, the IGBT module samples include qualified samples and defective samples;

[0021] A multispectral dataset was constructed based on the IGBT module samples.

[0022] The multispectral dataset is divided into a training set and a validation set;

[0023] The detection model is trained and validated using the training set and validation set to obtain a trained detection model.

[0024] Furthermore, the ratio of the training set to the validation set is 7:3.

[0025] Furthermore, the backbone network includes three parallel branches, each of which is serially connected to multiple C2f modules combined with coordinate attention modules and SPPF modules; the neck network includes serially connected feature alignment modules, adaptive weight fusion modules, and scale recovery modules.

[0026] Furthermore, the preprocessed three-band image is input into a pre-trained detection model to obtain preliminary detection results including defect type, location, and confidence level, specifically including:

[0027] The input layer receives the preprocessed three-band image;

[0028] Each branch of the backbone network receives images of the corresponding band, and iterates and repeats multiple times through the combination of the C2f module and the coordinate attention module to gradually improve the feature abstraction. Then, the SPPF module fuses multi-scale features to obtain the core feature map of each band.

[0029] The feature alignment module unifies the dimensions of the core feature maps of each band through convolution operations; the adaptive weight fusion module calculates the confidence weights of each branch and sums them up; and the scale recovery module optimizes the feature resolution through upsampling and feature stitching to obtain the fused feature map.

[0030] The head network completes defect type identification and bounding box localization of the fused feature map through classification and regression branches, respectively. Combined with cross-entropy loss and loss optimization parameters, it obtains preliminary detection results including defect type, location and confidence.

[0031] The output layer outputs preliminary detection results containing defect type, location, and confidence level.

[0032] Secondly, this invention proposes a machine vision-based automotive parts inspection system, comprising:

[0033] An image acquisition module is used to acquire three-band images; wherein the three-band images include ultraviolet light images, visible light images, and near-infrared light images;

[0034] The preprocessing module is used to preprocess the three-band images respectively;

[0035] The detection module is used to input the preprocessed three-band image into a pre-trained detection model to obtain preliminary detection results including defect type, location, and confidence level; wherein, the detection model is built based on the YOLOv8 model, and the detection model includes a serially connected input layer, backbone network, neck network, head network, and output layer;

[0036] The judgment module is used to compare the confidence level of the preliminary detection result with a preset confidence level threshold to obtain the final detection result.

[0037] The beneficial effects of this invention are:

[0038] The method of this invention constructs an improved YOLOv8 model and combines ultraviolet, visible, and near-infrared light images for detection. The ultraviolet light image is adapted for detecting surface defects in the packaging layer, the visible light image clearly presents the bonding line morphology, and the near-infrared light image can penetrate the packaging layer to capture internal structural information. Through multi-branch feature extraction, fusion, and accurate prediction by the model, defects such as micro-cracks in IGBT module solder joints and bonding line misalignment can be identified, which can effectively solve the problem that the existing technology with a single light source is insufficient for identifying core internal defects.

[0039] The method of this invention extracts features by inputting ultraviolet, visible, and near-infrared light images into the three branches of the detection model, and fuses multi-scale features through an adaptive weight fusion module, automatically assigning weights to features of different bands, which can improve the recognition accuracy of complex defects.

[0040] The method of the present invention enhances the detection model’s sensitivity to the location of minute defects (such as microcracks in weld joints) by adding a coordinate attention module to the detection model and encoding the horizontal and vertical coordinate information of the image, thus solving the problem of insufficient feature extraction of small-sized targets in the prior art.

[0041] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description and the drawings. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart of a machine vision-based automotive parts inspection method according to the present invention is shown;

[0044] Figure 2 A framework diagram of a machine vision-based automotive parts inspection system according to the present invention is shown.

[0045] Figure 3 A schematic diagram of the device structure of the present invention is shown;

[0046] Figure 4 A schematic diagram of the structure of the computer-readable storage medium of the present invention is shown. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Reference Figure 1 As shown, a machine vision-based method for inspecting automotive parts, specifically IGBT modules (Insulated Gate Bipolar Transistors), includes the following steps:

[0049] S10. Acquire three-band images; wherein, the three-band images include ultraviolet light images, visible light images, and near-infrared light images;

[0050] S20. Preprocess the three-band images respectively;

[0051] S30. Input the preprocessed three-band image into the pre-trained detection model to obtain preliminary detection results including defect type, location and confidence level; wherein, the detection model is built based on the YOLOv8 model and includes a serially connected input layer, backbone network, neck network, head network and output layer;

[0052] S40. Compare the confidence level of the preliminary detection result with the preset confidence level threshold to obtain the final detection result.

[0053] In step S10 above, the method for obtaining the three-band image is as follows: ultraviolet, visible, and near-infrared light sources are turned on respectively, and the corresponding band reflection signals are captured to obtain the three-band image. Specifically, a three-band light source array containing ultraviolet (200-400nm), visible (400-760nm), and near-infrared (760-1100nm) light sources is first constructed. The light sources, camera, and lens are arranged coaxially to ensure that the light perpendicularly illuminates the surface of the IGBT module, and three positioning points are fixed to establish a unified image coordinate system. In a specific embodiment, ultraviolet (exposure 50ms), visible (exposure 30ms), and near-infrared (exposure 80ms) light sources are turned on respectively, and the corresponding band reflection signals are captured by the camera and lens to generate a 640×640×3 three-band image.

[0054] Ultraviolet light effectively highlights the reflectivity differences of scratches on the IGBT module's encapsulation layer, enhancing the identification of surface defects. Visible light provides high fidelity for bond line morphology, clearly capturing details of misalignment or breakage. Near-infrared light penetrates the epoxy resin encapsulation layer, reaching internal solder joints and the encapsulation layer, thus enabling the identification of defects such as microcracks and bubbles. By constructing a three-band light source array, independent acquisition of the three bands of light can be achieved, thereby addressing the problem of insufficient identification capability of core internal defects using a single light source in existing technologies.

[0055] In step S20 above, preprocessing the three-band images separately means performing preprocessing operations on each image individually, rather than merging the ultraviolet, visible, and near-infrared images and processing them uniformly. The method for preprocessing the three-band images separately is as follows: first, Gaussian filtering is used to remove high-frequency noise from each single-band image; then, adaptive histogram equalization is used to enhance the contrast between defect areas and the background in each single-band image; finally, the three-band images are standardized: the pixel values ​​of the three-band images are normalized to the [0,1] interval to eliminate the interference caused by differences in illumination intensity between different bands to subsequent feature extraction. The final standardized three-band images (preprocessed three-band images) are obtained, and these standardized three-band images serve as inputs to the three parallel branches of the subsequent backbone network.

[0056] In step S30 above, the detection model is trained to obtain a trained detection model, which specifically includes the following steps:

[0057] S3011. Obtain IGBT module samples; wherein, the IGBT module samples include qualified samples and defective samples;

[0058] S3012. Construct a multispectral dataset based on the IGBT module samples;

[0059] S3013. Divide the multispectral dataset into a training set and a validation set in a 7:3 ratio;

[0060] S3014. Train and validate the detection model using the training set and validation set to obtain a trained detection model;

[0061] In one specific embodiment, 5000 IGBT module samples were collected, including 3000 qualified samples and 2000 defective samples (including defects such as solder joint microcracks, bond line misalignment, and encapsulation layer bubbles). Ultraviolet, visible, and near-infrared images were acquired for each sample, and defect information was manually labeled to construct a multispectral dataset. This dataset was divided into a training set and a validation set in a 7:3 ratio. The detection model was trained using the training set and validated using the validation set. The parameters of the detection model were then fine-tuned to obtain the finally trained detection model.

[0062] In step S30 above, the backbone network includes three parallel branches (corresponding to ultraviolet, visible, and near-infrared light images, respectively), and each branch is serially connected to multiple C2f modules and coordinate attention modules, as well as SPPF modules; the neck network includes serially connected feature alignment modules, adaptive weight fusion modules, and scale recovery modules.

[0063] The preprocessed three-band image is input into a pre-trained detection model to obtain preliminary detection results containing defect type, location, and confidence level. The specific steps include the following:

[0064] S3021, The input layer receives the preprocessed three-band image;

[0065] S3022. Each branch of the backbone network receives the corresponding band image, and iterates and repeats multiple times through the combination of the C2f module and the coordinate attention module to gradually improve the feature abstraction. Then, the SPPF module fuses multi-scale features to obtain the core feature map of each band.

[0066] S3023, the feature alignment module unifies the dimension of the core feature maps of each band through 1×1 convolution, the adaptive weight fusion module calculates the confidence weight of each branch and sums them up, and the scale recovery module optimizes the feature resolution through upsampling and feature stitching to obtain the fused feature map.

[0067] S3024. The head network completes defect type identification and bounding box localization of the fused feature map through classification and regression branches respectively. Combined with cross-entropy loss and loss optimization parameters, it obtains preliminary detection results including defect type, location and confidence.

[0068] S3025, The output layer outputs preliminary detection results including defect type, location, and confidence level.

[0069] In step S3022 above, the backbone network is a three-branch parallel structure. Each branch corresponds to the feature extraction of one spectral image. The original C2f module and SPPF (spatial pyramid pooling fusion) module of YOLOv8 are used inside the branch. However, a coordinate attention module (CA module) is inserted after each C2f module to enhance the features of small defects.

[0070] Specifically, by inputting ultraviolet, visible, and near-infrared images into the three branches of the detection model for feature extraction, and then using an adaptive weight fusion module to fuse multi-scale features, the model automatically assigns weights to features of different bands (e.g., the weight coefficient of near-infrared image for weld defects is set to 0.6, and the weight coefficient of visible light for bonding lines is set to 0.5), which can improve the recognition accuracy of complex defects.

[0071] Specifically, by adding a coordinate attention module to the detection model and encoding the horizontal and vertical coordinate information of the image, the detection model's sensitivity to the location of minute defects (such as microcracks in weld joints) is enhanced, solving the problem of insufficient feature extraction for small-sized targets in existing technologies.

[0072] In a specific embodiment, taking the near-infrared light branch (for detecting microcracks in weld joints) as an example: First stage: Input the preprocessed near-infrared image (640×640×3) into the first C2f module (64 convolution kernels, stride 2), and extract the basic texture features through the "convolution-residual connection-feature stitching" operation, and output the feature map with a size of 320×320×64;

[0073] Attention Enhancement: The C2f output feature map is input into the CA module. First, global average pooling is used to encode the horizontal and vertical coordinates respectively, resulting in horizontal features of dimension (320, 64) and vertical features of dimension (320, 64). Then, the feature dimension is compressed to (320, 16) through 1×1 convolution, and coordinate attention weights are generated by Sigmoid activation function. Finally, the weights are multiplied element-wise with the original feature map to enhance the positional features of linear defects such as microcracks, and the enhanced feature map (320×320×64) is output.

[0074] The coordinate attention module calculation process is implemented using the following formula:

[0075] Global average pooling formula for coordinates: for the input feature map (in, (These are the feature map height, width, and number of channels, respectively). Global average pooling is performed along the horizontal and vertical directions to obtain the horizontal feature map. and longitudinal features Their expressions are as follows:

[0076] ;

[0077] ;

[0078] In the formula, Indicates the first The first channel, the first The results of horizontal pooling of rows, Indicates the first The first channel, the first The vertical pooling result of the column.

[0079] Feature fusion and activation formula: and After concatenation, the dimensions are compressed by a 1×1 convolution, and then coordinate attention weights are generated by a Sigmoid activation function. :

[0080] ;

[0081] In the formula, For splicing operations, This is a 1×1 convolution operation. This is the Sigmoid activation function.

[0082] Feature enhancement formula: weights Compared with the original feature map Element-wise multiplication yields the enhanced feature map. :

[0083] ;

[0084] Second to fourth stages: Repeat the "C2f module-CA module" combination, and compress the feature map size to 160×160×128, 80×80×256, and 40×40×512 in turn, gradually improving the feature abstraction level;

[0085] SPPF Pooling: The 40×40×512 feature map output by the final C2f module is input into the SPPF module. Multi-scale features are fused through pooling operations at three scales: 5×5, 9×9, and 13×13, and the core feature map of the near-infrared branch (40×40×512) is output.

[0086] It should be further explained that the ultraviolet light branch (encapsulation layer surface defects) and the visible light branch (bonding line defects) use the same structural parameters, and are only adaptively adjusted in the weight coefficient of the CA module (such as increasing the weight coefficient of the visible light branch CA module to 0.7 to enhance the position sensitivity of the linear bonding line), and finally output 40×40×512 ultraviolet feature maps and visible light feature maps respectively.

[0087] In step S3023 above, the neck network is the core layer for realizing multispectral feature fusion. It adopts an architecture of "upsampling-feature concatenation-adaptive weight allocation" to fuse the feature maps of the three branches into a unified multispectral feature map (fused feature map). Specifically, it includes the following steps:

[0088] S30231, Feature Alignment: The 40×40×512 feature maps output from the ultraviolet light, visible light, and near-infrared light branches are unified to 40×40×256 through 1×1 convolution to eliminate the difference in feature dimension between different branches.

[0089] S30232, Adaptive Fusion: Construct an adaptive weighted fusion module. Input three aligned feature maps, and calculate the confidence weight of each feature map through a fully connected layer (e.g., near-infrared branch solder joint defect weight 0.6, visible light branch bonding line weight 0.5, ultraviolet light branch surface defect weight 0.4). The fused feature map (40×40×256) is obtained by weighted summation. The core fusion formula is: ;

[0090] In the formula, This is the fused feature map. These are alignment feature maps for the ultraviolet, visible, and near-infrared light branches, respectively. Adaptive weights for each branch (satisfying) This is obtained through dynamic learning via fully connected layers. This represents the number of channels in the feature map.

[0091] S30233, Scale Restoration: The fused feature map size is restored to 80×80×128 by the upsampling module (step size 0.5), and then it is spliced ​​with the intermediate feature map of 80×80×256 in the backbone network. After optimization by the C2f module, the final neck feature map (80×80×192) is output, providing high-resolution, multi-dimensional fused features for the head network.

[0092] In step S3024 above, the head network adopts the YOLOv8 decoupledHead structure to separate the classification task from the regression task, and introduces the CIoU loss function to optimize the bounding box prediction accuracy. Specifically, it includes the following steps:

[0093] S30241, Feature Input: Receive the 80×80×192 fused feature map output by the neck network, and input it into the classification branch and regression branch respectively;

[0094] S30242, Classification Branch: The feature dimension is compressed to 80×80×64 through two 1×1 convolutions. The probability of each pixel belonging to defects such as "micro-cracks in solder joints", "bonding line offset", and "bubbles in encapsulation layer" is calculated by the Softmax activation function, and the classification feature map (80×80×6) is output.

[0095] S30243, Regression Branch: Adopts the same convolutional structure as the classification branch, and outputs a regression feature map (80×80×5) containing the x and y coordinates, width, height and confidence of the defect bounding box.

[0096] S30244, Loss Optimization: Calculate the classification loss (cross-entropy loss) and regression loss (CIoU loss), and update the entire model parameters through backpropagation. The CIoU loss incorporates metrics such as the distance to the bounding box center point and aspect ratio error, making the prediction of defects requiring precise localization, such as bond line offset, more accurate. The core loss function expression is:

[0097] Classification loss (cross-entropy loss): used to optimize the accuracy of defect type classification, the formula is: ;

[0098] In the formula, For the sample size, This represents the total number of defect categories. This is a single defect category number. For the k-th sample The actual label of the class (0 or 1). For the k-th sample The predicted probability of a class.

[0099] Regression loss (CIoU loss): Used to optimize the accuracy of defect bounding box localization, the formula is: ;

[0100] In the formula: To predict bounding boxes With the true bounding box The intersection and union ratio, The square of the Euclidean distance between the centers of the two bounding boxes; The length of the diagonal of the smallest rectangle that encloses the two bounding boxes; For balance coefficient, ; For aspect ratio consistency parameters, ( To predict the width and height of the bounding box, (This refers to the actual frame width and height).

[0101] The total loss function is a weighted sum of the classification loss and the regression loss, and its expression is: ;

[0102] In the formula, , These are all loss weighting coefficients. It should be further noted that... and It can be adapted to meet the needs of the testing task.

[0103] By replacing the traditional GIoU loss function with the CIoU loss function, and by incorporating information such as the distance to the center point of the target and the aspect ratio, the prediction accuracy of the detection model for the defect bounding box can be improved, making it more suitable for defects that require precise localization, such as bond line offset.

[0104] In step S40 above, the confidence level of each defect in the preliminary detection result is compared with a pre-set confidence threshold (e.g., 0.5). If the confidence level of the defect is greater than or equal to 0.5, it is a reliable result, and the type, location coordinates and confidence level information of the defect are retained. If the confidence level of the defect is less than 0.5, it is an unreliable result, and the defect record is directly removed. This can avoid the interference of low confidence prediction on quality judgment. Finally, the final detection result after confidence verification is output.

[0105] Reference Figure 2 As shown, based on the same inventive concept as the above method, the present invention also proposes a machine vision-based automotive parts inspection system, comprising:

[0106] The image acquisition module 110 is used to acquire three-band images; wherein, the three-band images include ultraviolet light images, visible light images, and near-infrared light images;

[0107] Preprocessing module 120 is used to preprocess the three-band images respectively;

[0108] The detection module 130 is used to input the preprocessed three-band image into a pre-trained detection model to obtain preliminary detection results including defect type, location and confidence level; wherein, the detection model is built based on the YOLOv8 model and includes a serially connected input layer, backbone network, neck network, head network and output layer;

[0109] The judgment module 140 is used to compare the confidence level of the preliminary detection result with a preset confidence threshold to obtain the final detection result.

[0110] It should be further clarified that the above system and method correspond one-to-one, and the design, architecture, and operating mechanism of each functional module are closely aligned with the steps and stages of the method described below. Therefore, this paper provides a complete exposition of the method and a brief introduction to the above system. It should be understood that, based on the complete exposition of the method, those skilled in the art can clearly understand the structure and operating logic of the system.

[0111] Reference Figure 3 As shown, based on the same inventive concept as the above method, the present invention also proposes a device including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and the processor executes the above-described machine vision-based automotive parts inspection method when executing the computer instructions.

[0112] Reference Figure 4As shown, based on the same inventive concept as the above method, the present invention also proposes a computer-readable storage medium storing computer instructions thereon, which, when executed, can realize the above-mentioned machine vision-based automotive parts inspection method.

[0113] Any references to memory, storage, database, or other media used in the embodiments provided in this invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0114] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine vision based method of detecting automotive parts, characterized in that, The method comprises the following steps: obtaining three-band images; wherein the three-band images include ultraviolet light images, visible light images and near-infrared light images; respectively pre-processing the three-band images; inputting the pre-processed three-band images into a pre-trained detection model to obtain preliminary detection results containing defect types, positions and confidence levels; wherein the detection model is constructed based on a YOLOv8 model, and the detection model comprises an input layer, a backbone network, a neck network, a head network and an output layer connected in series; the backbone network comprises three branches connected in parallel, and each branch is connected in series with a combination of a C2f module and a coordinate attention module and an SPPF module; the neck network comprises a feature alignment module, an adaptive weight fusion module and a scale recovery module connected in series; the input layer receives the pre-processed three-band images; each branch of the backbone network receives the corresponding band image, iteratively repeats multiple rounds through the C2f module and the coordinate attention module combination, gradually improves the feature abstraction degree, and then fuses multi-scale features through the SPPF module to obtain core feature maps of each band; the feature alignment module unifies the dimensions of the core feature maps of each band through convolution operation, the adaptive weight fusion module calculates and weights the confidence weight of each branch and sums it up, and the scale recovery module optimizes the feature resolution through upsampling and feature splicing to obtain a fusion feature map; compare the confidence level of the preliminary detection results with the pre-set confidence threshold to obtain the final detection results.

2. The machine vision based automobile component detection method as claimed in claim 1, wherein, The automobile accessory is an IGBT module.

3. The machine vision based automobile component detection method as claimed in claim 2, wherein, The three-band images are obtained by sequentially turning on the ultraviolet light, visible light and near-infrared light sources according to instructions, and capturing the corresponding band reflection signals by the camera according to instructions, and receiving the image data transmitted by the camera to obtain the three-band images.

4. The machine vision based automobile component detection method as claimed in claim 2, wherein, The pre-processing of the three-band images specifically includes: using Gaussian filtering to remove high-frequency noise from each single-band image; enhancing the contrast between the defect area and the background of each single-band image through adaptive histogram equalization; normalizing the pixel values of the three-band images.

5. The machine vision based automobile component detection method as claimed in claim 2, wherein, Training the detection model to obtain the trained detection model, specifically including: obtaining IGBT module samples; wherein the IGBT module samples include qualified samples and defect samples; constructing a multi-spectral dataset based on the IGBT module samples; dividing the multi-spectral dataset into a training set and a validation set; training and verifying the detection model using the training set and the validation set to obtain the trained detection model.

6. The machine vision based automobile component detection method as claimed in claim 5, wherein, The division ratio of the training set and the validation set is 7:

3.

7. The machine vision based automobile component detection method as claimed in claim 1 wherein, The input layer receives the pre-processed three-band images; each branch of the backbone network receives the corresponding band image, iteratively repeats multiple rounds through the C2f module and the coordinate attention module combination, gradually improves the feature abstraction degree, and then fuses multi-scale features through the SPPF module to obtain core feature maps of each band; ​ The feature alignment module unifies the dimensions of the core feature maps of each waveband through a convolution operation, the adaptive weight fusion module calculates confidence weights of each branch and performs weighted summation, and the scale recovery module optimizes the feature resolution through upsampling and feature concatenation to obtain a fused feature map; The head network completes defect type identification and bounding box positioning of the fused feature map through a classification branch and a regression branch respectively, combines cross-entropy loss and loss optimization parameters to obtain a preliminary detection result containing defect type, position and confidence; An output layer outputs a preliminary detection result containing defect type, position and confidence.

8. A machine vision based automotive component inspection system characterized in that, The method comprises the following steps: An image acquisition module is configured to acquire a three-waveband image; wherein the three-waveband image comprises an ultraviolet image, a visible light image and a near-infrared image; A preprocessing module is configured to preprocess the three-waveband images respectively; A detection module is configured to input the preprocessed three-waveband images into a pre-trained detection model to obtain a preliminary detection result containing defect type, position and confidence; wherein the detection model is constructed based on a YOLOv8 model, and the detection model comprises an input layer, a backbone network, a neck network, a head network and an output layer connected in series; The backbone network comprises three branches connected in parallel, and each branch is connected in series with a plurality of C2f modules and a coordinate attention module combination and an SPPF module; The neck network comprises a feature alignment module, an adaptive weight fusion module and a scale recovery module connected in series; The input layer receives the preprocessed three-waveband images; Each branch of the backbone network receives the corresponding waveband image, iteratively repeats multiple rounds through the C2f module and the coordinate attention module combination to gradually improve the feature abstraction degree, and then fuses multi-scale features through the SPPF module to obtain core feature maps of each waveband; The feature alignment module unifies the dimensions of the core feature maps of each waveband through a convolution operation, the adaptive weight fusion module calculates confidence weights of each branch and performs weighted summation, and the scale recovery module optimizes the feature resolution through upsampling and feature concatenation to obtain a fused feature map; A judgment module is configured to compare the confidence of the preliminary detection result with a pre-set confidence threshold to obtain a final detection result.

Images