A Defect Classification Method and System for Acoustic Modules Based on Multi-Feature Fusion

Abstract

This invention belongs to the field of industrial automation inspection technology, specifically relating to a method and system for classifying acoustic module defects based on multi-feature fusion. The method includes: acquiring multi-angle polarization images of the acoustic module under test; calculating and generating basic feature maps such as illumination intensity, degree of polarization, and polarization angle; decomposing the light intensity into diffuse reflection and specular reflection components using the degree of polarization; and constructing a material inhomogeneity index and a geometric distortion index to enhance weak defect signals. Through online prototype learning, this method can generate a robust material distribution map unaffected by illumination and batch variations. This invention uses this material distribution map as a spatial context, invoking specific classification rules based on the different material regions where the defect is located, thereby distinguishing defects from normal structures, reducing the false alarm rate of traditional detection methods, and achieving accurate and robust classification of complex surface defects.

Description

Technical Field

[0001] This invention relates to the field of industrial automation inspection technology. More specifically, this invention relates to a method and system for classifying acoustic module defects based on multi-feature fusion. Background Technology

[0002] As precision electronic components, acoustic modules integrate various materials such as diaphragms, metal shells, and solder pads on their surfaces, and contain complex curved and inclined surface structures. Traditional detection methods based on two-dimensional grayscale images struggle to effectively distinguish between variations caused by inherent differences in material reflectivity and highlights or shadows caused by surface geometry. This confusion between material and geometric information results in persistently high false alarm and false negative rates for defect detection when dealing with such complex surfaces, constituting a pressing technical problem that needs to be solved.

[0003] To address the aforementioned issues, existing technologies have introduced polarization imaging techniques, which decouple material and geometric information by analyzing the polarization state of light. This primarily employs a physics-based reflection model algorithm, which uses information such as polarization degree to decompose the total collected light intensity into a diffuse reflection component related to material properties and a specular reflection component related to surface geometry.

[0004] However, in the detection scenarios of acoustic modules, many minute defects, such as tiny stains or scratches, only exhibit weak signal changes in diffuse or specular reflection component maps, resulting in low signal-to-noise ratios that are easily masked by the material's natural texture or background noise. Therefore, directly using feature maps separated by physics-based reflection model algorithms for defect segmentation and identification still falls short of the high accuracy and robustness required for industrial production. Summary of the Invention

[0005] To address the aforementioned technical problems of insufficient accuracy and robustness in defect detection and classification of acoustic modules, this invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a method for classifying acoustic module defects based on multi-feature fusion, comprising:

[0007] A multi-angle polarization image sequence of the acoustic module under test is acquired, and a basic feature map set including an illumination intensity map, a polarization degree map, and a polarization angle map is generated. Using the polarization degree map as weights, the specular reflection component is separated from the illumination intensity map to obtain a specular reflection component map, and the remaining portion of the illumination intensity map constitutes a diffuse reflection component map. The diffuse reflection component map, the material inhomogeneity index map, and the geometric distortion index map constitute a multi-dimensional feature map set, and online prototype learning is performed based on the multi-dimensional feature map set to obtain a material distribution map. All values ​​of the material inhomogeneity index map are positively correlated with the neighborhood standard deviation of the corresponding diffuse reflection component map and negatively correlated with the neighborhood mean. All values ​​of the geometric distortion index map are positively correlated with the product of the gradient magnitude of the corresponding polarization angle map and the specular reflection component. Each material label in the material distribution map is used as a spatial context. In response to the existence of any defect region, the spatial context of the defect region is acquired, and classification is performed based on the spatial context and the characteristics of the defect region.

[0008] This invention generates a robust material distribution map by constructing a multi-dimensional feature map that is more sensitive to material and geometric anomalies, and combining it with online prototype learning. This material distribution map is unaffected by illumination and batch variations. Using this material distribution map as spatial context, this invention applies different classification rules based on the different material regions where defects are located, reducing the classification ambiguity and high false alarm rate problems caused by the lack of contextual information in traditional methods. This achieves accurate and stable classification of complex surface defects in acoustic modules.

[0009] Preferably, the calculation to generate a basic feature map set including an illumination intensity map, a polarization degree map, and a polarization angle map includes:

[0010] Perform a Stokes transform on the multi-angle polarization image sequence to obtain a Stokes vector image;

[0011] Based on the Stokes vector diagram, the illumination intensity diagram, polarization degree diagram, and polarization angle diagram are obtained.

[0012] Preferably, the step of using the polarization degree map as weights to separate the specular reflection component from the illumination intensity map to obtain the specular reflection component map, and the remaining part of the illumination intensity map constituting the diffuse reflection component map, includes:

[0013] The specular reflection component and diffuse reflection component at any coordinate satisfy the following expression:

[0014] ;

[0015] ;

[0016] In the formula, , Representing coordinates The specular reflection component and the diffuse reflection component; Representing coordinates Total light intensity; Representing coordinates The degree of polarization.

[0017] This invention uses polarization degree as a physical weight to linearly decompose the mixed illumination information into a diffuse reflection component strongly correlated with material properties and a specular reflection component strongly correlated with surface geometry. This separates material information from geometric information, providing interference-free input for subsequent feature enhancement and detection of material defects and geometric defects, respectively.

[0018] Preferably, the material non-uniformity index satisfies the expression:

[0019] ;

[0020] In the formula, Representing coordinates Material unevenness index; , Represents the coordinates of the diffuse reflection component map The neighborhood mean and neighborhood standard deviation; This represents the second minimum value.

[0021] This invention calculates local variation coefficients, including neighborhood standard deviation and neighborhood mean, which can amplify the relative changes of local material anomalies such as stains and color differences, while suppressing large-area slow changes in illumination, thereby improving the signal-to-noise ratio of weak material defects.

[0022] Preferably, the geometric distortion index satisfies the expression:

[0023] ;

[0024] In the formula, Representing coordinates The geometric distortion index; Represents a polarization angle diagram; Representing coordinates The specular reflection component.

[0025] This invention achieves the logical "AND" function by multiplying the polarization angle gradient with the specular reflection intensity, ensuring that only areas with drastic changes in surface normal and accompanied by specular reflection are identified as geometric defects, thereby effectively eliminating interference from normal structural edges or defect-free highlight areas.

[0026] Preferably, the step of obtaining the material distribution map through online prototype learning based on the multidimensional feature map set includes:

[0027] Based on the structural design drawing of the acoustic module, the material prototype area of ​​each known material is delineated in the illumination intensity diagram;

[0028] Calculate the mean vector and covariance matrix of the multidimensional feature vectors of all pixels within each material prototype area;

[0029] A Gaussian classifier model based on Mahalanobis distance is used to calculate the membership degree between the multidimensional feature vector of any pixel and each material prototype region.

[0030] The material category with the highest membership degree is used as the final material label to generate the material distribution map.

[0031] This invention learns the multidimensional feature statistical distribution of various materials in the current image in real time and uses Mahalanobis distance, which considers feature correlation, for classification. This can overcome the interference of factors such as lighting fluctuations on the production line and differences in the optical properties of materials from different batches, and generate a continuously stable and highly accurate pixel-level material segmentation map.

[0032] Preferably, the step of obtaining the spatial context of the defect region and classifying it based on the spatial context and the features of the defect region includes:

[0033] Dynamic threshold segmentation is performed on the material inhomogeneity index map to obtain material defect masks;

[0034] Dynamic threshold segmentation is performed on the geometric distortion index map to obtain the geometric defect mask;

[0035] Perform a logical OR operation on the material defect mask and the geometric defect mask to obtain the potential defect mask region;

[0036] The potential defect mask region is used as the defect region. The spatial context of the defect region is obtained, and the region is classified according to the spatial context and the features of the defect region.

[0037] Preferably, the classification based on the spatial context and the characteristics of the defect region includes:

[0038] Based on the characteristics of the defective region and its spatial context, the corresponding rules are matched and executed from the classification rule base, and the classification result of the defect is output.

[0039] Preferably, the classification rule base includes:

[0040] When the spatial context is the diaphragm region, if the average response value of the defect region on the specular reflection component map is greater than the preset metal specular threshold, then the defect region is classified as a metal foreign object.

[0041] When the spatial context is the diaphragm region, if the defective region appears as a thin strip on the diffuse reflection component map, then the defective region is classified as a scratch.

[0042] When the spatial context is a solder pad area, if the average response value of the defect area on the specular reflection component map is greater than the preset metal specular threshold, and the defect is circular and located in the geometric center area of ​​the solder pad, then the defect area is determined to be a normal solder joint.

[0043] Secondly, the present invention provides an acoustic module defect classification system based on multi-feature fusion, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned acoustic module defect classification method based on multi-feature fusion is implemented.

[0044] By adopting the above technical solution, a computer program is generated from the above-mentioned acoustic module defect classification method based on multi-feature fusion and stored in the memory so that it can be loaded and executed by the processor. In this way, a terminal device can be made based on the memory and the processor for convenient use.

[0045] The beneficial effects of this invention are as follows:

[0046] (1) This invention enhances weak defect signals through polarization imaging and defect sensitivity index;

[0047] (2) The present invention generates a pixel-level material distribution map with high robustness that is not affected by environmental changes through online prototype learning, and uses the material distribution map as spatial context information;

[0048] (3) Based on the specific location of the defect on the material map, the present invention calls the corresponding classification rules to distinguish the defect from the normal structure with similar appearance, thereby reducing the classification ambiguity problem and high false alarm rate problem in the detection of complex surface defects. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating an acoustic module defect classification method based on multi-feature fusion according to the present invention;

[0050] Figure 2 This is a schematic diagram illustrating the separation effect of material defects;

[0051] Figure 3 This is a schematic diagram illustrating the effect of separating geometric defects. Detailed Implementation

[0052] This invention discloses a method for classifying acoustic module defects based on multi-feature fusion, referring to... Figure 1 This includes steps S1-S4:

[0053] S1: Obtain the multi-angle polarization image sequence of the acoustic module under test, and calculate and generate the basic feature map set.

[0054] It should be noted that in automated detection scenarios for acoustic modules, the diverse surface materials, including curved and sloping structures, make it impossible for two-dimensional grayscale images to distinguish between changes caused by inherent material reflectivity differences and highlights and shadows caused by surface geometry. This leads to certain deviations in subsequent region segmentation. Considering that polarization optical information can deeply reveal the physical processes of light-material surface interaction, it provides a new dimension for decomposing material and geometry. Therefore, this invention acquires multi-angle polarization images and calculates Stokes vectors to provide multi-dimensional basic optical data, including total illumination intensity, degree of polarization, and polarization angle, for subsequent separation of material and geometric information.

[0055] Specifically, the multi-angle polarization image sequence of the acoustic module under test is acquired, and a basic feature map set is calculated and generated, including:

[0056] A dedicated polarization camera integrating a micro-polarizer array is used to photograph the acoustic module under test. A single exposure can simultaneously acquire multiple polarization images of the acoustic module under test at different polarization angles, forming a multi-angle polarization image sequence. For example, the acquired polarization angles are as follows: The four polarization images are denoted as follows: .

[0057] The multi-angle polarization image sequence is input as a whole, and pixel-by-pixel fusion calculation is performed using Stokes transform to obtain a Stokes vector image, which consists of three component images. It should be noted that the Stokes transform is existing technology and will not be elaborated upon here.

[0058] For example, with polarization angles respectively Taking four polarization images as an example, the values ​​of any coordinates in the three component images satisfy the expression:

[0059] ;

[0060] ;

[0061] ;

[0062] In the formula, , , Representing coordinates The values ​​in the first component plot, the second component plot, and the third component plot; , , , Representing coordinates At polarization angles respectively , , , The grayscale value of the polarization image.

[0063] In the formula, Representing coordinates Total illuminance at the location; Representing coordinates At this location, there is a difference in illumination intensity between the linearly polarized light components in the horizontal and vertical directions; Representing coordinates Place, and Difference in illumination intensity between linearly polarized light components in different directions.

[0064] Create a light intensity map by combining the total light intensity at all coordinates.

[0065] It should be noted that although the three component diagrams in the Stokes vector above contain complete linear polarization information, they are descriptions based on the Cartesian coordinate system, such as horizontal, vertical, and 45°. 135 The physical meaning is not intuitive enough to be directly used for subsequent physical modeling. For example, the polarization intensity of the acoustic module is relatively large, but because its principal direction is exactly 22.5... This can lead to smaller values ​​in the second and third component maps, thus masking their true physical properties. Considering the use of polar coordinates—in this invention, intensity and angle—which more fundamentally describe the polarization state, the total intensity of polarization reflects the absolute amount of specular reflection, while the principal direction of polarization is directly related to the geometric orientation of the object's surface. Therefore, converting the second and third component maps into two physically orthogonal and meaningful features—intensity and angle—makes subsequent features more robust and interpretable.

[0066] Based on the Stokes vector diagram, obtain the degree of polarization map and the angle of polarization map, coordinates. The degree of polarization and the polarization angle at a given point satisfy the following expression:

[0067] ;

[0068] ;

[0069] In the formula, Representing coordinates degree of polarization; Representing coordinates The polarization angle; To represent a very small number, used to prevent the denominator from being zero, for example, ; It is a bivariate arctangent function.

[0070] In the formula, This represents the total illumination intensity of linearly polarized light. It measures the proportion of polarization component in a light wave; the larger the value, the more specular reflection component is received at that point. This indicates the principal direction of linearly polarized light, which is directly related to the direction of the normal vector of the object's surface.

[0071] Thus, a basic feature map containing an illumination intensity map, a polarization degree map, and a polarization angle map has been obtained.

[0072] S2: Using the polarization degree map as the weighting basis, the illumination intensity map is decomposed into a diffuse reflection component map and a specular reflection component map.

[0073] It should be noted that on the surface of the acoustic module, the frosted diaphragm mainly produces diffuse reflection, while the smooth metal shell produces specular reflection. The illumination intensity map mixes these two types of reflected light from completely different physical sources into a single pixel value, causing threshold-based material segmentation methods to be inaccurate when dealing with specular highlights. According to physical optics theory, after natural light is reflected from an object's surface, the diffuse reflection portion remains unpolarized, while the specular reflection portion produces strong linear polarization. This means that the total light intensity of a pixel can be modeled as a linear superposition of the unpolarized diffuse light intensity and the polarized specular reflection intensity. The degree of polarization measures the proportion of specular reflection. Therefore, this invention constructs a linear decomposition model based on the degree of polarization, using polarization as a weight to separate the diffuse reflection component, which is only related to the material, and the specular reflection component, which is only related to geometric specular highlights, from the mixed total illumination intensity.

[0074] Specifically, using the polarization degree map as the weighting basis, the illumination intensity map is decomposed into a diffuse reflection component map and a specular reflection component map, including:

[0075] The specular reflection component and diffuse reflection component at any coordinate satisfy the following expression:

[0076] ;

[0077] ;

[0078] In the formula, , Representing coordinates The specular reflection component and the diffuse reflection component; Representing coordinates Total light intensity; Representing coordinates The degree of polarization.

[0079] In the formula, This indicates that the degree of polarization is used as the weight of the specular reflection component. The specular reflection intensity is extracted from the total illumination intensity. The specular reflection component can highlight the highlights and geometric edges of objects. This means that after removing the specular reflection component from the total light intensity, the remaining part is the diffuse reflection component, which suppresses the highlights. The pixel value can better reflect the inherent reflectivity of the object's surface, that is, the material itself.

[0080] The specular reflection components of all coordinates form a specular reflection component map, and the diffuse reflection components of all coordinates form a diffuse reflection component map.

[0081] At this point, the diffuse reflection component map and the specular reflection component map have been obtained.

[0082] S3: Based on defect sensitivity feature fusion and online prototype learning, obtain the material distribution map of the acoustic module under test.

[0083] It should be noted that in actual production lines, slow shifts in ambient light or subtle differences in the optical properties of different batches of raw materials can cause a shift in the overall brightness of the separated diffuse reflection component. This makes it difficult to maintain a stable, offline-calibrated segmentation threshold over a long period. Furthermore, a single diffuse reflection feature is insufficient to distinguish regions with significantly different geometric characteristics but similar materials. Considering that the diffuse reflection component characterizing the material, the degree of polarization characterizing metallic and specular properties, and the polarization angle characterizing surface orientation together constitute a multi-dimensional feature space that can comprehensively describe the physical properties of a pixel, segmenting this multi-dimensional feature space is more robust than using only a single feature. However, while directly using raw physical quantities such as diffuse reflection and degree of polarization as features for segmentation is feasible, the signal-to-noise ratio of these features may not be high for minute defects. For example, a tiny stain may only cause a slight change in the diffuse reflection value, easily masked by the texture noise of the material itself. The essence of a defect is an "anomaly" in a local area. These anomalies can be categorized into two types: material anomalies, such as stains and discoloration, manifested as localized non-uniformity in diffuse reflectance; and geometric anomalies, such as scratches and pits, manifested as drastic changes in surface normals, often accompanied by specular reflection. Therefore, this invention creates two meta-features: a material non-uniformity index, used to amplify local anomalies in the diffuse reflectance map, and a geometric distortion index, used to amplify anomalies caused by both geometric abrupt changes and specular reflection. Furthermore, through online learning, the joint probability distribution of these multi-dimensional features is statistically analyzed in real-time within the known prototype region of each test image, enabling the dynamic establishment of a segmentation model for the current image. Therefore, this invention combines multi-feature fusion with online prototype learning to adaptively segment the multi-dimensional feature space, thereby achieving adaptation to various environmental and material changes and generating more robust region segmentation results.

[0084] Specifically, obtain the material distribution map of the acoustic module under test, including:

[0085] Material defects, such as blemishes, are reflected in diffuse reflection maps. However, the challenge lies in distinguishing them from inherent, large-area, slow changes in brightness in the background, such as those caused by uneven lighting or inherent material characteristics. Simple thresholding methods cannot handle such background variations. Considering that defects are essentially localized relative anomalies, effective features should be sensitive to the differences between a pixel and its immediate neighbors, but insensitive to the overall brightness of that area. Therefore, this invention proposes a material non-uniformity index, which converts the absolute brightness information of a pixel into relative non-uniformity information by calculating the coefficient of variation of the image within a local window.

[0086] The material non-uniformity index at any coordinate satisfies the expression:

[0087] ;

[0088] In the formula, Representing coordinates Material unevenness index; , Represents the coordinates of the diffuse reflection component map The neighborhood mean and neighborhood standard deviation; This represents the second minimum value, used to avoid a denominator of 0. For example, For example, the neighborhood is a pixel. Neighborhood. It should be noted that the material non-uniformity index is sensitive to the relative changes in local brightness, which can highlight material defects such as blemishes and color differences, but is not sensitive to slow changes in brightness over a large area, thus effectively extracting weak material defect signals from the changing background.

[0089] The material non-uniformity indices of all coordinates are used to construct a material non-uniformity index map. It should be noted that, for example... Figure 2 This is a diagram showing the separation effect of material defects, illustrating the separation effect between a normal diaphragm and blemish defects.

[0090] It should be noted that geometric defects such as scratches have dual physical characteristics, indicating both changes in orientation and the generation of specular reflection. While the gradient of the polarization angle alone can detect all geometric changes, this can confuse normal structural edges with defects. Conversely, using only the specular reflection map cannot distinguish between defect highlights and highlights generated by normal surfaces. True geometric defects, however, are manifestations of the simultaneous occurrence and coexistence of these two physical phenomena. Therefore, it is necessary to capture this symbiotic relationship. Thus, this invention proposes a geometric distortion index, which multiplicatively fuses the gradient magnitude of the polarization angle map with the intensity of the specular reflection component.

[0091] The geometric distortion index of any coordinate satisfies the expression:

[0092] ;

[0093] In the formula, Representing coordinates The geometric distortion index; Represents a polarization angle diagram; Representing coordinates The specular reflection component.

[0094] In the formula, The partial derivative term is used to calculate the gradient magnitude of the polarization angle diagram using the gradient operator, representing the degree of drastic geometric change. Then it is used as the weight; The multiplication operation here acts as a logical AND. The exponent will only increase when the gradient magnitude and specular reflection intensity are both large, thus enabling the location of geometric anomalies such as scratches, pits, and edge burrs, while effectively suppressing the interference of normal structural edges and non-defect specular highlights.

[0095] Construct a geometric distortion index plot using the geometric distortion indices of all coordinates. It should be noted that, for example... Figure 3 This is a diagram illustrating the separation effect of geometric defects, showing the separation effect between a normal metal casing and scratch defects.

[0096] The diffuse reflection component map, material inhomogeneity index map, and geometric distortion index map are denoted as a multidimensional feature map set, and the coordinates are... The multidimensional feature vector is denoted as ,in, Representing coordinates The diffuse reflection component, Representing coordinates Material unevenness index, Representing coordinates The geometric distortion index is used for the pixel's material composition. It's important to note that the diffuse component provides macroscopic information about the material type of a pixel, such as a high-brightness metal or a low-brightness diaphragm; the material inhomogeneity index provides microscopic texture or anomalous information within the material region, such as a smooth diaphragm or a diaphragm with blemishes; and the geometric distortion index provides information about geometric anomalies. These three feature dimensions are physically orthogonal and functionally complementary.

[0097] Based on the structural design drawing of the acoustic module, prototype regions for each known material are delineated in the illumination intensity map. These materials include, for example, the diaphragm, metal shell, and solder pads. The prototype regions are then mapped onto a feature map set. The mean vector and covariance matrix of the multidimensional feature vectors of all pixels in each prototype region are calculated. It should be noted that delineating the prototype regions for each known material in the illumination intensity map requires consideration of the acoustic module's CAD design drawing. Template matching is used to locate the regions of interest (ROIs) for the diaphragm, solder pads, and shell in the image, ensuring that the prototype regions contain at least... Pure material samples of pixels are used to ensure the statistical stability of the covariance matrix calculation.

[0098] It's important to note that after obtaining the statistical distribution of each material prototype, a classifier is needed to determine which material category any pixel in the image belongs to. Euclidean distance classifiers ignore the correlation between features; for example, metallic materials may simultaneously have high specular reflection components and high polarization, and are sensitive to feature scale, leading to low classification accuracy. In contrast, modeling each material prototype as a multidimensional Gaussian distribution and using Mahalanobis distance as a similarity measure is a better statistical classification method. Mahalanobis distance considers the correlation between features and is scale-independent, more accurately measuring the statistical distance between a sample point and the center of its distribution. Therefore, this invention employs a Gaussian classifier model based on Mahalanobis distance to calculate the Mahalanobis distance between the multidimensional feature vector of any pixel and its distribution for each material prototype, and transforms this distance into a probabilistic membership degree using an exponential function, thereby achieving accurate pixel-level classification.

[0099] The membership degree of any coordinate to any material satisfies the expression:

[0100] ;

[0101] In the formula, Representing coordinates The membership degree of the k-th material; Representing coordinates The multidimensional feature vector; and Let represent the eigenvalue vector and covariance matrix of the multidimensional eigenvectors of all pixels in the material prototype region for the k-th material, respectively.

[0102] In the formula, The calculation is a pixel multidimensional feature vector. The Mahalanobis distance between the material prototype distribution of the k-th material and the material type, which takes into account the correlation between features; This represents the membership degree in probabilistic form, where the smaller the Mahalanobis distance, the higher the membership degree.

[0103] For each coordinate The material category with the highest membership degree is used as the final material label, and the material labels of all coordinates constitute a material distribution map. It should be noted that the above membership degree calculation obtains the probability value of each pixel belonging to all material categories, which is a soft classification result. In order to obtain a clear material segmentation map, a unique and definite material label needs to be assigned to each pixel.

[0104] At this point, the material distribution map has been obtained.

[0105] S4: Integrate material context information and material distribution map to accurately classify defects.

[0106] It should be noted that the final classification of defects depends on their spatial context. For example, a bright spot might be a serious metallic foreign object in the diaphragm area, but a normal solder joint in the pad area. Traditional methods suffer from unstable region segmentation, leading to errors in context information and thus misclassification. This invention fuses high signal-to-noise ratio defect detection results with highly robust material context information to construct a context-dependent classification rule base, thereby achieving accurate defect classification.

[0107] Specifically, integrate material context information and Material distribution map ， Accurately classify defects, including:

[0108] Dynamic threshold segmentation is performed on the material inhomogeneity index map to obtain material defect masks; dynamic threshold segmentation is also performed on the geometric distortion index map to obtain geometric defect masks. The dynamic threshold segmentation method, such as the Otsu method, is existing technology and will not be elaborated upon here.

[0109] Perform a logical OR operation on the material defect mask and the geometric defect mask to obtain the final potential defect mask region.

[0110] For any potential defect mask region, the potential defect mask region is mapped onto the material distribution map. The material labels within the mapped region of the material distribution map are counted, and the material label with the highest frequency is determined as the spatial context of the potential defect mask region.

[0111] A classification rule base associated with each spatial context is constructed. Based on the features of the potential defect mask region and its spatial context, the corresponding rules are matched and executed from the classification rule base, ultimately outputting the accurate classification result of the defect. For example, the features are area, shape, and average response on the diffuse and specular reflection component maps.

[0112] For example, the classification rule base is set as follows:

[0113] When the potential defect mask region is the diaphragm region, if the average response value of the potential defect mask region on the specular reflection component map is greater than the preset metallic specular threshold... If the defect appears as a thin strip on the diffuse reflection component map, the potential defect mask area is classified as a scratch; otherwise, it is classified as dirt. For example, the metallic highlight threshold can be adjusted according to the actual lighting conditions and material reflectivity. If the image has 8 bits of grayscale value and the grayscale value range is 0 to 255, then... .

[0114] When the potential defect mask area is a pad area, if the average response value of the potential defect mask area on the specular reflection component map is greater than... If the defect is circular and located in the geometric center of the pad, then the potential defect mask area is determined to be a normal solder joint.

[0115] At this point, the classification results of surface defects of the acoustic module have been obtained and output.

[0116] This invention also discloses an acoustic module defect classification system based on multi-feature fusion, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement an acoustic module defect classification method based on multi-feature fusion according to the present invention.

[0117] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0118] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A method for classifying defects of an acoustic module based on multi-feature fusion, characterized in that, The method comprises the following steps: Obtain a multi-angle polarized image sequence of an acoustic module to be tested, and calculate a basic feature set comprising an illumination intensity map, a degree of polarization map, and a polarization angle map; Use the degree of polarization map as a weight to separate a specular reflection component from the illumination intensity map to obtain a specular reflection component map, and the remaining part of the illumination intensity map constitutes a diffuse reflection component map, wherein the specular reflection component and the diffuse reflection component of any coordinate satisfy the expression: ； ； In the formula, , represents the specular reflection component, the diffuse reflection component of the coordinate ; represents the total illumination intensity of the coordinate ; represents the degree of polarization of the coordinate ; Construct a multi-dimensional feature set comprising the diffuse reflection component map, a material non-uniformity index map, and a geometric distortion index map, and perform online prototype learning based on the multi-dimensional feature set to obtain a material distribution map; all values of the material non-uniformity index map are positively correlated with the neighborhood standard deviation of the corresponding diffuse reflection component map and negatively correlated with the neighborhood mean; all values of the geometric distortion index map are positively correlated with the product of the gradient amplitude of the corresponding polarization angle map and the specular reflection component; The material non-uniformity index satisfies the expression: ； wherein denotes the coordinate of the material non-uniformity index; , denotes the diffuse reflection component map coordinate of the neighborhood mean, neighborhood standard deviation; denotes the second minimum value; The geometric distortion index satisfies the expression: ； wherein denotes the geometric distortion index of the coordinates . denotes the polar angle diagram; denotes the specular reflection component of the coordinates . Use each material label in the material distribution map as a spatial context; in response to the presence of any defect region, obtain the spatial context of the defect region, and classify the defect region according to the spatial context and the characteristics of the defect region.

2. The acoustic module defect classification method based on multi-feature fusion according to claim 1, characterized in that, The calculation of the basic feature set comprising the illumination intensity map, the degree of polarization map, and the polarization angle map comprises the following steps: Perform Stokes transformation on the multi-angle polarized image sequence to obtain a Stokes vector map; Based on the Stokes vector map, obtain the illumination intensity map, the degree of polarization map, and the polarization angle map. 3.The acoustic module defect classification method based on multi-feature fusion according to claim 1, characterized in that, The online prototype learning based on the multi-dimensional feature set to obtain the material distribution map comprises the following steps: According to a structure design drawing of the acoustic module, demarcate material prototype areas of known materials in the illumination intensity map; Calculate the mean vector and the covariance matrix of the multi-dimensional feature vectors of all pixels in each material prototype area; Use a Gaussian classifier model based on Mahalanobis distance to calculate the membership degree between the multi-dimensional feature vector of any pixel and each material prototype area thereof; The material class with the maximum membership degree is taken as the final material label to generate the material distribution map.

4. The acoustic module defect classification method based on multi-feature fusion according to claim 1, characterized in that, The obtaining of the spatial context of the defect region and the classification according to the spatial context and the characteristics of the defect region comprise the following steps: Perform dynamic threshold segmentation on the material non-uniformity index map to obtain a material class defect mask; Perform dynamic threshold segmentation on the geometric distortion index map to obtain a geometric class defect mask; Perform a logical OR operation on the material class defect mask and the geometric class defect mask to obtain a potential defect mask region; The potential defect mask region is taken as the defect region, the spatial context of the defect region is obtained, and the defect region is classified according to the spatial context and the characteristics of the defect region.

5. The acoustic module defect classification method based on multi-feature fusion according to claim 4, characterized in that, The classification according to the spatial context and the characteristics of the defect region comprises the following steps: According to the characteristics of the defect region and the spatial context to which the defect region belongs, match and execute corresponding rules from a classification rule library to output the classification result of the defect.

6. The acoustic module defect classification method based on multi-feature fusion according to claim 5, characterized in that, The classification rule library comprises the following rules: When the spatial context is a diaphragm area, if the average response value of the defect region on the specular reflection component map is greater than a preset metal highlight threshold, the defect region is classified as a metal foreign object. when the spatial context is the diaphragm area, if the defect area is in the form of an elongated strip on the diffuse reflection component map, the defect area is classified as a scratch; when the spatial context is the pad area, if the average response value of the defect area on the specular reflection component map is greater than a preset metal highlight threshold, and the defect is circular and located in the geometric center area of the pad, the defect area is determined as a normal solder joint.

7. An acoustic module defect classification system based on multi-feature fusion, characterized in that, comprising: a processor and a memory, the memory storing computer program instructions which, when executed by the processor, implement a multi-feature fusion-based acoustic module defect classification method according to any one of claims 1-6.

Images