Defect detection method combining vision and X-ray detection technology

Abstract

The invention relates to the technical field of industrial product defect detection, and discloses a defect detection method combining vision and X-ray detection technologies, which comprises the following steps of: synchronously acquiring vision image data and X-ray transmission data in a surface coverage area of a detection object, performing combined preprocessing to complete noise suppression, artifact elimination and time alignment, and then acquiring the X-ray transmission data; according to the method, surface texture features and internal structure features are extracted respectively, cross-dimensional information mapping is performed through a multi-modal association module to generate a fusion feature set, surface damage, internal holes and boundary discontinuous defects are identified, and finally, a hierarchical detection report is generated according to defect space distribution and severity and is updated continuously. According to the method, the advantages of vision and X-ray detection are integrated, cooperative detection of surface and internal defects is achieved, the comprehensiveness, accuracy and real-time performance of defect detection are improved through multi-modal data synchronous collection, feature fusion and dynamic report generation, and the method is suitable for part quality detection in the industrial manufacturing field.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial product defect detection, and in particular to a defect detection method combining vision and X-ray detection technologies. Background Art

[0002] In the manufacturing process of industrial products, defect detection is a key link in ensuring product quality. Traditional defect detection technologies often use a single-modality detection method, which makes it difficult to fully cover the surface and internal defects of the product. For example, although simple visual inspection technology can effectively identify appearance defects such as scratches and pits on the surface of the product, it cannot penetrate the surface of the object and is powerless against hidden defects such as cavities and cracks inside. Although single X-ray detection technology can obtain internal structural information by penetrating the object through rays, it easily ignores subtle changes in surface texture and has limited ability to identify some non-density surface defects. With the increasing complexity of industrial products, higher requirements are placed on the comprehensiveness and accuracy of defect detection, and the limitations of traditional single-modality detection technology are becoming increasingly prominent.

[0003] In the research and application of multimodal detection, how to achieve effective fusion of data from different modalities remains an urgent problem to be solved. Existing multimodal detection methods often suffer from problems such as asynchronous data acquisition and inaccurate feature association, resulting in the inability of the fused detection results to accurately reflect the actual defects of the detected object. For example, the difference in the acquisition time of visual images and X-ray transmission data may lead to deviations in the spatial position correspondence. Poor noise suppression and artifact removal during preprocessing will affect the accuracy of subsequent feature extraction. The lack of an effective cross-dimensional information mapping mechanism during feature fusion makes it difficult to organically combine surface features with internal features, which in turn affects the accuracy and completeness of defect identification.

[0004] Traditional defect detection methods also have shortcomings in the presentation and updating of test results. Test reports often lack a systematic, hierarchical layout, making it difficult to clearly display the spatial distribution and severity of defects. They also lack real-time updates as the test progresses, making it difficult for users to fully and dynamically understand the defect status of the test object. These issues severely restrict the application and development of defect detection technology in high-end manufacturing. Therefore, a new defect detection method is urgently needed that can integrate multimodal test data, improve detection accuracy, and optimize result presentation. Summary of the Invention

[0005] The object of the present invention is to provide a defect detection method combining vision and X-ray detection technologies to solve the problems raised in the above background technology.

[0006] To achieve the above objectives, the present invention provides a defect detection method combining vision and X-ray detection technologies, the method comprising: Synchronously collect visual image data and X-ray transmission data from the surface coverage area of ​​the detection object as multimodal detection input; The collected visual image data and X-ray transmission data are jointly preprocessed, wherein the visual image data is subjected to noise suppression and contrast adjustment, and the X-ray transmission data is subjected to artifact removal and grayscale standardization; Perform feature extraction operations on the pre-processed visual image data and X-ray transmission data to form surface texture features and internal structure features; The surface texture features and internal structure features are mapped across dimensions through a multimodal association module to generate a fusion feature set containing spatial correspondences. Based on the fusion feature set, the surface damage position, internal cavity area and boundary discontinuity defects of the inspection object are identified to form a defect location set; Defect location sets are hierarchically arranged based on their spatial distribution and severity, generating a test result report with defect locations marked. The report content is continuously updated as the test progresses.

[0007] Preferably, the collected visual image data and X-ray transmission data are jointly preprocessed, and the specific steps are as follows: The visual image data is processed frame by frame using a median filtering algorithm, while the coordinates of the detection area corresponding to each frame of the image are recorded; The X-ray transmission data uses an adaptive threshold method to remove inherent artifacts of the equipment, and the remaining data is processed by histogram equalization to retain the grayscale distribution with detail resolution; A time synchronization mechanism is established to mark the visual image data and X-ray transmission data with a unified acquisition timestamp to form a time-aligned preprocessed data group.

[0008] Preferably, feature extraction operations are performed on the pre-processed visual image data and X-ray transmission data respectively, and the specific steps are as follows: The visual image data is processed by edge detection operators to extract the contour features of surface scratches and pits as surface texture features; The X-ray transmission data is analyzed by gray-level co-occurrence matrix to extract the distribution characteristics of the density abnormality area as the internal structure characteristics; The surface texture features and internal structure features are normalized to unify the numerical dimensions of the feature vectors.

[0009] Preferably, the surface texture features and the internal structure features are mapped across dimensions through a multimodal association module. The specific steps are as follows: Construct a spatial coordinate mapping table to establish a one-to-one correspondence between the pixel coordinates of the visual image data and the detection point coordinates of the X-ray transmission data; The similarity value between the surface texture features and the internal structure features at the same spatial position is calculated by feature matching algorithm; The feature pairs whose similarity values ​​are higher than the set threshold are associated and bound to generate a fused feature set containing spatial location information.

[0010] Preferably, the surface damage position, internal cavity area and boundary discontinuity defect of the inspection object are identified based on the fusion feature set, and the specific steps are as follows: The fused feature set is divided into regions using a threshold segmentation algorithm to determine candidate regions of suspected defects; Extract the mutation points of surface texture features in the candidate area as the surface damage locations; Extract the low grayscale value area of ​​the internal structural features as the internal cavity area; The breaking points of the feature-associated boundary are extracted as boundary discontinuity defects to form a defect location set.

[0011] Preferably, the defect location set is hierarchically arranged according to its spatial distribution and severity. The specific steps are as follows: Arrange the defect location set according to the structural partition of the inspection object, and list the corresponding defect coordinate range under each structural partition; Within each structural partition, the damage types that exceed the design allowable values ​​are prioritized based on the length and depth of the surface damage; Internal void areas are classified separately, and the cross-sectional area and depth parameters of the voids are marked to form a clearly hierarchical defect information list.

[0012] Preferably, a test result report with defect locations marked is generated and continuously updated. The specific steps are as follows: Convert the defect information list into graphic annotation text, and organize the content using the expression logic of "structure partition-defect type-coordinate parameter"; When the inspection covers a new area, the multimodal data acquisition, joint preprocessing, feature extraction, association mapping and defect identification steps are repeated to obtain the newly added defect information; Insert the new defect information into the corresponding structural partition or defect type position, adjust the layout and content of the original report, and keep the report synchronized with the inspection process.

[0013] Preferably, when synchronously collecting visual image data and X-ray transmission data, the specific collection steps are as follows: Visual image data is captured by an industrial camera array, and the photosensitivity parameters are adjusted using an automatic exposure algorithm; X-ray transmission data is collected by setting the ray source and detector opposite to each other, and the ray energy value and the detector response signal are recorded synchronously; A data synchronization acquisition mechanism is established to ensure that the frame period of the visual image data is consistent with the sampling period of the X-ray transmission data, forming a multi-source synchronous detection input data group.

[0014] Preferably, when calculating the similarity value between the surface texture feature and the internal structure feature at the same spatial position by a feature matching algorithm, the specific calculation steps are as follows: Based on the spatial coordinates, the surface texture feature vector and the internal structure feature vector at the same position are combined into a feature pair; Calculate the Euclidean distance for each feature pair as the feature difference; Calculate the cosine similarity of the feature vector direction as feature consistency; The similarity value is calculated based on the feature difference and feature consistency results. The lower the similarity value, the closer the feature association.

[0015] Preferably, when the visual image data is processed frame by frame by the median filtering algorithm, the specific processing steps are as follows: Divide the visual image data into fixed-size pixel windows and sort the pixel values ​​within each window; The sorted middle value is selected as the new value of the center pixel of the window to complete the single window filtering process; The filtering operation is repeated through all pixel windows until the noise intensity of the entire frame image is lower than the set threshold, and the visual image data after noise suppression is obtained.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This defect detection method constructs a multimodal detection input system by synchronously collecting visual image data and X-ray transmission data. This system enables the coordinated analysis of surface texture features and internal structural features, effectively overcoming the limitations of single detection technologies. During the data preprocessing stage, a median filter algorithm is used to suppress visual image noise, an adaptive threshold method is used to eliminate X-ray artifacts, and a time synchronization mechanism is established to achieve data alignment, laying a high-quality data foundation for subsequent feature extraction. During feature extraction, edge detection operators and gray-level co-occurrence matrices are used to obtain surface and internal features, respectively. Normalization is then used to unify the dimensions, ensuring feature accuracy and comparability.

[0017] The multimodal association module achieves precise mapping of cross-dimensional information by constructing a spatial coordinate mapping table and feature matching algorithm. This enables the fused feature set to simultaneously encompass both surface and internal spatial correspondences. This allows for simultaneous localization of surface damage, internal voids, and boundary discontinuities during defect identification, significantly improving the comprehensiveness and accuracy of defect detection. The hierarchical organization of defect location sets, organized by structural partition, defect type, and severity, makes detection results more logical and readable, enabling users to quickly understand defect distribution.

[0018] The dynamic update mechanism of the test result report continuously integrates new defect information as the inspection progresses, ensuring that the report content is synchronized with the actual inspection progress, providing a real-time and reliable reference for quality control during the production process. Through the deep integration and systematic processing of multimodal data, this method not only improves the accuracy and efficiency of defect detection, but also provides comprehensive and dynamic technical support for the quality assessment and improvement of industrial products, with significant engineering application value and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A diagram showing the working principle of the defect detection method combining vision and X-ray detection technologies according to the present invention; Figure 2 Flowchart for multimodal association mapping; Figure 3 Flowchart for defect identification; Figure 4 Generate updated flow chart for inspection reports. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] See also Figures 1-4 The present invention provides a defect detection method combining vision and X-ray detection technology, the method comprising:

[0022] Visual image data and X-ray transmission data are synchronously collected from the covered area of ​​the inspection object's surface as multimodal inspection input. The visual image data is captured by an industrial camera array, with the photosensitivity parameters adjusted using an automatic exposure algorithm. The X-ray transmission data is collected by aligning the radiation source with the detector, and the radiation energy value and the detector response signal are synchronously recorded. A data synchronization acquisition mechanism is established to ensure that the frame period of the visual image data is consistent with the sampling period of the X-ray transmission data, forming a multi-source synchronous inspection input data set.

[0023] The collected visual image data and X-ray transmission data are jointly preprocessed. The visual image data is processed frame by frame using a median filtering algorithm. The visual image data is divided into fixed-size pixel windows. The pixel values ​​in each window are sorted, and the median value after sorting is selected as the new value of the center pixel of the window to complete the single-window filtering process. The filtering operation is repeated through all pixel windows until the noise intensity of the entire frame image is lower than the set threshold. The noise-suppressed visual image data is obtained, and the detection area coordinates corresponding to each frame image are recorded. The X-ray transmission data uses an adaptive threshold method to remove inherent device artifacts. The remaining data is histogram-equalized to retain a grayscale distribution with detail resolution. A time synchronization mechanism is established to label the visual image data and X-ray transmission data with a unified acquisition timestamp to form a time-aligned preprocessed data set.

[0024] Feature extraction is performed on the preprocessed visual image data and X-ray transmission data. The visual image data is processed using an edge detection operator to extract the contour features of surface scratches and pits as surface texture features. The X-ray transmission data is analyzed using a gray-level co-occurrence matrix to extract the distribution characteristics of density anomalies as internal structural features. The surface texture features and internal structural features are normalized to unify the numerical dimensions of the feature vectors.

[0025] The surface texture features and internal structure features are mapped across dimensions through a multimodal association module to generate a fused feature set containing spatial correspondences. A spatial coordinate mapping table is constructed to establish a one-to-one correspondence between the pixel coordinates of the visual image data and the detection point coordinates of the X-ray transmission data. The similarity values ​​of the surface texture features and internal structure features at the same spatial position are calculated using a feature matching algorithm. Based on the spatial coordinates, the surface texture feature vectors and internal structure feature vectors at the same position are combined into feature pairs. For each feature pair, the Euclidean distance is calculated as the feature difference, and the cosine similarity of the feature vector direction is calculated as the feature consistency. The similarity value is calculated based on the feature difference and feature consistency results. The lower the similarity value, the closer the feature association. Feature pairs with similarity values ​​above the set threshold are associated and bound to generate a fused feature set containing spatial position information.

[0026] Based on the fused feature set, the surface damage location, internal void area, and boundary discontinuity defects of the inspection object are identified to form a defect location set. The fused feature set is divided into regions using a threshold segmentation algorithm to determine candidate regions for suspected defects. Within the candidate regions, the mutation points of the surface texture features are extracted as surface damage locations; the low-value grayscale areas of the internal structural features are extracted as internal void areas; and the break points of the feature-related boundaries are extracted as boundary discontinuity defects to form a defect location set.

[0027] According to the spatial distribution and severity of the defect location set, a hierarchical arrangement is performed to generate a test result report with the defect location marked, and the report content is continuously updated as the inspection progresses. The defect location set is arranged according to the structural partition of the inspection object, and the corresponding defect coordinate range is listed under each structural partition; within each structural partition, the surface damage is sorted according to the length and depth, and the damage type that exceeds the design allowable value is prioritized; the internal void area is classified separately, and the cross-sectional area and depth parameters of the void are marked to form a clearly hierarchical defect information list. The defect information list is converted into graphic annotation text, and the content is organized using the expression logic of "structural partition-defect type-coordinate parameter"; when the inspection covers a new area, the multimodal data acquisition, joint preprocessing, feature extraction, association mapping and defect identification steps are repeated to obtain the newly added defect information; the newly added defect information is inserted into the corresponding structural partition or defect type position, and the layout and content of the original report are adjusted to keep the report synchronized with the inspection process.

[0028] Embodiment 1:

[0029] When jointly preprocessing the collected visual image data and X-ray transmission data, it is necessary to complete noise suppression and contrast adjustment of the visual image data, as well as artifact elimination and grayscale standardization of the X-ray transmission data, and establish a time synchronization mechanism.

[0030] To process visual image data, it is first divided into pixel windows of fixed size. The pixel window size here can be determined based on the image resolution and noise characteristics. For example, for common industrial inspection images, it may be divided into 3×3 or 5×5 pixel windows. The pixel values ​​within each window are sorted, either from large to small or from small to large, depending on the specific processing requirements and algorithm design. After the sorting is completed, the middle value of the sorting is selected as the new value of the center pixel of the window. For example, a 3×3 window has 9 pixel values. The value in the middle position after sorting will be used to replace the original value of the center pixel of the window. This method completes single-window filtering processing. This operation can effectively remove impulse noise such as salt and pepper noise in the image. Because noise points are usually isolated pixel values ​​that differ greatly from surrounding pixels, they can be replaced with the values ​​of surrounding normal pixels by taking the middle value.

[0031] After completing the single-window filtering process, it is necessary to traverse all pixel windows of the entire frame and repeat the filtering operation described above. During the traversal process, each window is processed sequentially in a certain order, such as from left to right or from top to bottom. After each window is processed, the next window is processed until all pixel windows of the entire frame have been processed. During this process, the image noise intensity must be continuously monitored. When the noise intensity falls below a set threshold, the filtering operation is terminated. Noise intensity detection can be achieved by calculating the image noise variance or other appropriate noise assessment metrics. The threshold is determined based on the specific detection requirements and image quality standards. This process ultimately produces noise-suppressed visual image data, making the image clearer and facilitating subsequent feature extraction and analysis. Furthermore, at each step in the processing of the visual image data, the coordinates of the detection area corresponding to each frame must be recorded. This coordinate information can be obtained using the calibration parameters of the industrial camera and the position information at the time of capture. This information can be recorded by adding the coordinate information to the image metadata or by establishing a dedicated coordinate mapping table. This allows subsequent data processing and analysis to accurately determine the actual detection location of each area in the image, achieving precise data positioning and matching.

[0032] When processing X-ray transmission data, an adaptive thresholding method is first used to remove inherent device artifacts. This method automatically determines a threshold based on local image features. It adapts to the grayscale distribution characteristics of different image regions and offers greater adaptability and accuracy than fixed thresholding methods. Specifically, this method divides the image into several small regions and calculates an appropriate threshold for each region. This threshold is then used to distinguish artifacts from valid data and remove them. This method effectively addresses inherent artifacts caused by factors such as device characteristics and imaging conditions, improving the quality of X-ray transmission data. After artifact removal, the remaining data undergoes histogram equalization. Histogram equalization adjusts the image histogram to achieve a more uniform distribution of grayscale levels within the image. Specifically, it remaps the image's grayscale values, expanding densely distributed grayscale levels and compressing sparsely distributed grayscale levels. This enhances image contrast, preserves the grayscale distribution that allows for detail resolution, and allows for clearer visualization of regions of varying density within the X-ray transmission image, facilitating subsequent extraction and analysis of internal structural features.

[0033] Furthermore, to ensure temporal consistency between visual image data and X-ray transmission data, a time synchronization mechanism is required. This is achieved by annotating both data types with a unified acquisition timestamp. Timestamps can be synchronized with the system clock, ensuring that each data sample has an accurate time stamp. During data acquisition, when a frame of visual image data and the corresponding X-ray transmission data are acquired, they are both assigned the same timestamp. This creates a time-aligned preprocessed data set. During subsequent processing and analysis, the two data types can be accurately matched and correlated based on the timestamps, ensuring temporal consistency of the multimodal data and providing a reliable time reference for subsequent operations such as feature extraction and correlation mapping.

[0034] Example 2:

[0035] When performing feature extraction operations on preprocessed visual image data and X-ray transmission data, it is necessary to extract surface texture features and internal structure features from the two types of data respectively, and normalize the features to unify the numerical dimensions of the feature vectors.

[0036] To extract features from visual image data, edge detection operators are used for traversal processing. Edge detection operators identify locations in an image where pixel values ​​suddenly change. These locations typically correspond to edges or surface defects. Common edge detection operators include the Sobel operator and the Canny operator. In practical applications, the appropriate edge detection operator is selected based on the characteristics of the object being inspected and the detection requirements. For example, the Sobel operator determines edge locations by calculating the horizontal and vertical gradients of the image. Specifically, the edge detection operator is applied to the entire visual image data, calculating the gradient value for each pixel. For each pixel, a convolution operation is performed with the operator template to obtain the horizontal and vertical gradient components. The magnitude and direction of the gradient are then calculated based on these two components. Locations with larger gradient values ​​are considered edges, allowing the contour features of surface defects such as scratches and pits to be extracted. These contour features intuitively reflect surface texture variations, such as the direction of scratches and the shape and size of pits. These contour features serve as important evidence for subsequent defect identification and analysis.

[0037] Gray-level co-occurrence matrix analysis is used to extract features from X-ray transmission data. A gray-level co-occurrence matrix describes the statistical relationships between pixels at different locations and grayscale levels within an image. Its basic concept is to consider the grayscale combinations of two pixels separated by a certain distance and orientation within the image and statistically analyze these combinations to form a matrix. In specific applications, two key parameters of the gray-level co-occurrence matrix must first be determined: the distance interval and the direction. The distance interval can be set based on the internal structure of the object being inspected and the defect size, for example, 1 pixel or 2 pixels. The direction typically includes multiple directions such as 0°, 45°, 90°, and 135°. Then, for each pixel in the X-ray transmission image, the number of occurrences of its grayscale combination with the corresponding pixel, according to the specified distance interval and orientation, is counted to construct a gray-level co-occurrence matrix. By analyzing the gray-level co-occurrence matrix, various feature parameters can be extracted, such as contrast, correlation, energy, and entropy. These feature parameters reflect information such as the uniformity of grayscale distribution and texture complexity within the image, thereby extracting the distribution characteristics of density anomalies. For example, in areas with internal voids or uneven density, the characteristic parameters of the grayscale co-occurrence matrix will be different from those of normal areas. By analyzing these differences, the location and distribution of the abnormal density area can be determined and used as internal structural features for subsequent defect identification and analysis.

[0038] After extracting surface texture features and internal structure features, they need to be normalized. Because surface texture features and internal structure features may come from different data sources and have different physical meanings and numerical ranges, their feature vectors may have different dimensions and numerical values. If these features are processed and analyzed directly, features of different dimensions and numerical values ​​may have different impacts on the results, resulting in inaccurate analysis results. The purpose of normalization is to eliminate these differences in dimensions and numerical values, converting feature vectors to a unified numerical range, so that different types of features can be compared and processed on the same dimension.

[0039] There are many specific methods for normalization, the most common of which are min-max normalization and Z-score normalization. Taking min-max normalization as an example, its basic principle is to linearly transform each value in the feature vector to a specified interval, usually [0, 1] or [-1, 1]. The specific calculation formula is: ,in is the original eigenvalue, is the minimum value in the eigenvector, is the maximum value in the eigenvector, is the normalized eigenvalue.

[0040] Through this method, the feature vectors of surface texture features and internal structure features are converted into the same numerical range, thereby unifying the numerical dimensions of the feature vectors.

[0041] When performing normalization, the following points need to be noted: First, ensure that surface texture features and internal structure features are normalized separately and cannot be mixed; second, during the processing, the original information of the feature vector must be retained and important feature details cannot be lost due to normalization; finally, the parameters of the normalization processing (such as minimum and maximum values) need to be determined according to the specific feature vector to ensure that the normalized feature vector can accurately reflect the distribution of the original features.

[0042] By extracting and normalizing features from visual image data and X-ray transmission data, we obtain surface texture features and internal structure features with unified dimensions. These features can more accurately reflect the surface and internal structure of the inspection object, providing reliable feature data support for subsequent multimodal association mapping, defect recognition and other operations, enabling the entire defect detection method to more accurately and effectively identify various defects of the inspection object.

[0043] Example 3: When mapping surface texture features and internal structure features across dimensions through a multimodal association module, it is necessary to complete operations such as constructing a spatial coordinate mapping table, calculating feature similarity values, and binding feature pairs in sequence to generate a fused feature set containing spatial location information.

[0044] Construct a spatial coordinate mapping table, which is used to establish the correspondence between the pixel coordinates of the visual image data and the detection point coordinates of the X-ray transmission data. In actual operation, the visual image data is obtained by an industrial camera array. Each pixel corresponds to a specific position on the surface of the detection object, and its coordinates can be expressed as ,in Represents the column index of the pixel in the horizontal direction of the image, Represents the row index of the pixel in the vertical direction of the image. X-ray transmission data is collected by setting the ray source and the detector opposite each other. Each detection point on the detector corresponds to a spatial position inside the detection object, and its coordinates can be expressed as ,in 、 are the horizontal and vertical coordinates of the detection point on the detector plane, The depth coordinate of the detection point along the ray direction. When constructing the mapping table, it is necessary to use calibration technology, such as using a calibration plate with a known geometric structure, simultaneously acquiring the visual image and the X-ray transmission image, and establishing the pixel coordinate by calculating the transformation matrix between the camera coordinate system and the detector coordinate system. With the detection point coordinates One-to-one correspondence, forming a form like The mapping entries are stored in the spatial coordinate mapping table.

[0045] The similarity value between the surface texture features and the internal structure features at the same spatial position is calculated by feature matching algorithm. Based on the spatial coordinates, the visual pixel coordinates corresponding to a certain spatial position are obtained from the spatial coordinate mapping table. and X-ray detection point coordinates , and then extract the surface texture feature vector at that location and the internal structure eigenvector , forming feature pairs .

[0046] For each feature pair, first calculate the Euclidean distance as the feature difference. The calculation formula of Euclidean distance is:

[0047] in, represents the Euclidean distance, is the dimension of the feature vector, is the surface texture feature vector No. A quantity, is the internal structure feature vector No. This formula is used to measure the difference between two eigenvectors. The straight-line distance in the dimensional space, the larger the distance, the greater the difference in the values ​​of the two features.

[0048] Then calculate the cosine similarity of the feature vector direction as feature consistency. The calculation formula of cosine similarity is:

[0049] in, represents the cosine similarity, is the dot product of two eigenvectors, and are surface texture feature vectors and the internal structure feature vector The formula measures the directional similarity of two vectors by calculating the cosine of the angle between them. The closer the value is to , which means the directions of the two vectors are more consistent.

[0050] According to the feature difference Calculate similarity value based on feature consistency , the calculation formula of the similarity value is:

[0051] in, is the weight coefficient, and its value range is , used to adjust the weight of Euclidean distance and cosine similarity in the similarity value calculation. In this formula, the feature difference Directly counted into the similarity value, while feature consistency pass After being converted into the degree of difference, the similarity value The lower the value, the closer the features are. For example, when the Euclidean distance between two feature vectors is Small and cosine similarity When it is large, the similarity value It will be lower, indicating that the two features are close in value and direction and closely related.

[0052] The similarity value Above the set threshold Set the threshold It can be determined according to the actual detection requirements and feature distribution, for example For satisfaction The feature pairs of , bind their surface texture features and internal structure features, and attach the corresponding spatial coordinate information, and finally generate a fusion feature set containing spatial position information. Each fusion feature element in the set can be expressed as ,in is the bound feature pair, The corresponding spatial coordinates are obtained, thereby realizing the mapping and fusion of cross-dimensional information and providing comprehensive data with both surface and internal features for subsequent defect identification.

[0053] During the entire implementation process, attention should be paid to the construction accuracy of the spatial coordinate mapping table, which will directly affect the spatial correspondence accuracy of the feature pairs; the quality of feature vector extraction is also crucial. If the early feature extraction is incomplete or noisy, it will lead to deviations in the calculation of similarity values; the weight coefficient The selection of needs to be combined with the characteristics of the inspection object. For example, for inspection scenarios that are more sensitive to surface defects, the value can be appropriately increased. To increase the weight of the Euclidean distance. In addition, the threshold The setting needs to be adjusted through multiple experiments to ensure that irrelevant feature pairs can be filtered out without missing important related features, thereby ensuring the effectiveness and accuracy of the fusion feature set and laying the foundation for subsequent defect localization based on fusion features.

[0054] Example 4: When identifying the surface damage location, internal void area, and boundary discontinuity defects of the inspection object based on the fused feature set, it is necessary to form a defect location set through operations such as threshold segmentation and feature extraction. The following is a detailed explanation with specific examples.

[0055] Taking metal casting inspection as an example, assume that the fused feature set is formed by the fusion of the visual image features and X-ray transmission features of the casting. First, the fused feature set is divided into regions using the threshold segmentation algorithm. The threshold segmentation algorithm sets a suitable threshold based on the numerical distribution characteristics of the fused features, and divides the data points in the fused feature set into different regions. For example, when the fused feature value is greater than the threshold, it is divided into one category, and when it is less than the threshold, it is divided into another category, thereby determining the candidate region of suspected defects. In the fused feature image of the metal casting, these candidate regions may appear as parts with significantly different brightness or feature values ​​from the surrounding areas. For example, at the flange connection of the casting, there may be an area with abnormal feature values. This is the preliminarily determined candidate region of suspected defects.

[0056] When extracting the location of surface damage within a candidate region, it is important to focus on the mutation points of the surface texture features. Taking scratch defects on the surface of a casting as an example, in the surface texture features of a visual image, the grayscale value of the pixel at the scratch will suddenly change, forming an edge feature. These mutation points can be identified through methods such as edge detection. For example, within a candidate region, scanning along the possible scratch direction, it is found that the gradient value in the surface texture feature vector at a certain position suddenly increases. This position is the mutation point of the surface damage, that is, the location of the scratch. Assume that a series of continuous mutation points are detected in the candidate region on the upper surface of the casting. These mutation points are connected to form the outline of a scratch, thereby determining the specific location of the surface damage.

[0057] To extract internal void areas, it is necessary to focus on the low-grayscale areas of the internal structural features. In X-ray transmission data, the void areas have low density and corresponding grayscale values ​​are also low. Taking the porosity defects inside the casting as an example, in the internal structural features of the X-ray transmission image, the grayscale value of the area where the pores are located will be significantly lower than the surrounding normal areas. By setting a grayscale threshold, the areas with grayscale values ​​below the threshold are extracted, which are the internal void areas. For example, in the candidate area of ​​the main part of the casting, a circular low-grayscale area is found, and its grayscale value is lower than the set threshold. This area is determined to be the internal void area, that is, the location and range of the pores.

[0058] When extracting boundary discontinuity defects, it's important to focus on the breakpoints in the feature association boundary. In a fused feature set, features in adjacent regions should normally be continuously associated. However, when a boundary discontinuity defect exists, the feature association boundary will be broken. For example, taking a crack defect at a casting's weld seam as an example, in the fused feature image, the feature association at the weld seam should be continuous. However, if a crack is present, the feature association will break at the crack. By detecting these breakpoints, the location of the boundary discontinuity defect can be determined. For example, in a candidate weld seam region of a casting, by checking the continuity of the feature association along the seam direction, a location where the feature association is interrupted is identified. This location is the breakpoint in the feature association boundary, i.e., the location of the crack.

[0059] Taking plastic shell detection as an example, the fused feature set comes from the visual image and X-ray transmission data of the shell. During threshold segmentation, a threshold is set based on the distribution of the fused features of the plastic shell to determine candidate areas, such as areas with abnormal feature values ​​at the corners of the shell. Within the candidate area, when extracting the location of surface damage, if there are pits on the shell surface, the mutation points of its surface texture features will reflect the outline of the pits; when extracting internal void areas, if there are bubbles inside the shell, the low grayscale value areas of the X-ray transmission data will show the location of the bubbles; when extracting boundary discontinuity defects, if there are gaps at the connection between the shell components, the fracture points of the feature-associated boundaries will indicate the location of the gaps.

[0060] In practice, the threshold setting for segmentation depends on the material, structure, and fusion features of the object being tested. Different threshold setting methods may be required for different objects. For example, for metal and plastic, due to their different physical properties, the distribution of fusion features will also be different, so appropriate threshold settings are required for each.

[0061] When extracting the mutation points at the location of surface damage, a suitable algorithm is needed to identify the mutations in the eigenvector. This can be detected by calculating the first or second derivative of the eigenvector. The peak of the first derivative or the zero crossing of the second derivative usually corresponds to the location of the mutation point.

[0062] When extracting low-value grayscale areas of internal structural features, it is necessary to consider the distribution range of grayscale values ​​and the influence of noise. X-ray transmission data can be preprocessed using filtering and other methods to remove noise before extracting low-value grayscale areas to improve extraction accuracy.

[0063] When extracting breakpoints along feature association boundaries, it's necessary to define a continuity criterion for the feature association. This can be determined by calculating the similarity or distance between adjacent feature points. When the similarity falls below a certain threshold or the distance exceeds a certain threshold, the feature association is considered broken, indicating a boundary discontinuity defect.

[0064] Through the above steps, surface damage locations, internal void areas, and boundary discontinuities are extracted from the fused feature set to form a defect location set. This set contains the specific location and relevant feature information for each defect, providing an accurate basis for subsequent defect analysis and treatment. In the example of inspecting metal castings and plastic casings, this method can effectively identify various defects, helping quality inspectors to promptly detect product problems and ensure product quality.

[0065] Example 5:

[0066] When generating a hierarchical report based on the spatial distribution and severity of defect location sets, it's important to consider the structural characteristics and defect types of the specific inspection object to ensure orderly organization and dynamic updating of report content. The following examples illustrate this using aircraft engine blades and automotive aluminum alloy wheels.

[0067] Taking aircraft engine blade inspection as an example, the defect location set contains defect information for different structural partitions, such as the blade tenon, blade body, and blade tip. First, the defect location set is arranged according to the structural partition of the inspection object. The tenon area is listed first in the report, and the coordinate range of the defects in this area is listed below it. For example, the coordinate range of the defect at the root of the tenon tooth is X1 to X2 horizontally and Y1 to Y2 vertically. Next, the blade body area is listed, and the specific coordinate range of the defect in this area is marked accordingly, such as the coordinate range of a certain position in the middle of the blade body. Then the blade tip area is listed, and the coordinate range of the defect is also recorded.

[0068] Within each structural partition, surface damage is sorted by length and depth, with damage types exceeding the design allowable values ​​prioritized. For example, in the blade area, if there are surface scratches of varying lengths and depths, the maximum allowable scratch length is 5mm and the maximum allowable depth is 0.1mm. If one scratch is 6mm long and 0.15mm deep, and another is 3mm long and 0.08mm deep, the 6mm long, 0.15mm deep scratch will be prioritized because it exceeds the design allowable values, followed by the 3mm long, 0.08mm deep scratch.

[0069] Internal voids are categorized separately, with their cross-sectional area and depth parameters noted. For example, a void detected in the blade tenon area has a cross-sectional area of ​​2mm² and a depth of 1.5mm. The report lists the relevant parameters for this void separately, creating a clearly structured list of defect information.

[0070] Convert defect information lists into graphical annotation text, organizing the content using the "structural partition - defect type - coordinate parameters" representation logic. For example, a defect in the tenon area could be described as: "Tenon - Surface scratch - Horizontal X1 to X2, Vertical Y1 to Y2, Length 6mm, Depth 0.15mm"; an internal void in the blade area could be described as: "Blade - Internal void - Coordinate range X3 to X4, Y3 to Y4, Cross-sectional area 2mm², Depth 1.5mm."

[0071] When the inspection covers a new area, the multimodal data acquisition, joint preprocessing, feature extraction, association mapping, and defect identification steps are repeated to obtain the newly added defect information. Suppose that in a subsequent inspection, a new surface pit is found in the blade tip area. Its coordinate range is X5 to X6, Y5 to Y6, and its depth is 0.2mm, which exceeds the design allowable depth of 0.15mm. The newly added defect information is inserted into the defect type position in the blade tip area, and the layout and content of the original report are adjusted to prioritize the pit defect under the blade tip area to maintain synchronization between the report and the inspection process.

[0072] Taking the inspection of automotive aluminum alloy wheels as an example, the structural partitions include areas such as the rim, spokes, and hub center. Within the defect location set in the rim area, there are different surface damage and internal defects. After being arranged according to the structural partitions, within the rim area, the surface damage is sorted by length and depth. For example, a crack with a length of 10mm and a depth of 0.3mm exceeds the design allowable length of 8mm and depth of 0.2mm, and is prioritized. Another scratch with a length of 5mm and a depth of 0.1mm meets the design requirements and is ranked at the back. Internal cavity areas, such as the cavity in the spoke, have a cross-sectional area of ​​3mm² and a depth of 2mm, and are separately classified and labeled.

[0073] When converted into graphic annotation text, the crack on the rim is described as: "Rim - Surface crack - Coordinate range X7 to X8, Y7 to Y8, length 10mm, depth 0.3mm"; the cavity on the spoke is described as: "Spoke - Internal cavity - Coordinate range X9 to X10, Y9 to Y10, cross-sectional area 3mm², depth 2mm".

[0074] If a new boundary discontinuity defect is found in the hub center area during subsequent inspections, with coordinates ranging from X11 to X12 and Y11 to Y12, insert it into the defect type position in the hub center area and adjust the report content so that the report reflects all detected defect information in real time.

[0075] In practice, structural partitioning must be based on the design drawings and functional modules of the inspection object to ensure targeted and traceable defect information for each partition. Measurements of parameters such as the length and depth of surface damage and the cross-sectional area and depth of internal cavities must be made using precision-matched measurement tools and methods to ensure data accuracy.

[0076] Sorting rules must be formulated in strict accordance with design specifications and industry standards, clearly defining the criteria for determining damage types exceeding design allowable values ​​and ensuring that defects with a significant impact on the performance of the inspected object are prioritized. Graphic annotation text should be concise, clear, accurate, and standardized to facilitate understanding and use by inspectors and related personnel.

[0077] When adding new defect information, carefully check its relationship with existing defect information and correctly insert it into the corresponding structural partition or defect type position to avoid information confusion and errors. At the same time, pay attention to the layout of the report, choose appropriate fonts and font sizes, and use appropriate methods to distinguish defect information in different areas and types, such as segmentation and indentation, to improve report readability.

[0078] Through this hierarchical organization and dynamic update mechanism, the generated inspection result report clearly and accurately displays the defects of the inspected object, providing a detailed basis for subsequent defect assessment and remediation decisions, thereby ensuring the effectiveness and reliability of the inspection work. In the inspection examples of aircraft engine blades and automotive aluminum alloy wheels, this approach systematically organizes defect information, helping relevant personnel fully understand the product's quality status and take timely measures to ensure product safety and performance.

[0079] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0080] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A defect detection method combining vision and X-ray detection technology, characterized in that: The following steps are involved: Synchronously collect visual image data and X-ray transmission data from the surface coverage area of ​​the detection object as multimodal detection input; The collected visual image data and X-ray transmission data are jointly preprocessed, wherein the visual image data is subjected to noise suppression and contrast adjustment, and the X-ray transmission data is subjected to artifact removal and grayscale standardization; Perform feature extraction operations on the pre-processed visual image data and X-ray transmission data to form surface texture features and internal structure features; The surface texture features and internal structure features are mapped across dimensions through a multimodal association module to generate a fusion feature set containing spatial correspondences. Based on the fusion feature set, the surface damage position, internal cavity area and boundary discontinuity defects of the inspection object are identified to form a defect location set; Defect location sets are hierarchically arranged based on their spatial distribution and severity, generating a test result report with defect locations marked. The report content is continuously updated as the test progresses.

2. The defect detection method combining vision and X-ray detection technology according to claim 1, characterized in that: The collected visual image data and X-ray transmission data are jointly preprocessed. The specific steps are as follows: The visual image data is processed frame by frame using a median filtering algorithm, while the coordinates of the detection area corresponding to each frame of the image are recorded; The X-ray transmission data uses an adaptive threshold method to remove inherent artifacts of the equipment, and the remaining data is processed by histogram equalization to retain the grayscale distribution with detail resolution; A time synchronization mechanism is established to mark the visual image data and X-ray transmission data with a unified acquisition timestamp to form a time-aligned preprocessed data group.

3. The defect detection method combining vision and X-ray detection technology according to claim 1, characterized in that: The feature extraction operations are performed on the preprocessed visual image data and X-ray transmission data respectively. The specific steps are as follows: The visual image data is processed by edge detection operators to extract the contour features of surface scratches and pits as surface texture features; The X-ray transmission data is analyzed by gray-level co-occurrence matrix to extract the distribution characteristics of the density abnormality area as the internal structure characteristics; The surface texture features and internal structure features are normalized to unify the numerical dimensions of the feature vectors.

4. The defect detection method combining vision and X-ray detection technology according to claim 3, characterized in that: The surface texture features and internal structure features are mapped across dimensions through a multimodal association module. The specific steps are as follows: Construct a spatial coordinate mapping table to establish a one-to-one correspondence between the pixel coordinates of the visual image data and the detection point coordinates of the X-ray transmission data; The similarity value between the surface texture features and the internal structure features at the same spatial position is calculated by feature matching algorithm; The feature pairs whose similarity values ​​are higher than the set threshold are associated and bound to generate a fused feature set containing spatial location information.

5. The defect detection method combining vision and X-ray detection technology according to claim 4, characterized in that: Based on the fusion feature set, the surface damage location, internal cavity area and boundary discontinuity defects of the inspection object are identified. The specific steps are as follows: The fused feature set is divided into regions using a threshold segmentation algorithm to determine candidate regions of suspected defects; Extract the mutation points of surface texture features in the candidate area as the surface damage locations; Extract the low grayscale value area of ​​the internal structural features as the internal cavity area; The breaking points of the feature-associated boundary are extracted as boundary discontinuity defects to form a defect location set.

6. The defect detection method combining vision and X-ray detection technology according to claim 5, characterized in that: Defect location sets are hierarchically arranged based on their spatial distribution and severity. The specific steps are as follows: Arrange the defect location set according to the structural partition of the inspection object, and list the corresponding defect coordinate range under each structural partition; Within each structural partition, the damage types that exceed the design allowable values ​​are prioritized based on the length and depth of the surface damage; Internal void areas are classified separately, and the cross-sectional area and depth parameters of the voids are marked to form a clearly hierarchical defect information list.

7. The defect detection method combining vision and X-ray detection technology according to claim 6, characterized in that: Generate and continuously update the inspection result report with defect locations marked. The specific steps are as follows: Convert the defect information list into graphic annotation text, and organize the content using the expression logic of "structure partition-defect type-coordinate parameter"; When the inspection covers a new area, the multimodal data acquisition, joint preprocessing, feature extraction, association mapping and defect identification steps are repeated to obtain the newly added defect information; Insert the new defect information into the corresponding structural partition or defect type position, adjust the layout and content of the original report, and keep the report synchronized with the inspection process.

8. The defect detection method combining vision and X-ray detection technology according to claim 1, characterized in that: When synchronously collecting visual image data and X-ray transmission data, the specific collection steps are as follows: Visual image data is captured by an industrial camera array, and the photosensitivity parameters are adjusted using an automatic exposure algorithm; X-ray transmission data is collected by setting the ray source and detector opposite to each other, and the ray energy value and the detector response signal are recorded synchronously; A data synchronization acquisition mechanism is established to ensure that the frame period of the visual image data is consistent with the sampling period of the X-ray transmission data, forming a multi-source synchronous detection input data group.

9. The defect detection method combining vision and X-ray detection technology according to claim 4, characterized in that: When calculating the similarity value between the surface texture features and the internal structure features at the same spatial position through the feature matching algorithm, the specific calculation steps are as follows: Based on the spatial coordinates, the surface texture feature vector and the internal structure feature vector at the same position are combined into a feature pair; Calculate the Euclidean distance for each feature pair as the feature difference; Calculate the cosine similarity of the feature vector direction as feature consistency; The similarity value is calculated based on the feature difference and feature consistency results. The lower the similarity value, the closer the feature association.

10. The defect detection method combining vision and X-ray detection technology according to claim 2, characterized in that: When visual image data is processed frame by frame using the median filtering algorithm, the specific processing steps are as follows: Divide the visual image data into fixed-size pixel windows and sort the pixel values ​​within each window; The sorted middle value is selected as the new value of the center pixel of the window to complete the single window filtering process; The filtering operation is repeated through all pixel windows until the noise intensity of the entire frame image is lower than the set threshold, and the visual image data after noise suppression is obtained.

Images