Bearing defect detection method and system based on machine vision and ultrasonic detection

Abstract

The invention discloses a bearing defect detection method and system based on machine vision and ultrasonic detection, and particularly relates to the technical field of industrial automatic detection, and the method comprises the steps: S1, a synchronous collection module: carrying out pulse triggering synchronous collection, and generating a time-space reference table; s2, a feature extraction module: performing image noise reduction segmentation and ultrasonic frequency domain decomposition, and outputting a defect feature vector; s3, a fusion identification module: performing cross-modal feature alignment fusion to generate a defect classification conclusion; s4, a size measurement module: performing contour fitting to calculate inner and outer diameters, and outputting a size deviation value; and S5, a comprehensive judgment module: carrying out threshold comparison logic judgment, and generating a multi-modal detection report. According to the method, a space-time reference is established through an encoder, images are segmented in a self-adaptive mode, features are extracted through wavelet decomposition ultrasound, feature weights are re-calibrated through a parallel network and an attention mechanism, composite defects are recognized through cross-modal fusion, comprehensive judgment is conducted in combination with dimensional deviation, and a multi-dimensional quality evaluation system is achieved.

Description

Technical Field

[0001] This invention relates to the field of industrial automation inspection technology, and in particular to a bearing defect detection method and system based on machine vision and ultrasonic testing. Background Technology

[0002] The field of industrial automation inspection technology encompasses key technological aspects such as bearing quality inspection, production process monitoring, and equipment condition diagnosis. Its core components involve machine vision recognition, multimodal sensor fusion, automated control algorithms, and industrial IoT technology. It is primarily applied to defect identification, dimensional measurement, and process optimization in the manufacturing of precision parts, achieving real-time quality monitoring and data feedback for bearing assemblies on the production line through integrated inspection systems.

[0003] Among them, the bearing defect detection method and system based on machine vision and ultrasonic testing refers to using visible light imaging technology to obtain the geometric features of the bearing surface, simultaneously using ultrasonic flaw detection technology to detect the integrity of the internal structure, and establishing a correlation model between surface defects and internal cracks through a multi-source data registration algorithm. The specific implementation process includes a hardware combination of a high-resolution image acquisition device, an ultrasonic transducer array, a feature extraction processor, and a defect classifier, and uses a dual verification mechanism of image grayscale analysis and acoustic wave reflection signal time-frequency analysis to complete the defect judgment.

[0004] The limitations of existing technologies lie in the superficiality of their data analysis and the weak intermodal synchronization. When the vision and ultrasound systems operate as independent units, the spatiotemporal synchronization mechanism lacks precision, easily leading to mismatches in the spatial positions of surface images and internal echo signals. For example, when the coordinates of a minor surface scratch and an internal impurity approximately overlap, the system may incorrectly associate two independent low-risk defects as a single high-risk defect. Furthermore, relying on image grayscale analysis and conventional time-frequency analysis makes it difficult to effectively distinguish defects with inconspicuous features, such as surface oil stains and shallow pits, or internal micropores and inclusions. This analytical limitation results in blind spots in quality assessment. Summary of the Invention

[0005] The main objective of this invention is to provide a bearing defect detection method and system based on machine vision and ultrasonic testing, which can effectively solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A bearing defect detection method based on machine vision and ultrasonic testing includes the following steps:

[0008] S1: Multi-source signal synchronous acquisition, captures rotary encoder pulse signal as trigger source, synchronously excites image sensor and ultrasonic probe, and calculates trigger frequency based on transmission speed to generate spatiotemporal synchronization reference table;

[0009] S2: Single-modal data preprocessing, based on the spatiotemporal synchronization reference table, filtering and segmenting the image to extract defect contours, synchronously decomposing the ultrasonic signal to extract feature parameters, and generating visual defect regions and ultrasonic feature vectors;

[0010] S3: Cross-modal data fusion, calling visual defect regions and ultrasonic feature vectors, inputting them into a parallel network to extract features, recalibrating feature map weights through an attention module, and then aligning and integrating heterogeneous features through an interaction module, outputting fusion criteria, and generating fusion defect classification conclusions;

[0011] S4: Bearing size deviation measurement. Edge extraction and contour filtering are performed on the image data to obtain the inner and outer ring point sets of the bearing. The inner and outer diameters are calculated by least squares circle fitting and compared with the standard size to obtain the difference, generating the size deviation judgment result.

[0012] S5: Comprehensive Judgment and Report Generation. Based on the fusion defect classification conclusion and dimensional deviation judgment result, the decision rule base is called to compare the confidence level, deviation value and acceptance threshold, determine the qualified status, and generate a bearing multimodal inspection report.

[0013] Preferably, the spatiotemporal synchronization reference table specifically includes event sequence number, acquisition timestamp, and spatial location code; the visual defect region and ultrasonic feature vector include defect pixel coordinate set, ultrasonic energy feature spectrum, and echo transit time data; the fused defect classification conclusion specifically refers to defect prediction category and classification result reliability; the size deviation judgment result includes contour fitting geometric parameters and size tolerance compliance conclusion; and the bearing multimodal inspection report includes final quality grade, defect location and type description, size measurement data list, and product traceability information.

[0014] Preferably, step S1 includes:

[0015] Capture the pulse signal output by the rotary encoder, continuously monitor its logic level status, determine the rising edge of the pulse when the level changes from a low logic bit to a high logic bit, and record the system timestamp at this moment. Combine all rising edge events and their corresponding timestamps to establish the original trigger event sequence.

[0016] Each event in the original trigger event sequence is called as the main trigger signal, and a pulse width modulation signal is synchronously output to the exposure control terminal of the image sensor to excite the ultrasonic probe crystal to oscillate. At the same time, the real-time velocity reading of the transmission velocity sensor is collected, and the reading is divided by the preset spatial sampling interval value to obtain the synchronous excitation frequency value.

[0017] For each timestamp in the original trigger event sequence, an incremental pulse count value is assigned as a discretized index of spatial location, and the obtained synchronization excitation frequency value is used as the system operating frequency setting value at the current moment. The pulse count value, timestamp and frequency setting value are structurally associated to generate a spatiotemporal synchronization reference table.

[0018] Preferably, step S2 includes:

[0019] Based on the spatiotemporal synchronization reference table, the image data with its timestamp index is called, a pixel neighborhood window is set, the image is traversed pixel by pixel, the gray values ​​of each pixel and all pixels in its neighborhood window are collected and sorted, and the middle value after sorting is selected to replace the original gray value of the center pixel, thereby reducing isolated noise points in the image and establishing smooth image data.

[0020] Call smooth image data, count the gray values ​​of all pixels in the image and draw a gray value distribution histogram. Calculate the global segmentation threshold according to the principle of maximizing inter-class variance. Set pixels with gray values ​​higher than the threshold as foreground and pixels with gray values ​​lower than the threshold as background. Then perform erosion and dilation operations on the generated foreground area in sequence to fill the internal holes and smooth the contour to obtain the defect binarization map.

[0021] Based on the spatiotemporal synchronization reference table, the ultrasonic echo signal synchronized with the smoothed image data is extracted. Wavelet basis functions are selected to perform multi-level decomposition on the time domain signal. The original signal is projected into a series of orthogonal frequency band subspaces. The signal energy in each subspace is calculated and the energy values ​​are arranged in frequency band order to obtain the acoustic energy feature sequence.

[0022] For the original ultrasonic echo signal, the peak point of its signal envelope is detected to obtain the peak amplitude, and the time difference between the transmitted pulse and the peak of the first echo is calculated to obtain the transit time. The peak amplitude, transit time and the acquired acoustic energy feature sequence are combined and associated with the foreground region marked in the defect binarization map to generate the visual defect region and the ultrasonic feature vector.

[0023] Preferably, step S3 includes:

[0024] By calling the visual defect region and the ultrasonic feature vector, the pixel matrix of the visual defect region is input into a multi-layer convolutional structure. The convolutional kernel with a preset stride and a non-linear activation function are used to perform layer-by-layer scanning and mapping to extract geometric texture information under different receptive fields. Simultaneously, the ultrasonic feature vector is input into a one-dimensional temporal network. The one-dimensional convolutional kernel is used to slide along the sequence dimension to analyze the temporal dependency relationship inside the signal and establish a multi-layer visual feature map and a temporal acoustic feature set.

[0025] The channel and spatial attention weights are calculated and recalibrated based on the multi-layer visual feature map. Simultaneously, the channel attention weights are calculated and recalibrated based on the temporal acoustic feature set. Then, the recalibrated visual features are used as query vectors and acoustic features are used as key vectors. These are input into the cross-modal interaction module. By calculating the dot product attention score of the query vector and the key vector, the two heterogeneous features are aligned to obtain the aligned fused feature stream.

[0026] The aligned and fused feature stream is invoked, and a global average pooling operation is performed on it. The resulting feature vector is input into a fully connected classification layer. A normalized exponential function is applied to the node activation value of each defect category in the output, and the probability of belonging to each preset defect type is calculated. The category with the highest probability value is selected as the defect type identifier, and its probability value is used as the classification confidence score to generate a fused defect classification conclusion.

[0027] Preferably, step S4 includes:

[0028] For the acquired raw image data, the gradient intensity and direction of each pixel are calculated, local maxima in non-gradient directions are suppressed, and double thresholds are applied to connect the broken edges. Then, all connected pixel chains are searched to find the contour. Based on the perimeter and area of ​​the contour, the contour that conforms to the geometric features of the bearing is selected, and the inner and outer closed contour point sets are established.

[0029] Call the inner and outer closed contour point sets, construct an overdetermined system of equations for the coordinates of the point sets belonging to the inner and outer circles respectively, and obtain the center coordinates and radius values ​​that minimize the variance of the distance from all points to the center by solving the system of equations. Multiply the calculated inner and outer circle radii by two to obtain the fitted measurement values ​​of the inner and outer diameters.

[0030] Based on the fitted measurement values ​​of the inner and outer diameters, the nominal inner and outer diameters of the current bearing model are queried from the standard size database. The fitted measurement values ​​of the inner and outer diameters are subtracted from the corresponding nominal dimensions to calculate the dimensional deviation of the inner and outer diameters. The measured values ​​and the dimensional deviations are then combined in a structured manner to generate the dimensional deviation judgment result.

[0031] Preferably, step S5 includes:

[0032] Based on the fusion defect classification conclusion and the size deviation judgment result, the preset decision rule library is called to extract the defect category confidence and size deviation value respectively, and compare them with the corresponding quality acceptance threshold in the rule library one by one. The Boolean output results of each comparison operation are arranged in order to obtain the single logical judgment value.

[0033] Call the single logic judgment value, traverse all Boolean values ​​in the sequence, and when any judgment value is found to be "true", define the output result of the judgment logic as "exceeding the threshold" and the system immediately assigns the overall qualified status of the bearing to unqualified; otherwise, assign it to qualified and establish the comprehensive quality judgment status.

[0034] Based on the overall quality assessment status, the defect categories and confidence levels in the defect classification conclusions are summarized and integrated, and the inner and outer diameter measurements and deviations in the dimensional deviation assessment results are integrated. All the above information and the overall quality assessment status are then structured and arranged to generate a bearing multimodal inspection report.

[0035] A bearing defect detection system based on machine vision and ultrasonic testing, comprising:

[0036] The multi-source synchronous acquisition module captures the rotary encoder pulse signal as a trigger source, synchronously controls the industrial camera to acquire the bearing surface image, excites the ultrasonic probe to obtain the internal echo signal, calculates the trigger frequency based on the transmission speed, and generates a spatiotemporal synchronization reference table.

[0037] The visual feature extraction module, based on the spatiotemporal synchronization reference table, performs median filtering noise reduction on the bearing surface image, extracts the defect contour through adaptive threshold segmentation, optimizes geometric features by combining morphological operations, and establishes a visual defect feature map.

[0038] The ultrasonic feature extraction module calls the ultrasonic echo signal corresponding to the spatiotemporal synchronization reference table, performs wavelet packet decomposition to obtain the energy distribution of each frequency band, calculates the signal peak amplitude and transit time parameters, and generates ultrasonic energy feature vectors.

[0039] The cross-modal fusion module inputs the visual defect feature map into the convolutional network to extract spatial features, and simultaneously inputs the ultrasonic energy feature vector into the temporal network to extract time-frequency features. The feature weights are recalibrated through the attention mechanism, and the geometric features and temporal features are aligned using the feature interaction module to output the multimodal fusion criteria.

[0040] The comprehensive judgment module, based on the defect classification confidence level of the multimodal fusion criterion and combined with the inner and outer diameter deviation of the dimensional measurement module, calls the quality acceptance rule library to perform logical judgment and generate a bearing inspection report that includes defect type, dimensional tolerance, and quality grade.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] This invention establishes a unified spatiotemporal reference through rotary encoder pulses, laying the foundation for accurate cross-modal data alignment. At the feature extraction level, adaptive image segmentation and ultrasonic signal wavelet packet decomposition are applied to effectively extract defect information from complex interference. Its core gain lies in utilizing parallel networks and attention mechanisms to deeply analyze visual spatial features and ultrasonic temporal features, assigning weights accordingly, and then integrating them through a cross-modal interaction module to accurately identify previously difficult-to-distinguish correlated defects. Finally, dimensional deviation measurements are fused for comprehensive logical evaluation, achieving a shift from single-dimensional identification to multi-dimensional quality assessment and improving the comprehensiveness of detection. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0044] Figure 2 This is a schematic diagram of the single-modal data preprocessing process of the present invention. Detailed Implementation

[0045] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0046] Example 1, as Figure 1 As shown, the bearing defect detection method based on machine vision and ultrasonic testing includes the following steps:

[0047] S1: Multi-source signal synchronous acquisition. Pulse signals are captured by a rotary encoder. When the rising edge of the pulse is detected, the signal serves as the main trigger source. It synchronously outputs control signals to the exposure control terminal of the image sensor and excites the ultrasonic probe to emit sound waves. At the same time, it calls the real-time speed data fed back by the transmission speed sensor, calculates and obtains the trigger frequency that ensures a fixed spatial sampling interval, and generates a spatiotemporal synchronization reference table.

[0048] S2: Single-modal data preprocessing: Based on the spatiotemporal synchronization reference table, the average filtering algorithm is applied to the acquired image data to reduce random noise. Then, the adaptive segmentation threshold is calculated based on the gray-level histogram and the potential defect contour is extracted by combining morphological opening and closing operations. Simultaneously, wavelet packet decomposition is performed on the ultrasonic echo signal to extract parameters such as energy, peak amplitude and signal transit time of each frequency band to form a feature vector, generating visual defect areas and ultrasonic feature vectors.

[0049] S3: Cross-modal data fusion, which inputs the visual defect region and the ultrasonic feature vector into the parallel convolutional neural network and the one-dimensional temporal network branch respectively. The visual branch is used to extract the deep spatial geometric features of the defect, and the acoustic branch is used to analyze the temporal dependence of the signal. Then, the feature maps of the two modalities are re-weighted through the embedded channel and spatial attention modules respectively. Finally, the two heterogeneous features are aligned and integrated through the cross-modal interaction module, and the fusion criteria are output to generate the fused defect classification conclusion.

[0050] S4: Bearing size deviation measurement. For the acquired raw image data, the Canny operator is called to extract edges and the contour discovery algorithm is used to filter out the closed contour point set belonging to the inner and outer rings of the bearing. Then, the least squares circle fitting algorithm is executed on the selected point set to calculate the measured values ​​of the inner and outer diameters of the bearing. Finally, the measured values ​​are compared with the standard size database entries of the bearing model, the difference between the two is calculated, and the size deviation judgment result is generated.

[0051] S5: Comprehensive Judgment and Report Generation. Based on the defect category confidence and dimensional deviation judgment results output from the fusion defect classification conclusion, the system calls the preset decision rule base for logical evaluation. This rule base contains multiple independent judgment logics, which compare the defect confidence and dimensional deviation values ​​with the corresponding quality acceptance thresholds. When the output result of any judgment logic is "exceeds the threshold", the system sets the final judgment status of the bearing to unqualified and summarizes all inspection information to generate a bearing multimodal inspection report.

[0052] The spatiotemporal synchronization reference table specifically includes event sequence number, acquisition timestamp, and spatial location code. The visual defect area and ultrasonic feature vector include defect pixel coordinate set, ultrasonic energy feature spectrum, and echo transit time data. The fusion defect classification conclusion specifically refers to the defect prediction category and the reliability of the classification result. The dimensional deviation judgment result includes the contour fitting geometric parameters and dimensional tolerance compliance conclusion. The bearing multimodal inspection report includes the final quality grade, defect location and type description, dimensional measurement data list, and product traceability information.

[0053] Step S1 includes:

[0054] Capture the pulse signal output by the rotary encoder, continuously monitor its logic level status, determine the rising edge of the pulse when the level changes from a low logic bit to a high logic bit, and record the system timestamp at this moment. Combine all rising edge events and their corresponding timestamps to establish the original trigger event sequence.

[0055] The voltage signal at the output port of the rotary encoder is monitored. The logic low-level threshold is set to 0.8V and the high-level threshold is set to 2.4V. When the voltage value jumps from below 0.8V to above 2.4V, an interrupt service routine is triggered. In the interrupt service routine, the current count value of the 64-bit system timer is read as a timestamp (for example, when the timer frequency is 10MHz, the measured rising edge time is 1025843975 clock cycles). The 10 consecutively captured rising edge events are arranged in chronological order to form a two-dimensional array containing timestamps and event numbers. For example, event number 1 corresponds to timestamp 1025843975, event number 2 corresponds to 1025844123, and so on, to establish the original trigger event sequence.

[0056] Each event in the original trigger event sequence is called as the main trigger signal, and a pulse width modulation signal is synchronously output to the exposure control terminal of the image sensor to excite the ultrasonic probe crystal to oscillate. At the same time, the real-time velocity reading of the transmission velocity sensor is collected, and the reading is divided by the preset spatial sampling interval value to obtain the synchronous excitation frequency value.

[0057] The sequence number and timestamp of the nth event in the original trigger event sequence are called, and a TTL trigger signal with a pulse width of 20μs is output to the exposure control terminal of the image sensor (for example, when event number 3 corresponds to timestamp 1025844287, a high level is output at 1025844287+50ns and maintained for 20μs). At the same time, an excitation pulse with a center frequency of 5MHz is sent to the ultrasonic probe drive circuit. Simultaneously, the real-time speed value of the photoelectric encoder installed on the side of the conveyor belt is read (for example, the measured linear speed is 0.5m / s). The speed value of 0.5m / s is divided by the preset 2mm spatial sampling interval to calculate the trigger frequency of 250Hz. This frequency value is written into the clock divider register of the FPGA to establish the synchronous excitation frequency value.

[0058] For each timestamp in the original trigger event sequence, an incremental pulse count value is assigned as a discretized index of spatial location, and the obtained synchronization excitation frequency value is used as the system operating frequency setting value at the current moment. The pulse count value, timestamp and frequency setting value are structurally associated to generate a spatiotemporal synchronization reference table.

[0059] Assign an incrementing pulse count value to each timestamp in the original trigger event sequence (e.g., timestamp 1025843975 corresponds to count value 1, and 1025844123 corresponds to count value 2). Convert the count value to hexadecimal address codes 0x0001 and 0x0002. At the same time, read the current 250Hz frequency setting value from the synchronization excitation frequency value and quantize it into a 16-bit integer 0x00FA. Arrange the address code, the original timestamp value (64-bit binary), and the frequency quantization value in byte order to form a structured data packet of length 88 bytes (e.g., count value 1 corresponds to data packet: 0x00010x0000003D8A3B470x00FA). Transfer the data packet to the shared memory buffer via DMA to generate a spatiotemporal synchronization reference table.

[0060] See Figure 2 Step S2 includes:

[0061] Based on the spatiotemporal synchronization reference table, the image data with its timestamp index is called, a pixel neighborhood window is set, the image is traversed pixel by pixel, the gray values ​​of each pixel and all pixels in its neighborhood window are collected and sorted, and the middle value after sorting is selected to replace the original gray value of the center pixel, thereby reducing isolated noise points in the image and establishing smooth image data.

[0062] Based on the index address 0x0001 corresponding to timestamp 1025843975 in the spatiotemporal synchronization reference table, a 1280x1024 resolution grayscale image of the bearing surface is read from the image buffer. A 5x5 pixel neighborhood window is set, and the grayscale values ​​of the surrounding 25 pixels are collected with pixel coordinates (256, 512) as the center (e.g., [45, 48, 50, 49, 47, 50, 210, 52, 48, 49, 51, 53, 49, 48, 47, 46, 50 ...9, 48, 47, 46, 50, 49, 49, 49, The values ​​[9, 48, 52] are sorted in ascending order as [45, 46, 47, 47, 48, 48, 48, 48, 49, 49, 49, 50, 50, 50, 51, 52, 52, 53, 210]. The 13th value, 50, is selected to replace the original value 210 of the center pixel. This operation is performed on all 2.3 million pixels of the image. For example, the grayscale of the neighborhood of the processed coordinates (256, 512) becomes [45, 48, 50, 49, 47, 50, 50, 52, 48, ...], thus creating smooth image data.

[0063] Call smooth image data, count the gray values ​​of all pixels in the image and draw a gray value distribution histogram. Calculate the global segmentation threshold according to the principle of maximizing inter-class variance. Set pixels with gray values ​​higher than the threshold as foreground and pixels with gray values ​​lower than the threshold as background. Then perform erosion and dilation operations on the generated foreground area in sequence to fill the internal holes and smooth the contour to obtain the defect binarization map.

[0064] The number of pixels in the smoothed image data with gray levels 0-255 is counted. When the gray value is 127, the cumulative number of pixels is 18543, and when it is 128, the cumulative number of pixels is 18497. The inter-class variance when 127 and 128 are used as segmentation thresholds is calculated as follows: ω0 = 18543 / (12801024) = 0.0141, μ0 = Σ(icount(i)) / ω0 (i = 0-127). Similarly, ω1 and μ1 are calculated, and the variance is ω0ω1(μ 0-μ1)^2, iterate through the gray levels from 0 to 255 to find the threshold of 168 that maximizes the variance, set pixels with a gray level ≥ 168 to 255, and set the rest to 0. Perform erosion (remove isolated points) on the binary image using a 3x3 cross-shaped structuring element, and then perform two dilation operations (fill holes). For example, the area of ​​a region in the original binary image changes from 358 pixels to 320 pixels after erosion, and then recovers to 345 pixels after dilation, thus obtaining the defect binarization map.

[0065] Based on the spatiotemporal synchronization reference table, the ultrasonic echo signal synchronized with the smoothed image data is extracted. Wavelet basis functions are selected to perform multi-level decomposition on the time domain signal. The original signal is projected into a series of orthogonal frequency band subspaces. The signal energy in each subspace is calculated and the energy values ​​are arranged in frequency band order to obtain the acoustic energy feature sequence.

[0066] Based on the index corresponding to timestamp 1025843975 in the spatiotemporal synchronization reference table, echo signals from 2048 sampling points (sampling rate 100MHz) were read from the ultrasonic acquisition card. A three-level decomposition was performed using the db4 wavelet basis function, resulting in eight sub-band signals. The energy value E = Σx_i of the fifth sub-band (frequency range 6.25-12.5MHz) in the third level was calculated. 2 (i = 1 to 256), for example, the sum of squares of the sampled values ​​of this sub-band [-12, 15, -8, ...] is 28450. Similarly, calculate the energy of the remaining 7 sub-bands and normalize (the sum is 100%) to obtain the acoustic energy characteristic sequence of [12.3%, 9.8%, 15.2%, 18.7%, 10.5%, 13.1%, 11.6%, 8.8%].

[0067] For the original ultrasonic echo signal, the peak point of its signal envelope is detected to obtain the peak amplitude, and the time difference between the transmitted pulse and the peak of the first echo is calculated to obtain the transit time. The peak amplitude, transit time and the acquired acoustic energy feature sequence are combined and associated with the foreground region marked in the defect binarization map to generate the visual defect region and the ultrasonic feature vector.

[0068] The envelope of the original ultrasonic echo signal was obtained by performing a Hilbert transform. The maximum amplitude of 125mV (corresponding to sampling point 516) was detected in the sampling interval of 512-520. The rise time of the transmitted pulse t1 = 1025843975ns and the peak time t2 = 1025844123ns were recorded. The transit time Δt = 148ns was calculated. The peak amplitude of 125mV, Δt = 148ns and the acoustic energy feature sequence [12.3%,...,8.8%] were combined into a 10-dimensional vector [125,148,12.3,9.8,15.2,18.7,10.5,13.1,11.6,8.8]. The foreground region identifier code 0xFF with coordinates (256,512) in the defect binarization map was associated with it to generate the visual defect region and the ultrasonic feature vector.

[0069] Step S3 includes:

[0070] By calling the visual defect region and the ultrasonic feature vector, the pixel matrix of the visual defect region is input into a multi-layer convolutional structure. The convolutional kernel with a preset stride and a non-linear activation function are used to perform layer-by-layer scanning and mapping to extract geometric texture information under different receptive fields. Simultaneously, the ultrasonic feature vector is input into a one-dimensional temporal network. The one-dimensional convolutional kernel is used to slide along the sequence dimension to analyze the temporal dependency relationship inside the signal and establish a multi-layer visual feature map and a temporal acoustic feature set.

[0071] The 256x256 pixel matrix of the visual defect region (e.g., the region with coordinates (50-305, 100-355)) and the ultrasound feature vector [125, 148, 12.3, ..., 8.8] are input into the first convolutional layer. A 3x3 convolution kernel (weight matrix [[0.1, 0.2, 0.1], [0.2, 0.3, 0.2], [0.1, 0.2, 0.1]]) is used to scan with a stride of 2. The ReLU function (threshold) is applied to the output feature map (127x127). The first layer uses a 5x5 convolution kernel with a stride of 1, outputting a 123x123 feature map. Simultaneously, the ultrasound vector is unfolded into a 10-dimensional temporal sequence, which is input into a one-dimensional convolutional layer (kernel size 3, weights [0.4, 0.3, 0.3]), outputting an 8-dimensional sequence. After max pooling, a 4-dimensional temporal acoustic feature set [0.56, 0.78, 0.34, 0.62] is obtained. The visual branch finally outputs a 32x32 multi-layer visual feature map with 64 channels (e.g., the activation value at position (0,0) of channel 1 is 0.87).

[0072] The channel and spatial attention weights are calculated and recalibrated based on the multi-layer visual feature map. Simultaneously, the channel attention weights are calculated and recalibrated based on the temporal acoustic feature set. Then, the recalibrated visual features are used as query vectors and acoustic features are used as key vectors. These are input into the cross-modal interaction module. By calculating the dot product attention score of the query vector and the key vector, the two heterogeneous features are aligned to obtain the aligned fused feature stream.

[0073] Global average pooling was performed on the 64 channels of the multi-layer visual feature map to obtain a 64-dimensional channel weight vector [0.12, 0.09, ..., 0.15]. After normalization using the sigmoid function, the channel attention weights [0.52, 0.48, ..., 0.58] were obtained. Each channel of the original feature map was multiplied by its corresponding weight. For the spatial attention part, a 3x3 neighborhood variance was calculated for each 32x32 position. Positions with a variance greater than 0.05 were assigned a weight of 1.2, and the rest were assigned 1.0, resulting in a spatial weight matrix. The mean of the four channels [0.56, 0.78, 0.05] was calculated for the temporal acoustic feature set. [0.34, 0.62], after normalization, the channel weights are [0.35, 0.45, 0.12, 0.08]. After recalibration, the acoustic features become [0.56 × 0.35 = 0.196, ..., 0.62 × 0.08 = 0.0496]. The visual features are reshaped into a 1024-dimensional query vector Q, and the acoustic features are used as an 8-dimensional key vector K. The attention score S = Q·K / √8 is calculated (for example, the product of the 100th bit of Q (0.45) and the 3rd bit of K (0.34) contributes 0.153). After performing softmax on the score matrix, a weighted sum is obtained to obtain a 1024-dimensional aligned and fused feature stream.

[0074] The aligned and fused feature stream is invoked, and a global average pooling operation is performed on it. The resulting feature vector is input into a fully connected classification layer. A normalized exponential function is applied to the node activation value of each defect category in the output, and the probability of belonging to each preset defect type is calculated. The category with the highest probability value is selected as the defect type identifier, and its probability value is used as the classification confidence score to generate a fused defect classification conclusion.

[0075] Global average pooling (1024-dimensional → 64-dimensional) is performed on the aligned and fused feature stream. The input is a fully connected layer (weight matrix 64x5), and the node output values ​​[1.2, 3.5, -0.8, 2.1, 0.5] are calculated. The softmax function is applied: exp(1.2) = 3.32, exp(3.5) = 33.12, sum = 3.32 + 33.12 + 0.45 + 8.17 + 1.65 = 46.71, resulting in the probability distribution [7.1%, 70.9%, 1.0%, 17.5%, 3.5%]. The maximum value of 70.9% is selected as category 3 (internal crack), and the fused defect classification conclusion is generated (category ID3, confidence level 0.709).

[0076] Step S4 includes:

[0077] For the acquired raw image data, the gradient intensity and direction of each pixel are calculated, local maxima in non-gradient directions are suppressed, and double thresholds are applied to connect the broken edges. Then, all connected pixel chains are searched to find the contour. Based on the perimeter and area of ​​the contour, the contour that conforms to the geometric features of the bearing is selected, and the inner and outer closed contour point sets are established.

[0078] For a 1280x1024 pixel grayscale image of the bearing end face, the 3x3 Sobel operator is used to calculate the horizontal gradient Gx (convolution kernel [-1,0,1; -2,0,2; -1,0,1]) and the vertical gradient Gy (convolution kernel [-1,-2,-1; 0,0,0; 1,2,1]). For example, for pixel (200,300), Gx = 45 and Gy = 32, and the gradient magnitude √(45) 2 +32 2 =55.2, direction θ = arctan(32 / 45) = 35.5°, compare the magnitude of two adjacent pixels along the θ direction for each pixel, and set the current pixel to zero if it is not the maximum value. Set the high threshold to 50 and the low threshold to 20. Mark pixels with magnitude ≥ 50 as strong edges and pixels with magnitude 20-50 as weak edges. Connect strong edges with adjacent weak edges to form a continuous edge chain. Search all 8 connected regions. Calculate the perimeter L of the contour (e.g., L = 258 pixels for contour A) and the area S = 1200 pixels. Filter contours with L ∈ [200, 300] and S ∈ [800, 1500]. Establish a closed contour point set containing 32 contour points.

[0079] Call the inner and outer closed contour point sets, construct an overdetermined system of equations for the coordinates of the point sets belonging to the inner and outer circles respectively, and obtain the center coordinates and radius values ​​that minimize the variance of the distance from all points to the center by solving the system of equations. Multiply the calculated inner and outer circle radii by two to obtain the fitted measurement values ​​of the inner and outer diameters.

[0080] Use the coordinates of 15 points (e.g., (100, 200), (101, 201), ...) from the inner circle of the closed contour point set to construct an overdetermined system of equations: (x_i-a) 2 +(y_i-b) 2 =r 2 Expanded to -2ax_i-2by_i+(a 2 +b 2 -r 2 )=-x_i 2 -y_i 2 Let c = a 2 +b 2 -r 2This forms the matrix equation AX = B, where A = [-2x_i, -2y_i, 1] and B = [-x_i... 2 -y_i 2 Using the least squares method, we can solve X = (A^TA)^-1A^TB. For example, if we calculate a = 150.2, b = 180.5, and c = 1250.3, then r = √(a^TA)^-1A^TB. 2 +b 2 -c)=√(150.2 2 +180.5 2 -1250.3)=25.1mm, the outer ring r=52.3mm is calculated, and the inner and outer radii are multiplied by 2 to obtain the fitted measurement values ​​of the inner and outer diameters (inner diameter 50.2mm, outer diameter 104.6mm).

[0081] Based on the fitted measurement values ​​of the inner and outer diameters, the nominal inner and outer diameters of the current bearing model are queried from the standard size database. The fitted measurement values ​​of the inner and outer diameters are subtracted from the corresponding nominal dimensions to calculate the dimensional deviation of the inner and outer diameters. The measured values ​​and the dimensional deviations are then combined in a structured manner to generate the dimensional deviation judgment result.

[0082] Based on bearing model 6205, the nominal inner diameter of 25mm and outer diameter of 52mm are obtained from the standard database. The fitted measurement values ​​of the inner and outer diameters, 50.2mm and 25mm, are compared, and the deviation of the inner diameter is found to be +0.2mm. The deviation of the outer diameter, 104.6mm and 52mm, is found to be +0.6mm. A tolerance threshold of ±0.5mm is set. An inner diameter deviation of 0.2mm ≤ 0.5mm is considered acceptable, while an outer diameter deviation of 0.6mm > 0.5mm is considered out of tolerance. The dimensional deviation judgment result containing [inner diameter 50.2mm (+0.2mm), outer diameter 104.6mm (+0.6mm)] is generated.

[0083] The S5 steps include:

[0084] Based on the fusion defect classification conclusion and the size deviation judgment result, the preset decision rule library is called to extract the defect category confidence and size deviation value respectively, and compare them with the corresponding quality acceptance threshold in the rule library one by one. The Boolean output results of each comparison operation are arranged in order to obtain the single logical judgment value.

[0085] Based on the confidence score of 0.709 for category 3 (internal crack) in the defect classification conclusion and the outer diameter deviation of +0.6mm in the size deviation judgment result, the decision rule base calls rule 5 (crack confidence threshold 0.7) and rule 8 (outer diameter tolerance ±0.5mm). 0.709 is compared with 0.7 and found to be False (0.709 < 0.7), and +0.6mm is compared with ±0.5mm and found to be True (0.6 > 0.5). A Boolean array [Rule5:False, Rule8:True] is generated according to the rule number. The 12 thresholds in the rule base are traversed (e.g., Rule1 surface scratch threshold 0.6, Rule2 corrosion threshold 0.65), generating a total of 12 Boolean values, of which the 8th item is True and the rest are False. A single logical judgment value [F,F,F,F,F,F,F,T,F,F,F,F,F] is established.

[0086] Call the single logic judgment value, traverse all Boolean values ​​in the sequence, and when any judgment value is found to be "true", define the output result of the judgment logic as "exceeding the threshold" and the system immediately assigns the overall qualified status of the bearing to unqualified; otherwise, assign it to qualified and establish the comprehensive quality judgment status.

[0087] Iterate through the 12 elements of the single-item logic judgment value array. If the value of index 7 (starting from 0) is True, the interrupt mechanism is triggered, and the bearing status is marked as unqualified. At the same time, the trigger rule ID8 (outer diameter out of tolerance) is recorded. If all elements in the array are False, the bearing is marked as qualified. For example, if the current array contains True, the status code "Reject-08" is generated. The status code is bound to the timestamp 20240629143015 and written to the database field quality_status. The comprehensive quality judgment status is established as {"status":"unqualified","rule_id":8,"timestamp":"2024-06-2914:30:15"}.

[0088] Based on the overall quality assessment status, the defect categories and confidence levels in the defect classification conclusions are summarized and integrated, and the inner and outer diameter measurements and deviations in the dimensional deviation assessment results are integrated. All the above information and the overall quality assessment status are then structured and arranged to generate a bearing multimodal inspection report.

[0089] The defect type "internal crack" with a confidence level of 70.9% was extracted from the defect classification conclusion. The dimensional deviation judgment results were inner diameter 50.2mm (+0.2mm) and outer diameter 104.6mm (+0.6mm). Combined with the comprehensive quality judgment status, the results were formatted in JSON as {"defect_type":"internal crack","confidence":0.709,"inner_dia":50.2,"inner_dev":+0.2,"outer_dia":104.6,"outer_dev":+0.6"judgment":"unqualified"}. After adding the inspection time and equipment number EQP-205, a bearing multimodal inspection report containing 23 fields was generated.

[0090] A bearing defect detection system based on machine vision and ultrasonic testing, comprising:

[0091] The multi-source synchronous acquisition module captures the rotary encoder pulse signal as a trigger source, synchronously controls the industrial camera to acquire images of the bearing surface, and excites the ultrasonic probe to acquire internal echo signals. It calculates the trigger frequency based on the transmission speed and generates a spatiotemporal synchronization reference table.

[0092] The visual feature extraction module, based on the spatiotemporal synchronization reference table, performs median filtering noise reduction on the bearing surface image, extracts the defect contour through adaptive threshold segmentation, optimizes geometric features by combining morphological operations, and establishes a visual defect feature map.

[0093] The ultrasonic feature extraction module calls the ultrasonic echo signal corresponding to the spatiotemporal synchronization reference table, performs wavelet packet decomposition to obtain the energy distribution of each frequency band, calculates the signal peak amplitude and transit time parameters, and generates ultrasonic energy feature vectors.

[0094] The cross-modal fusion module inputs the visual defect feature map into the convolutional network to extract spatial features, and simultaneously inputs the ultrasonic energy feature vector into the temporal network to extract time-frequency features. The feature weights are recalibrated through the attention mechanism, and the geometric features and temporal features are aligned using the feature interaction module to output the multimodal fusion criteria.

[0095] The comprehensive judgment module, based on the defect classification confidence level of the multimodal fusion criterion and combined with the inner and outer diameter deviation of the dimensional measurement module, calls the quality acceptance rule library to perform logical judgment and generate a bearing inspection report that includes defect type, dimensional tolerance, and quality grade.

[0096] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A bearing defect detection method based on machine vision and ultrasonic testing, comprising the following steps: S1: Multi-source signal synchronous acquisition, captures rotary encoder pulse signal as trigger source, synchronously excites image sensor and ultrasonic probe, and calculates trigger frequency based on transmission speed to generate spatiotemporal synchronization reference table; S2: Single-modal data preprocessing, based on the spatiotemporal synchronization reference table, filtering and segmenting the image to extract defect contours, synchronously decomposing the ultrasonic signal to extract feature parameters, and generating visual defect regions and ultrasonic feature vectors; S3: Cross-modal data fusion, calling visual defect regions and ultrasonic feature vectors, inputting them into a parallel network to extract features, recalibrating feature map weights through an attention module, and then aligning and integrating heterogeneous features through an interaction module, outputting fusion criteria, and generating fusion defect classification conclusions; S4: Bearing size deviation measurement. Edge extraction and contour filtering are performed on the image data to obtain the inner and outer ring point sets of the bearing. The inner and outer diameters are calculated by least squares circle fitting and compared with the standard size to obtain the difference, generating the size deviation judgment result. S5: Comprehensive Judgment and Report Generation. Based on the fusion defect classification conclusion and dimensional deviation judgment result, the decision rule base is called to compare the confidence level, deviation value and acceptance threshold, determine the qualified status, and generate a bearing multimodal inspection report.

2. The bearing defect detection method based on machine vision and ultrasonic testing according to claim 1, characterized in that: The spatiotemporal synchronization reference table specifically includes event sequence number, acquisition timestamp, and spatial location code. The visual defect region and ultrasonic feature vector include defect pixel coordinate set, ultrasonic energy feature spectrum, and echo transit time data. The fused defect classification conclusion specifically refers to the defect prediction category and the reliability of the classification result. The size deviation judgment result includes the contour fitting geometric parameters and the size tolerance compliance conclusion. The bearing multimodal inspection report includes the final quality grade, defect location and type description, size measurement data list, and product traceability information.

3. The bearing defect detection method based on machine vision and ultrasonic testing according to claim 1, characterized in that: Step S1 includes: Capture the pulse signal output by the rotary encoder, continuously monitor its logic level status, determine the rising edge of the pulse when the level changes from a low logic bit to a high logic bit, and record the system timestamp at this moment. Combine all rising edge events and their corresponding timestamps to establish the original trigger event sequence. Each event in the original trigger event sequence is called as the main trigger signal, and a pulse width modulation signal is synchronously output to the exposure control terminal of the image sensor to excite the ultrasonic probe crystal to oscillate. At the same time, the real-time velocity reading of the transmission velocity sensor is collected, and the reading is divided by the preset spatial sampling interval value to obtain the synchronous excitation frequency value. For each timestamp in the original trigger event sequence, an incremental pulse count value is assigned as a discretized index of spatial location, and the obtained synchronization excitation frequency value is used as the system operating frequency setting value at the current moment. The pulse count value, timestamp and frequency setting value are structurally associated to generate a spatiotemporal synchronization reference table.

4. The bearing defect detection method based on machine vision and ultrasonic testing according to claim 1, characterized in that: Step S2 includes: Based on the spatiotemporal synchronization reference table, the image data with its timestamp index is called, a pixel neighborhood window is set, the image is traversed pixel by pixel, the gray values ​​of each pixel and all pixels in its neighborhood window are collected and sorted, and the middle value after sorting is selected to replace the original gray value of the center pixel, thereby reducing isolated noise points in the image and establishing smooth image data. Call smooth image data, count the gray values ​​of all pixels in the image and draw a gray value distribution histogram. Calculate the global segmentation threshold according to the principle of maximizing inter-class variance. Set pixels with gray values ​​higher than the threshold as foreground and pixels with gray values ​​lower than the threshold as background. Then perform erosion and dilation operations on the generated foreground area in sequence to fill the internal holes and smooth the contour to obtain the defect binarization map. Based on the spatiotemporal synchronization reference table, the ultrasonic echo signal synchronized with the smoothed image data is extracted. Wavelet basis functions are selected to perform multi-level decomposition on the time domain signal. The original signal is projected into a series of orthogonal frequency band subspaces. The signal energy in each subspace is calculated and the energy values ​​are arranged in frequency band order to obtain the acoustic energy feature sequence. For the original ultrasonic echo signal, the peak point of its signal envelope is detected to obtain the peak amplitude, and the time difference between the transmitted pulse and the peak of the first echo is calculated to obtain the transit time. The peak amplitude, transit time and the acquired acoustic energy feature sequence are combined and associated with the foreground region marked in the defect binarization map to generate the visual defect region and the ultrasonic feature vector.

5. The bearing defect detection method based on machine vision and ultrasonic testing according to claim 1, characterized in that: Step S3 includes: By calling the visual defect region and the ultrasonic feature vector, the pixel matrix of the visual defect region is input into a multi-layer convolutional structure. The convolutional kernel with a preset stride and a non-linear activation function are used to perform layer-by-layer scanning and mapping to extract geometric texture information under different receptive fields. Simultaneously, the ultrasonic feature vector is input into a one-dimensional temporal network. The one-dimensional convolutional kernel is used to slide along the sequence dimension to analyze the temporal dependency relationship inside the signal and establish a multi-layer visual feature map and a temporal acoustic feature set. The channel and spatial attention weights are calculated and recalibrated based on the multi-layer visual feature map. Simultaneously, the channel attention weights are calculated and recalibrated based on the temporal acoustic feature set. Then, the recalibrated visual features are used as query vectors and acoustic features are used as key vectors. These are input into the cross-modal interaction module. By calculating the dot product attention score of the query vector and the key vector, the two heterogeneous features are aligned to obtain the aligned fused feature stream. The aligned and fused feature stream is invoked, and a global average pooling operation is performed on it. The resulting feature vector is input into a fully connected classification layer. A normalized exponential function is applied to the node activation value of each defect category in the output, and the probability of belonging to each preset defect type is calculated. The category with the highest probability value is selected as the defect type identifier, and its probability value is used as the classification confidence score to generate a fused defect classification conclusion.

6. The bearing defect detection method based on machine vision and ultrasonic testing according to claim 1, characterized in that: The S4 step includes: For the acquired raw image data, the gradient intensity and direction of each pixel are calculated, local maxima in non-gradient directions are suppressed, and double thresholds are applied to connect the broken edges. Then, all connected pixel chains are searched to find the contour. Based on the perimeter and area of ​​the contour, the contour that conforms to the geometric features of the bearing is selected, and the inner and outer closed contour point sets are established. Call the inner and outer closed contour point sets, construct an overdetermined system of equations for the coordinates of the point sets belonging to the inner and outer circles respectively, and obtain the center coordinates and radius values ​​that minimize the variance of the distance from all points to the center by solving the system of equations. Multiply the calculated inner and outer circle radii by two to obtain the fitted measurement values ​​of the inner and outer diameters. Based on the fitted measurement values ​​of the inner and outer diameters, the nominal inner and outer diameters of the current bearing model are queried from the standard size database. The fitted measurement values ​​of the inner and outer diameters are subtracted from the corresponding nominal dimensions to calculate the dimensional deviation of the inner and outer diameters. The measured values ​​and the dimensional deviations are then combined in a structured manner to generate the dimensional deviation judgment result.

7. The bearing defect detection method based on machine vision and ultrasonic testing according to claim 1, characterized in that: Step S5 includes: Based on the fusion defect classification conclusion and the size deviation judgment result, the preset decision rule library is called to extract the defect category confidence and size deviation value respectively, and compare them with the corresponding quality acceptance threshold in the rule library one by one. The Boolean output results of each comparison operation are arranged in order to obtain the single logical judgment value. Call the single logic judgment value, traverse all Boolean values ​​in the sequence, and when any judgment value is found to be "true", define the output result of the judgment logic as "exceeding the threshold". The system immediately assigns the overall qualified status of the bearing to unqualified. Otherwise, it assigns the value to qualified and establishes the comprehensive quality judgment status. Based on the overall quality assessment status, the defect categories and confidence levels in the defect classification conclusions are summarized and integrated, and the inner and outer diameter measurements and deviations in the dimensional deviation assessment results are integrated. All the above information and the overall quality assessment status are then structured and arranged to generate a bearing multimodal inspection report.

8. A bearing defect detection system based on machine vision and ultrasonic testing, characterized in that, The method is used in the bearing defect detection system based on machine vision and ultrasonic testing as described in any one of claims 1-7, the system comprising: The multi-source synchronous acquisition module captures the rotary encoder pulse signal as a trigger source, synchronously controls the industrial camera to acquire images of the bearing surface, and excites the ultrasonic probe to acquire internal echo signals. It calculates the trigger frequency based on the transmission speed and generates a spatiotemporal synchronization reference table. The visual feature extraction module, based on the spatiotemporal synchronization reference table, performs median filtering noise reduction on the bearing surface image, extracts the defect contour through adaptive threshold segmentation, optimizes geometric features by combining morphological operations, and establishes a visual defect feature map. The ultrasonic feature extraction module calls the ultrasonic echo signal corresponding to the spatiotemporal synchronization reference table, performs wavelet packet decomposition to obtain the energy distribution of each frequency band, calculates the signal peak amplitude and transit time parameters, and generates ultrasonic energy feature vectors. The cross-modal fusion module inputs the visual defect feature map into the convolutional network to extract spatial features, and simultaneously inputs the ultrasonic energy feature vector into the temporal network to extract time-frequency features. The feature weights are recalibrated through the attention mechanism, and the geometric features and temporal features are aligned using the feature interaction module to output the multimodal fusion criteria. The comprehensive judgment module, based on the defect classification confidence level of the multimodal fusion criterion and combined with the inner and outer diameter deviation of the dimensional measurement module, calls the quality acceptance rule library to perform logical judgment and generate a bearing inspection report that includes defect type, dimensional tolerance, and quality grade.

Images