A bearing steel ball surface pit identification method and system based on machine vision

Abstract

The application relates to the technical field of bearing steel ball detection, in particular to a bearing steel ball surface pit identification method and system based on machine vision, which comprises the following steps: projecting a multi-frequency phase shift stripe on the surface of a steel ball and collecting an image sequence, constructing an absolute phase matrix through multi-frequency heterodyne phase demodulation, introducing a reference phase model for differential comparison, stripping a macro-surface component to extract a residual phase anomaly distribution graph, calculating a phase gradient feature as a guide signal, inputting the guide signal into a convolutional neural network embedded with an adaptive attention mechanism, dynamically adjusting a convolution kernel to extract abnormal features and outputting a suspected pit boundary box, and adopting a dynamic contour evolution logic to perform energy functional minimization iteration on local data blocks to extract a final pit area. The technical scheme can effectively eliminate the interference of macro-surface and reflection, and realize high-precision and automatic accurate identification of the pit defects on the surface of the steel ball.

Description

Technical Field

[0001] This application relates to the field of bearing steel ball detection technology, specifically a method and system for identifying surface pits on bearing steel balls based on machine vision. Background Technology

[0002] As a core component in precision mechanical transmission, the surface quality of bearing steel balls directly determines the bearing's operating accuracy, fatigue life, and vibration and noise levels. During the processing and distribution of steel balls, micron-sized pits are easily generated on the surface. Therefore, rigorous quality inspection of these pits is an essential step.

[0003] In practical industrial applications, the surface of high-precision bearing steel balls exhibits extremely omnidirectional, highly reflective mirror characteristics. Faced with such extremely reflective targets, existing machine vision inspection methods, during image acquisition and feature extraction, completely obscure the subtle morphological features of extremely shallow, micrometer-sized pits within the strong specular reflection and the macroscopic spherical background of the steel ball itself. This results in the inability to effectively image and characterize the abrupt changes in the local morphology of these shallow pits. Current technologies struggle to effectively isolate and capture the true morphological boundaries of micrometer-sized pits from the highly reflective macroscopic spherical background, leading to an extremely high false negative rate for identifying these extremely shallow, tiny pits on the surface of such extremely reflective steel balls, thus hindering automated and high-precision identification.

[0004] In view of this, this application proposes a machine vision-based method and system for identifying surface pits on bearing steel balls. Summary of the Invention

[0005] To achieve the above objectives, this application provides a machine vision-based method and system for identifying surface pits on bearing steel balls, the specific technical solution of which is as follows:

[0006] A machine vision-based method for identifying surface pits on bearing steel balls, comprising:

[0007] Multi-frequency phase-shifted sinusoidal fringes are projected onto the surface of the bearing steel ball, and a sequence of deformation fringe images reflected from the surface of the steel ball is acquired to obtain a two-dimensional digital image matrix containing the surface topography modulation information of the steel ball.

[0008] Multi-frequency heterodyne phase deconvolution processing is performed on the two-dimensional digital image matrix to expand the relative phase data wrapped in a fixed period into a continuously distributed absolute phase, and construct an absolute phase spatial distribution matrix that characterizes the geometric morphology of the steel ball surface.

[0009] A reference phase model characterizing the macroscopic spherical basic phase components of a steel ball is introduced. The absolute phase spatial distribution matrix is ​​compared with the reference phase model by difference. The macroscopic spherical basic phase components are stripped away, and the residual phase anomaly distribution map containing only local morphological change information is extracted.

[0010] The phase gradient feature information of the residual phase anomaly distribution map is calculated. The residual phase anomaly distribution map is input into a convolutional neural network with an adaptive attention mechanism. The phase gradient feature information is used as the guiding signal of the adaptive attention mechanism. The phase gradient direction is used to dynamically adjust the offset of the convolutional kernel to extract the anomaly features and output the coordinates of the two-dimensional bounding box of the suspected pit.

[0011] In the residual phase anomaly distribution map, the local phase data block corresponding to the coordinates of the two-dimensional bounding box of the suspected pit is extracted. The local phase data block is subjected to energy functional minimization iteration using dynamic contour evolution logic, and the pixel set of the energy convergence region is extracted as the final pit region.

[0012] Furthermore, the projection of multi-frequency phase-shifted sinusoidal fringes onto the surface of the bearing steel balls includes:

[0013] Multiple sets of sinusoidal fringe patterns with different spatial frequencies are pre-configured. The sinusoidal fringe patterns at each spatial frequency are sequentially phase-shifted at equal intervals according to a preset number of phase shift steps to generate a multi-frequency phase-shifted sinusoidal fringe sequence covering multiple frequencies and multiple phase shifts. The multi-frequency phase-shifted sinusoidal fringe sequence is projected frame by frame onto the surface of the bearing steel ball according to a preset time sequence, so that each frame of fringe pattern is modulated by the steel ball surface to form deformation stripes carrying surface morphology information.

[0014] Furthermore, the acquisition of the deformation stripe image sequence reflected from the surface of the steel ball includes:

[0015] The camera is driven to acquire frame-by-frame images of the deformation stripes reflected from the surface of the steel ball according to the trigger signal synchronized with the projection timing of the projection device. The gray value distribution of each frame of the acquired deformation stripe image is processed by gray value quantization, and the gray value matrix is ​​converted into a discrete gray value matrix. The gray value matrix is ​​grouped and arranged according to frequency group and phase shift number to construct the two-dimensional digital image matrix.

[0016] Furthermore, the multi-frequency heterodyne phase deconvolution processing of the two-dimensional digital image matrix includes:

[0017] For each set of spatial frequency corresponding to the phase-shifted fringe image, a phase-shift demodulation algorithm is used to extract the wrapped phase distribution at each frequency from the gray values ​​of the multi-step phase-shifted image, resulting in multiple wrapped phase maps that correspond one-to-one with each spatial frequency. The multiple wrapped phase maps are then subjected to step-by-step heterodyne operation, and the equivalent wavelength is gradually expanded by the phase difference between adjacent frequencies, so that the high-frequency wrapped phase is gradually expanded to the low-frequency range, eliminating the periodic phase cutoff jump.

[0018] Furthermore, the construction of the absolute phase space distribution matrix characterizing the geometric morphological features of the steel ball surface includes:

[0019] The unfolded phase obtained by the successive heterodyne operation is used as the guiding phase. The wrapping phase at the highest spatial frequency is recovered by integer number of cycles to obtain continuously distributed absolute phase values. The absolute phase values ​​are mapped and arranged according to the pixel spatial coordinate position to form a two-dimensional absolute phase spatial distribution matrix corresponding to each pixel of the sampling area on the steel ball surface. The value of each element in the two-dimensional absolute phase spatial distribution matrix represents the morphological feature information at the corresponding surface position.

[0020] Furthermore, the construction of the absolute phase space distribution matrix characterizing the geometric morphological features of the steel ball surface includes:

[0021] After forming a two-dimensional absolute phase spatial distribution matrix corresponding to each pixel of the sampling area on the steel ball surface, the absolute phase spatial distribution matrix is ​​preprocessed by spatial filtering. An adaptive smoothing filter is used to suppress the high-frequency phase jitter components introduced by the acquisition noise in the matrix, while retaining the mid-to-low frequency phase change characteristics corresponding to the surface morphology.

[0022] The effective region mask is extracted from the filtered absolute phase spatial distribution matrix. Invalid phase data outside the effective reflection area of ​​the steel ball surface are removed according to the modulation index criterion to obtain the absolute phase spatial distribution matrix that only covers the effective measurement area of ​​the steel ball.

[0023] Furthermore, the introduction of a reference phase model characterizing the fundamental phase components of the macroscopic spherical surface of the steel ball includes:

[0024] Using the phase data located within the effective measurement area of ​​the steel ball in the absolute phase spatial distribution matrix as fitting samples, the least squares bivariate polynomial surface fitting algorithm is used to perform overall fitting operations on the fitting samples to determine the polynomial coefficients of the reference phase model.

[0025] Based on the polynomial coefficients, a reference phase model of the same size as the absolute phase space distribution matrix is ​​generated pixel by pixel according to the bivariate polynomial surface equation.

[0026] Furthermore, the step of stripping the macroscopic spherical fundamental phase component and extracting a residual phase anomaly distribution map containing only local morphological abrupt changes includes:

[0027] The absolute phase spatial distribution matrix and the reference phase model are subjected to element-wise difference operations at corresponding pixel positions to obtain the residual phase distribution after stripping the macroscopic spherical basic phase components; the residual phase distribution is subjected to numerical normalization mapping to uniformly map the residual phase values ​​to a preset numerical range to generate the residual phase anomaly distribution map, and the local phase change region in the residual phase anomaly distribution map corresponds to the candidate region of morphological defects on the steel ball surface.

[0028] Furthermore, the extraction of the residual phase anomaly distribution map containing only local morphological abrupt changes includes:

[0029] After generating the residual phase anomaly distribution map, morphological preprocessing is performed on the residual phase anomaly distribution map. Isolated noise disturbance regions with an area smaller than a preset threshold are identified and removed through connected component analysis, while retaining local phase anomaly regions with spatial continuity.

[0030] The residual phase anomaly distribution map after morphological preprocessing is used as input data for calculating phase gradient feature information.

[0031] Furthermore, the phase gradient feature information includes phase gradient magnitude information and phase gradient direction information;

[0032] The convolutional neural network with embedded adaptive attention mechanism includes: a feature extraction backbone unit, a multi-scale feature fusion unit, and a detection head unit connected in sequence;

[0033] The feature extraction backbone unit uses the residual phase anomaly distribution map as the main input, and the phase gradient direction information as the auxiliary guiding input of the adaptive attention mechanism embedded in the feature extraction backbone unit; the feature extraction backbone unit is a residual neural network backbone unit, which is composed of multiple cascaded residual convolutional units, each of which contains a convolutional layer, a normalization layer and a skip connection path.

[0034] The multi-scale feature fusion unit receives feature maps output from different depth levels in the feature extraction backbone unit and performs cross-scale feature aggregation through upsampling and channel concatenation operations.

[0035] The detection head unit outputs the coordinates of the two-dimensional bounding box of the suspected pit and the corresponding confidence score based on the aggregated multi-scale features.

[0036] Furthermore, the embedded adaptive attention mechanism includes:

[0037] An attention subunit is embedded in the residual convolution unit of the feature extraction backbone unit. The attention subunit uses the phase gradient feature information at the corresponding position of the current feature map as a guiding signal to dynamically generate the spatial offset parameter of each sampling position.

[0038] The spatial position of the regular sampling grid of the convolution kernel is adjusted according to the spatial offset parameter, so that the receptive field of the convolution kernel is adaptively expanded along the direction of increasing phase gradient magnitude, covering local abnormal features distributed along the edge of abrupt morphological changes.

[0039] Furthermore, the output of the two-dimensional bounding box coordinates of the suspected pit includes:

[0040] The two-dimensional bounding boxes of suspected pits output by the detection head unit are post-processed and filtered. A non-maximum suppression strategy is used to redundantly remove candidate bounding boxes with overlapping spatial positions. The bounding box with the highest confidence score in each group of overlapping bounding boxes is retained as the localization result of each suspected pit region. The coordinates of the filtered two-dimensional bounding boxes are expanded outward according to a preset expansion ratio to expand the truncation range to ensure that the subsequent truncation of local phase data blocks completely covers the pit and its surrounding edge transition region.

[0041] Furthermore, the step of using dynamic contour evolution logic to perform energy functional minimization iteration on local phase data blocks includes:

[0042] A closed evolution curve is initialized in the local phase data block, and an energy functional is constructed that includes an external energy term driven by the phase gradient features and an internal energy term constrained by the curve's own geometry. The energy functional is iteratively minimized to drive the closed evolution curve to gradually evolve along the phase gradient direction under the traction of the external energy term until the curve converges to the phase abrupt boundary of the pit region.

[0043] Furthermore, the set of pixels in the extracted energy convergence region, as the final pit region, includes:

[0044] The convergence criterion for determining that the closed evolution curve has reached the energy convergence state is that the change in curve position between adjacent iteration steps is less than a preset convergence threshold.

[0045] When the convergence criterion is met, the set of pixels corresponding to the internal region enclosed by the closed evolution curve is determined as the precise segmentation region of a single pit.

[0046] The dynamic contour evolution logic is executed on the local phase data blocks corresponding to the coordinates of the two-dimensional bounding boxes of all suspected pits. The pixel set of the energy convergence region in each local phase data block is summarized to form a complete recognition result of all pit areas on the surface of the bearing steel ball.

[0047] A machine vision-based bearing steel ball surface pit recognition system, used to implement the aforementioned machine vision-based bearing steel ball surface pit recognition method, includes: a stripe image acquisition module, a multi-frequency heterodyne phase resolution module, a residual phase extraction module, a suspected pit detection module, and a pit contour extraction module.

[0048] The stripe image acquisition module is used to project multi-frequency phase-shift sinusoidal stripes onto the surface of the bearing steel ball, acquire the deformation stripe image sequence reflected by the steel ball surface, and obtain a two-dimensional digital image matrix containing the surface topography modulation information of the steel ball.

[0049] The multi-frequency heterodyne phase unification module is used to perform multi-frequency heterodyne phase unification processing on the two-dimensional digital image matrix, expand the relative phase data wrapped in a fixed period into a continuously distributed absolute phase, and construct an absolute phase spatial distribution matrix that characterizes the geometric morphology of the steel ball surface.

[0050] The residual phase extraction module is used to introduce a reference phase model that characterizes the basic phase components of the macroscopic spherical surface of the steel ball, perform a difference comparison operation between the absolute phase spatial distribution matrix and the reference phase model, strip away the basic phase components of the macroscopic spherical surface, and extract a residual phase anomaly distribution map that contains only local morphological change information.

[0051] The suspected pit detection module is used to calculate the phase gradient feature information of the residual phase anomaly distribution map, input the residual phase anomaly distribution map into a convolutional neural network with an adaptive attention mechanism, and use the phase gradient feature information as the guiding signal of the adaptive attention mechanism. It uses the phase gradient direction to dynamically adjust the convolution kernel offset to extract anomaly features and outputs the coordinates of the two-dimensional bounding box of the suspected pit.

[0052] The pit contour extraction module is used to extract the local phase data block corresponding to the coordinates of the two-dimensional bounding box of the suspected pit in the residual phase anomaly distribution map, and to perform energy functional minimization iteration on the local phase data block using dynamic contour evolution logic, and extract the pixel set of the energy convergence region as the final pit region.

[0053] The beneficial effects of this application are as follows: By projecting multi-frequency phase-shifting sinusoidal fringes and acquiring deformation image sequences, this application overcomes the problem of easy overexposure of extreme mirror surfaces under traditional lighting conditions. This application utilizes the high-sensitivity modulation characteristics of structured light to transform the three-dimensional deformation of extremely shallow micron-level pits into a two-dimensional digital image matrix, laying a reliable data foundation.

[0054] This application employs multi-frequency heterodyne phase unification processing to solve the problems of phase truncation and envelopment ambiguity in single high-frequency fringes. By unfolding the relative phase into continuous absolute phases, a high-precision absolute phase spatial distribution matrix is ​​constructed, enabling unambiguous digital reconstruction of the three-dimensional morphology of the steel ball surface.

[0055] This application introduces a reference phase model for differential comparison to accurately remove the dominant macroscopic spherical fundamental phase component. This operation eliminates the masking of minute defects by the inherent curved surface background, separates the weak signal of extremely shallow pits from the strong background, and extracts a high signal-to-noise ratio residual abnormal distribution map, thus reducing the difficulty of identification.

[0056] This application uses phase gradient features as a guiding signal to input into the neural network and dynamically adjusts the convolution kernel offset, so that the network adaptively focuses on the edge region with drastic changes in shape. This overcomes the problem that conventional convolution is insufficient for extracting small pit features and achieves high-accuracy localization of suspected pit bounding boxes.

[0057] This application introduces dynamic contour evolution logic into local data blocks, finds the optimal boundary through energy functional minimization iteration, and uses local phase change information to constrain the evolution, avoiding edge blurring of traditional threshold segmentation, and achieving pixel-level accurate extraction of the real physical boundary of extremely shallow pits.

[0058] The technical solution of this application breaks through the industrial bottleneck of the difficulty in imaging and identifying extremely shallow pits on highly reflective mirror steel balls, realizes the leap of defect detection towards full automation and high precision, reduces the missed detection rate and labor costs, ensures the operating accuracy and life of precision bearings, and provides core quality control support for high-end equipment manufacturing. Attached Figure Description

[0059] Figure 1 A flowchart of a machine vision-based method for identifying surface pits on bearing steel balls provided in this application;

[0060] Figure 2 This is a schematic diagram of the structured light projection measurement scene provided in this application;

[0061] Figure 3 A schematic diagram of the multi-frequency phase-shift sinusoidal fringe sequence provided in this application;

[0062] Figure 4 A schematic diagram illustrating the construction of the reference phase model and the extraction of residuals provided in this application;

[0063] Figure 5 A schematic diagram of the dynamic contour evolution process provided for this application;

[0064] Figure 6 This application provides a structural diagram of a machine vision-based bearing steel ball surface pit recognition system. Detailed Implementation

[0065] To make the above-mentioned objectives, features and advantages of this application more readily understood, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0066] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0067] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of this application. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0068] Example 1

[0069] Reference Figures 1 to 5 This is the first embodiment of the present application, such as Figure 1 As shown, a machine vision-based method for identifying surface pits on bearing steel balls is provided.

[0070] Step 1: Project multi-frequency phase-shifted sinusoidal fringes onto the surface of the bearing steel ball, acquire the image sequence of deformation fringes reflected from the surface of the steel ball, and obtain a two-dimensional digital image matrix containing the surface topography modulation information of the steel ball.

[0071] To obtain raw image data carrying three-dimensional morphological information of the steel ball surface, the structured light projection measurement principle is adopted. A sinusoidal stripe pattern with a known coding rule is projected onto the surface of the bearing steel ball through a digital projection device. The deformation stripes formed after being modulated by the geometric morphology of the steel ball surface carry the surface height undulation information. The deformation stripe images are acquired frame by frame by a synchronously triggered industrial camera and digitally processed to provide a data foundation for subsequent phase demodulation and morphological reconstruction.

[0072] See Figure 2 This is a schematic diagram of a structured light projection measurement scenario. Figure 2 The layout of a projection measurement scene based on the principle of structured light triangulation is demonstrated. Figure 2 A digital projector (with a built-in DLP chip) on the left projects a sinusoidal stripe pattern onto the surface of the central bearing steel ball using a conical lens assembly. An industrial camera (CMOS sensor) on the right captures images of the deformed stripes reflected from the steel ball's surface through a lens assembly. Multiple nearly vertical projected stripes are visible on the steel ball's surface. In the recessed areas (marked by dashed ellipses), the stripes exhibit significant bending distortion, demonstrating the geometric modulation effect of abrupt changes in surface height on the stripes. A top-mounted control computer sends stripe pattern data to the projector and acquires image data from the camera via data channels. The projector and camera are connected by a hardware synchronization trigger signal (dashed arrow), ensuring precise timing correspondence between each frame of projected stripes and the corresponding acquired image. The entire system is mounted on a stable platform, with the projector and camera positioned on either side of the steel ball, forming a triangulation geometry.

[0073] Specifically, during fringe projection, multiple sets of sinusoidal fringe patterns with different spatial frequencies need to be pre-configured. The number of spatial frequency sets is determined based on the measurement accuracy requirements of the steel ball surface and the constraints on the frequency order of the multi-frequency hierarchical phase demodulation algorithm. For example, three spatial frequency sets are sufficient to meet the accuracy requirements for industrial-grade bearing steel ball detection. The sinusoidal fringe patterns at each spatial frequency are sequentially phase-shifted at equal intervals according to a preset number of phase shift steps. The number of phase shift steps determines the accuracy of single-frequency phase demodulation and its resistance to ambient light noise interference. For example, four phase shift steps are used for each spatial frequency set. The spatial frequency values ​​of each frequency set are arranged in ascending order, and the frequency ratio between adjacent frequency sets must meet the requirements for hierarchical phase demodulation unfolding accuracy.

[0074] Spatial frequency is described by normalizing the number of complete fringe periods contained within the full width of the projected image. Let the total number of horizontal pixels in the projected image be... Then frequency group The corresponding spatial frequency is Period per pixel, where For frequency group The number of complete fringe periods across the full width of the projected image. For example, the three sets of fringe period numbers are 1, 8, and 64, respectively, corresponding to 1, 8, and 64 complete fringe periods across the full width of the projected image, with a frequency ratio of 8 between adjacent frequency groups. For each phase shift step at each spatial frequency group, a frame of sinusoidal fringe pattern is generated. The light intensity distribution at any pixel position in a single frame of sinusoidal fringe pattern satisfies a sinusoidal modulation relationship. Let the horizontal pixel coordinates in the projected image be... Vertical pixel coordinates are The spatial frequency of the frequency group to which the current stripe pattern belongs is The current phase shift number is The phase shift step size between adjacent phase shift steps is Then the projected stripes are at the pixel position Design light intensity at the location Represented as: ;in This represents the DC background light intensity component of the projected fringes. This represents the modulation amplitude component of the projected fringes. For example, the phase shift step size corresponds to a four-step phase shift. Pick Based on the combination relationship between the number of frequency groups and the number of phase shift steps, a multi-frequency phase-shifted sinusoidal fringe sequence covering all frequency groups and all phase shift steps is generated. For example, three spatial frequencies are combined with four phase shift steps to generate a total of 12 frames of fringe patterns.

[0075] See Figure 3 This is a schematic diagram of a multi-frequency phase-shifted sinusoidal fringe sequence. Figure 3The complete encoding scheme of the multi-frequency phase-shift sinusoidal stripe sequence is presented in matrix form. Figure 3 The three rows correspond to the low-frequency group (1 fringe period), mid-frequency group (8 fringe periods), and high-frequency group (64 fringe periods), respectively, with spatial frequencies increasing sequentially from top to bottom. The frequency ratio between adjacent frequency groups is 8 times (indicated by the double-headed arrows on the left). Each row of four columns represents four equally spaced phase-shifted fringe patterns within the same frequency group, with phase shifts of 0°, 90°, 180°, and 270°, respectively. The brightness distribution of the fringes within a row shifts horizontally with each phase shift step. The low-frequency group exhibits a slow gradient in brightness, presenting only one complete bright-dark cycle across the entire image; the mid-frequency group increases to eight fringe periods, with more dense alternations between bright and dark; the high-frequency group has extremely fine fringes, presenting 64 complete cycles within the same image width. The curly braces on the right indicate three groups of four-step phase shifts, totaling 12 frames of fringe patterns, constituting a complete multi-frequency phase-shifted projection sequence.

[0076] Furthermore, the generated multi-frequency phase-shift sinusoidal fringe sequence is loaded frame by frame into the frame buffer of a digital projection device according to a preset timing sequence. The digital projection device then projects each frame of the fringe pattern onto the surface of the bearing steel ball at fixed frame intervals. The sinusoidal fringes projected onto the steel ball surface deform due to the modulation of the surface geometry. The fringes in flat areas maintain an approximately equidistant distribution, while in areas with surface defects such as pits, the fringes undergo significant bending or local spacing distortion due to abrupt changes in local height. A hardware trigger signal, strictly synchronized with the projection timing sequence of the digital projection device, drives an industrial camera to acquire frame by frame images of the deformed fringe patterns reflected from the steel ball surface, ensuring a precise temporal correspondence between each frame of projected fringe pattern and the corresponding acquired image.

[0077] Each frame of the deformation stripe image captured by the industrial camera undergoes grayscale quantization processing. The reflected light intensity distribution sensed by the image sensor is transformed into a discrete grayscale value matrix. For example, the grayscale quantization bit depth is 8 bits, corresponding to a grayscale value range of 0 to 255. The grayscale value matrices of all captured frames are grouped and arranged according to frequency group and phase shift sequence to construct a two-dimensional digital image matrix. The two-dimensional digital image matrix is ​​divided into multiple subsets according to frequency group, and each subset contains grayscale images of all phase shift steps under the corresponding frequency group.

[0078] Through the above-described stripe projection and image acquisition process, the three-dimensional morphology information of the steel ball surface is completely encoded into a two-dimensional grayscale image sequence. The multi-frequency, multi-phase-shift encoding strategy provides sufficient redundant information for subsequent heterodyne deconvolution to eliminate phase ambiguity.

[0079] Step 2: Perform multi-frequency heterodyne phase deconvolution on the two-dimensional digital image matrix to expand the relative phase data wrapped in a fixed period into a continuously distributed absolute phase, and construct an absolute phase spatial distribution matrix that characterizes the geometric features of the steel ball surface.

[0080] Specifically, in the multi-frequency heterodyne phase demodulation process, the first step is to extract the wrapped phase. For each subset of phase-shifted fringe images corresponding to a spatial frequency in the two-dimensional digital image matrix, a phase-shift demodulation algorithm is used to extract the wrapped phase distribution at each frequency from the grayscale values ​​of the multi-step phase-shifted images. For frequency group number... A subset of the stripe image, assuming the number of phase shift steps at each spatial frequency group is . The phase shift step size between adjacent phase shift steps is , No. The phase shift corresponds to the pixel position of the acquired image. The gray value at that location is denoted as Then frequency group At pixel position The wrapping phase at the location Calculated using the arctangent function in the four quadrants:

[0081] ;

[0082] in For frequency group In pixels The enclosed phase value obtained by the arctangent operation in the four quadrants; For frequency group The Phase-shift image acquisition at pixel The grayscale value at that location; This represents the number of phase shift steps at each spatial frequency group. The phase offset step size between adjacent phase shift steps; This is a four-quadrant arctangent function, which can determine the true phase quadrant based on the signs of the sine and cosine terms. The first parameter in the formula is the negative of the sum of the products of the phase shift grayscale values ​​and the corresponding sine weights in each step. This is because the phase shift in the projected light intensity model defined in step 1 is superimposed with a positive sign within the phase term of the cosine function. The numerator term of the phase shift demodulation operation needs to be negative to maintain consistency with the positive phase shift convention of the projected light intensity model, ensuring the correct mapping relationship between the extracted encapsulated phase polarity and the true morphology of the steel ball surface.

[0083] Due to the range characteristics of the four-quadrant arctangent operation, the resulting wrapped phase value is precisely truncated within... Within the interval, the portion of the true phase corresponding to the surface morphology that exceeds a single period exhibits periodic truncation transitions. The aforementioned phase-shift demodulation operation is performed on all frequency groups in the two-dimensional digital image matrix, yielding multiple wrap-around phase maps corresponding one-to-one with each spatial frequency. Although each of the multiple wrap-around phase maps with different spatial frequencies exhibits periodic truncation, the truncation transition positions between different frequencies are distinct. This complementarity provides a sufficient information basis for subsequent heterodyne operations to progressively eliminate phase ambiguity.

[0084] Furthermore, heterodyne calculations and absolute phase recovery are performed. A stepwise heterodyne operation is performed on multiple wrapped phase maps, progressively expanding the equivalent wavelength by using the wrapped phase difference between adjacent frequency groups, gradually unfolding the high-frequency wrapped phase to the low-frequency range. Specifically, the phase difference is calculated at the same pixel location for wrapped phase maps corresponding to two adjacent spatial frequencies. The difference result corresponds to the wrapped phase at the equivalent spatial frequency, which is equal to the difference between the two original spatial frequencies, thus significantly increasing the equivalent wavelength. Phase wrapping correction is then applied to the difference result to ensure that the heterodyne phase value remains within the required range. Within the interval, heterodyne operations are performed sequentially from low to high frequency. Each heterodyne operation uses the expanded phase obtained from the previous level as guiding information to perform integer-cycle recovery of the wrapped phase at higher frequencies in the current level. In the final heterodyne operation, the expanded phase obtained from each level's heterodyne operation is used as the guiding phase to perform integer-cycle recovery of the wrapped phase under the highest spatial frequency group, obtaining continuously distributed absolute phase values. Let the index of the highest spatial frequency group be . The number of recoveries in the whole cycle is Then the pixel position absolute phase at Represented as: ;in For the highest spatial frequency group in pixels The phase value of the package at that location, This is the integer-cycle recovery number determined based on the guiding phase. The absolute phase values ​​at all pixel locations are mapped and arranged according to their pixel spatial coordinates, forming a two-dimensional absolute phase spatial distribution matrix corresponding to each pixel of the sampling area on the steel ball surface. Each element in the two-dimensional absolute phase spatial distribution matrix represents the morphological features at the corresponding surface location. The spatial distribution trend of the phase values ​​reflects the macroscopic spherical contour and local morphological changes such as pits on the steel ball surface. The multi-frequency heterodyne stepwise expansion strategy allows the final absolute phase to simultaneously possess the high sensitivity of the highest frequency group and the large range advantage of the lowest frequency group, achieving phase resolution matching the highest spatial frequency without sacrificing the measurement range.

[0085] Furthermore, after constructing the two-dimensional absolute phase spatial distribution matrix, it undergoes spatial domain filtering preprocessing. An adaptive smoothing filter is used to suppress high-frequency phase jitter components introduced by image acquisition noise and quantization errors in the absolute phase spatial distribution matrix. The window size of the adaptive smoothing filter is automatically adjusted according to the local phase gradient amplitude. A larger window is used in areas with gentle phase changes to enhance the smoothing effect, while the window is automatically reduced in edge areas with sharp phase changes to avoid blurring effective morphological feature information. For example, the maximum size of the filtering window is 7×7 pixels. The adaptive window adjustment mechanism achieves a good balance between noise suppression and edge preservation in the filtering operation. The absolute phase spatial distribution matrix after spatial domain filtering preprocessing retains the mid-to-low frequency phase change features corresponding to the surface morphology of the steel ball, while effectively suppressing high-frequency noise interference. Subsequently, effective region mask extraction is performed on the filtered absolute phase spatial distribution matrix. The effective reflection area of ​​the steel ball surface is identified based on the modulation index criterion. Pixel positions with a modulation index lower than a preset criterion threshold are determined as invalid data outside the effective reflection area of ​​the steel ball. For example, the modulation index criterion threshold is 10% of the maximum modulation index. The positions of invalid data pixels are marked as masks and zeroed out to obtain an absolute phase spatial distribution matrix that only covers the effective measurement area of ​​the steel ball.

[0086] This step extracts high-precision, low-noise, and continuously distributed absolute phase information from the original grayscale image sequence through the above-mentioned multi-frequency heterodyne phase deconvolution and preprocessing operations. The absolute phase spatial distribution matrix accurately records the geometric morphological features of the steel ball surface in the form of quantized phase values, providing reliable input data for the subsequent construction of the reference phase model and morphological deviation analysis.

[0087] Step 3: Introduce a reference phase model that characterizes the basic phase components of the macroscopic spherical surface of the steel ball. Perform a difference comparison operation between the absolute phase spatial distribution matrix and the reference phase model to strip away the basic phase components of the macroscopic spherical surface and extract the residual phase anomaly distribution map that contains only local morphological change information.

[0088] Specifically, the construction process of the baseline phase model includes: using phase data located within the effective reflection region of the steel ball in the absolute phase space distribution matrix as fitting samples, and performing an overall fitting operation on the fitting samples using a least-squares bivariate polynomial surface fitting algorithm. Since the mapping relationship between the absolute phase and the physical height of the steel ball surface is nonlinear due to the influence of the geometric parameters of the projection-imaging system, the distribution shape of the macroscopic spherical contour of the steel ball in the absolute phase space is not a strictly spherical function, but a smooth continuous surface determined by the system's geometric parameters and the spherical curvature. The bivariate polynomial surface model does not require pre-setting specific geometric shape assumptions and can fit any smooth basic shape in the absolute phase space with sufficient flexibility. The order of the bivariate polynomial used for fitting needs to strike a balance between the accuracy of the baseline fitting and the risk of overfitting: too low an order cannot accurately fit the curvature characteristics of the spherical phase distribution, while too high an order may include local phase abrupt changes in the pit region in the baseline model, leading to the defect information being incorrectly fitted and eliminated. For example, the order of the bivariate polynomial is 4, corresponding to 15 undetermined coefficients. The goal of least-squares fitting is to minimize the sum of squared residuals between the absolute phase observations and the predictions of the polynomial surface model at all pixel locations within the effective region. Let the pixel locations in the reference phase model be... Reference phase value at for: ;in Let the order of the fitting of the bivariate polynomial be denoted as . In polynomials of Power and of The undetermined coefficients corresponding to the power-law cross terms are all determined by performing least squares calculation on the absolute phase values ​​of all pixels within the effective region.

[0089] Since local defects such as pits occupy a very small proportion of the overall surface area of ​​the steel ball, and the vast majority of pixels in the fitted sample correspond to normal spherical areas, the phase deviation caused by local defects has a negligible impact on the overall fitting result. Therefore, the least squares polynomial surface fitting algorithm can robustly extract the smooth reference distribution of the macroscopic spherical contour of the steel ball in phase space. The polynomial coefficients determined by the least squares solution are... A reference phase model of the same size as the absolute phase space distribution matrix is ​​generated by calculating pixel-by-pixel according to the polynomial surface equations described above. The reference phase value at each pixel location in the reference phase model... Determined by the polynomial coefficients and the spatial coordinates of the pixel itself, it represents the theoretical phase value that should be presented at the corresponding position when the surface of the steel ball is assumed to be a defect-free ideal sphere. The spatial distribution of the reference phase model presents a smooth and continuous curved surface feature, without containing any information on abrupt changes in local morphology.

[0090] Furthermore, the absolute phase spatial distribution matrix and the reference phase model are subjected to element-wise difference operations at corresponding pixel positions to obtain the residual phase distribution after stripping the macroscopic spherical fundamental phase components. Pixel position residual phase value at Represented as: ;in For pixels in the absolute phase space distribution matrix The absolute phase value at that point, For pixels in the reference phase model The reference phase theoretical value is used. In the normal spherical region of the steel ball surface, the difference between the absolute phase value and the reference phase theoretical value is minimal, and the residual phase value approaches zero. In the region of morphological defects such as pits, the absolute phase value deviates significantly from the reference phase theoretical value due to abrupt changes in local height, and the residual phase value exhibits obvious positive or negative abrupt changes. Through differential operations, the large phase changes contributed by the macroscopic spherical contour are completely eliminated, and only the phase anomaly information related to local morphological defects is retained in the residual phase distribution. This transforms the steel ball surface morphology detection problem from the difficult task of identifying minute defects against a macroscopic spherical background into the simple task of detecting local phase abrupt changes against an approximately zero background.

[0091] See Figure 4 This is a schematic diagram of the construction of the reference phase model and residual extraction. Figure 4 The core principles of constructing the reference phase model and extracting the residual phase are illustrated in two parts. In the upper one-dimensional cross-sectional view, the "absolute phase distribution" curve on the left shows a smooth, arc-shaped overall trend (corresponding to the macroscopic spherical contour of the steel ball), with significant phase depressions appearing in local locations (the gray-filled areas are labeled as phase depressions caused by pits). The middle "reference phase model" is generated by fitting a fourth-order bivariate polynomial surface, exhibiting a smooth, continuous arc shape without any local abrupt changes. After difference operations, the "residual phase distribution" on the right is obtained. The spherical background phase is completely removed, and the residual approaches zero in the normal region (labeled as near-zero background), retaining significant negative anomalous abrupt changes only at the pit locations. The lower two-dimensional grayscale view simulates the phase space distribution with radial gradients. In the absolute phase image, pits appear as dark spots. After subtracting the smooth reference model, the residual image clearly highlights isolated dark pit anomalous areas on a uniform grayscale background.

[0092] The residual phase distribution is numerically normalized to map the residual phase values ​​to a preset numerical range, typically between zero and one. The normalization mapping employs a linear stretching method, using the minimum and maximum values ​​in the residual phase distribution as the starting and ending points of the mapping. All residual phase values ​​are scaled proportionally to the preset numerical range, generating a residual phase anomaly distribution map. This normalization process eliminates the absolute dimensional differences in the residual phase values, enabling subsequent convolutional neural networks to learn features on a uniform numerical scale. Local phase abrupt change regions in the residual phase anomaly distribution map correspond to candidate morphological defects on the steel ball surface. Normalized residual values ​​in the normal spherical region cluster around a specific normalization level determined by the range of residual numerical distribution because the original residuals are close to zero.

[0093] After generating the residual phase anomaly distribution map, morphological preprocessing is performed to further purify the residual phase data. A connected component analysis algorithm is used to identify all connected regions in the residual phase anomaly distribution map where the normalized residual values ​​exceed the normal fluctuation range. Isolated connected regions with an area smaller than a preset area threshold are identified as false anomaly responses caused by noise disturbances, and the residual values ​​of the corresponding pixels are reset to the background mean. For example, the preset area threshold is set to 5 pixels. Local phase anomaly regions with spatial continuity and an area not less than the preset area threshold are retained; these regions correspond to actual morphological deviations on the steel ball surface.

[0094] The residual phase anomaly distribution map after morphological preprocessing in this step eliminates scattered noise interference points, making the phase anomaly features of real defect areas such as pits more prominent, and significantly reducing the feature extraction difficulty of subsequent pit detection algorithms.

[0095] Step 4: Calculate the phase gradient feature information of the residual phase anomaly distribution map, input the residual phase anomaly distribution map into the convolutional neural network with an adaptive attention mechanism, and use the phase gradient feature information as the guiding signal of the adaptive attention mechanism. Dynamically adjust the convolutional kernel offset using the phase gradient direction to extract anomaly features and output the coordinates of the two-dimensional bounding box of the suspected pit.

[0096] Specifically, the first-order partial derivatives are calculated in both the horizontal and vertical directions of the residual phase anomaly distribution map after morphological preprocessing to obtain the phase gradient components at each pixel location. Let the pixel locations in the residual phase anomaly distribution map be... The normalized residual value at is Horizontal phase gradient component and the phase gradient component in the vertical direction Pixel positions are obtained through calculations using the difference operator. Phase gradient magnitude at and phase gradient direction angle They are represented as follows:

[0097] ;

[0098] ;

[0099] in The residual phase anomaly distribution map at the pixel The value of the first-order partial derivative along the horizontal direction. The residual phase anomaly distribution map at the pixel The value of the first-order partial derivative along the vertical direction. This represents the combined magnitude of the gradient components in both directions. As a four-quadrant arctangent function, it can determine the true gradient direction based on the signs of the two gradient components, avoiding computational singularities when the gradient component in a single direction is zero. It should be noted that when the horizontal gradient component... gradient components in the vertical direction As the gradient approaches zero, the gradient vector degenerates into a zero vector, and the gradient direction angle... Mathematically, this situation lacks definite physical meaning. It corresponds to a normal spherical region where the residual phase distribution is flat. In the subsequent adaptive attention mechanism, this is resolved through multiplicative gating of the gradient magnitude: when the gradient magnitude approaches zero, the spatial offset output by the offset prediction network is suppressed by the magnitude feature and automatically approaches zero. The uncertain orientation angle does not have a practical impact on the convolution operation. In the pit edge region, the phase gradient magnitude increases significantly, and the gradient direction is highly consistent along the normal direction of the pit edge. The orientation angle information is reliable and has a clear physical meaning. The phase gradient magnitude and orientation angle at each pixel location are combined to form a dual-channel phase gradient feature map of the same size as the residual phase anomaly distribution map. This dual-channel phase gradient feature map serves as an auxiliary guiding input to the adaptive attention mechanism in the convolutional neural network.

[0100] The convolutional neural network with embedded adaptive attention mechanism consists of three sequentially connected parts: a feature extraction backbone unit, a multi-scale feature fusion unit, and a detection head unit. The feature extraction backbone unit uses the residual phase anomaly distribution map as its main input channel and a dual-channel phase gradient feature map as the auxiliary guiding input channel for the adaptive attention mechanism embedded within it. The feature extraction backbone unit adopts a residual neural network backbone architecture, consisting of multiple cascaded residual convolutional units; for example, 16 residual convolutional units are used. Each residual convolutional unit contains two convolutional layers, a batch normalization layer following the convolutional layers, and a skip connection path spanning the two convolutional layers. This skip connection path directly adds the input of the residual convolutional unit to the convolutional output element-wise, alleviating the gradient decay problem in deep network training. For example, the kernel size of the convolutional layers in the residual convolutional unit is 3×3 pixels, with an initial channel count of 64. As the network depth increases, spatial downsampling is performed through convolutional layers with a stride of two, and the channel count is multiplied.

[0101] Furthermore, the adaptive attention mechanism is embedded in the residual convolutional unit of the feature extraction backbone unit. Before the first convolutional operation of each residual convolutional unit, an attention sub-unit is set up. The attention sub-unit receives the dual-channel phase gradient feature map at the corresponding pixel position of the current feature map as a guiding signal. The attention sub-unit contains an offset prediction network consisting of two fully connected layers. The offset prediction network uses the gradient magnitude and orientation angle data of the dual-channel phase gradient feature map after being processed by the same downsampling layer as the current feature map as joint input, and dynamically generates the spatial offset parameters at each sampling position in the current residual convolutional unit. Let the horizontal offset corresponding to the original sampling position of the convolution kernel on the standard regular sampling grid be denoted as . Vertical offset is The offset prediction network is based on pixels Phase gradient direction angle at and phase gradient magnitude Together, they generate the predicted value of the offset.

[0102] In the pit edge region, a strong gradient magnitude activates the offset prediction network, causing it to generate spatial offsets based on the orientation angle. This shifts the sampling points of the convolutional kernel along the direction of the drastic phase gradient change, adaptively expanding the receptive field of the kernel along the normal direction of the pit edge and capturing complete local anomaly features distributed along the abrupt change in shape. In the normal spherical region, because the phase gradient magnitude is close to zero, the spatial offsets generated by the offset prediction network are suppressed by the gradient magnitude feature and tend to zero. The sampling grid of the convolutional kernel maintains a standard and regular arrangement, preserving the spatial response characteristics of conventional convolution operations. Based on the spatial offset parameters output by the offset prediction network, the regular sampling grid of the convolutional kernel is spatially offset position by position. Bilinear interpolation is used to calculate the feature values ​​at the non-integer coordinate positions after the offset, completing the convolution operation with adaptive receptive field deformation capability. The adaptive attention mechanism enables the convolutional neural network to fully utilize the inherent gradient prior knowledge of the residual phase data to guide the feature extraction process. Compared with standard convolution operations using a fixed geometric receptive field, it has a stronger feature capture capability in scenarios with irregular pit edge shapes and varying scales.

[0103] Furthermore, the multi-scale feature fusion unit receives feature maps output from different depth levels in the feature extraction backbone unit. Shallow feature maps retain high spatial resolution, containing fine-grained localization information of the pit edges; deep feature maps, after multiple spatial downsampling operations, have a larger receptive field, containing overall semantic discrimination information of the pit region. The multi-scale feature fusion unit performs upsampling on the deep feature maps, restoring their spatial dimensions to a level comparable to the shallow feature maps. Figure 1 Then, the feature maps of different levels are aggregated along the channel direction through a channel-dimensional concatenation operation. A 1×1 convolution layer is then applied to the concatenated multi-channel features for channel compression and information integration, outputting a feature map containing multi-scale fusion information. The multi-scale feature fusion unit performs upsampling and concatenation operations on feature maps of multiple different depth levels in the feature extraction backbone unit in a top-down path. For example, the depth levels involved in fusion are the outputs of the 4th, 8th, and 16th residual convolution units of the feature extraction backbone unit. Through multi-scale feature fusion, pits and defects of different sizes can be effectively represented in the fused feature map.

[0104] Furthermore, the detection head unit performs localization and confidence assessment of the pit region based on the aggregated feature map output by the multi-scale feature fusion unit. The detection head unit contains two parallel convolutional branches. One branch predicts the coordinates of the two-dimensional bounding box of the suspected pit region, encoded with four parameters: the x-coordinate of the center point, the y-coordinate of the center point, the width of the bounding box, and the height of the bounding box. The other branch predicts the pit confidence score corresponding to each two-dimensional bounding box, representing the probability that the current bounding box actually contains a pit defect. Each branch contains two 3×3 convolutional layers and one 1×1 convolutional output layer. Post-processing is performed on all candidate two-dimensional bounding boxes output by the detection head unit. First, a minimum confidence threshold is set based on the pit confidence score. Candidate bounding boxes with pit confidence scores lower than the minimum confidence threshold are directly eliminated. For example, the minimum confidence threshold is set to 0.5. Subsequently, a non-maximum suppression (NMS) strategy is employed to redundantly eliminate candidate bounding boxes with spatial overlap. The intersection-union ratio (IUR) between any two candidate bounding boxes is calculated. When the IUR exceeds a preset overlap threshold, the bounding box with the higher pit confidence score is retained while the bounding box with the lower score is eliminated. For example, the preset overlap threshold is set to 0.45. After filtering by the NMS strategy, the retained bounding boxes are the localization results for each suspected pit region. The coordinates of the filtered 2D bounding boxes are then expanded outward according to a preset expansion ratio. The width and height of the bounding box are multiplied by the expansion ratio coefficient and then expanded proportionally in all directions. For example, the preset expansion ratio coefficient is set to 1.3, expanding the truncated range outward by 30% from the original bounding box. The boundary expansion process ensures that the truncated local phase data block completely covers the pit contour and its surrounding edge transition region, providing input data containing complete pit boundary information for the dynamic contour evolution logic.

[0105] This step introduces the gradient prior unique to the phase domain into the adaptive attention mechanism of the convolutional neural network, thereby realizing adaptive feature extraction and coarse localization detection of pit defects. The detection results are labeled in the form of two-dimensional bounding boxes to indicate the spatial location range of each suspected pit in the residual phase anomaly distribution map.

[0106] Step 5: Extract the local phase data block corresponding to the coordinates of the two-dimensional bounding box of the suspected pit from the residual phase anomaly distribution map. Use dynamic contour evolution logic to perform energy functional minimization iteration on the local phase data block and extract the pixel set of the energy convergence region as the final pit region.

[0107] Specifically, for each suspected pit's two-dimensional bounding box coordinate obtained after post-processing filtering and boundary expansion in step 4, a rectangular sub-region is extracted from the residual phase anomaly distribution map after morphological preprocessing according to the spatial coordinate range of the bounding box. The extracted rectangular sub-region is the local phase data block. The local phase data block contains the phase abrupt change information of the pit region and the stable phase background information of the normal surface region around the pit. The pit edge in the local phase data block is represented by a gradient jump band where the normalized residual value transitions sharply from the background level to the interior of the pit.

[0108] Dynamic contour evolution logic is executed in each local phase data block. First, a closed evolution curve is initialized within the local phase data block. The initial closed evolution curve is set as a rectangular loop with a preset inward shrinkage distance from the boundary of the local phase data block. For example, the preset inward shrinkage distance is 10% of the length of the shorter side of the local phase data block. The initial closed evolution curve is set outside the pit and completely surrounds the pit region, providing the starting condition for subsequent curve shrinkage evolution from the outside in.

[0109] See Figure 5 This is a schematic diagram of the dynamic contour evolution process. Figure 5 The entire process of dynamic contour evolution from initialization to convergence is illustrated in four stages. In the first panel, the initial state, the gray irregular area represents the actual extent of the pit, and the thick dashed rectangle is the initial closed evolution curve, shrinking inward by 10% to the boundary of the local phase data block. Four inward arrows indicate the direction of the contraction driving force. In the second panel, the intermediate state, the evolution curve has shrunk from a rectangle to an irregular closed curve that approximates the shape of the pit, but it has not yet completely conformed to the edge. The arrows point in the direction of continued contraction. In the third panel, the convergence state, the evolution curve (solid line) closely conforms to the actual boundary of the pit, and the edge attraction and contraction driving force reach a dynamic balance (marked with bidirectional short lines), and the curve no longer moves. The fourth panel, the segmentation result, shows the final extracted set of pit pixels (dark gray filled area), accurate to pixel-level resolution.

[0110] Furthermore, an energy functional containing internal and external energy terms is constructed. Minimizing this energy functional drives the closed evolution curve to gradually shrink towards the true boundary of the pit. Let the closed evolution curve be defined by the arc length parameter. For parameterization, the position vector of any point on the curve is: Then the energy functional From internal energy terms and external energy terms The total energy is obtained by integrating the two energy densities over the entire arc length of the closed evolution curve. The internal energy term constrains the geometry of the closed evolution curve itself and consists of smoothing constraint components and rigid constraint components. The smoothing constraint components penalize the magnitude of the first derivative of the closed evolution curve to suppress local stretching, while the rigid constraint components penalize the magnitude of the second derivative of the closed evolution curve to suppress sharp bends. Internal energy density Represented as: ;in, For closed evolution curves in arc length parameter Internal energy density at that location To smooth the weighting coefficients of the constraint components and control the ability of the closed evolution curve to resist local tensile deformation; The weighting coefficients of the rigid constraint components control the ability of the closed evolution curve to resist bending deformation; For closed evolution curves in arc length parameter The first derivative vector at point ; For closed evolution curves in arc length parameter The second derivative vector at that point. Smoothness constraint weight coefficients. and rigid constraint weight coefficient The value of this parameter needs to strike a balance between the regularity of the curve shape and the flexibility of the boundary fit. For example, the smoothing constraint weight coefficient... Take zero point one, the rigid constraint weight coefficient We take 0.01. The smoothing constraint and rigid constraint of the internal energy term work together to ensure that the closed evolution curve maintains the continuity and smoothness of the overall contour as it shrinks towards the edge of the pit, suppressing the fragmentation or sharp-cornered distortion of the curve caused by local noise interference.

[0111] The external energy term consists of two parts: an edge attraction component and a contraction driving component. The edge attraction component is constructed based on the gradient magnitude of the normalized residual value at the corresponding position in the local phase data block. A larger gradient magnitude results in lower edge attraction energy, causing the closure evolution curve to tend towards the pit edge position with the largest gradient magnitude. The contraction driving component provides a constant contraction force along the curve normal direction to the closure evolution curve, ensuring that the closure evolution curve continues to contract inward in a flat background region far from the pit edge. The location of the Euler-Lagrange force corresponding to the external energy term on the closure evolution curve is shown below. The location is represented as: The first item is marginal attraction. Represents the gradient operator. The gradient field of the normalized residual values ​​in the local phase data block. The spatial gradient points in the direction of increasing gradient magnitude, i.e., the edge of the pit, so it is positive to drive the curve towards the edge; the second term is the contraction driving force. This is the strength coefficient of the contraction driving force; a negative value indicates the direction of contraction. For closed evolution curves in arc length parameter The unit outward normal vector at that location; Indicates the action on any point on the curve. Total external driving force at the location. Contraction driving force intensity coefficient. The absolute value must be less than the typical amplitude of the edge attraction at the edge of the pit to ensure that the contraction driving force is balanced by the edge attraction after the closed evolution curve reaches the edge of the pit and will not continue to contract inward across the edge. For example, the contraction driving force intensity coefficient We set it to -0.2. In flat background regions, the gradient magnitude approaches zero, causing the edge attraction to become zero, and the contraction driving force dominates, driving the closed evolution curve to move continuously inward. When the closed evolution curve approaches the edge of the pit, the gradient magnitude increases sharply, making the edge attraction dominant. The closed evolution curve is attracted to the phase abrupt boundary at the edge of the pit, forming a dynamic equilibrium between the edge attraction and the contraction driving force. The external energy term directly utilizes the inherent gradient jump characteristics of the residual phase data to drive the curve evolution, avoiding the boundary bias that may be introduced by relying on manually set fixed thresholds for binarization segmentation.

[0112] Furthermore, the constructed energy functional is iteratively minimized, and the variational method is used to transform the minimization problem of the energy functional into a problem of updating the positions of discrete control points on the closed evolution curve. In each iteration, based on the direction of the resultant force of the internal and external energy gradients at the current position of the closed evolution curve, the positions of each discrete control point are slightly adjusted along the direction of the fastest energy decrease. Under the pull of external energy, the closed evolution curve gradually contracts inward from the initial rectangular ring position, while the constraint of internal energy ensures that the closed evolution curve maintains the smoothness of its overall shape during the contraction process. When the closed evolution curve reaches the edge of the pit, the edge attraction energy trap generated by the strong phase gradient at the pit edge position forms a dynamic balance with the contraction driving force, and the closed evolution curve is stably locked at the pit edge.

[0113] During the iteration process, the convergence criterion for determining whether the closed evolution curve has reached the energy convergence state is that the maximum value of the change in the position of all discrete control points on the closed evolution curve between adjacent iteration steps is less than a preset convergence threshold. For example, a preset convergence threshold Take 1 pixel. Simultaneously set a maximum iteration step limit to prevent the iteration process from failing to converge and falling into an infinite loop under special circumstances; for example, the maximum iteration step limit is set to 300 steps. When the maximum value of the total change in the position of all control points between adjacent iteration steps is less than a preset convergence threshold... The iteration process terminates when the number of iterations reaches the maximum limit.

[0114] Once the closed evolution curve stops evolving after satisfying the convergence criterion, all pixel positions corresponding to the internal region enclosed by the closed evolution curve are determined as the precise segmentation region of a single pit. A scan-line filling algorithm is used to determine whether each pixel in the local phase data block is located inside the converged closed evolution curve. The coordinate set of all pixels located inside the closed evolution curve is marked as the pit pixel set in the current local phase data block. The coordinates of the pit pixel set are transformed from the local coordinate system of the local phase data block back to the global coordinate system of the residual phase anomaly distribution map, completing the pixel-level precise segmentation of a single suspected pit region.

[0115] The dynamic contour evolution logic described above is executed one by one on the local phase data blocks corresponding to the coordinates of the two-dimensional bounding boxes of all suspected pits output in step 4. The pixel set of the energy convergence region in each local phase data block is summarized to form a complete identification result of all pit regions on the surface of the bearing steel ball. The complete identification result accurately describes the spatial position and contour shape of each pit defect in the sampling area on the steel ball surface in the form of a pixel set, and the identification accuracy reaches pixel-level resolution.

[0116] This step refines the coarse localization result of the rectangular bounding box output in step 4 into a pixel-level segmentation result that fits the true contour of the pit through dynamic contour evolution logic. The edge attraction component and the contraction driving component in the energy functional work together to drive the closed evolution curve to adaptively lock the pit boundary, and can achieve accurate boundary extraction for pit defects of different depths and shapes.

[0117] Example 2

[0118] Reference Figure 6 This is the second embodiment of the present application, which provides a machine vision-based system for recognizing surface pits on bearing steel balls.

[0119] The system includes: a stripe image acquisition module, a multi-frequency heterodyne phase resolution module, a residual phase extraction module, a suspected pit detection module, and a pit contour extraction module.

[0120] The stripe image acquisition module is used to project multi-frequency phase-shift sinusoidal stripes onto the surface of the bearing steel ball, acquire the deformation stripe image sequence reflected by the steel ball surface, and obtain a two-dimensional digital image matrix containing the surface topography modulation information of the steel ball.

[0121] The multi-frequency heterodyne phase unification module is used to perform multi-frequency heterodyne phase unification processing on the two-dimensional digital image matrix, unfolding the relative phase data wrapped in a fixed period into a continuously distributed absolute phase, and constructing an absolute phase spatial distribution matrix that characterizes the geometric morphology of the steel ball surface.

[0122] The residual phase extraction module is used to introduce a reference phase model that characterizes the basic phase components of the macroscopic spherical surface of the steel ball, perform a difference comparison operation between the absolute phase spatial distribution matrix and the reference phase model, strip away the basic phase components of the macroscopic spherical surface, and extract a residual phase anomaly distribution map that contains only local morphological change information.

[0123] The suspected pit detection module is used to calculate the phase gradient feature information of the residual phase anomaly distribution map, input the residual phase anomaly distribution map into a convolutional neural network with an adaptive attention mechanism, and use the phase gradient feature information as the guiding signal of the adaptive attention mechanism. It uses the phase gradient direction to dynamically adjust the convolution kernel offset to extract anomaly features and outputs the coordinates of the two-dimensional bounding box of the suspected pit.

[0124] The pit contour extraction module is used to extract the local phase data block corresponding to the coordinates of the two-dimensional bounding box of the suspected pit in the residual phase anomaly distribution map, and to perform energy functional minimization iteration on the local phase data block using dynamic contour evolution logic, and extract the pixel set of the energy convergence region as the final pit region.

[0125] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0126] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of this application without departing from the spirit and scope of protection of the claims. All of these variations are within the protection scope of this application.

Claims

1. A method for identifying surface pits on bearing steel balls based on machine vision, characterized in that, include: Multi-frequency phase-shifted sinusoidal fringes are projected onto the surface of the bearing steel ball, and a sequence of deformation fringe images reflected from the surface of the steel ball is acquired to obtain a two-dimensional digital image matrix containing the surface topography modulation information of the steel ball. Multi-frequency heterodyne phase deconvolution processing is performed on the two-dimensional digital image matrix to expand the relative phase data wrapped in a fixed period into a continuously distributed absolute phase, and construct an absolute phase spatial distribution matrix that characterizes the geometric morphology of the steel ball surface. A reference phase model characterizing the macroscopic spherical basic phase components of a steel ball is introduced. The absolute phase spatial distribution matrix is ​​compared with the reference phase model by difference. The macroscopic spherical basic phase components are stripped away, and the residual phase anomaly distribution map containing only local morphological change information is extracted. The phase gradient feature information of the residual phase anomaly distribution map is calculated. The residual phase anomaly distribution map is input into a convolutional neural network with an adaptive attention mechanism. The phase gradient feature information is used as the guiding signal of the adaptive attention mechanism. The phase gradient direction is used to dynamically adjust the offset of the convolutional kernel to extract the anomaly features and output the coordinates of the two-dimensional bounding box of the suspected pit. The phase gradient feature information includes phase gradient magnitude information and phase gradient direction information; The convolutional neural network with embedded adaptive attention mechanism includes: a feature extraction backbone unit, a multi-scale feature fusion unit, and a detection head unit connected in sequence; The feature extraction backbone unit uses the residual phase anomaly distribution map as the main input, and the phase gradient direction information as the auxiliary guiding input of the adaptive attention mechanism embedded in the feature extraction backbone unit; the feature extraction backbone unit is a residual neural network backbone unit, which is composed of multiple cascaded residual convolutional units, each of which contains a convolutional layer, a normalization layer and a skip connection path. The multi-scale feature fusion unit receives feature maps output from different depth levels in the feature extraction backbone unit and performs cross-scale feature aggregation through upsampling and channel concatenation operations. The detection head unit outputs the coordinates of the two-dimensional bounding box of the suspected pit and the corresponding confidence score based on the aggregated multi-scale features. The embedded adaptive attention mechanism includes: embedding an attention sub-unit in the residual convolution unit of the feature extraction backbone unit, wherein the attention sub-unit uses the phase gradient feature information at the corresponding position of the current feature map as a guiding signal to dynamically generate spatial offset parameters at each sampling position; The spatial position of the regular sampling grid of the convolution kernel is adjusted according to the spatial offset parameter, so that the receptive field of the convolution kernel is adaptively expanded along the direction of increasing phase gradient magnitude, covering local abnormal features distributed along the edge of abrupt morphological changes. In the residual phase anomaly distribution map, the local phase data block corresponding to the coordinates of the two-dimensional bounding box of the suspected pit is extracted. The local phase data block is subjected to energy functional minimization iteration using dynamic contour evolution logic, and the pixel set of the energy convergence region is extracted as the final pit region.

2. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 1, characterized in that, The projection of multi-frequency phase-shifted sinusoidal fringes onto the surface of the bearing steel balls includes: Multiple sets of sinusoidal fringe patterns with different spatial frequencies are pre-configured. The sinusoidal fringe patterns at each spatial frequency are sequentially phase-shifted at equal intervals according to a preset number of phase shift steps to generate a multi-frequency phase-shifted sinusoidal fringe sequence covering multiple frequencies and multiple phase shifts. The multi-frequency phase-shifted sinusoidal fringe sequence is projected frame by frame onto the surface of the bearing steel ball according to a preset time sequence, so that each frame of fringe pattern is modulated by the steel ball surface to form deformation stripes carrying surface morphology information.

3. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 2, characterized in that, The acquisition of the deformation stripe image sequence reflected from the surface of the steel ball includes: The camera is driven to acquire frame-by-frame images of the deformation stripes reflected from the surface of the steel ball according to the trigger signal synchronized with the projection timing of the projection device. The gray value distribution of each frame of the acquired deformation stripe image is processed by gray value quantization, and the gray value matrix is ​​converted into a discrete gray value matrix. The gray value matrix is ​​grouped and arranged according to frequency group and phase shift number to construct the two-dimensional digital image matrix.

4. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 1, characterized in that, The multi-frequency heterodyne phase deconvolution processing of the two-dimensional digital image matrix includes: For each set of spatial frequency corresponding to the phase-shifted fringe image, a phase-shift demodulation algorithm is used to extract the wrapped phase distribution at each frequency from the gray values ​​of the multi-step phase-shifted image, resulting in multiple wrapped phase maps that correspond one-to-one with each spatial frequency. The multiple wrapped phase maps are then subjected to step-by-step heterodyne operation, and the equivalent wavelength is gradually expanded by the phase difference between adjacent frequencies, so that the high-frequency wrapped phase is gradually expanded to the low-frequency range, eliminating the periodic phase cutoff jump.

5. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 4, characterized in that, The construction of the absolute phase space distribution matrix characterizing the geometric features of the steel ball surface includes: The unfolded phase obtained by the successive heterodyne operation is used as the guiding phase. The wrapping phase at the highest spatial frequency is recovered by integer number of cycles to obtain continuously distributed absolute phase values. The absolute phase values ​​are mapped and arranged according to the pixel spatial coordinate position to form a two-dimensional absolute phase spatial distribution matrix corresponding to each pixel of the sampling area on the steel ball surface. The value of each element in the two-dimensional absolute phase spatial distribution matrix represents the morphological feature information at the corresponding surface position.

6. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 5, characterized in that, The construction of the absolute phase space distribution matrix characterizing the geometric features of the steel ball surface includes: After forming a two-dimensional absolute phase spatial distribution matrix corresponding to each pixel of the sampling area on the steel ball surface, the absolute phase spatial distribution matrix is ​​preprocessed by spatial filtering. An adaptive smoothing filter is used to suppress the high-frequency phase jitter components introduced by the acquisition noise in the matrix, while retaining the mid-to-low frequency phase change characteristics corresponding to the surface morphology. The effective region mask is extracted from the filtered absolute phase spatial distribution matrix. Invalid phase data outside the effective reflection area of ​​the steel ball surface are removed according to the modulation index criterion to obtain the absolute phase spatial distribution matrix that only covers the effective measurement area of ​​the steel ball.

7. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 1, characterized in that, The reference phase model introduced to characterize the fundamental phase components of the macroscopic spherical surface of the steel ball includes: Using the phase data located within the effective measurement area of ​​the steel ball in the absolute phase spatial distribution matrix as fitting samples, the least squares bivariate polynomial surface fitting algorithm is used to perform overall fitting operations on the fitting samples to determine the polynomial coefficients of the reference phase model. Based on the polynomial coefficients, a reference phase model of the same size as the absolute phase space distribution matrix is ​​generated pixel by pixel according to the bivariate polynomial surface equation.

8. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 7, characterized in that, The process of stripping the macroscopic spherical fundamental phase components and extracting a residual phase anomaly distribution map containing only local morphological abrupt changes includes: The absolute phase spatial distribution matrix and the reference phase model are subjected to element-wise difference operations at corresponding pixel positions to obtain the residual phase distribution after stripping the macroscopic spherical basic phase components; the residual phase distribution is subjected to numerical normalization mapping to uniformly map the residual phase values ​​to a preset numerical range to generate the residual phase anomaly distribution map, and the local phase change region in the residual phase anomaly distribution map corresponds to the candidate region of morphological defects on the steel ball surface.

9. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 8, characterized in that, The extracted residual phase anomaly distribution map, which contains only information on local morphological abrupt changes, includes: After generating the residual phase anomaly distribution map, morphological preprocessing is performed on the residual phase anomaly distribution map. Isolated noise disturbance regions with an area smaller than a preset threshold are identified and removed through connected component analysis, while retaining local phase anomaly regions with spatial continuity. The residual phase anomaly distribution map after morphological preprocessing is used as input data for calculating phase gradient feature information.

10. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 1, characterized in that, The output of the coordinates of the two-dimensional bounding box of the suspected pit includes: The two-dimensional bounding boxes of suspected pits output by the detection head unit are post-processed and filtered. A non-maximum suppression strategy is used to redundantly remove candidate bounding boxes with overlapping spatial positions. The bounding box with the highest confidence score in each group of overlapping bounding boxes is retained as the localization result of each suspected pit region. The coordinates of the filtered two-dimensional bounding boxes are expanded outward according to a preset expansion ratio to expand the truncation range to ensure that the subsequent truncation of local phase data blocks completely covers the pit and its surrounding edge transition region.

11. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 1, characterized in that, The method of using dynamic contour evolution logic to perform energy functional minimization iteration on local phase data blocks includes: A closed evolution curve is initialized in the local phase data block, and an energy functional is constructed that includes an external energy term driven by the phase gradient features and an internal energy term constrained by the curve's own geometry. The energy functional is iteratively minimized to drive the closed evolution curve to gradually evolve along the phase gradient direction under the traction of the external energy term until the curve converges to the phase abrupt boundary of the pit region.

12. The method for identifying surface pits of bearing steel balls based on machine vision according to claim 11, characterized in that, The set of pixels in the energy convergence region is used as the final pit region, including: The convergence criterion for determining that the closed evolution curve has reached the energy convergence state is that the change in curve position between adjacent iteration steps is less than a preset convergence threshold. When the convergence criterion is met, the set of pixels corresponding to the internal region enclosed by the closed evolution curve is determined as the precise segmentation region of a single pit. The dynamic contour evolution logic is executed on the local phase data blocks corresponding to the coordinates of the two-dimensional bounding boxes of all suspected pits. The pixel set of the energy convergence region in each local phase data block is summarized to form a complete recognition result of all pit areas on the surface of the bearing steel ball.

13. A machine vision-based bearing steel ball surface pit recognition system, used to implement the machine vision-based bearing steel ball surface pit recognition method according to any one of claims 1 to 12, characterized in that, include: The module includes a stripe image acquisition module, a multi-frequency heterodyne phase deconvolution module, a residual phase extraction module, a suspected pit detection module, and a pit contour extraction module. The stripe image acquisition module is used to project multi-frequency phase-shift sinusoidal stripes onto the surface of the bearing steel ball, acquire the deformation stripe image sequence reflected by the steel ball surface, and obtain a two-dimensional digital image matrix containing the surface topography modulation information of the steel ball. The multi-frequency heterodyne phase unification module is used to perform multi-frequency heterodyne phase unification processing on the two-dimensional digital image matrix, expand the relative phase data wrapped in a fixed period into a continuously distributed absolute phase, and construct an absolute phase spatial distribution matrix that characterizes the geometric morphology of the steel ball surface. The residual phase extraction module is used to introduce a reference phase model that characterizes the basic phase components of the macroscopic spherical surface of the steel ball, perform a difference comparison operation between the absolute phase spatial distribution matrix and the reference phase model, strip away the basic phase components of the macroscopic spherical surface, and extract a residual phase anomaly distribution map that contains only local morphological change information. The suspected pit detection module is used to calculate the phase gradient feature information of the residual phase anomaly distribution map, input the residual phase anomaly distribution map into a convolutional neural network with an adaptive attention mechanism, and use the phase gradient feature information as the guiding signal of the adaptive attention mechanism. It uses the phase gradient direction to dynamically adjust the convolution kernel offset to extract anomaly features and outputs the coordinates of the two-dimensional bounding box of the suspected pit. The pit contour extraction module is used to extract the local phase data block corresponding to the coordinates of the two-dimensional bounding box of the suspected pit in the residual phase anomaly distribution map, and to perform energy functional minimization iteration on the local phase data block using dynamic contour evolution logic, and extract the pixel set of the energy convergence region as the final pit region.

Images