A method and system for detecting wear of a transfer case component based on a semantic segmentation network

Abstract

The present disclosure relates to the technical field of intersection of computer vision and overhaul of transfer equipment, and discloses a transfer component wear detection method and system based on a semantic segmentation network, which comprises the following steps: collecting a depth image and a structured light projection image of a component at a standardized shooting station; inputting the depth image into a pre-trained semantic segmentation network to output a wear area probability map and generate a wear area mask; performing phase decoding on the structured light projection image, reconstructing a three-dimensional point cloud and a continuous surface of the wear area based on the principle of triangulation; after registering the reconstructed surface with a reference surface, calculating the signed distance point by point to obtain the maximum wear depth, the average wear depth, the wear area and the wear volume; outputting the wear grade according to a preset grading standard; and entering the monitoring data into a wear database, fitting the wear trend based on a double exponential model and predicting the remaining life.

Description

Technical Field

[0001] This disclosure relates to the interdisciplinary field of turnaround equipment maintenance and computer vision, specifically to a method and system for detecting wear of turnaround components based on semantic segmentation networks. Background Technology

[0002] Rotating equipment (such as fans, coal mills, circulating pumps, and motors) is critical equipment in thermal power plants, and its key components inevitably experience wear during long-term operation. Taking fans as an example, common wear areas include bearing raceways, impeller blades, journal surfaces, and gear teeth. The degree of wear on these components directly affects the operational safety of the rotating equipment and the timing of maintenance.

[0003] Mainstream methods for detecting wear on rotating parts each have their limitations. Traditional maintenance often combines visual inspection with simple measuring tools such as vernier calipers and feeler gauges. This method relies heavily on the subjective experience of maintenance personnel, resulting in significant discrepancies in judgment among personnel. The degree of wear can only be described using vague qualitative terms, making it difficult to achieve standardization and quantification. Furthermore, it cannot be linked to historical inspection data for comparison and analysis of wear rate. Two-dimensional image detection solutions that rely on industrial cameras can identify wear areas using image processing algorithms, but they can only obtain surface projection information, lacking key data such as wear depth and three-dimensional morphology. This makes it impossible to accurately calculate the wear volume, and the shooting angle and ambient lighting can easily interfere with the imaging effect, resulting in poor comparability of multiple measurement results. The most accurate three-dimensional scanning detection relies on scanning equipment to collect surface point clouds and compare them with the original model to calculate the amount of wear. However, due to the high cost of equipment, high operating threshold, and long scanning time, it is difficult to implement on a large scale in maintenance sites. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this disclosure is to provide a method and system for detecting wear on rotating parts based on semantic segmentation networks. By standardizing image acquisition, automatically identifying wear areas using semantic segmentation networks, reconstructing three-dimensional morphology using structured light, and quantifying wear, the method achieves objective quantitative detection of wear on rotating parts, eliminating the subjectivity of manual visual inspection and improving detection efficiency and accuracy.

[0005] To achieve the above objectives, the present disclosure adopts the following technical solution: In a first aspect, this disclosure provides a method for detecting wear of rotating parts based on semantic segmentation networks, including: Acquire depth images and structured light projection images of rotating components under standardized shooting conditions; The depth image is input into a pre-trained semantic segmentation network, which classifies each pixel in the depth image and outputs a wear region probability map. A wear region mask is then generated based on the wear region probability map. Phase decoding is performed on the structured light projection image, and a three-dimensional point cloud of the area corresponding to the wear area mask is reconstructed based on the triangulation principle. The three-dimensional surface of the wear area is then reconstructed based on the three-dimensional point cloud. Obtain the reference surface of the component, register the three-dimensional surface of the wear area with the reference surface, calculate the signed distance from the three-dimensional surface of the wear area to the reference surface point by point, and calculate the wear quantification index based on the signed distance. The wear quantification index includes the maximum wear depth, average wear depth, wear area and wear volume. The wear level is output based on the wear quantification index and the preset grading standard.

[0006] Preferably, the method further includes: after outputting the wear level, recording the wear quantification index and the corresponding detection time into a wear database, fitting the wear trend based on historical detection data of the same component in the wear database, and predicting the remaining life of the component based on the fitting result. The above technical solution, by storing historical detection data in a database and fitting wear trends, upgrades maintenance from "retroactive maintenance" or "periodic maintenance" to "condition-based predictive maintenance," enabling advance prediction of component replacement timing.

[0007] Preferably, acquiring the depth image and structured light projection image of the rotating parts under standardized shooting conditions includes: simultaneously acquiring the depth image, grayscale image, and structured light projection image modulated by the surface of the parts at a standardized shooting station. The standardized shooting station has fixed lighting conditions, a fixed camera shooting distance, and a fixed camera shooting angle. This technical solution, by setting standardized shooting conditions, ensures the comparability of image data from different inspections and eliminates measurement errors introduced by differences in shooting conditions.

[0008] Preferably, the semantic segmentation network adopts an encoder-decoder structure. The encoder is used to extract multi-scale feature maps of the depth image step by step, and the decoder is used to restore the multi-scale feature maps to the resolution of the depth image step by step. The encoder and decoder pass feature maps of corresponding scales through skip connections. In the above technical solution, the encoder-decoder structure combined with skip connections can accurately locate the boundary of the wear area while maintaining high segmentation accuracy, solving the problem of blurred edges in the wear area making accurate segmentation difficult.

[0009] Preferably, generating a wear region mask based on the wear region probability map includes: setting a probability threshold, determining pixels with probability values ​​greater than or equal to the probability threshold in the wear region probability map as worn pixels, determining pixels with probability values ​​less than the probability threshold as non-worn pixels, and the set of all worn pixels constitutes the wear region mask.

[0010] Preferably, the step of performing phase decoding on the structured light projection image and reconstructing the three-dimensional point cloud of the area corresponding to the wear area mask based on the triangulation principle includes: performing phase-shift decoding on the structured light projection image to obtain the wrapping phase; using the multi-frequency heterodyne method to perform phase unwrapping on the wrapping phase to obtain the absolute phase; and calculating the spatial three-dimensional coordinates corresponding to each pixel in the wear area mask based on the triangulation principle according to the absolute phase and camera calibration parameters, with the set of all spatial three-dimensional coordinates constituting the three-dimensional point cloud.

[0011] Preferably, obtaining the reference surface of the component includes: obtaining the original model of the component, and extracting the theoretical surface corresponding to the wear area mask from the original model as the reference surface; or, when the original model cannot be obtained, if there is an unworn area within a preset width range around the wear area mask, the reference surface is obtained by fitting the surface of the unworn area and extrapolating it to the wear area; if there is no unworn area within a preset width range around the wear area mask, the historical 3D scan data of the component or the 3D data of an unworn component of the same model is obtained as the reference surface. The above technical solution provides three methods for obtaining the reference surface. When there is a model of the component, the theoretical surface is directly extracted, which has the highest accuracy; when there is no model, the reference surface is obtained by extrapolating from the surrounding unworn area, ensuring the versatility of the method.

[0012] Preferably, the registration of the three-dimensional surface of the wear region with the reference surface includes: rigidly registering the point cloud of the three-dimensional surface of the wear region with the point cloud of the reference surface using an iterative nearest-neighbor (ICP) registration algorithm; solving for the rotation matrix and translation vector that minimizes the distance between the two sets of point clouds; and transforming the three-dimensional surface of the wear region to align with the reference surface based on the rotation matrix and translation vector. The above technical solution achieves precise alignment between the three-dimensional surface of the wear region and the reference surface through the iterative nearest-neighbor (ICP) registration algorithm, eliminating the influence of component placement pose differences on the measurement results.

[0013] Preferably, the step of fitting wear trend data of the same component in the wear database and predicting the remaining life of the component based on the fitting result includes: extracting the maximum wear depth and corresponding cumulative operating hours of the same component in each test from the wear database; fitting the curve of the maximum wear depth as a function of cumulative operating hours using a double exponential model; calculating the cumulative operating hours required for the component to reach the scrap threshold based on the fitted wear trend model and a preset scrap threshold; and obtaining the remaining life based on the difference between the cumulative operating hours and the current cumulative operating hours.

[0014] The above technical solution adopts a double-exponential model consisting of a constant term and two exponential terms, which respectively describe the initial surface roughness of the machined surface of the parts, the rapid wear during the break-in period, and the accelerated wear after the stable wear period. The three are superimposed to fully reflect the wear evolution law of the parts from the initial break-in, stable wear to accelerated failure. Compared with linear fitting or single-exponential fitting, it is closer to the actual wear physical process and has higher prediction accuracy.

[0015] Secondly, this disclosure provides a wear detection system for rotating parts based on a semantic segmentation network, comprising: An image acquisition module is used to acquire depth images and structured light projection images of rotating parts under standardized shooting conditions. The image acquisition module includes an industrial camera, a structured light projector, and a standardized shooting station. The standardized shooting station is equipped with a fixed light source, a positioning bracket with a fixed shooting distance, and a positioning fixture for fixing the parts. The relative position between the industrial camera and the structured light projector is fixed. The wear region segmentation module is used to input the depth image into a pre-trained semantic segmentation network, classify each pixel in the depth image through the semantic segmentation network, output a wear region probability map, and generate a wear region mask based on the wear region probability map. The three-dimensional topography reconstruction module is used to perform phase decoding on the structured light projection image, reconstruct the three-dimensional point cloud of the area corresponding to the wear area mask based on the triangulation principle, and reconstruct the three-dimensional surface of the wear area based on the three-dimensional point cloud. The wear quantification calculation module is used to obtain the reference surface of the component, register the three-dimensional surface of the wear area with the reference surface, calculate the signed distance from the three-dimensional surface of the wear area to the reference surface point by point, calculate the wear quantification index based on the signed distance, the wear quantification index includes the maximum wear depth, average wear depth, wear area and wear volume, and output the wear level according to the wear quantification index and the preset grading standard. The trend prediction module is used to input the wear quantification index and the corresponding detection time into the wear database after the wear quantification calculation module outputs the wear level, perform wear trend fitting based on the historical detection data of the same component in the wear database, and predict the remaining life of the component based on the fitting result.

[0016] Compared with the prior art, the beneficial effects achieved by this disclosure are as follows: This disclosure establishes a standardized shooting station to simultaneously acquire depth images and structured light projection images under fixed lighting, distance, and angle. This ensures the comparability of image data from different tests, eliminates measurement errors caused by differences in shooting conditions, and lays a data foundation for achieving accurate quantitative detection.

[0017] This disclosure classifies each pixel in a depth image using a semantic segmentation network, automatically identifies the precise boundaries of wear areas and generates wear area masks, eliminating the subjective differences of manual visual inspection and achieving automated, objective, and pixel-level accuracy in wear area identification.

[0018] This disclosure extends the detection dimension from a two-dimensional plane to a three-dimensional space by performing phase decoding on structured light projection images and reconstructing the three-dimensional surface of the wear area based on the principle of triangulation. This enables the accurate acquisition of wear depth information and three-dimensional morphological features, thereby supporting the calculation of key three-dimensional quantitative indicators such as wear volume.

[0019] This disclosure accurately calculates multi-dimensional quantitative indicators such as maximum wear depth, average wear depth, wear area, and wear volume by performing ICP registration between the reconstructed three-dimensional surface of the wear area and the reference surface and calculating the signed distance point by point. Compared with a single indicator, it can more comprehensively reflect the wear status of the parts and provide accurate multi-dimensional data support for maintenance timing.

[0020] This disclosure, by recording historical test data into a wear database and using a dual-exponential model to fit the wear trend, can predict the remaining life of components, upgrading traditional "post-inspection" or "periodic inspection" to "condition-based predictive maintenance," thus avoiding unplanned downtime caused by sudden component failure. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the wear detection method for rotating components in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the module connection of the wear detection system for rotating components in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the encoder-decoder structure of the semantic segmentation network in an embodiment of this disclosure; Figure 4 This is a schematic diagram illustrating the process of three-dimensional surface reconstruction of the wear area in an embodiment of this disclosure. Detailed Implementation

[0022] The technical solution of this disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this disclosure and the specific features in the embodiments are detailed descriptions of the technical solution of this disclosure, rather than limitations thereof.

[0023] Example 1: like Figure 1 As shown, this embodiment provides a method for detecting wear of rotating components based on semantic segmentation networks, taking the wear detection of the inner ring raceway of a wind turbine bearing as an example. The method includes the following steps: In this embodiment, the standardized shooting station is fixed in place. The standardization of the shooting station is based on calibration experiments conducted at different shooting angles and distances to determine the optimal range of shooting parameters. Specifically, the angle between the camera's optical axis and the normal direction of the surface to be measured is controlled within 5° to reduce the impact of perspective distortion on the accuracy of the depth image. The working distance from the camera lens to the surface of the component is determined based on the size of the component and the depth-of-field range of the depth camera, ensuring that the surface to be measured is within the camera's optimal imaging depth-of-field range. The ring LED light source uses standard white light, and a diffuser plate ensures that the illumination light is evenly distributed on the surface of the component, avoiding phase calculation errors in the structured light projection image caused by local reflections or shadows. The bearing inner ring is fixed in the station using a dedicated V-shaped positioning fixture to ensure that the axial position and circumferential orientation of the bearing inner ring are consistent each time it is placed.

[0024] Specifically, depth image acquisition employs an active infrared binocular structured light camera. This camera has a built-in infrared projector that projects an invisible speckle pattern. The speckle images are acquired by left and right infrared cameras, and the parallax is calculated to output the depth image. ,in Image pixel coordinates, The value is the depth from the spatial point corresponding to the pixel to the camera's optical center along the optical axis, in millimeters. Grayscale images are acquired simultaneously within the same field of view. .

[0025] Specifically, the structured light projection image acquisition method is as follows: a three-step phase-shift sinusoidal fringe pattern is projected onto the raceway surface by an independent structured light projector, and three phase-shift images modulated by the raceway surface are simultaneously acquired by a high-resolution industrial camera. .

[0026] According to an exemplary embodiment, three different frequency three-step phase-shift sinusoidal fringe patterns can also be sequentially projected onto the raceway surface by an independent structured light projector. The number of fringe periods corresponding to the three frequency fringe patterns are different within the field of view. For each frequency fringe pattern, three phase-shift images modulated by the raceway surface are sequentially acquired by a high-resolution industrial camera, that is, a total of nine phase-shift images are acquired for the three frequencies.

[0027] The structured light projector uses a digital light processing (DLP) projection module. The relative positions of the structured light projector and the industrial camera are fixed, and they are rigidly connected to the same base plate via a bracket. Depth images and structured light projection images are acquired using time-division multiplexing, meaning acquisition is completed in two time sequences at the same shooting position: In the first sequence, the infrared projector of the depth camera is turned on and the structured light projector is turned off, allowing the depth camera to acquire depth and grayscale images; in the second sequence, the infrared projector of the depth camera is turned off and the structured light projector is turned on, allowing the industrial camera to acquire structured light projection images modulated by the component surface. The interval between the two time sequences is controlled within 50ms to reduce component pose changes between the two images caused by environmental vibrations or operational disturbances. Since the components are stationary at the shooting position in the inspection base, the pose deviation introduced by time-division acquisition is negligible. Before the inspection begins, the industrial camera and structured light projector need to be jointly calibrated. The calibration method uses Zhang Zhengyou's planar calibration method, using a checkerboard calibration board to acquire images at different poses, and calibrating to obtain the intrinsic parameter matrix of the industrial camera. Distortion coefficient and the intrinsic parameter matrix of the structured light projector And the rotation matrix between the industrial camera and the structured light projector Translation vector .

[0028] According to an exemplary embodiment, the acquisition process of depth images, grayscale images, and structured light projection images, completed sequentially under the same standardized shooting position and within the same time window (the interval between two time sequences does not exceed 50ms), is collectively referred to as synchronous acquisition. That is, synchronous acquisition in the embodiments of this disclosure refers to an acquisition method where the acquired depth images, grayscale images, and structured light projection images correspond to the same clamping pose of the component, and the acquisition time interval is sufficiently small so that the influence of component pose changes on the measurement results is negligible. Specific implementation forms include, but are not limited to, the aforementioned time-division multiplexing acquisition, and other acquisition methods that can ensure the consistency of the poses of the three types of images can also be used.

[0029] According to an exemplary implementation, since the wear area mask is generated based on the depth image, while the structured light projection image used for 3D reconstruction is acquired by the industrial camera, and their image coordinate systems are different, the wear area mask needs to be mapped to the pixel coordinate system of the structured light projection image before phase decoding. Therefore, dual-target calibration of the depth camera and industrial camera is required before detection begins. Specifically, the checkerboard calibration board is fixed in a standardized shooting position, and the depth camera and industrial camera are controlled to acquire images of the same calibration board in different poses, extracting the pixel coordinates of the same checkerboard corner points in both images. Using the depth camera coordinate system as the reference coordinate system, the Zhang Zhengyou planar calibration method is used to solve for the intrinsic parameter matrix and distortion coefficients of the depth camera and the industrial camera, respectively, and the rotation matrix and translation vector from the depth camera coordinate system to the industrial camera coordinate system are also solved.

[0030] According to an exemplary embodiment, after completing dual-target calibration, the mapping method of the wear region mask to the structured light projection image coordinate system is as follows: For any pixel in the wear region mask, the depth value corresponding to the pixel is read from the depth image; the pixel coordinates and depth value of the pixel are back-projected to the depth camera coordinate system according to the intrinsic parameter matrix and distortion coefficient of the depth camera to obtain the first spatial point coordinates corresponding to the pixel; the first spatial point coordinates are transformed to the industrial camera coordinate system according to the rotation matrix and translation vector from the depth camera coordinate system to the industrial camera coordinate system to obtain the second spatial point coordinates; the second spatial point coordinates are projected to the pixel coordinate system of the industrial camera according to the intrinsic parameter matrix and distortion coefficient of the industrial camera to obtain the corresponding pixel coordinates of the pixel in the structured light projection image. The above mapping is performed on all pixels in the wear region mask to obtain the corresponding region of the wear region mask in the structured light projection image coordinate system; if the corresponding pixel coordinates of a certain pixel exceed the pixel range of the structured light projection image, the pixel is removed from the corresponding region. Subsequent phase decoding and 3D reconstruction are performed only on pixels belonging to the corresponding region in the structured light projection image, thereby ensuring that the reconstructed 3D point cloud strictly corresponds to the wear region identified by the wear region mask.

[0031] The standardized shooting conditions set above ensure that the image data from different tests are comparable, eliminating measurement errors caused by changes in factors such as shooting angle, light intensity, and working distance.

[0032] In this embodiment, the depth image As the primary input, grayscale image As auxiliary inputs, the two are superimposed along the channel dimension (forming a dual-channel input tensor) and then input into a pre-trained semantic segmentation network, which outputs a probability map of the wear region. ,in Represents pixels The probability of belonging to a wear zone.

[0033] Specifically, the semantic segmentation network adopts an improved U-Net structure, such as... Figure 3 As shown. The specific structure of this network is as follows: The encoder consists of five downsampling blocks. The first downsampling block comprises two 3×3 convolutional layers (64 kernels, stride 1, padding 1) and a 2×2 max-pooling layer (stride 2), followed by a batch normalization layer and a ReLU activation function after each convolutional layer. The subsequent four downsampling blocks have the same structure, but the number of convolutional kernels increases progressively (128, 256, 512, and 1024 respectively), and each downsampling block also contains two 3×3 convolutional layers and a 2×2 max-pooling layer. By progressively downsampling, the encoder halves the spatial resolution of the input image step by step (from the original resolution to 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32), while the number of channels increases progressively, to extract multi-scale feature maps from shallow texture features to deep semantic features.

[0034] The decoder section contains five upsampling blocks. The first upsampling block (corresponding to the output of the encoder's 5th level) consists of a 2×2 transposed convolutional layer (used to double the spatial size of the feature map and halve the number of channels) and two 3×3 convolutional layers (with the same number of kernels as the corresponding number of channels in the current level encoder). Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. The subsequent four upsampling blocks have the same structure, gradually restoring the feature map to the original image resolution.

[0035] In this embodiment, the last layer of the network uses a 1×1 convolutional layer (mapping the number of channels in the feature map to 1) plus a sigmoid activation function to output the probability value of each pixel belonging to the wear region, i.e., the wear region probability map. The size of the probability map is the same as that of the input image.

[0036] Specifically, the total loss function used during network training is a weighted sum of the Dice loss and the binary cross-entropy loss: , in, Let this be the total loss function during the network training phase. For Dice's loss, For binary cross-entropy loss, For weighting coefficients. According to one exemplary implementation, The value is 0.5.

[0037] Dice loss is defined as: , in, These are the coordinates of pixels in the image. To iterate through and sum all pixels in the image, For pixels in the probability map of wear areas predicted by the network The corresponding probability predicted by the network. A truth mask labeled manually. This indicates that the pixel belongs to a worn area. This indicates that the pixel belongs to a non-worn area.

[0038] The binary cross-entropy loss is defined as: , Specifically, the training dataset consists of 5000 sets of depth images of rotating parts and corresponding pixel-level wear annotations. Annotations were completed pixel-by-pixel by domain experts using specialized annotation tools. The annotation specifications followed these principles: the criteria for determining wear areas were areas of material loss on the surface of the parts due to friction, fatigue, or corrosion, where the surface morphology was visually distinguishable from the unworn areas; for areas with blurred wear boundaries, the intersection of annotations by three or more annotators was taken as the final annotation ground truth to ensure consistency and accuracy; annotation software supporting both pen and lasso annotation modes was used, allowing annotators to flexibly choose the annotation mode based on the shape of the wear area; after annotation, morphological closing operations were performed on the annotation results to fill any small voids within the annotated areas, making the edges of the annotation mask smoother and more closely matching the actual wear area boundaries. The training, validation, and test sets were divided in an 8:1:1 ratio. The network training used the Adam optimizer with an initial learning rate of 1×10⁻⁶. -4 The batch size is 8, the training rounds are 200, and training is stopped early when the validation set loss does not decrease for 10 consecutive rounds.

[0039] Furthermore, a probability threshold is set. The probability map of the wear area Binarization: The pixel is identified as a worn pixel (with a value of 1). Pixels with a value of 0 are considered non-worn pixels. The set of all pixels with a value of 1 constitutes the worn area mask. ,Right now:

[0040] In component detection scenarios without an original model, the same segmentation method described above can also be used. The semantic segmentation network has strong generalization ability and can effectively segment the wear areas of different types of components.

[0041] In this embodiment, phase decoding and phase unwrapping are performed on three phase-shifted images, and the 3D point cloud of the wear area is reconstructed by combining the camera calibration parameters, thereby reconstructing the 3D surface. Specifically, phase decoding and 3D reconstruction are performed only on pixels in the structured light projection image that belong to the region corresponding to the wear area mask.

[0042] Specifically, the grayscale value expressions for the three phase-shifted images are as follows: , in, For the first Amplitude phase-shifted image at pixel points grayscale value at that location Background light intensity, For modulation amplitude, The phase of the enclosed space to be solved. The number of phase shift steps. For the first The known phase shift of the first image relative to the second image.

[0043] The wrapped phase can be solved from the three phase-shifted images:

[0044] Specifically, due to the wrapping phase Cut off at Within the interval, there exists Phase blurring necessitates phase unwrapping. This embodiment employs a multi-frequency heterodyne method for phase unwrapping, the principle of which is as follows: projecting three different frequency stripe patterns (frequency values ​​are respectively...) , , (i.e., the number of fringe periods within the field of view), calculate the wrap phase at three frequencies respectively. According to an exemplary embodiment, the stripe patterns of the three frequencies are projected sequentially by the same structured light projector, with three phase-shifted images projected for each frequency, i.e., the aforementioned nine phase-shifted images; the number of stripe periods corresponding to the three frequencies are coprime, so as to maximize the number of equivalent stripe periods obtained by heterodyne, thereby expanding the reliable range of phase unfolding and ensuring the uniqueness of absolute phase.

[0045] The fringe order is determined by the difference in the wrapping phase at different frequencies. First, the equivalent wavelength is calculated. The absolute phase is obtained through stepwise extrapolation. In actual implementation, when there are oil stains, rust, or highly reflective areas on the surface of components, the signal-to-noise ratio of local areas in the structured light projection image may decrease, leading to phase calculation failure or phase unfolding errors in those areas. To address this, this embodiment adds a confidence level determination mechanism to the phase unfolding step: for each pixel, the normalized value of its modulation amplitude is calculated. When this value is lower than a preset threshold, the pixel is determined to be a low-confidence pixel and is removed in subsequent 3D reconstruction to avoid 3D coordinate noise introduced by phase calculation errors. Simultaneously, for phase jump points occurring during phase unfolding, a median filtering algorithm is used to smooth the absolute phase map, eliminating isolated phase anomalies caused by noise and ensuring the continuity of the 3D reconstruction results.

[0046] absolute phase The correspondence with three-dimensional spatial coordinates is established using the principle of triangulation. For the wear area mask... Each pixel in Its corresponding three-dimensional spatial coordinates Satisfying the projection equation of an industrial camera: , in, This represents the depth value of a 3D point in the camera coordinate system. This is the intrinsic parameter matrix of the industrial camera.

[0047] At the same time, according to the calibration parameters of the structured light projector, the same spatial point Pixel coordinates on the projector image plane with absolute phase The following conditions must be met: ,in The fringe spacing (in pixels) is used. Combining the projector's projection equation with the above relationships, the three-dimensional spatial coordinates can be solved. .

[0048] Specifically, for all masks belonging to the worn area The above calculations are performed on the pixels to obtain a three-dimensional point cloud set of the wear area. : , in, For the first The spatial coordinates of a three-dimensional point For the first The pixel coordinates of a point on a two-dimensional image This is a mask for the worn area.

[0049] Furthermore, to verify the accuracy and precision of the 3D reconstruction, this embodiment calibrates the system using standard gauge blocks before each test. The specific method is as follows: place the standard gauge block on the workstation, execute the same structured light projection and 3D reconstruction process, compare the reconstructed surface dimensions of the gauge block with its nominal dimensions, and calculate the measurement deviation of the current system. If the deviation is within the preset accuracy tolerance range (e.g., ±0.01mm), the system is considered to be in normal condition and subsequent tests can proceed; if the deviation exceeds the tolerance range, a system calibration prompt is output, requiring the operator to check the workstation settings and equipment status. Testing can only proceed after the calibration is passed. The above standard calibration steps ensure the stability and consistency of the system's measurement accuracy during long-term use. The moving least squares (MLS) method is used to convert the discrete point cloud... Reconstructed as a continuous three-dimensional surface of the wear region. The basic idea of ​​the MLS method is to establish a local fitting plane in the point cloud space, and for each local neighborhood, perform weighted least squares fitting using quadratic polynomial basis functions, with Gaussian weights used as the weights. : , in, The local neighborhood radius, Let be the distance from a point in the point cloud space to the center of the local fitted neighborhood. The above method converts the discrete point cloud into a continuous parametric surface representation. Figure 4 The complete reconstruction process from the structured light projection image to the three-dimensional surface of the wear area is shown above.

[0050] In this embodiment, the theoretical reference surface of the bearing inner ring is obtained. In this embodiment, the acquisition method is as follows: read the original model of the bearing inner ring, and based on the wear area mask... Based on the location in the image, and combined with the camera imaging model, the corresponding surface region in the model space is calculated, and the NURBS surface of this region is extracted from the model as a reference surface. .

[0051] It should be noted that for component inspection scenarios where the model cannot be obtained, this embodiment also provides an alternative method for obtaining the reference surface: First, it is determined whether there is an unworn area within a preset width range around the wear area mask. If so, the pixels of the unworn area within the preset width range around the wear area mask are used as a reference to extract the spatial three-dimensional coordinates corresponding to these unworn areas. The surface equation of the unworn area is fitted using a quadratic polynomial surface fitting method. This surface equation is then extrapolated to the wear area to obtain the complete reference surface. If there is no unworn area within a preset width range around the wear area mask, for example, if the wear area extends to the edge of the component or covers the entire surface being measured, then the historical three-dimensional scan data of the component or the three-dimensional data of an unworn component of the same model is obtained as a reference surface.

[0052] Specifically, the Iterative Closest Point (ICP) algorithm is used to analyze the three-dimensional surface of the wear region. Point cloud and reference surface The point cloud is rigidly registered to eliminate coordinate system deviations caused by differences in component placement poses. The specific process of the ICP algorithm is as follows: Initialization: Use the identity matrix as the rotation matrix The initial values ​​are determined by using the zero vector as the translation vector. The initial value of .

[0053] Iterative steps: (1) For the point cloud of the wear area Each point in In the reference surface point cloud Using KD-trees to accelerate the search for the nearest point , forming a matching point pair (2) Calculate the rotation matrix that minimizes the sum of squared distances between all pairs of matched points. Translation vector The objective function is The problem is solved by singular value decomposition (SVD); (3) the point cloud of the wear area is transformed. All points according to Perform the transformation; (4) Calculate the average registration error of this iteration. (5) If ( The value is 1×10 - 6 If the maximum number of iterations (set to 50) is reached (mm), the iteration is terminated; otherwise, return to step (1).

[0054] Furthermore, after registration, the three-dimensional surface of the wear area is... Each point in (Transformed to a coordinate system aligned with the reference surface), calculations are performed on the reference surface. Signed distance The signed distance is calculated as follows: on the reference surface... Find the distance nearest point ,calculate The distance to the projection point, and according to The sign is assigned to which side of the reference surface it is located on. The method for determining the sign is: calculate the vector. Normal vector of the reference surface at the projection point The dot product, if the dot product is greater than 0, then Take a positive value (convexity); if the dot product is less than 0, then... Take a negative value (dent / wear). The effective wear point is... point.

[0055] Maximum wear depth Take the maximum absolute value of the signed distance among all valid wear points.

[0056] The average wear depth is obtained using the following expression. : , in, This represents the number of effective wear points.

[0057] The wear area is obtained by analyzing the three-dimensional surface of the wear region. By performing area integral, we obtain: , in, For the area of ​​wear, Mask for worn areas The corresponding region in the surface parameter domain In the wear area Perform a double integral on top. Let be the magnitude of the vector.

[0058] Wear volume is the reference surface With wear surface The volume enclosed between: , The above integrals are only applicable to Integrate the regions. For the reference surface and the wear surface in terms of parameters The height difference at the point is the wear depth. The above double integral is calculated using numerical integration methods (such as the Gauss-Legend quadrature formula).

[0059] In this embodiment, based on a preset grading standard, the maximum wear depth is considered. Average wear depth Wear area and wear volume Output wear level.

[0060] According to an exemplary embodiment, the wear grading standard and its judgment rules for the bearing inner ring raceway are as follows: The preset grading standard has three levels in descending order of severity: severe wear, moderate wear, and light wear; during judgment, the wear is compared in the order of severe wear, moderate wear, and light wear. When the wear quantification index meets the judgment conditions of a certain level, it is judged as that level and the subsequent comparison is terminated. That is, when the maximum wear depth and wear volume fall into the judgment range of different levels, the level with the higher severity is taken as the standard.

[0061] For example: Maximum wear depth less than 0.05 mm and wear volume less than 1.0 mm³: slight wear; Maximum wear depth greater than or equal to 0.05 mm and less than 0.15 mm, or wear volume greater than or equal to 1.0 mm³ and less than 5.0 mm³: moderate wear; Maximum wear depth greater than or equal to 0.15 mm, or wear volume greater than or equal to 5.0 mm³: severe wear.

[0062] When the maximum wear depth is 0.10 mm (corresponding to moderate wear) and the wear volume is 6.0 mm³ (corresponding to severe wear), the wear level is determined to be severe wear according to the above judgment rules.

[0063] A test report is generated based on the wear level. The test report includes the following: component information (component ID, model, material, cumulative operating hours), a 3D topographic map of the wear area (displaying the wear depth distribution in pseudo-color), and values ​​for each wear quantification index. , , , The wear level and corresponding maintenance recommendations are as follows: For minor wear, continue operation and increase monitoring; for moderate wear, replace during the next maintenance; for severe wear, replace immediately.

[0064] According to one exemplary implementation, for mild wear, it is recommended to continue operation and strengthen monitoring, and arrange re-inspection according to the regular maintenance cycle; for moderate wear, it is recommended to increase the monitoring frequency and arrange replacement during the next maintenance in conjunction with the equipment operation plan; for severe wear, it is recommended to further shorten the monitoring cycle, prioritize replacement during the most recent shutdown maintenance window, and review the replacement timing in conjunction with the remaining life prediction results below.

[0065] It should be noted that the maintenance recommendations and the scrap thresholds set in the life prediction are used to guide maintenance decisions at different levels: maintenance recommendations are given based on wear levels and are used for maintenance plan preparation; scrap thresholds are used in life prediction to determine whether components have reached their unusable limit. When the maximum wear depth is between 0.15mm and 0.20mm and the wear level is determined to be severe wear, it indicates that the component has entered the accelerated wear stage but has not yet reached the scrap limit. At this time, the monitoring frequency should be increased, and the replacement timing should be determined in conjunction with the remaining life prediction results; when the maximum wear depth reaches or exceeds 0.20mm, it is determined that the component has reached the scrap limit, and a warning message of "immediate replacement recommended" is output.

[0066] In this embodiment, after outputting the wear level, the bearing inner ring information (part ID, model, material, cumulative operating hours) and wear quantification index () are included in this test. , , , The wear level and inspection time are entered into the wear database. Each inspection record is a data tuple: , in, For the first The data tuples of the next detection record For the detection time, This refers to the cumulative operating hours of the components up to the time of this inspection. For the first The wear level of the second test, For the first The maximum wear depth detected in this test. For the first The average wear depth of the test. For the first The wear area measured in the second test. For the first The wear volume measured in the second test. For the test serial number, ,in This represents the total number of inspections. The maximum wear depth of the same bearing inner ring across all inspections is extracted from the wear database. and the corresponding cumulative operating hours (When K≥5, the life prediction function is triggered; when K<5, only the wear level and quantitative index of this test are output).

[0067] Specifically, a double exponential model was used to fit the curve of maximum wear depth as a function of cumulative operating hours [mathematical formula]: , Among them, the first item Initial surface roughness ( The equivalent wear depth at that time reflects the initial roughness of the machined surface of the component); the second item Describes the rapid wear phase of the break-in period, in which This represents the total wear during the break-in period. This is the wear rate reduction coefficient during the break-in period; the wear rate during this stage gradually decreases with increasing operating time. (The third term...) Describes the accelerated wear phase following the stable wear period, in which For acceleration period amplitude coefficient, The acceleration growth factor represents the wear rate during this stage, which increases exponentially with operating time, reflecting the accelerated degradation characteristics of components after entering the severe wear stage. The entire model can comprehensively describe the complete lifecycle of the wear process from "initial break-in → stable wear → accelerated failure".

[0068] It should be noted that the model parameters , , , , The Levenberg-Marquardt nonlinear least squares method was used to fit historical data. To ensure the stability and reliability of the fitting results, at least 5 independent historical detection data points are required. Therefore, a trigger condition of K≥5 is set. When K<5, the fitting problem is underdetermined, and unique and reliable model parameters cannot be obtained. Therefore, lifetime prediction is not performed. The specific steps of the algorithm are: (1) give initial values ​​of model parameters; (2) calculate the residual vector under the current parameters. (3) Calculate the Jacobian matrix Similar to Hessian Calculate the gradient (4) Solve the incremental equation ,in (5) Update the parameters and calculate the new residuals. If the residuals decrease, accept the update and decrease the value. Otherwise increase And recalculate; (6) Iterate until convergence.

[0069] Specifically, set the maximum permissible wear depth of the bearing inner ring. =0.20mm (This value is determined by the bearing design specifications and corresponds to the critical value for bearing clearance deviation). Solve the equation. = The solution obtained This is the predicted scrapping time. If the equation is in If there is a unique solution within the range, then the remaining lifetime... ,in This represents the current cumulative running hours. If the equation has no real solution (i.e., the fitted curve is in...), then... If the slope at a certain point tends to flatten out and the wear amount never reaches the scrap threshold, then the output will be "The scrap threshold will not be reached within the predictable range." If the component is found to have exceeded its scrap threshold, a warning message suggesting immediate replacement is issued. Based on the residuals between historical data and the fitted model, the confidence interval of the predicted value is calculated, providing quantitative information on the uncertainty of maintenance timing. The final prediction report includes: a fitted curve, the predicted scrap time, the remaining life, the prediction confidence interval, and maintenance recommendations.

[0070] The above technical solution can scientifically predict the remaining life of parts by recording the test data into a database and using a double exponential model to fit the wear trend. This upgrades the traditional "post-inspection" or "periodic inspection" to "condition-based predictive maintenance", avoiding unplanned downtime accidents caused by sudden component failure.

[0071] This embodiment uses the wear detection of the inner ring raceway of a fan bearing as an example for detailed explanation. However, those skilled in the art should understand that this method is also applicable to the wear detection of other rotating parts such as coal mill roller sleeves, journals, and gear tooth surfaces. It only requires adjusting the positioning fixture of the standardized imaging station according to the shape characteristics of different parts. The scope of protection of this disclosure is not limited to the specific embodiment described above.

[0072] Example 2: like Figure 2 As shown, this embodiment provides a wear detection system for rotating parts based on a semantic segmentation network, used to implement the method described in Embodiment 1. The system includes: Image acquisition module 10 is used to acquire depth images and structured light projection images of rotating parts under standardized shooting conditions. Image acquisition module 10 specifically includes an industrial camera 11, a structured light projector 12, and a standardized shooting station 13. The standardized shooting station 13 is equipped with a fixed light source, a positioning bracket with a fixed shooting distance, and a positioning fixture for fixing the parts. The industrial camera 11 and the structured light projector 12 are rigidly connected and fixed to the same base plate, with their relative positions fixed. The industrial camera 11 is a high-resolution industrial camera with a built-in infrared cutoff filter, used to acquire grayscale images and structured light projection images. The structured light projector 12 is a DLP digital projection module used to project coded structured light patterns onto the surface of the parts.

[0073] According to one exemplary embodiment, the image acquisition module 10 further includes a depth camera for acquiring depth images.

[0074] The wear region segmentation module 20, connected to the image acquisition module 10 via an Ethernet interface, receives depth images and inputs them into a pre-trained semantic segmentation network. The semantic segmentation network classifies each pixel in the depth image and outputs a wear region probability map. A wear region mask is then generated based on the wear region probability map. The specific structure of the semantic segmentation network has been described in detail in Embodiment 1.

[0075] The 3D topography reconstruction module 30 is connected to the image acquisition module 10 via a gigabit Ethernet interface and to the wear area segmentation module 20 via an internal data bus. It receives structured light projection images and wear area masks, performs phase decoding on the structured light projection images, reconstructs the 3D point cloud of the area corresponding to the wear area mask based on the triangulation principle, and reconstructs the 3D surface of the wear area based on the 3D point cloud. The 3D topography reconstruction module 30 is also deployed on an industrial control computer.

[0076] The wear quantification calculation module 40 is connected to the three-dimensional shape reconstruction module 30 through an internal data bus. It is used to obtain the reference surface of the component, register the three-dimensional surface of the wear area with the reference surface, calculate the signed distance from the three-dimensional surface of the wear area to the reference surface point by point, calculate the maximum wear depth, average wear depth, wear area and wear volume based on the signed distance, and output the wear level according to the preset grading standard.

[0077] The trend prediction module 50, connected to the wear quantification calculation module 40 via an internal data bus, is used to input the wear quantification index and corresponding detection time into the wear database after the wear quantification calculation module outputs the wear level. Based on the historical detection data of the same component in the wear database, a dual-exponential model is used to fit the wear trend, and the remaining life of the component is predicted based on the fitting result. The wear database is deployed locally on the industrial control computer and stored using an SQLite database, while also supporting data synchronization with the factory MES system via industrial Ethernet.

[0078] The data flow between the functional modules is as follows: the image acquisition module 10 transmits the acquired depth image to the wear area segmentation module 20 and the structured light projection image to the three-dimensional topography reconstruction module 30; the wear area segmentation module 20 transmits the generated wear area mask to the three-dimensional topography reconstruction module 30; the three-dimensional topography reconstruction module 30 transmits the reconstructed three-dimensional surface to the wear quantization calculation module 40; the wear quantization calculation module 40 transmits the wear level and quantification index to the trend prediction module 50, and outputs a detection report.

[0079] In this embodiment, each module in the system is controlled by computer program instructions to perform corresponding data processing operations on the corresponding computer hardware, and the modules interact with each other through a data bus or network interface.

[0080] In summary, this disclosure ensures data consistency and comparability through standardized image acquisition, achieves automatic and accurate identification of wear areas through semantic segmentation networks, acquires the three-dimensional morphology of wear areas through structured light 3D reconstruction, achieves multi-dimensional quantification of wear through ICP registration and signed distance calculation, and realizes wear trend prediction and remaining life estimation through databases and dual-exponential models. This forms a complete technical closed loop from data acquisition to life prediction, providing an objective, accurate, efficient, and traceable complete solution for wear detection of rotating parts.

[0081] The above description is merely a preferred embodiment of this disclosure. The scope of protection of this disclosure is not limited to the above embodiments. All technical solutions falling within the scope of this disclosure are protected. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of this disclosure should also be considered within the scope of protection of this disclosure.

Claims

1. A method for detecting wear of rotating components based on semantic segmentation networks, characterized in that, include: Acquire depth images and structured light projection images of rotating components under standardized shooting conditions; The depth image is input into a pre-trained semantic segmentation network, which classifies each pixel in the depth image and outputs a wear region probability map. A wear region mask is then generated based on the wear region probability map. Phase decoding is performed on the structured light projection image, and a three-dimensional point cloud of the area corresponding to the wear area mask is reconstructed based on the triangulation principle. The three-dimensional surface of the wear area is then reconstructed based on the three-dimensional point cloud. Obtain the reference surface of the component, register the three-dimensional surface of the wear area with the reference surface, calculate the signed distance from the three-dimensional surface of the wear area to the reference surface point by point, and calculate the wear quantification index based on the signed distance. The wear quantification index includes the maximum wear depth, average wear depth, wear area and wear volume. The wear level is output based on the wear quantification index and the preset grading standard.

2. The wear detection method for rotating components based on semantic segmentation networks according to claim 1, characterized in that, Also includes: After outputting the wear level, the wear quantification index and the corresponding detection time are entered into the wear database. Based on the historical detection data of the same component in the wear database, the wear trend is fitted, and the remaining life of the component is predicted according to the fitting result.

3. The wear detection method for rotating components based on semantic segmentation networks according to claim 1, characterized in that, The acquisition of depth images and structured light projection images of the rotating parts under standardized shooting conditions includes: simultaneously acquiring depth images, grayscale images, and structured light projection images modulated on the surface of the parts at a standardized shooting station, wherein the standardized shooting station has fixed lighting conditions, fixed camera shooting distance, and fixed camera shooting angle.

4. The wear detection method for rotating components based on semantic segmentation networks according to claim 1, characterized in that, The semantic segmentation network adopts an encoder-decoder structure. The encoder is used to extract multi-scale feature maps of the depth image step by step, and the decoder is used to restore the multi-scale feature maps to the resolution of the depth image step by step. The encoder and the decoder pass feature maps of corresponding scales through skip connections.

5. The wear detection method for rotating components based on semantic segmentation networks according to claim 1, characterized in that, The step of generating a wear region mask based on the wear region probability map includes: setting a probability threshold, determining pixels with probability values ​​greater than or equal to the probability threshold in the wear region probability map as worn pixels, determining pixels with probability values ​​less than the probability threshold as non-worn pixels, and the set of all worn pixels constitutes the wear region mask.

6. The wear detection method for rotating components based on semantic segmentation networks according to claim 1, characterized in that, The step of performing phase decoding on the structured light projection image and reconstructing the three-dimensional point cloud of the area corresponding to the wear area mask based on the triangulation principle includes: performing phase-shift decoding on the structured light projection image to obtain the wrapping phase; using the multi-frequency heterodyne method to perform phase unwrapping on the wrapping phase to obtain the absolute phase; and calculating the spatial three-dimensional coordinates corresponding to each pixel in the wear area mask based on the triangulation principle according to the absolute phase and camera calibration parameters. The set of all spatial three-dimensional coordinates constitutes the three-dimensional point cloud.

7. The wear detection method for rotating components based on semantic segmentation networks according to claim 1, characterized in that, The step of obtaining the reference surface of the component includes: obtaining the original model of the component, and extracting the theoretical surface corresponding to the wear area mask from the original model as the reference surface; or, when the original model cannot be obtained, if there is an unworn area within a preset width range around the wear area mask, the reference surface is obtained by fitting the surface of the unworn area and extrapolating it to the wear area; if there is no unworn area within a preset width range around the wear area mask, the historical three-dimensional scan data of the component or the three-dimensional data of an unworn component of the same model is obtained as the reference surface.

8. The wear detection method for rotating components based on semantic segmentation networks according to claim 1, characterized in that, The process of registering the three-dimensional surface of the wear region with the reference surface includes: using an iterative nearest-point algorithm to rigidly register the point cloud of the three-dimensional surface of the wear region with the point cloud of the reference surface; solving for the rotation matrix and translation vector that minimizes the distance between the two sets of point clouds; and transforming the three-dimensional surface of the wear region to align with the reference surface based on the rotation matrix and translation vector.

9. The wear detection method for rotating components based on semantic segmentation networks according to claim 2, characterized in that, The step of fitting wear trend data of the same component in the wear database and predicting the remaining life of the component based on the fitting result includes: extracting the maximum wear depth and corresponding cumulative operating hours of the same component in each test from the wear database; fitting the curve of the maximum wear depth as a function of cumulative operating hours using a double exponential model; calculating the cumulative operating hours required for the component to reach the scrap threshold based on the fitted wear trend model and a preset scrap threshold; and obtaining the remaining life based on the difference between the cumulative operating hours and the current cumulative operating hours.

10. A wear detection system for rotating parts based on semantic segmentation networks, used to implement the wear detection method for rotating parts based on semantic segmentation networks as described in any one of claims 1 to 9, characterized in that, include: An image acquisition module is used to acquire depth images and structured light projection images of rotating parts under standardized shooting conditions. The image acquisition module includes an industrial camera, a structured light projector, and a standardized shooting station. The standardized shooting station is equipped with a fixed light source, a positioning bracket with a fixed shooting distance, and a positioning fixture for fixing the parts. The relative position between the industrial camera and the structured light projector is fixed. The wear region segmentation module is used to input the depth image into a pre-trained semantic segmentation network, classify each pixel in the depth image through the semantic segmentation network, output a wear region probability map, and generate a wear region mask based on the wear region probability map. The three-dimensional topography reconstruction module is used to perform phase decoding on the structured light projection image, reconstruct the three-dimensional point cloud of the area corresponding to the wear area mask based on the triangulation principle, and reconstruct the three-dimensional surface of the wear area based on the three-dimensional point cloud. The wear quantification calculation module is used to obtain the reference surface of the component, register the three-dimensional surface of the wear area with the reference surface, calculate the signed distance from the three-dimensional surface of the wear area to the reference surface point by point, calculate the wear quantification index based on the signed distance, the wear quantification index includes the maximum wear depth, average wear depth, wear area and wear volume, and output the wear level according to the wear quantification index and the preset grading standard. The trend prediction module is used to input the wear quantification index and the corresponding detection time into the wear database after the wear quantification calculation module outputs the wear level, perform wear trend fitting based on the historical detection data of the same component in the wear database, and predict the remaining life of the component based on the fitting result.

Images