A ferrograph image wear particle chain growth process segmentation method

Abstract

The application claims a ferrography image abrasive particle chain growth process segmentation method, and relates to the technical field of ferrography image segmentation, and comprises the following steps: S1, inputting a training picture, and training an abrasive particle target detection model; S2, inputting a predicted ferrography image, and acquiring object position information and category information in a predicted picture; S3, adopting an improved marker watershed segmentation algorithm to perform image segmentation on the ferrography image, and segmenting the abrasive particle chain adhered together into single abrasive particles. The method can accurately segment the abrasive particles in the abrasive particle chain one by one, and the abrasive particle chain marked with the abrasive particle and the background is segmented by using the marker watershed algorithm, so that the abrasive particle chain is accurately segmented, and the over-segmentation and under-segmentation problems in the abrasive particle chain segmentation are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ferrography image segmentation, and particularly relates to a ferrography image segmentation method for growth process of abrasive particles in oil detection. BACKGROUND

[0002] Mechanical equipment is the basis of industrial production, and safe operation of equipment, reduction of fault downtime and prolongation of service life can maximize the use value of equipment. For most mechanical equipment, a lubrication system is an indispensable part. In addition to the functions of lubrication and cooling, flowing lubricating oil also carries rich information about the wear state of mechanical friction pairs. Mechanical equipment often operates in harsh environments, and friction between contacting surfaces in the relative motion trend causes material wear and forms abrasive particles. The abrasive particles flow with the lubricating oil in the mechanical equipment, which may cause hidden troubles. 80% of mechanical failures are caused by abrasive particles and lubrication failure.

[0003] Ferrography analysis uses a high-gradient strong magnetic field to orderly separate abrasive particles in the detection oil circuit, is a kind of abrasive particle analysis technology in tribology, and is also one of important means for mechanical wear detection and diagnosis. Processing of ferrography images is an important content in the field of ferrography analysis technology. With rapid upgrading and improvement of hardware technology, the processes of spectrum sheet fabrication and ferrography image acquisition have been basically automated. The types of abrasive particles in the ferrography image are diversified, and the characteristics of different abrasive particles are quite different. Segmentation and recognition of the ferrography image are difficult, and it is difficult to accurately segment the ferrography image, so that the quality of the segmentation result is uneven. Personnel with professional knowledge and experience are still required to manually extract typical abrasive particles from the ferrography image collected in the segmentation of abrasive particle chains, which greatly limits the wide application of ferrography technology.

[0004] CN110223299B, a method for dividing abrasive particles based on deposition process, comprising: decomposing the abrasive particle deposition process video obtained from the online visual iron spectrum to obtain the abrasive particle chain image at each time; the abrasive particle chain deposition image obtained at each time is divided into abrasive particle chain, and the abrasive particle chain block is obtained; the nearest neighbor method is used to match the abrasive particle chain block at each time of the abrasive particle, and the deposition change process of the abrasive particle chain is obtained; according to the length change of the abrasive particle deposition chain, the redundant abrasive particle block is removed; the pixel value of each column of the abrasive particle chain block is added respectively, the pixel value of the adjacent abrasive particle chain block is subtracted, and the same part of the adjacent abrasive particle chain block is matched; the newly deposited abrasive particle is marked by distance transformation, and the segmentation image of the abrasive particle is obtained by using the marker watershed segmentation; the abrasive particle in the abrasive particle deposition chain can be accurately segmented, which has important significance for realizing the intelligentization and automation of online iron spectrum image analysis technology. In the problem of abrasive particle chain accumulation, the abrasive particle may not increase at both ends of the abrasive particle chain during deposition, but may be deposited from the upper and lower directions of the abrasive particle chain. The method described in this patent cannot mark the abrasive particles deposited from the upper and lower ends independently, which will cause the phenomenon of under segmentation. In the process of abrasive particle deposition, the slow movement of the abrasive particle chain causes two abrasive particle chains to stack together, and the method mentioned in the patent for matching the abrasive particle chain will cause errors, which cannot match the subsequent deposited abrasive particle chain, and cannot complete the segmentation of the abrasive particle chain. The method mentioned in this patent has no direction restriction, and cannot be segmented because two abrasive particle chains are stacked together.

[0005] CN111832608A discloses a method for multi-abrasive grain recognition in ferrography images based on the single-stage detection model YOLOv3. This invention first improves the YOLOv3 model backbone network by adding a spatial pyramid pooling module, replacing the original IOU loss function with a GIOU loss function, and expanding the YOLO layer scale. This improvement effectively overcomes the problems of low recognition rate for similar abrasive grains and high false negative rate for small abrasive grains in ferrography abrasive grain recognition. Furthermore, this invention simplifies the network structure by fusing the model's BN and convolutional layers, limiting the additional computational load from added modules and ensuring the real-time performance of the YOLOv3 model at its lowest detection speed. With this method, multiple abrasive grain images only need to be input into the trained module to obtain intelligent recognition results, eliminating the need for further manual operation and achieving intelligent recognition. This patent can perform preliminary segmentation of abrasive grain chains; however, if multiple prediction boxes exist within the same area, resulting in stacked targets, the recognition effect is still poor. This patented method, based on obtaining the position information of the detected object through a target detection network, uses the acquired positional region information to perform segmentation using a watershed algorithm, achieving more detailed segmentation. It can also obtain the boundary information of abrasive particles, and for oil monitoring, this boundary information reflects the machine's operating status. Summary of the Invention

[0006] This invention aims to solve the problems of the prior art. It proposes a method for segmenting the abrasive grain chain growth process in ferrographic images. The technical solution of this invention is as follows:

[0007] A method for segmenting the grain chain growth process in ferrographic images, comprising the following steps:

[0008] S1. Obtain ferrography video images and filter them; input the training images into the target detection model to train the abrasive target detection model;

[0009] S2. Input the ferrography image to be predicted into the trained abrasive target detection model to obtain the object location information and category information in the image to be predicted.

[0010] S3, the coordinate position information obtained after the predicted image is processed, and the ferrography image is segmented by using an improved marker watershed segmentation algorithm. The improved marker watershed segmentation algorithm is improved in that the watershed segmentation algorithm is suitable for extracting adjacent or overlapping objects in a picture. A gray-scale image can be regarded as a terrain surface, and a high peak represents high intensity, and a valley represents low intensity. Noise and other abnormalities in the image will have a bad influence on the segmentation result of the image. The improvement of the present method on the general marker watershed is that the marker is enlarged as much as possible, so that the segmentation error at the end is less affected by the previous noise interference. According to the segmentation result, the mechanical wear condition is judged. When the ferrography video image is frame-by-frame differentiated, the stacking order of each abrasive particle in the entire abrasive particle chain growth process can be obtained. According to the order and the final segmentation result, the segmentation effect can be judged.

[0011] Further, the step S1 specifically comprises the following steps:

[0012] The ferrography video image is collected by a camera. The collected ferrography video image is frame-by-frame split into pictures, and the split pictures are screened to retain ferrography image pictures with good quality and less interference. Each frame abrasive particle chain change graph is obtained by using a labelimg online visual ferrography image video decomposition. The deposition change order of the abrasive particle chain is obtained according to the length change of the abrasive particle chain. The ferrography image picture is marked according to the deposition change order. The marked categories are debris, chain and bubble, which respectively represent abrasive particles, abrasive particle chains and water bubbles in the oil. The data set format is made in VOC format. Then, the ferrography image picture data is divided into a training set, a verification set and a test set. The sample image is obtained by frame-by-frame differentiation of the previously collected ferrography video image. The sample image is labeled and made by using the previous labelimg to picture. The sample image contains the position size information, confidence and target category of the real box of the abrasive particle in the ferrography image picture. Then, the training set in the data set is sent into a target detection network model for training.

[0013] Further, the abrasive particle target detection model of the step S1 adopts a YOLOv4 model.

[0014] Further, in the step S2, after the ferrography image to be detected is input into the abrasive particle target detection model, the repeated prediction boxes output by the target detection model are removed by using a non-maximum suppression method, and a detection result of the abrasive particle chain in the ferrography image to be detected is obtained. The detection result of the abrasive particle chain includes the position size information, confidence and target category of the prediction box.

[0015] Further, the repeated prediction box output by the target detection model is removed by the non-maximum suppression method, and specifically comprises:

[0016] The non-maximum suppression method is used to find the best target boundary box and eliminate redundant boundary boxes in the target detection process.

[0017] Further, the step S3 improved marker watershed segmentation algorithm specifically comprises:

[0018] S41, according to the coordinate information of the prediction result, the center point of the detected object coordinate is drawn; the intersection region in the coordinate information of the prediction result is erased, the threshold is used for image binarization, and then morphological opening and closing operation is adopted to eliminate the noise in the binarized image; finally, the image of the center point of the drawn object coordinate and the erased intersection region is added as a marker image;

[0019] S42, the marker image and the input prediction image are sent into the watershed algorithm to complete the segmentation of the abrasive grain chain image.

[0020] Further, the watershed algorithm specifically comprises the following steps:

[0021] The image gradient is calculated, the area with smaller gradient value is obtained as the catchment point through the image gradient, and the direction is found through the catchment point to obtain all the pixels of the same catchment point and number the pixels of different catchment points to obtain the catchment area. All the catchment areas are traversed to obtain the boundary of the catchment area. Wherein, the boundary pixels of the catchment area are judged to detect whether they can be submerged, and then the catchment area and the boundary pixels of the area are recalculated, and the boundary of two catchment areas is the watershed.

[0022] The advantages and beneficial effects of the present application are as follows:

[0023] The application provides a segmentation method for the growth process of abrasive particle chains in ferrograph images.

[0024] Compared with general target detection networks, the method can perform more fine segmentation, and it is not easy to think of further utilizing the object prediction region information of the target detection network to more accurately segment the abrasive particle chains. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The application provides a segmentation method for the growth process of abrasive particle chains in ferrograph images, and a step diagram of the method is shown in the figure;

[0026] Figure 2 The figure is a picture sent into a YOLOv4 network model for prediction;

[0027] Figure 3 The figure is a prediction result picture;

[0028] Figure 4 The figure is a center point marking picture;

[0029] Figure 5 The figure is a picture in which a common intersection region of a prediction target is erased;

[0030] Figure 6 The figure is a ferrograph image segmentation result picture. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0032] The technical solution of the application to solve the above technical problems is:

[0033] Referring to Figure 1 The embodiment provides a segmentation method for the growth process of abrasive particle chains in ferrograph images, and the method includes the following steps:

[0034] (1) Collect the ferrography image pictures through the camera. The collected ferrography image pictures are screened to retain the ferrography image pictures with good quality and less interference. The labelimg labeling software is used to obtain the change diagram of each frame of abrasive grain chain through online image video decomposition. The deposition change order of the abrasive grain chain is obtained according to the length change of the abrasive grain chain. The ferrography image pictures are labeled according to the deposition change order. The labeled categories are debris, chain and bubble. The data set format is made in VOC format. Then, the ferrography image picture data is divided into a training set, a verification set and a test set.

[0035] (2) The training set is input, and the abrasive grain target detection model is trained. The optimal YOLOv4 model is selected to output the target recognition result.

[0036] (3) As shown in Figure 3 , the selected ferrography image to be detected is sent into the target detection model. The repeated prediction boxes output by the target detection model are removed through the non-maximum suppression method. As shown in Figure 3 , the detection result of the abrasive grain chain in the image to be detected is obtained, including the prediction box position size information, confidence and target category.

[0037] Embodiment 2

[0038] The application provides a method for segmenting the abrasive grain chain in the ferrography image by using an improved marker watershed segmentation algorithm, which comprises the following steps:

[0039] S1, as shown in Figures 5-6 , the center point of the detected object coordinate is drawn according to the coordinate information of the prediction result. The intersection area in the coordinate information in the prediction result is removed. A suitable threshold is found to binarize the image. Then, the morphological opening and closing operation is performed to eliminate the noise in the binarized image. Finally, the image of the intersection area removed after the center point of the object coordinate is drawn is added as the foreground image;

[0040] S2, the labeled image and the input prediction image are sent into the watershed algorithm. The marker watershed algorithm completes the segmentation of the abrasive grain chain image according to the prior knowledge obtained by labeling. Each abrasive grain of the abrasive grain chain in the ferrography image is segmented.

[0041] The system, device, module or unit illustrated in the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may, for example, be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an electronic mail device, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0042] It is also to be noted that the terms "comprising", "including", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a..." does not, without further restriction, exclude the existence of additional elements of a like kind in the process, method, article, or apparatus that comprises the element.

[0043] The above examples are to be understood only as illustrative of the application and not a limitation of the scope of protection for the present application. After reading the specification, those skilled in the art will be able to make various modifications or alterations to the application without departing from the scope of the application, which is defined by the claims.

Claims

1. A method for segmenting ferrographic image wear particle chain growth processes, characterized by, It comprises the following steps: S1, acquiring an iron spectrum video image and performing screening; inputting a training picture into a target detection model to train a wear particle target detection model; S2, inputting a to-be-predicted iron spectrum image into the trained wear particle target detection model to obtain object position information and category information in the to-be-predicted picture; S3, processing coordinate position information obtained after the predicted image, and performing image segmentation on the iron spectrum image by using an improved marker watershed segmentation algorithm to segment the wear particle chain adhered together into single wear particles, the improved marker watershed segmentation algorithm specifically being: S31, drawing a center point of a detected object coordinate according to coordinate information of a prediction result; erasing an area where coordinate information in the prediction result exists intersection, finding a threshold to binarize the image, and then performing a morphological opening and closing operation to eliminate noise in the binarized image; finally, adding the image of the erased intersection area and the center point of the object coordinate to serve as a marker image; S32, inputting the marker image and the input prediction image into a watershed algorithm to complete segmentation of the wear particle chain image; The watershed algorithm specifically comprises the following steps: Calculating an image gradient, obtaining a region with smaller gradient values as a catchment point through the image gradient, and obtaining all pixels of the same catchment point and numbering pixels of different catchment points through direction finding to obtain a catchment region; traversing all catchment regions to obtain a boundary of the catchment region; wherein, judging whether the boundary pixel of the catchment region can be submerged, and then recalculating the boundary pixel of the catchment region and the region, and the boundary of two catchment regions is a watershed.

2. The method according to claim 1, wherein, The step S1 specifically comprises the following steps: An iron spectrum video image is collected through a camera; the collected iron spectrum video image is frame by frame split into pictures, the split pictures are screened, and iron spectrum image pictures with good quality and less interference are retained; an online visual iron spectrum image video is decomposed through a labelimg software to obtain a wear particle chain change image of each frame, a deposition change sequence of the wear particle chain is obtained according to the length change of the wear particle chain, the iron spectrum image pictures are labeled according to the deposition change sequence, and the labeled categories are debris, chain and bubble, which respectively represent wear particles, wear particle chains and water bubbles in oil, only a category is distinguished, a data set format is made in a VOC format, and then the iron spectrum image picture data is divided into a training set, a verification set and a test set, sample images are obtained by frame by frame difference of the previously collected iron spectrum video image, and the sample images are labeled and made through the labelimg to contain position size information, confidence and target category of a real box of the wear particles in the iron spectrum image pictures; and then the training set in the data set is input into a target detection network model for training.

3. The method according to claim 1, wherein, The wear particle target detection model of the step S1 adopts a YOLOv4 model.

4. The method according to claim 1, wherein, In the step S2, the ferrograph image to be predicted is input into the abrasive particle target detection model, and repeated prediction boxes output by the target detection model are removed by a non-maximum suppression method to obtain a detection result of the abrasive particle chain in the image to be predicted, and the detection result of the abrasive particle chain includes prediction box position and size information, confidence and target category.

5. The method according to claim 4, wherein the method is characterized by, The repeated prediction boxes output by the target detection model are removed by the non-maximum suppression method, and specifically includes the following steps: In the step of the non-maximum suppression method, a large number of candidate boxes will be generated at the same position in the target detection process, and these candidate boxes will overlap to a certain extent, so it is necessary to find the best target bounding box by using the non-maximum suppression method to eliminate redundant bounding boxes; 1. First, sorting is performed according to the confidence; 2. The bounding box with the highest confidence is added to the final output list, and is deleted from the bounding box list; 3. The area of all bounding boxes is calculated; 4. The IOU of the bounding box with the highest confidence and other candidate boxes is calculated; 5. The bounding boxes with the IOU greater than the threshold are deleted until the bounding box list is empty; IOU: the intersection part of two bounding boxes divided by the union part of the two bounding boxes, and the larger the value is, the more the overlapping part of the two bounding boxes is.

Images