An engine oil abrasive particle image analysis method and device, medium and equipment

Abstract

The present application relates to the technical field of lubricating oil wear particle detection, and particularly relates to an engine lubricating oil wear particle image analysis method and device, medium and equipment, the method comprising: acquiring a ferrograph image and constructing a data set, then dividing the data set into a training set and a validation set; constructing a U-net network model, then training and verifying the data set to obtain a trained lubricating oil wear particle detection model; inputting a ferrograph image to be analyzed into the lubricating oil wear particle detection model to obtain a recognized wear particle image; then detecting the edge of the ferrograph image using a Canny operator, and segmenting in combination with the recognized wear particle image; and finally performing contour detection on the segmented wear particle image to obtain contour information of the wear particle. The analysis method provided by the present application processes the image using a U-net network model and a Canny operator, can effectively recognize and extract wear particle information in the image, not only simplifies the ferrograph analysis process, but also greatly improves the accuracy of wear particle analysis.

Description

Technical Field

[0001] This invention relates to the field of lubricating oil abrasive detection technology, and in particular to an image analysis method, apparatus, medium, and equipment for engine lubricating oil abrasive particles. Background Technology

[0002] The lubricating oil system is one of the most important components of an engine. If abrasive particles in the lubricating oil are not filtered and collected in a timely manner, they may cause secondary wear between rotating parts and the lubricating oil system. Therefore, timely collection and analysis of abrasive particles in the lubricating oil is beneficial for timely and effective maintenance of the engine's rotating parts, avoiding the adverse effects of secondary wear, and also providing information on the engine's wear status.

[0003] Currently, commonly used methods for detecting wear particles in engine lubricating oil are mainly divided into two categories: online monitoring and offline testing. Offline testing methods utilize spectroscopy and ferrography to analyze the morphology, size, and type of wear particles. In recent years, ferrography has proven to be an effective technique for monitoring wear conditions and diagnosing faults by qualitatively and quantitatively analyzing wear particles (referred to as abrasive particles or wear debris).

[0004] However, current ferrography results suffer from problems such as abrasive particle accumulation and poor image acquisition, which greatly limits their application. Specifically, for example... Figure 1 and Figure 2 The images shown are discrete abrasive grain images and overlapping abrasive grain images, respectively. Some researchers have proposed methods for segmenting discrete abrasive grain images, but traditional computer image processing methods still have difficulty segmenting overlapping abrasive grains due to the problem of overlapping abrasive grain stacking. This limits the automated analysis of abrasive grain morphology, size, and type, resulting in a significant reduction in the accuracy of abrasive grain analysis. Summary of the Invention

[0005] To address the inaccuracy of lubricating oil abrasive particle detection in the prior art, this invention provides an image analysis method for engine lubricating oil abrasive particles, comprising the following steps:

[0006] Step S100: Obtain ferrography images and crop them to construct a dataset, then divide the dataset into a training set and a validation set;

[0007] Step S200: Construct a U-net network model and input the training set into the U-net network model for training to obtain a trained lubricating oil abrasive detection model; then input the validation set into the lubricating oil abrasive detection model for testing and evaluate the training effect.

[0008] Step S300: Preprocess the ferrography image to be analyzed, and then input the preprocessed ferrography image into the lubricating oil abrasive detection model to obtain the identified abrasive image contours.

[0009] Step S400: Use the Canny operator to detect the edges of the preprocessed ferrography image to obtain edge pixels. If the detected edge pixels are located within the outline of the abrasive image identified by the lubricating oil abrasive detection model, they are considered as overlapping abrasive particles, and the Canny operator is used to segment the overlapping abrasive particles. If the detected edge pixels are located outside the outline of the abrasive image identified by the lubricating oil abrasive detection model, they are considered as false detections and are removed.

[0010] Step S500: Perform contour detection on the segmented abrasive grain image to obtain the contour information of the abrasive grains in the image.

[0011] In one embodiment, preprocessing the ferrography image to be analyzed includes one or more of the following: denoising, enhancement, sharpening, restoration, and filtering.

[0012] In one embodiment, the training set is input into the U-net network model for training, including the following steps: inputting the images of the training set into convolutional blocks and CBAM attention modules to obtain feature layers, and then performing multiple convolutions, CBAM attention module training, and pooling to obtain a primary training feature map; performing multiple upsampling, fusing preliminary effective feature layers, and convolutions on the primary training feature map to obtain a secondary training feature map; adjusting the channels of the secondary training feature map according to the number of target categories detected in the feature map, and using the features to obtain prediction results; wherein, the cross-entropy loss function and the DiceLoss loss function are used to obtain the loss.

[0013] In one embodiment, the CBAM attention module includes a channel attention mechanism module and a spatial attention mechanism module. Inputting an image into the CBAM attention module includes the following steps:

[0014] The image is input into the channel attention mechanism module for global max pooling and global average pooling to obtain two sets of first feature maps, which are then input into a shared two-layer neural network. Channel attention features are generated based on element-wise summation and sigmoid function activation. Finally, the channel attention features and the image are multiplied to generate the input feature map.

[0015] The input feature map is input into the spatial attention module, and then channel-based global max pooling and global average pooling are performed to obtain two sets of second feature maps. The two sets of second feature maps are then concatenated and convolved based on channels to reduce the dimensionality to a single channel. The spatial attention features are then generated by the sigmoid function. Finally, the spatial attention features and the input feature map are multiplied to obtain the feature layer.

[0016] In one embodiment, the image detection target types are divided into background parts and abrasive parts.

[0017] In one embodiment, the step of evaluating the training effect is to calculate the Intersection over Union (IoU) of the background portion, the IoU of the abrasive portion, and the average IoU, respectively, using the following formula:

[0018]

[0019]

[0020] Where A1 and B1 are the predicted and actual results for the abrasive particles, respectively, and A2 and B2 are the predicted and actual results for the background, respectively.

[0021] In one embodiment, edge detection of an abrasive image using the Canny operator includes the following steps:

[0022] The image noise reduction step involves performing Gaussian filtering on the abrasive grain image;

[0023] The step of calculating image gradients involves using the Sobel operator to calculate the edge gradient magnitude and gradient direction of each pixel in the abrasive image.

[0024] The non-maximum suppression step involves a full scan of the abrasive image to remove irrelevant pixels that do not form edges. Then, it checks whether each pixel is a local maximum among its neighboring pixels in the gradient direction. If so, it is retained; otherwise, it is removed.

[0025] The dual threshold filtering step involves selecting a preset high threshold and a preset low threshold, and determining the relationship between the edge gradient magnitude of each pixel and the preset high threshold and preset low threshold.

[0026] If the edge gradient magnitude is greater than the preset high threshold, the pixel is determined to be an edge pixel; if the edge gradient magnitude is lower than the preset low threshold, the pixel is determined to be a non-edge pixel and is discarded; if the edge gradient magnitude is between the preset low threshold and the preset high threshold, the pixel is determined to be a pixel to be determined, and then the continuity of the pixel to be determined is judged. If it is connected to an edge pixel, it is determined to be an edge pixel; otherwise, it is discarded.

[0027] The present invention also provides an engine lubricating oil abrasive particle image analysis device, comprising:

[0028] The dataset construction module acquires ferrography images and crops them to construct a dataset, and then divides the dataset into a training set and a validation set;

[0029] The training module constructs a U-net network model and inputs the training set into the U-net network model for training to obtain a trained lubricating oil abrasive detection model; then, the validation set is input into the lubricating oil abrasive detection model for testing, and the training effect is evaluated.

[0030] The segmentation module preprocesses the ferrography image to be analyzed, and then inputs the preprocessed ferrography image into the lubricating oil abrasive detection model to obtain the identified abrasive image contours. The Canny operator is used to detect the edges of the preprocessed ferrography image to obtain edge pixels. If the detected edge pixels are located within the abrasive image contours identified by the lubricating oil abrasive detection model, they are considered overlapping abrasive particles, and the Canny operator is used to segment the overlapping abrasive particles. If the detected edge pixels are located outside the abrasive image contours identified by the lubricating oil abrasive detection model, they are considered false detections and are removed.

[0031] The analysis module performs contour detection on the segmented abrasive grain image to obtain the contour information of the abrasive grains in the image.

[0032] The present invention also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the engine lubricating oil abrasive particle image analysis method as described in any of the above embodiments.

[0033] The present invention also provides an electronic device, including at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the processor to perform the engine lubricating oil abrasive particle image analysis method as described in any of the above embodiments.

[0034] Based on the above, compared with the prior art, the engine lubricating oil wear particle image analysis method provided by the present invention uses the U-net network model and Canny operator to process the image, which can effectively identify and extract wear particle information in the image, not only simplifying the ferrography analysis process, but also greatly improving the accuracy of wear particle analysis.

[0035] Other features and beneficial effects of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other beneficial effects of the invention can be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Unless otherwise specified, the positional relationships shown in the drawings in the following description are based on the direction in which the components are drawn in the figure.

[0037] Figure 1 A schematic diagram of the ferrographic image of discrete abrasive particles;

[0038] Figure 2 A schematic diagram of the ferrographic image of overlapping abrasive grains;

[0039] Figure 3 A flowchart of the engine lubricating oil abrasive particle image analysis method provided by the present invention;

[0040] Figure 4 The diagram shows the network structure of U-net, which is improved based on the attention mechanism.

[0041] Figure 5 This is a schematic diagram of the ferrographic image of the engine lubricating oil wear particles to be analyzed.

[0042] Figure 6 This is a schematic diagram of the ferrography image after preprocessing and cropping.

[0043] Figure 7 This is a schematic diagram of the outline of the identified abrasive particles obtained in the lubricating oil abrasive particle detection model.

[0044] Figure 8 This is a schematic diagram of an image with edge pixels obtained by edge detection using the Canny operator.

[0045] Figure 9 This is a schematic diagram of the segmented abrasive grain image result. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The technical features designed in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0047] In the description of this invention, it should be noted that all terms used in this invention (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains, and should not be construed as limiting the invention; it should be further understood that the terms used in this invention should be understood to have the same meaning as those in the context of this specification and in the relevant field, and should not be understood in an idealized or overly formal sense, except as expressly defined in this invention.

[0048] Current ferrography results suffer from issues such as abrasive grain accumulation and poor image acquisition quality, limiting their application. The abrasive grain accumulation problem, in particular, is difficult to segment using traditional computer image processing methods, hindering automated analysis of grain morphology, size, and type. Therefore, to address any of these issues, please refer to [the relevant documentation / reference]. Figure 3 This invention provides a method for image analysis of engine lubricating oil abrasive particles, comprising the following steps:

[0049] Step S100: Obtain ferrography images, crop the ferrography images to construct a dataset, and then divide the dataset into a training set and a validation set; preferably, in order to ensure the image quality of the dataset, it is necessary to crop the abrasive grain images in the middle region as the dataset.

[0050] Step S200: Construct a U-net network model and input the training set into the U-net network model for training to obtain a trained lubricating oil abrasive detection model; then input the validation set into the lubricating oil abrasive detection model for testing and evaluate the training effect.

[0051] In practical implementation, the U-net network model provided in this embodiment is structured into two parts: downsampling and upsampling. Downsampling consists of four layers, each including three convolutions and one downsampling operation, used for feature extraction, extracting information such as image location and semantics. Upsampling involves deconvolution to decode the abstract features back to the original image size, ultimately obtaining the segmentation result. This embodiment effectively utilizes the advantages of the U-net network in extracting shallow features and its small network structure. It can achieve invariance and robustness with a small dataset, resulting in higher image pixel accuracy and providing a better recognition foundation for subsequent image segmentation.

[0052] The lubricating oil abrasive detection model obtained through the above method can intuitively understand the training status of the model after evaluating its training effect. It effectively solves the problem that the ferrographic image samples of lubricating oil abrasives are inherently scarce, and the gray values ​​of the identified abrasives are close to the background, making them difficult to identify and segment.

[0053] Step S300: Preprocess the ferrography image to be analyzed, and then input the preprocessed ferrography image into the lubricating oil abrasive detection model to obtain the identified abrasive image contours.

[0054] In practice, the purpose of preprocessing the ferrography image to be analyzed is to enhance image contrast, improve image quality, and reduce the difficulty of segmentation. This preprocessing may include one or more of the following: denoising, enhancement, sharpening, restoration, and filtering. These are all commonly used image processing methods and will not be elaborated upon here. Of course, those skilled in the art can use other preprocessing methods to improve image quality instead of the methods provided in this embodiment.

[0055] Step S400: Use the Canny operator to detect the edges of the preprocessed ferrography image to obtain edge pixels. If the detected edge pixels are located within the outline of the abrasive image identified by the lubricating oil abrasive detection model, they are considered as overlapping abrasive particles, and the Canny operator is used to segment the overlapping abrasive particles. If the detected edge pixels are located outside the outline of the abrasive image identified by the lubricating oil abrasive detection model, they are considered as false detections and are removed.

[0056] In this specific implementation, the Canny operator is used for edge detection of the image, which features a low false detection rate, high positioning accuracy, and fine detected edge pixels. The edge detection using the Canny operator obtains edge pixels, which, combined with the abrasive image contours identified by the lubricating oil abrasive detection model, allows for the identification of overlapping abrasive image regions through image fusion. Using the abrasive image contours detected by the U-net network as a benchmark, segmenting the overlapping abrasive image regions yields more accurate abrasive contours, thereby effectively extracting the shape, size, and type of all abrasive particles in the image.

[0057] The engine lubricating oil abrasive image segmentation method obtained by fusing the U-net network model and the Canny operator can avoid the problems of existing technology in segmenting ferrography images, such as difficulty in segmenting overlapping abrasive particles, poor edge morphology, missegmentation between background and abrasive particles, false edges of abrasive particles, and low positioning accuracy.

[0058] Step S400: Perform contour detection on the segmented abrasive grain image to obtain the contour information of the abrasive grains in the image.

[0059] In practice, the area of ​​the abrasive grain outline and the length and width of the largest circumscribed rectangle can be calculated based on the scale of the original drawing, which will facilitate the automated analysis of the morphology, size and type of engine lubricating oil abrasive grains by technicians.

[0060] The method described above for analyzing the wear particles in engine lubricating oil can effectively and automatically extract the necessary information from ferrographic images, simplifying the ferrographic analysis process and reducing the requirements for technical personnel.

[0061] Preferably, the training set is input into the U-net network model for training, including the following steps:

[0062] The images from the training set are input into convolutional blocks and CBAM attention modules to obtain feature layers. Multiple convolutions, CBAM attention module training, and pooling are then performed to obtain a primary training feature map. This primary training feature map is then upsampled multiple times, fused with preliminary effective feature layers, and convolved to obtain a secondary training feature map. The secondary training feature map is then channel-adjusted based on the number of target categories detected in the feature map, and the prediction result is obtained using these features. The cross-entropy loss function and the DiceLoss loss function are used to obtain the loss.

[0063] Preferably, the image detection target types are divided into background and abrasive particles, that is, the number of target types detected by the feature map is 2, and the image is only distinguished between the background and abrasive particles, which facilitates identification.

[0064] Preferably, the CBAM attention module includes a channel attention mechanism module and a spatial attention mechanism module, and inputting an image into the CBAM attention module includes the following steps:

[0065] The image is input into the channel attention mechanism module for global max pooling and global average pooling to obtain two sets of first feature maps, which are then input into a shared two-layer neural network. Channel attention features are generated based on element-wise summation and sigmoid function activation. Finally, the channel attention features and the image are multiplied to generate the input feature map.

[0066] The input feature map is input into the spatial attention module, and then channel-based global max pooling and global average pooling are performed to obtain two sets of second feature maps. The two sets of second feature maps are then concatenated and convolved based on channels to reduce the dimensionality to a single channel. The spatial attention features are then generated by the sigmoid function. Finally, the spatial attention features and the input feature map are multiplied to obtain the feature layer.

[0067] The improved U-net network model based on the attention mechanism described above achieves superior training results. Specifically, under the same training conditions, using 74 images and 100 batches of training on ferrography images for training and validation, the researchers tested the original U-net network model and the improved U-net network model in this embodiment, obtaining the performance metrics as shown in the table below:

[0068] method Average pixel accuracy Target intersection and union ratio Average crossover ratio Original U-net network model 95.94 0.9028 0.9182 Improved U-net network model 96.35 0.9088 0.9227

[0069] As can be seen from the table, the U-net network model improved based on the attention mechanism in this embodiment has higher image pixel accuracy, target cross-union ratio, and average cross-union ratio than the original U-net network model. These indices further indicate that the U-net network model proposed in this embodiment has better image segmentation accuracy when segmenting abrasive images in the subsequent process.

[0070] Preferably, the steps for evaluating the training effect are as follows: calculating the Intersection over Union (IoU) of the background portion, the IoU of the abrasive portion, and the average IoU, respectively, using the following formulas:

[0071]

[0072]

[0073] Where A1 and B1 are the predicted and actual results for the abrasive particles, respectively, and A2 and B2 are the predicted and actual results for the background, respectively.

[0074] Preferably, edge detection of abrasive grain images using the Canny operator includes the following steps:

[0075] The image denoising step involves applying a Gaussian filter to the abrasive grain image to reduce noise and smooth the image.

[0076] The image gradient calculation step involves using the Sobel operator to calculate the edge gradient magnitude and direction of each pixel in the abrasive grain image; specifically, the Sobel operator is used to calculate the horizontal gradient G in both the horizontal and vertical directions. x and vertical direction G y Then, the edge gradient magnitude and gradient direction of each pixel are calculated, as shown in the following formula:

[0077]

[0078] G x = f(x,y)*Sobel x G y = f(x,y)*Sobel y ;

[0079]

[0080] The non-maximum suppression step involves a full scan of the abrasive image to remove irrelevant pixels that do not form edges, followed by an analysis of whether each pixel is a local maximum among its neighboring pixels along the gradient direction. Specifically, a template, such as a 3x3 template, is used, with the comparator center point and its adjacent points along the gradient direction within the template. If the gradient magnitude of the center point is not greater than the gradient magnitude of its neighboring points along the gradient direction, it is set to zero; otherwise, the pixel is considered a local maximum and is retained.

[0081] The dual threshold filtering step involves selecting a preset high threshold and a preset low threshold, and determining the relationship between the edge gradient magnitude of each pixel and the preset high threshold and preset low threshold.

[0082] If the edge gradient magnitude is greater than the preset high threshold, the pixel is determined to be an edge pixel; if the edge gradient magnitude is lower than the preset low threshold, the pixel is determined to be a non-edge pixel and is discarded; if the edge gradient magnitude is between the preset low threshold and the preset high threshold, the pixel is determined to be a pixel to be determined, and then the continuity of the pixel to be determined is judged. If it is connected to an edge pixel, it is determined to be an edge pixel; otherwise, it is discarded.

[0083] The method described above, which uses the Canny operator for edge detection, can accurately detect the edge pixels of abrasive particles in an image, and ensures a low false detection rate and high positioning accuracy during the detection process.

[0084] The following example, using a specific ferrographic image analysis process of engine lubricating oil abrasive particles, further illustrates the abrasive particle image analysis method of the present invention.

[0085] (1) First, as Figure 5 As shown, the ferrographic images of 24 engine lubricating oil particles were preprocessed, and as follows... Figure 6 As shown, images of lubricating oil abrasive particles in the middle region are cropped to construct a dataset, which is then divided into a training set and a validation set, with 20 images in the training set and 4 images in the validation set.

[0086] (2) Input the training set into the U-net model for training, such as Figure 4 As shown, the specific training process is as follows:

[0087] 1) Input the original image [512,512,3] (width, height, channels) into convolution block 1. The convolution block contains two 3×3 64-channel convolutions to obtain a preliminary effective feature layer of [512,512,64]. Then perform 2×2 max pooling to obtain a feature layer of [256,256,64]. Input the feature layer into the CBAM attention module to obtain a feature layer of [256,256,64].

[0088] The CBAM attention mechanism is derived by concatenating a Channel Attention Module (CAM) and a Spartial Attention Module (SAM).

[0089] The training process for its channel attention mechanism is as follows: The input feature map (size H (height) × W (width) × C (channels)) is processed by global max pooling and global average pooling based on width and height, respectively, to obtain two 1×1×C (channel) first feature maps. These are then fed into a two-layer neural network (MLP). The first layer has C / r neurons (r is the reduction rate), using the ReLU function as the activation function. The second layer has C neurons, and these two layers are shared. The MLP output features are then summed element-wise, followed by activation using the sigmoid function to generate the final channel attention features. Finally, these features are multiplied element-wise with the input feature map to generate the input feature map required by the spatial attention module.

[0090] The training process of its spatial attention mechanism is as follows: The input feature map generated by the channel attention module is input into this mechanism. First, a channel-based global max pooling and global average pooling are performed to obtain two H×W×1 second feature maps. Then, these two second feature maps are concatenated based on channels. Next, a 7×7 convolution operation is performed to reduce the dimensionality to one channel, i.e., H×W×1. Then, the spatial attention feature is generated by the sigmoid function. Finally, the spatial attention feature is multiplied by the input feature map of this mechanism to obtain the final generated feature layer.

[0091] 2) Input the feature layer [256,256,64] into convolution block 2. The convolution block contains two 3×3 128-channel convolutions. Input the result into the CBAM attention module to obtain the feature map [256,256,128]. Perform a 2×2 pooling step to obtain the feature map [128,128,128].

[0092] 3) Input the feature layer [128,128,128] into convolution block 3. The convolution block contains three 3×3 256-channel convolutions. Input the result into the CBAM attention module to obtain the feature map [128,128,256]. Perform a 2×2 pooling step to obtain the feature map [64,64,256].

[0093] 4) Input the feature layer [64,64,256] into convolution block 4. The convolution block contains three 3×3 512-channel convolutions. Input the result into the CBAM attention module to obtain the feature map [64,64,512]. Perform a 2×2 pooling step to obtain the feature map [32,32,512].

[0094] 5) Input the feature layer [32,32,512] into convolution block 5. The convolution block contains three 3×3 512-channel convolutions to obtain a preliminary effective feature layer [32,32,512]. Upsample it to obtain an image [64,64,512], and fuse it with the feature layer extracted by the corresponding attention module to obtain an image [64,64,1024]. Then input it into convolution block 6 for two 3×3 512-channel convolutions to obtain an image [64,64,512].

[0095] 6) Upsample the image [64,64,512] to obtain the image [128,128,512], and perform a skip connection with the feature layer extracted by the corresponding attention module to obtain the image [128,128,768]. Then input it into convolution block 7 for two 3×3 256-channel convolutions to obtain the image [128,128,256].

[0096] 7) Upsample the image [128,128,256] to obtain the image [256,256,256], and fuse it with the feature layer extracted by the corresponding attention module to obtain the image [256,256,384]. Then input it into convolution block 8 for two 3×3 128-channel convolutions to obtain the image [256,256,128].

[0097] 8) Upsample the image [256,256,128] to obtain the image [512,512,128], and fuse it with the feature layer extracted by the corresponding attention module to obtain the image [512,512,192]. Then input it into convolution block 9 for two 3×3 64-channel convolutions to obtain the image [512,512,64].

[0098] 9) Perform 1×1 convolution to adjust the channels. The number of channels is the number of target types to be detected (in this method, only background and abrasive particles are distinguished, so it is 2). The prediction results are obtained using the features.

[0099] During the entire training process, the loss can be obtained through the cross-entropy loss function and the Dice loss function. The formula for the cross-entropy loss function is as follows:

[0100]

[0101] Where Loss is the loss value, and y is the actual result. For the predicted results;

[0102] The formula for the Dice Loss function is:

[0103]

[0104] Where Loss is the loss value, X is the predicted result, and Y is the actual result;

[0105] (3) Based on the idea of ​​transfer learning, the U-net network is frozen for 50 batches without changing the feature extraction network, preventing damage to the weights of the backbone network, saving training time and computing power. Then, 50 unfrozen training cycles are performed to adjust all parameters.

[0106] (4) Obtain the trained lubricating oil abrasive detection model and test it on the validation set. Calculate the cross-union ratio (IOU) of the background part, the cross-union ratio (IOU) of the abrasive part, and the average cross-union ratio (mIOU) to evaluate the training effect.

[0107] The specific formula is as follows:

[0108]

[0109]

[0110] Where A1 and B1 are the predicted and actual results for the abrasive particles, respectively, and A2 and B2 are the predicted and actual results for the background, respectively.

[0111] (5) Preprocess the ferrography image to be analyzed, and then input the preprocessed ferrography image into the lubricating oil abrasive detection model to obtain the identified abrasive image contours, specifically as follows: Figure 7 As shown.

[0112] (6) Then, the Canny operator is used to perform edge detection on the preprocessed ferrography image, and the resulting image is as follows: Figure 8 As shown, edge detection using the Canny operator includes: image denoising, image gradient calculation, non-maximum suppression, and dual threshold filtering. For details, please refer to the aforementioned steps, which will not be elaborated here.

[0113] (7) If the detected edge pixel is located within the outline of the abrasive image identified by the lubricating oil abrasive detection model, it is considered an overlapping abrasive, and the Canny operator is used to segment the overlapping abrasive; if the detected edge pixel is located outside the outline of the abrasive image identified by the lubricating oil abrasive detection model, it is considered a false detection and is removed. The specific recognition formula is as follows:

[0114]

[0115] Where X1(i,j), X2(i,j), and x3(i,j) are the pixel values ​​at coordinates (i,j) of the U-net segmentation result image, the Canny edge detection result image, and the final segmentation result image, respectively (where 1 represents the abrasive part and 0 represents the background part), i∈[0,w],j∈[0,h].

[0116] (8) The segmented abrasive grain image is as follows Figure 9 As shown, a contour detection is then performed on it, and the area of ​​the contour, the length and width of the largest circumscribed rectangle, and other parameters are calculated based on the scale of the original image.

[0117] The present invention also provides an engine lubricating oil abrasive particle image analysis device, comprising:

[0118] The dataset construction module acquires ferrography images and crops them to construct a dataset, and then divides the dataset into a training set and a validation set;

[0119] The training module constructs a U-net network model and inputs the training set into the U-net network model for training to obtain a trained lubricating oil abrasive detection model; then, the validation set is input into the lubricating oil abrasive detection model for testing, and the training effect is evaluated.

[0120] The segmentation module preprocesses the ferrography image to be analyzed, and then inputs the preprocessed ferrography image into the lubricating oil abrasive detection model to obtain the identified abrasive image contours. The Canny operator is used to detect the edges of the preprocessed ferrography image to obtain edge pixels. If the detected edge pixels are located within the abrasive image contours identified by the lubricating oil abrasive detection model, they are considered overlapping abrasive particles, and the Canny operator is used to segment the overlapping abrasive particles. If the detected edge pixels are located outside the abrasive image contours identified by the lubricating oil abrasive detection model, they are considered false detections and are removed.

[0121] The analysis module performs contour detection on the segmented abrasive grain image to obtain the contour information of the abrasive grains in the image.

[0122] By employing the aforementioned engine lubricating oil abrasive particle image analysis device, the edges of overlapping abrasive particles can be effectively identified, improving the accuracy of detection and analysis, which is of great significance for analyzing the engine's operating status.

[0123] The present invention also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the engine lubricating oil abrasive particle image analysis method as described in any of the above embodiments.

[0124] In specific implementations, computer-readable storage media may include magnetic disks, optical disks, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drives (HDDs), or solid-state drives (SSDs); computer-readable storage media may also include combinations of the above types of memory.

[0125] The present invention also provides an electronic device, including at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the processor to perform the engine lubricating oil abrasive particle image analysis method as described in any of the above embodiments.

[0126] In practice, the number of processors can be one or more, and the processor can be a Central Processing Unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips. A general-purpose processor can be a microprocessor or any conventional processor.

[0127] The memory and the processor can be connected to communicate via a bus or other means. The memory stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor to cause the processor to perform the engine lubricating oil abrasive particle image analysis method as described in any of the above embodiments.

[0128] In summary, compared with the prior art, the engine lubricating oil abrasive particle image analysis method, apparatus, medium, and equipment provided by the present invention have the following advantages:

[0129] (1) The advantages of U-net network in extracting shallow features and small network structure are effectively utilized to train the lubricating oil abrasive detection model. Further improvements are made to the U-net network model to obtain better training results, so that the lubricating oil abrasive detection model can extract better image accuracy.

[0130] (2) Use the Canny operator to perform edge detection on the image, which makes it easier to segment the overlapping area of ​​the abrasive particles, and then extract the shape, size and type of all abrasive particles on the image.

[0131] (3) The engine lubricating oil abrasive image segmentation method obtained by integrating the U-net network model and the Canny operator can avoid the problems of existing technology in segmenting ferrography images, such as difficulty in segmenting overlapping abrasive particles, poor edge morphology, missegmentation between background and abrasive particles, false edges of abrasive particles, and low positioning accuracy.

[0132] (4) The method developed based on neural network technology can automatically extract the required information from ferrography images, simplify the ferrography analysis process, and reduce the requirements for technical personnel.

[0133] Furthermore, those skilled in the art should understand that although many problems exist in the prior art, each embodiment or technical solution of the present invention can be improved in only one or a few aspects, without necessarily solving all the technical problems listed in the prior art or the background art simultaneously. Those skilled in the art should understand that any content not mentioned in a claim should not be construed as a limitation on that claim.

[0134] Although this paper frequently uses terms such as ferrography image, dataset, U-net network model, lubricating oil abrasive detection model, Canny operator, edge pixel, CBAM attention module, and intersection-over-union ratio, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of the invention; interpreting them as any additional limitation would contradict the spirit of the invention. The terms "first," "second," etc. (if present) in the specification, claims, and accompanying drawings of the embodiments of the invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An engine oil wear particle image analysis method, characterized by, The method comprises the following steps: Step S100, acquiring an iron spectrum image and performing cropping to construct a data set, and then dividing the data set into a training set and a verification set; Step S200, constructing a U-net network model, inputting the training set into the U-net network model for training to obtain a trained lubricating oil abrasive particle detection model, inputting the verification set into the lubricating oil abrasive particle detection model for testing, and evaluating the training effect; Step S300, preprocessing an iron spectrum image to be analyzed, and inputting the preprocessed iron spectrum image into the lubricating oil abrasive particle detection model to obtain a recognized abrasive particle image contour; Step S400, detecting the edge of the preprocessed iron spectrum image using a Canny operator to obtain edge pixels, regarding the detected edge pixels as overlapping abrasive particles if the edge pixels are located within the abrasive particle image contour recognized by the lubricating oil abrasive particle detection model, segmenting the overlapping abrasive particles using the Canny operator, regarding the detected edge pixels as false detection if the edge pixels are located outside the abrasive particle image contour recognized by the lubricating oil abrasive particle detection model, and removing the false detection; Step S500, performing contour detection on the segmented abrasive particle image to obtain contour information of the abrasive particles on the image.

2. The engine oil wear particle image analysis method of claim 1, wherein: The preprocessing of the iron spectrum image to be analyzed comprises one or more of denoising, enhancement, sharpening, restoration and filtering.

3. The engine oil wear particle image analysis method of claim 1, wherein: The training of the training set in the U-net network model comprises the following steps: inputting the image of the training set into a convolution block and a CBAM attention module to obtain a feature layer, and performing multiple times of convolution, CBAM attention module training and pooling to obtain a first training feature map; performing multiple times of upsampling, fusing a preliminary effective feature layer and convolution on the first training feature map to obtain a second training feature map, adjusting the channels of the second training feature map according to the number of target types of the feature map, and obtaining a prediction result by using the feature. The cross-entropy loss function and the DiceLoss loss function are used to obtain the loss.

4. The engine oil wear particle image analysis method of claim 3, wherein: The CBAM attention module comprises a channel attention mechanism module and a spatial attention mechanism module, and inputting the image into the CBAM attention module comprises the following steps: inputting the image into the channel attention mechanism module to obtain two groups of first feature maps through global maximum pooling and global average pooling, inputting the two groups of first feature maps into two shared neural networks respectively, generating channel attention features based on element-wise addition operation and sigmoid function activation operation, and finally performing multiplication operation on the channel attention features and the image to generate an input feature map; inputting the input feature map into the spatial attention mechanism module, performing global maximum pooling and global average pooling based on the channel to obtain two groups of second feature maps, performing channel splicing operation and convolution on the two groups of second feature maps based on the channel to reduce the dimension to a single channel, generating spatial attention features through a sigmoid function, and finally performing multiplication operation on the spatial attention features and the input feature map to obtain a feature layer.

5. The engine oil wear particle image analysis method of claim 3, wherein: The target types of the image are divided into a background part and an abrasive particle part.

6. The engine oil wear particle image analysis method of claim 5, wherein: The step of evaluating the training effect is to calculate the intersection over union (IOU) of the background part, the intersection over union (IOU) of the abrasive grain part and the average intersection over union (mIOU) respectively, and the formula is as follows: Wherein A1 and B1 are the prediction result and the true result of the abrasive grain part respectively, and A2 and B2 are the prediction result and the true result of the background part respectively.

7. The engine oil wear particle image analysis method of claim 1, wherein The edge of the abrasive grain image is detected by using the Canny operator, including the following steps: The image noise reduction step is to perform Gaussian filtering processing on the abrasive grain image. The image gradient calculation step is to calculate the edge gradient amplitude and gradient direction of each pixel in the abrasive grain image by using the Sobel operator. The non-maximum suppression step is to comprehensively scan the abrasive grain image to remove irrelevant pixel points that do not constitute an edge, and then detect whether each pixel is a local maximum value in the adjacent pixel points in the gradient direction, if yes, it is retained, otherwise, it is removed. The double-threshold screening step is to select a preset high threshold and a preset low threshold, and determine the size relationship between the edge gradient amplitude of each pixel and the preset high threshold and the preset low threshold. If the edge gradient amplitude is greater than the preset high threshold, the pixel is determined as an edge pixel; if the edge gradient amplitude is lower than the preset low threshold, the pixel is determined as a non-edge pixel and discarded; if the edge gradient amplitude is between the preset low threshold and the preset high threshold, the pixel is determined as a to-be-determined pixel, and then the continuity of the to-be-determined pixel is judged, if it is connected with an edge pixel, it is determined as an edge pixel, otherwise, it is discarded.

8. An engine oil wear particle image analysis device characterized by comprising: It includes: A data set construction module acquires an iron spectrum image and performs cropping to construct a data set, and then divides the data set into a training set and a validation set; A training module constructs a U-net network model, inputs the training set into the U-net network model for training to obtain a trained lubricating oil abrasive grain detection model, and then inputs the validation set into the lubricating oil abrasive grain detection model for testing and evaluating the training effect; A segmentation module pre-processes an iron spectrum image to be analyzed, inputs the pre-processed iron spectrum image into the lubricating oil abrasive grain detection model to obtain a recognized abrasive grain image contour, uses a Canny operator to detect the edge of the pre-processed iron spectrum image to obtain edge pixels, if the detected edge pixels are located within the abrasive grain image contour recognized by the lubricating oil abrasive grain detection model, they are regarded as overlapping abrasive grains, and the Canny operator is used to segment the overlapping abrasive grains, and if the detected edge pixels are located outside the abrasive grain image contour recognized by the lubricating oil abrasive grain detection model, they are regarded as false positives and are removed; An analysis module performs contour detection on the segmented abrasive grain image to obtain the contour information of the abrasive grains on the image.

9. A computer-readable storage medium, characterized in that: The computer readable storage medium stores computer instructions, and the computer is executed by the processor to realize the engine lubricating oil abrasive grain image analysis method of any one of claims 1-7.

10. An electronic device, comprising: An engine oil wear particle image analysis method as claimed in any one of claims 1-7, comprising at least one processor, and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the processor to perform the engine oil wear particle image analysis method.

Images