Ferromagnetic abrasive particle identification method and device based on multi-dimensional communication aggregation mechanism, medium

Abstract

This invention relates to a method, apparatus, and medium for identifying ferromagnetic abrasive particles based on a multidimensional communication aggregation mechanism. The method acquires an image to be detected, converts the image to a preset format, inputs it into a pre-trained neural network model, acquires output data, and obtains a recognition result based on the output data. The acquisition process of the pre-trained neural network model includes the following steps: acquiring multiple original ferrospectral images; for each original ferrospectral image, acquiring an aggregated image; generating a training sample set based on the multiple original ferrospectral images and the corresponding aggregated images; training the neural network model based on the training sample set; and obtaining the pre-trained neural network model after the loss function value reaches a preset convergence condition. Compared with existing technologies, this invention provides a larger receptive field for the neural network model while reducing computational resource consumption and accelerating the recognition speed.

Description

Technical Field

[0001] This invention relates to the field of equipment fault diagnosis and condition monitoring, and in particular to a method, device, and medium for identifying ferromagnetic abrasive particles based on a multidimensional communication aggregation mechanism. Background Technology

[0002] Ferrography is a technique for extracting and observing wear particles in a lubrication system. By analyzing their quantity, size, shape, and texture, it determines the lubrication condition, wear mechanism, and severity of wear in friction pairs. Compared to other fault diagnosis techniques, it offers advantages such as strong foresight, a wide range of wear particle detection, and direct reflection of major wear mechanisms. However, ferrographography currently lacks automation, its application relies heavily on expert experience, and it is time-consuming and expensive. These drawbacks limit its widespread industrial application.

[0003] The limitations of offline abrasive particle analysis have driven the development of this technology towards intelligence and online processing. Intelligent ferrography analysis can circumvent these drawbacks. The morphological characteristics of wear particles are complex, including shapes such as flaky, blocky, layered, and curled; surface textures such as smooth, cracked, pitted, and scratched; and varying colors and sizes. Small particles aggregate into chains due to magnetization, overlapping and obscuring each other, making automated analysis difficult. Therefore, intelligent recognition of ferrographic abrasive particle images is currently a technological bottleneck in ferrography development. It is the core and foundation for real-time condition monitoring and fault diagnosis of mechanical equipment, and is of great significance in improving both the reliability and economy of mechanical equipment.

[0004] To address the aforementioned problems in intelligent recognition of ferrographic abrasive images, existing technologies propose using deep neural networks, especially convolutional neural networks and their derivative models developed in recent years. Deep neural networks are well-suited for solving the two problems mentioned above. First, deep neural networks directly input raw data into the network and learn from the training data, avoiding manual feature extraction. Second, deep neural networks can be used for foreground target localization in images, thus possessing the potential to segment overlapping particles. However, research on its application to wear particle image recognition and detection remains limited. Peng P et al. used a relatively simple convolutional neural network, FECNN, and trained it with 420 wear particle images, achieving good accuracy, but did not explore the problem of abrasive segmentation in complex backgrounds. Peng Y et al. also developed a similar model, combining CNN and SVM to identify and classify cutting abrasives, spherical abrasives, and fatigue abrasives, but the samples used were still single-target images, and as can be seen above, spherical abrasives and cutting abrasives have more obvious features and are relatively easy to distinguish. In a study at Zhejiang University, CNN could identify whether a certain type of particle was present in an image, but it could not accurately mark the location of the particles in the image. Cai Junhao et al. from Shenzhen University proposed a research method for target detection and recognition based on ferrography images using hybrid convolutional neural networks. The method achieved good results when applied to small-sized, low-resolution abrasive images, but it still has shortcomings such as low accuracy in classifying similar particles and lack of ability to mark abrasive particle contours.

[0005] In summary, deep learning is very powerful in solving problems such as image segmentation and image recognition, and has already entered the practical application stage in many fields. It has broad prospects for application in intelligent analysis of wear particles. However, so far, research on intelligent identification of wear particles using deep neural networks at home and abroad still has the following two problems: (1) Research on deep learning technology in the field of ferrography images is an interdisciplinary study. Therefore, the depth of research on related dedicated models is very limited, and the accuracy of model recognition is relatively low; (2) Although new complex model training methods have improved the accuracy of classification, they have also significantly increased the computational cost of the model. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art by providing a method, device, and medium for identifying ferromagnetic abrasive particles based on a multidimensional communication aggregation mechanism. By adopting a multidimensional communication aggregation mechanism, the receptive field in each dimension is expanded, which is beneficial for the neural network model to fully extract information while reducing the amount of information processing required by the model.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] One aspect of the present invention provides a method for identifying ferromagnetic abrasive particles based on a multidimensional communication aggregation mechanism, comprising the following steps:

[0009] The process involves acquiring an image to be detected, converting it to a preset format, inputting it into a pre-trained neural network model, obtaining output data, and then obtaining the recognition result based on the output data.

[0010] The process of obtaining the pre-trained neural network model includes the following steps:

[0011] Multiple original ferrography images are acquired. For each original ferrography image, the width of the reconstructed image is obtained based on the width of the original ferrography image. Based on the width of the original ferrography image and the width of the reconstructed image, segmentation and reconstruction are performed on the channel dimension of the original ferrography image. Data of each channel after segmentation and reconstruction of the current original ferrography image are obtained. The data on each channel are placed on the corresponding channel at the same position to obtain the aggregated image.

[0012] A training sample set is generated based on the multiple original ferrography images and the corresponding aggregated images;

[0013] The neural network model is trained based on the training sample set. After the loss function value reaches the preset convergence condition, the pre-trained neural network model is obtained.

[0014] As a preferred technical solution, the width of the reconstructed image is calculated using the following formula:

[0015] New_width = Orig_height + 1

[0016] In the formula, New_width is the width of the reconstructed image, and Orig_height is the height of the original ferrography image.

[0017] As a preferred technical solution, segmentation and reconstruction in the channel dimension of the original ferrography image includes the following steps:

[0018] The R, G, and B channels of the original ferrography image are segmented and reconstructed respectively.

[0019] As a preferred technical solution, the segmentation and reconstruction of the R channel is performed using the following formula:

[0020] New_R=Orig_R[0:New_width / / 3+1]

[0021] In the formula, Orig_R is the R channel data of the original ferrography image, and New_width is the width of the reconstructed image.

[0022] As a preferred technical solution, the segmentation and reconstruction of the G channel is performed using the following formula:

[0023] New_G=Orig_R[New_width / / 3:New_width / / 3*2+1]

[0024] In the formula, Orig_R is the R channel data of the original ferrography image, and New_width is the width of the reconstructed image.

[0025] As a preferred technical solution, the segmentation and reconstruction of channel B is performed using the following formula:

[0026] New_B=Orig_R[New_width / / 3*2:New_width+1]

[0027] In the formula, Orig_R is the R channel data of the original ferrography image, and New_width is the width of the reconstructed image.

[0028] As a preferred technical solution, the neural network model is a convolutional neural network model.

[0029] As a preferred technical solution, the neural network model includes the ResNet50 network.

[0030] In another aspect, an electronic device is provided, including one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the above-described ferromagnetic abrasive particle identification method based on a multidimensional communication aggregation mechanism.

[0031] In another aspect, the present invention provides a computer-readable storage medium comprising one or more programs executable by one or more processors of an electronic device, the one or more programs comprising instructions for performing the above-described ferromagnetic abrasive particle identification method based on a multidimensional communication aggregation mechanism.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] (1) By adding multidimensional communication aggregation to the ferromagnetic abrasive particle recognition model based on multidimensional communication aggregation mechanism, the receptive field in each dimension is expanded by segmenting and reconstructing each channel of the input image. Compared with the optimal model of traditional artificial intelligence, it has higher accuracy and recall.

[0034] (2) The data segmentation and reconstruction method provided by this invention ensures information communication between each dimension and the previous dimension, while only requiring the original Figure 1 The information content of 3 / 3 was used to reconstruct a new ferrography image, reducing the computing power of the computer and speeding up the recognition process. Attached Figure Description

[0035] Figure 1 A schematic diagram of the multidimensional communication aggregation convolution model provided in Example 1;

[0036] Figure 2 This is a schematic diagram of the original three-channel image in Example 1;

[0037] Figure 3 The original image of the R channel in Example 1;

[0038] Figure 4 The original image of the G channel in Example 1;

[0039] Figure 5 This is the original image of channel B in Example 1;

[0040] Figure 6 The image shown is the R-channel image after segmentation and reconstruction in Example 1;

[0041] Figure 7 The image shown is the segmented and reconstructed G-channel image from Example 1.

[0042] Figure 8 The image shown is the B-channel image after segmentation and reconstruction in Example 1;

[0043] Figure 9 This is the new three-channel original image in Example 1;

[0044] Figure 10 This is a comparative diagram of MCFCNN, LMCNN, SBCNN, and the traditional optimal artificial intelligence model in Example 1;

[0045] Figure 11 This is a comparison of the average performance of MCFCNN, LMCNN, SBCNN, and the traditional optimal artificial intelligence model in Example 1;

[0046] Figure 12 This is a schematic diagram of the receptive field and convolution process in a traditional artificial intelligence model.

[0047] Figure 13 This is a schematic diagram of the receptive field and computation process of a multidimensional communication aggregation mechanism. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0049] Example 1

[0050] This embodiment provides a method for identifying ferromagnetic abrasive particles based on a multidimensional communication aggregation mechanism, including the following steps:

[0051] The process involves acquiring the image to be detected, converting it to a preset format, inputting it into a pre-trained ferromagnetic abrasive particle recognition model based on a multi-dimensional communication aggregation mechanism, obtaining the output data, and then obtaining the recognition result based on the output data.

[0052] The ferromagnetic abrasive particle identification model includes:

[0053] The multidimensional communication aggregation module is used to reconstruct the width of the image based on the input image of the ferromagnetic abrasive particle recognition model, and to segment and reconstruct each channel of the input image based on the input image and the reconstructed width to obtain aggregated data information.

[0054] The neural network module is used to obtain output data based on the input image of the ferromagnetic abrasive particle recognition model and the aggregated data information.

[0055] This embodiment designs a multidimensional communication aggregation algorithm. This algorithm needs to maximize the receptive field in each dimension based on the communication mechanism. Therefore, it no longer focuses on the positional information of the image. Specifically, it retains only one column of image overlap in each dimension, thereby performing image transformation on an original ferrography image. The specific algorithm is as follows:

[0056] The new width reconstruction of the ferrographic image is as follows:

[0057] New_width = Orig_height + 1

[0058] The segmentation and reconstruction process on the R channel of the image is as follows:

[0059] New_R=Orig_R[0:New_width / / 3+1]

[0060] The segmentation and reconstruction process on the G channel of the image is as follows:

[0061] New_G=Orig_R[New_width / / 3:New_width / / 3*2+1]

[0062] The segmentation and reconstruction process on the B channel of the image is as follows:

[0063] New_B=Orig_R[New_width / / 3*2:New_width+1]

[0064] Wherein, New_width is the reconstructed width, and Orig_R is the R channel data of the input image.

[0065] This algorithm only requires dividing the data in one R channel of an image into three parts and arranging them respectively in the R, G, and B three-dimensional channels. This data rearrangement method ensures information communication between each dimension and the previous dimension, while utilizing only the original... Figure 1 The information content of 3 / 3 is used to reconstruct a new ferrographic image. The specific operational effect is as follows: Figure 2-9 As shown, Figure 3 , Figure 4 and Figure 5 The original R, G, and B channel information are displayed respectively. Figure 6 , Figure 7 and Figure 8 This demonstrates how information from the original R channel is reconstructed into new R, G, and B channels using a multidimensional communication aggregation mechanism. Figure 2 It is the original image. Figure 9 It is a new 3-channel image.

[0066] like Figure 1 The above is a schematic diagram of the structure of the ferromagnetic abrasive particle recognition model, namely the multidimensional communication aggregation convolutional neural network model (CMFCNN), which includes CMF Blook, ResNet50 and output layer Output, wherein CMF Blook has built-in instructions of the above-mentioned multidimensional communication aggregation algorithm.

[0067] Because ferrography images have complex morphological features, varying colors, different sizes, and overlapping and occlusion, human recognition requires a comprehensive judgment based on the overall image, rather than focusing solely on local information. Therefore, ferrography image recognition needs to provide the model with a larger receptive field.

[0068] like Figure 12 The above is a schematic diagram of the receptive field of a traditional artificial intelligence model. Figure 5 It can be seen that the model has significant information redundancy in the receptive fields of the three channels (from the human perspective, the image information in the three channels is basically the same). This redundancy wastes the computing power of the computer and is not conducive to the model extracting comprehensive information.

[0069] like Figure 13 The diagram illustrates the receptive field and computation process of the multidimensional communication aggregation mechanism in this embodiment. The original image's one-dimensional data is divided into three parts along its width (maintaining overlap of one column of information), and then evenly distributed across the three channel dimensions to form a reconstructed image. This method not only reduces the model's information processing load to one-third of the original, avoiding redundant information processing, but also increases the model's local receptive field, expands the model's information richness across the three dimensions, and retains overlap of one column of information to ensure the model's memory of the original image's positional information.

[0070] Since the image in each dimension is significantly cropped, the multidimensional communication aggregation mechanism can achieve higher model performance with less computational power consumption. This also provides a powerful solution for the large computational power consumption required when selecting other complex models in the field of ferrography image recognition in the future.

[0071] like Figure 10 , 11 Compared to MCFCNN, LMCNN, SBCNN, and traditional optimal AI models, this algorithm constructs a new image based on 1 / 3 of the information in the original image, reducing the model training computation by 2 / 3. The CMF Block, or Multidimensional Communication Aggregation Algorithm Module, processes the original image into... Figure 9 The reconstructed image shown, together with the original ferrography image, constitutes the training set and participates in the subsequent training process of the ResNet50 model. After the model completes training, CMFBlock will not participate in the inference phase, thereby reducing the computational cost during inference. Its inference performance advantages are as follows: Figure 10 , 11 As shown.

[0072] Thus, the method for ferromagnetic abrasive particle identification based on a multi-dimensional communication aggregation mechanism can achieve superior ferrospectral image-specific model training compared to traditional artificial intelligence models while reducing model computation by 2 / 3. After averaging the three-channel data, the data from these three channels are placed in the same position across different channels, thereby reducing the overall image data size and redundant computation. Furthermore, the cropping operation transforms a receptive field from sensing similar information at the same position across all three channels to sensing completely different information at the same position, thus enhancing the receptive field's perceptual capabilities.

[0073] Example 2

[0074] This embodiment provides an electronic device, including one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the ferromagnetic abrasive particle identification method based on the multidimensional communication aggregation mechanism in Embodiment 1.

[0075] Example 3

[0076] This embodiment provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for executing the ferromagnetic abrasive particle identification method based on a multidimensional communication aggregation mechanism as described in Embodiment 1.

[0077] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for identifying ferromagnetic abrasive particles based on a multidimensional communication aggregation mechanism, characterized in that, Includes the following steps: The process involves acquiring an image to be detected, converting it to a preset format, inputting it into a pre-trained neural network model, obtaining output data, and then obtaining the recognition result based on the output data. The process of obtaining the pre-trained neural network model includes the following steps: Multiple original ferrography images are acquired. For each original ferrography image, the width of the reconstructed image is obtained based on the width of the original ferrography image. Based on the width of the original ferrography image and the width of the reconstructed image, segmentation and reconstruction are performed on the channel dimension of the original ferrography image. Data of each channel after segmentation and reconstruction of the current original ferrography image are obtained. The data on each channel are placed on the corresponding channel at the same position to obtain the aggregated image. A training sample set is generated based on the multiple original ferrography images and the corresponding aggregated images; The neural network model is trained based on the training sample set. Once the loss function value reaches a preset convergence condition, the pre-trained neural network model is obtained. The width of the reconstructed image is calculated using the following formula: In the formula, The width of the reconstructed image. The height of the original ferrography image. Segmentation and reconstruction along the channel dimension of the original ferrography image includes the following steps: The R, G, and B channels of the original ferrography image were segmented and reconstructed respectively. The segmentation and reconstruction of the R channel is performed using the following formula: In the formula, The R channel data of the original ferrography image. The width of the reconstructed image. The segmentation and reconstruction of the G channel is performed using the following formula: In the formula, The R channel data of the original ferrography image. The width of the reconstructed image. The segmentation and reconstruction of channel B is performed using the following formula: In the formula, The R channel data of the original ferrography image. The width of the reconstructed image.

2. The ferromagnetic abrasive particle identification method based on a multidimensional communication aggregation mechanism according to claim 1, characterized in that, The neural network model described is a convolutional neural network model.

3. The ferromagnetic abrasive particle identification method based on a multidimensional communication aggregation mechanism according to claim 1, characterized in that, The neural network model mentioned includes the ResNet50 network.

4. An electronic device, characterized in that, It includes one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the ferromagnetic abrasive particle identification method based on the multidimensional communication aggregation mechanism as described in any one of claims 1-3.

5. A computer-readable storage medium, characterized in that, It includes one or more programs that are executed by one or more processors of an electronic device, the one or more programs including instructions for executing the ferromagnetic abrasive particle identification method based on the multidimensional communication aggregation mechanism as described in any one of claims 1-3.

Images