Motor housing rust spot identification method, device and equipment and computer readable medium

Abstract

The application relates to a motor shell rust spot identification method, device, equipment and computer readable medium. The method comprises the following steps: acquiring a first motor shell image before a salt spray test and a second motor shell image after the salt spray test; performing image registration on the first motor shell image and the second motor shell image to register a rust spot area in the first motor shell image and a rust spot area in the second motor shell image and obtain an effective motor shell image; performing difference detection on the effective motor shell images before and after the salt spray test, extracting a new rust spot area based on the difference detection result; constructing a rust spot identification model according to the new rust spot area and the corresponding motor shell image; identifying the rust spot type of the new rust spot area based on the rust spot identification model, and calculating the area corresponding to the new rust spot area. The application solves the problem that manual motor shell rust spot identification detection leads to low detection efficiency and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of rust spot identification, and in particular to a motor shell rust spot identification method, device, equipment and computer readable medium. BACKGROUND

[0002] In the field of motor manufacturing and maintenance, the corrosion resistance of the motor shell is a crucial performance indicator, directly related to the service life and reliability of the motor. Especially in harsh conditions such as marine environment and chemical industry, the motor shell is exposed to salt-containing humid air for a long time, which is prone to salt spray corrosion and thus rust spots. Rust spots not only affect the appearance of the motor, but more seriously, they gradually erode the shell material, leading to a decrease in structural strength and ultimately affecting the normal operation of the motor. Therefore, accurately and quickly identifying and quantifying the rust spots on the motor shell is of great significance for assessing the state of the motor and developing maintenance plans.

[0003] Currently, the detection and identification of rust spots on the motor shell mainly rely on manual visual inspection methods. This method requires the inspector to use experience to visually observe the shell surface to determine the presence or absence of rust spots and roughly estimate their area and type (such as red rust spots, white rust spots, etc.). Although this method is simple to implement, it is inefficient and inaccurate when faced with large-scale and high-frequency detection requirements.

[0004] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0005] The present application provides a motor shell rust spot identification method, device, equipment and computer readable medium to solve the technical problem of low detection efficiency and accuracy of manual detection of motor shell rust spot detection.

[0006] According to one aspect of the embodiments of the present application, the present application provides a motor shell rust spot identification method, comprising: acquiring a first motor shell image before a salt spray test and a second motor shell image after the salt spray test; performing image registration on the first motor shell image and the second motor shell image to register the rust spot area in the first motor shell image and the rust spot area in the second motor shell image to obtain an effective motor shell image; performing difference detection on the effective motor shell images before and after the salt spray test, and extracting a new rust spot area based on the difference detection result; constructing a rust spot identification model according to the new rust spot area and the corresponding motor shell image; identifying the rust spot type of the new rust spot area based on the rust spot identification model, and calculating the area corresponding to the new rust spot area.

[0007] Optionally, the first motor shell image and the second motor shell image are image-registered to register the rust area in the first motor shell image and the rust area in the second motor shell image to obtain an effective motor shell image, including: performing feature point detection on the first motor shell image and the second motor shell image respectively by a scale-invariant feature transform algorithm to obtain a first detection result and a second detection result, extracting a corresponding first descriptor based on the first detection result, and extracting a corresponding second descriptor based on the second detection result; combining the first detection result and the first descriptor, the second detection result and the second descriptor, and matching the first motor shell image and the second motor shell image by a brute force matcher to obtain a similar feature point pair; calculating a transformation matrix based on the similar feature point pair, and transforming the first motor shell image into a coordinate system corresponding to the second motor shell image based on the transformation matrix; and transforming the first motor shell image and the second motor shell image respectively based on the transformation matrix to realize image registration and obtain an effective motor shell image, the effective motor shell image including a first effective motor shell image before a salt spray test and a second effective motor shell image after the salt spray test.

[0008] Optionally, the effective motor shell images before and after the salt spray test are subjected to difference detection, and a newly added rust area is extracted based on a difference detection result, including: extracting a first pixel feature corresponding to the first effective motor shell image and a second pixel feature corresponding to the second effective motor shell image; performing pixel subtraction on the first effective motor shell image and the second effective motor shell image based on the first pixel feature and the second pixel feature to obtain a difference image; segmenting the difference image according to a preset threshold to obtain a corresponding binary image; and extracting the newly added rust area based on the binary image and labeling the newly added rust area.

[0009] Optionally, a rust recognition model is constructed according to the newly added rust area and the corresponding motor shell image, including: generating a pixel mask corresponding to the newly added rust area; setting a data path and traversing the second motor shell image, reading a pixel mask corresponding to each second motor shell image and returning in an array form; preprocessing the second motor shell image and the pixel mask; setting a data enhancement parameter and configuring a data enhancement logic based on the data enhancement parameter; iteratively running the data enhancement logic to perform data enhancement on the second motor shell image and the pixel mask, and returning the second motor shell image and the pixel mask after data enhancement; and training a U-Net model using the second motor shell image and the pixel mask after data enhancement to obtain a rust recognition model.

[0010] Optionally, the size of the second motor housing image and the pixel mask is adjusted to the target size by image scaling; a normalization factor is calculated using the maximum and minimum pixel values ​​in the pixel array corresponding to the second motor housing image, and each pixel value in the pixel value array corresponding to the second motor housing image is normalized using the normalization factor; the mask array corresponding to the pixel mask is dimensionally expanded so that the dimension of the mask array reaches the target dimension.

[0011] Optionally, a rust spot recognition model is obtained by training a U-Net model using the data-augmented second motor housing image and pixel mask, including: using the U-Net model as a base model and adding a global average pooling layer, a fully connected layer, and an output layer of the base model to obtain an initial model; setting the optimizer, loss function, and evaluation metric corresponding to the initial model; training the model by using the second motor housing image as the input of the initial model and the pixel mask corresponding to the rust spot region as the output of the initial model; and iteratively training the initial model using the optimizer and the loss function until the output of the initial model meets the evaluation metric to obtain the rust spot recognition model.

[0012] Optionally, identifying the rust type of the newly added rust area based on the rust recognition model and calculating the area corresponding to the newly added rust area includes: acquiring a real-time motor housing image; segmenting the real-time motor housing image using the rust recognition model; identifying and extracting the newly added rust area; performing morphological operations on the newly added rust area, including dilation and erosion, to eliminate noise, fill holes, and preserve the main structure of the rust; converting the real-time motor housing image from the RGB color space to the Lab color space, and distinguishing between red and white rust spots based on the corresponding color components; and using a convolutional neural network to process the real-time motor housing image in the Lab color space. Feature extraction is performed to obtain the color, texture, and shape features of the newly added rust spots. A multi-label classification model is constructed by combining the color, texture, and shape features of the newly added rust spots to simultaneously output the probabilities of red and white rust spots. The multi-label classification model outputs the probability of red rust spots and the probability of white rust spots for each red rust spot. The area of ​​red rust spots and the area of ​​white rust spots are calculated using a pixel counting method. The actual rust spot area of ​​each newly added rust spot region is obtained by weighting the red rust spot probability, red rust spot area, white rust spot probability, and white rust spot area.

[0013] Optionally, according to another aspect of the embodiments of the present application, the present application provides an apparatus, comprising: an image preprocessing module, configured to collect motor shell images before and after a salt spray test and pre-process the images to obtain to-be-registered images; an image registration module, configured to perform image registration on the to-be-registered images to obtain effective motor shell images; a difference detection module, configured to perform difference detection on the effective motor shell images before and after the salt spray test, extract newly-added rust stain regions based on the difference detection result, and label the newly-added rust stain regions; a model construction module, configured to construct a rust stain identification model according to the newly-added rust stain regions and corresponding motor shell images; and a rust stain identification module, configured to identify a rust stain type of the newly-added rust stain regions based on the rust stain identification model, and calculate an area corresponding to the newly-added rust stain regions.

[0014] According to another aspect of the embodiments of the present application, the present application provides an electronic device, comprising a memory, a processor, a communication interface and a communication bus, the memory stores a computer program executable on the processor, the memory, the processor and the communication interface communicate through the communication bus, and the processor executes the computer program to implement the steps of the motor shell rust stain identification method.

[0015] According to another aspect of the embodiments of the present application, the present application further provides a computer readable medium having non-volatile program codes executable by a processor, the program codes causing the processor to execute the motor shell rust stain identification method.

[0016] The above technical solutions provided by the embodiments of the present application have the following advantages compared with related art:

[0017] By performing image registration and difference detection on the motor shell images before and after the salt spray test, the manual intervention is greatly reduced, the rust stain detection is quickly performed, multiple images can be processed in parallel, and the detection period is significantly shortened. Real-time monitoring is supported, the rust stain condition of the motor shell can be fed back in time, which helps to respond in time and take corresponding measures. The deep learning model is used for rust stain segmentation and classification, the powerful capability of the deep learning model in image processing and pattern recognition is utilized, and high-precision identification of the rust stain region is realized. The red rust stain and the white rust stain can be accurately distinguished, and even in the case of complex background or rust stain overlap, the identification accuracy can be maintained. By comparing the motor shell images before and after the salt spray test, the newly-added rust stain region is extracted, the misjudgment and omission are effectively avoided, and the consistency of the identification result is improved. The accurate rust stain detection result helps to discover potential problems in time and avoid major failures, thereby optimizing the production process and improving the overall operation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or the related art description. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0020] Figure 1 A hardware environment schematic diagram of a motor shell rust spot identification method provided according to an embodiment of the present application;

[0021] Figure 2 A flowchart of a motor shell rust spot identification method provided according to an embodiment of the present application;

[0022] Figure 3 A rust spot area updating flowchart of a motor shell rust spot identification method provided according to an embodiment of the present application;

[0023] Figure 4 A block diagram of a motor shell rust spot identification device provided according to an embodiment of the present application;

[0024] Figure 5 An optional electronic device structure schematic diagram provided by the present application. DETAILED DESCRIPTION

[0025] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort fall within the scope of protection of the present application.

[0026] In the subsequent description, the suffixes such as "module", "component", or "unit" used to represent elements are only for the convenience of the description of the present application, and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0027] In the related art, the manual method is used for the identification and detection of motor rust spots, which has the problems of low detection efficiency and low detection accuracy.

[0028] In order to solve the problems mentioned in the background art, according to an aspect of an embodiment of the present application, an embodiment of a method is provided.

[0029] Optionally, in the embodiments of the present application, the above-mentioned method can be applied to a hardware environment composed of a terminal 101 and a server 103 as shown in the following. Figure 1 ​Figure 1 As shown, the server 103 is connected with the terminal 101 through a network, which can be used to provide services for the terminal or the client installed on the terminal, and a database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The network includes but is not limited to a wide area network, a metropolitan area network or a local area network, and the terminal 101 includes but is not limited to a PC, a mobile phone, a tablet computer and the like.

[0030] The motor shell rust spot identification method in the embodiment of the application can be executed by the server 103, and can also be executed by the server 103 and the terminal 101 together, as shown in the figure. Figure 2 The method can include the following steps:

[0031] Step S202: acquiring a first motor shell image before a salt spray test and a second motor shell image after the salt spray test;

[0032] Specifically, after step S202, the following steps are further included:

[0033] The collected first motor shell image and the second motor shell image are respectively denoised;

[0034] The denoised first motor shell image and the denoised second motor shell image are respectively subjected to contrast enhancement;

[0035] The first motor shell image and the second motor shell image subjected to contrast enhancement are respectively subjected to grayscale processing.

[0036] In the above embodiment, by performing the denoising step on the collected motor shell image, the noise interference in the motor shell image, such as sensor noise and environmental light interference, can be effectively removed, so that the image is clearer. The contrast enhancement can improve the visual effect of the image, so that the details in the image are more prominent, especially the rust spots and other small features, thereby improving the sensitivity of rust spot detection. Converting the color image into a grayscale image not only simplifies the image data and reduces the processing time, but also avoids the interference of color information on rust spot identification, so that the identification process is more focused on the texture and shape features of the image.

[0037] Step S204: performing image registration on the first motor shell image and the second motor shell image to register the rust spot area in the first motor shell image and the rust spot area in the second motor shell image to obtain an effective motor shell image;

[0038] Specifically, step S204 includes:

[0039] The first motor shell image and the second motor shell image are subjected to feature point detection by a scale-invariant feature transform algorithm to obtain a first detection result and a second detection result, a first descriptor corresponding to the first detection result is extracted, and a second descriptor corresponding to the second detection result is extracted;

[0040] The first motor shell image and the second motor shell image are subjected to matching by a brute force matcher based on the first detection result and the first descriptor, the second detection result and the second descriptor to obtain a similar feature point pair;

[0041] A transformation matrix is calculated based on the similar feature point pair, and the first motor shell image is transformed into a coordinate system corresponding to the second motor shell image based on the transformation matrix;

[0042] The first motor shell image and the second motor shell image are subjected to transformation based on the transformation matrix to realize image registration and obtain an effective motor shell image, the effective motor shell image including a first effective motor shell image before a salt spray test and a second effective motor shell image after the salt spray test.

[0043] Further, feature point detection is performed by a scale construction extreme value detection algorithm (SIFT algorithm). First, a series of scale space images are generated by applying different scale Gaussian blur filters to the first motor shell image and the second motor shell image. Difference images between adjacent scale space images are calculated, and local extreme points are found in the difference images to obtain potential key points. For each candidate key point, Gaussian fitting is used to accurately locate the position of the candidate key point and remove key points with unstable edge responses. The positions of the key points are refined by fitting a cubic polynomial function, and key points with low contrast and key points greatly affected by edge effects are removed. Directions are assigned to the key points, the gradient size and direction of the pixels around the key points are calculated, and the gradient directions are divided into multiple direction histograms. The direction with the largest peak value is selected as the main direction of the key point, and multiple directions can also be assigned to the key point if the peak value exceeds a certain threshold. Descriptors are calculated for each key point, and the descriptors are fixed-length vectors based on the gradient information of the pixels around the key points. A rectangular region with a predetermined pixel size is selected around the key point, and the region is divided into smaller sub-regions. The gradient size and direction of the pixels in each sub-region are calculated, and a gradient histogram is constructed. The obtained gradient histogram is merged into a vector. The nearest neighbor search is used to find the most similar descriptor, and the Euclidean distance is used to measure the similarity between the descriptors in this embodiment. The ratio test is applied to reduce false matches, and if the ratio of the distance of the nearest neighbor to the distance of the second nearest neighbor is less than a threshold, the matching is successful.

[0044] Further, the feature point matching process based on the Brute-Force algorithm first extracts feature points and descriptors, and then detects feature points, including corner points, interest points, etc. from two input images. A descriptor is calculated for each feature point, which is a vector representing the local image content around the feature point. For each feature point descriptor in the first image, the distance between it and all feature point descriptors in the second image is calculated to obtain a distance matrix between the descriptors, and the Euclidean distance is used in this embodiment. For each feature point descriptor in the first image, the feature point descriptor corresponding to the smallest distance in the distance matrix is found, which represents the best match of the original feature point. A cross-check is set to filter out some incorrect matches. Finally, each feature point is assigned an existing match, and the assigned match is used to estimate the geometric relationship between the two images to obtain a feature point pair.

[0045] Further, the specific process of calculating the transformation matrix is as follows. First, a minimum number of corresponding point pairs are randomly selected from the source image and the target image. At least 3 pairs of points that are not collinear are needed for affine transformation. The number of iterations N and the inlier threshold t are set. In each iteration, a group of samples is randomly selected from the corresponding point pairs. The affine transformation matrix A is estimated using the selected group of samples. All remaining corresponding point pairs are applied to the affine transformation matrix A, and the distance between the transformed points and the target points is calculated. If the distance is less than the threshold t, the corresponding point pair is marked as an inlier. The foregoing process is repeated N times. The affine transformation matrix obtained in the iteration with the most inliers is selected as the final transformation matrix.

[0046] In the above embodiments, the scale-invariant feature transform algorithm can be used to detect key points in the images and extract scale-invariant feature descriptors of the key points. The descriptors have good robustness to image rotation, scaling, brightness changes, etc., and thus can ensure accurate matching of the same feature points in images under different conditions (e.g., before and after salt spray testing). The brute-force matcher iterates through all possible pairs of feature points to find the best match, but can ensure that the most similar pair of feature points is found, thereby improving the accuracy of registration. Based on the matched feature point pairs, a transformation matrix from one image to another can be calculated. The transformation matrix describes the geometric transformation relationship between the images, such as translation, rotation, scaling, etc. Using the transformation matrix, the image before the salt spray test can be transformed into the same coordinate system as the image after the salt spray test, or vice versa. In this way, the same positions in the two images correspond to the same part of the motor shell. After image registration, the changes such as rust spots in the images can be more intuitively displayed. Because the images have been accurately aligned, the pixel values or texture features at the same positions in the two images can be directly compared, thereby quickly and accurately detecting changes such as rust spots. Accurate image registration can reduce false positives and false negatives caused by image misalignment or distortion, and improve the reliability of rust spot detection.

[0047] In step S206, difference detection is performed on the effective motor shell images before and after the salt spray test, and a newly added rust spot region is extracted based on the difference detection result.

[0048] Specifically, step S206 includes:

[0049] extracting first pixel features corresponding to the first effective motor shell image and second pixel features corresponding to the second effective motor shell image;

[0050] performing pixel subtraction on the first effective motor shell image and the second effective motor shell image based on the first pixel features and the second pixel features to obtain a difference image;

[0051] segmenting the difference image according to a preset threshold to obtain a corresponding binary image;

[0052] extracting a newly added rust spot region based on the binary image, and labeling the newly added rust spot region.

[0053] In this embodiment, pixel features for describing the surface state of the motor shell are selected, including color features (RGB values), brightness features, and texture features. Using MATLAB image processing software, each pixel in the first effective motor shell image and each pixel in the second effective motor shell image is traversed respectively, the corresponding feature values of the color features, brightness features, and texture features are extracted, and are saved in array form. The first effective motor shell image and the second effective motor shell image are aligned in space, and the first effective motor shell image and the second effective motor shell image that have been aligned are subjected to pixel-by-pixel subtraction operation to obtain a difference image. Each pixel value in the difference image represents the change amount of the corresponding position pixel before and after the salt spray test. According to a statistical method, a threshold value is set to distinguish newly added rust stains. The pixel value in the difference image is compared with the set threshold value, and the pixel value greater than the threshold value is the changed area, and the pixel value less than or equal to the threshold value is the unchanged area. The segmented image is converted into a binary image, wherein the changed area is represented by white color and the unchanged area is represented by black color. In the binary image, the shape of the rust stain area is optimized by morphological operation, such as corrosion, expansion, hole filling, etc., so that it is more continuous and accurate. The contour detection algorithm is used to extract the contour of the rust stain area in the binary image. In the original first effective motor shell image, the second effective motor shell image or the difference image, the newly added rust stain area is highlighted by using a rectangular frame, a circular marker or a specific color, and a text label is added.

[0054] In the above embodiment, pixel features are extracted from the first effective motor shell image before the salt spray test and the second effective motor shell image after the salt spray test. Based on the pixel features, pixel subtraction is performed to directly generate a difference image, and the generated difference image can clearly represent the newly added rust stain area on the motor shell after the salt spray test. By pre-setting a threshold value, the difference image is segmented and converted into a binary image, so that the newly added rust stain area is more prominent in the image, facilitating subsequent extraction and labeling work. On the basis of the binary image, the newly added rust stain area can be easily extracted and labeled in the effective motor shell image or the difference image, thereby improving the efficiency of rust stain recognition.

[0055] Step S208, constructing a rust stain recognition model according to the newly added rust stain area and the corresponding motor shell image;

[0056] Specifically, step S208 includes:

[0057] generating a pixel mask corresponding to the newly added rust stain area;

[0058] setting a data path and traversing the second motor shell image, reading the pixel mask corresponding to each second motor shell image and returning in array form;

[0059] preprocessing the second motor shell image and the pixel mask;

[0060] Set data augmentation parameters and configure data augmentation logic based on the data augmentation parameters;

[0061] The iterative data augmentation logic performs data augmentation on the second motor housing image and pixel mask, and returns the data-augmented second motor housing image and pixel mask;

[0062] A rust spot recognition model was obtained by training a U-Net model using a data-augmented image of the second motor housing and a pixel mask.

[0063] In the above embodiments, by generating pixel masks corresponding to the newly added rust spots, the pixel masks accurately identify the rust spot areas in the image. The model can learn the specific features and boundaries of the rust spots, thereby improving detection accuracy. A data path is set and the second motor housing image is traversed. The pixel mask corresponding to each image is read and returned as an array for subsequent programming processing and batch operations. Preprocessing the images and pixel masks helps eliminate irrelevant information in the images, highlighting rust spot features and improving the model's generalization ability and detection effect. By setting reasonable data augmentation parameters, a large amount of new data similar to but different from the original data can be generated, effectively avoiding overfitting and improving the model's robustness. By iteratively running the data augmentation logic, a large amount of new training data can be generated in batches, maintaining the features of the original data while increasing diversity. Using the data-augmented second motor housing image and pixel mask, the U-Net model is trained to obtain a rust spot recognition model, which can accurately identify and segment the rust spot areas in the image.

[0064] Furthermore, the image and pixel mask of the second motor housing are preprocessed, including:

[0065] The size of the second motor housing image and pixel mask is adjusted to the target size using image scaling.

[0066] The normalization factor is calculated by taking the maximum and minimum pixel values ​​in the pixel array corresponding to the second motor housing image, and then the normalization factor is used to normalize each pixel value in the pixel value array corresponding to the second motor housing image.

[0067] Expand the dimensions of the mask array corresponding to the pixel mask so that the dimensions of the mask array reach the target dimension.

[0068] Specifically, the second motor shell image to be processed is stored in a digital matrix form, where each element represents the brightness or color value of a pixel. The pixel mask is a binary mask corresponding to the second motor shell image, used to identify specific regions in the image, such as the motor shell itself. The target width and height of the second motor shell image are set. An image scaling algorithm is applied to adjust the second motor shell image and its corresponding pixel mask to the target size. Optionally, the image scaling algorithm can use bilinear interpolation or nearest neighbor interpolation. For example, the target size is: width = 256 pixels, height = 256 pixels; the original image size of the second motor shell image is: width = 512 pixels, height = 512 pixels. The bilinear interpolation algorithm is selected for scaling, adjusting the original image size to the target size. The pixel array corresponding to the second motor shell image is traversed to find the maximum pixel value and the minimum pixel value. For each pixel value in the pixel array, it is converted to the range of 0 to 1 using a normalization formula based on the maximum pixel value and the minimum pixel value. The pixel mask of this embodiment uses a binary mask, so no normalization is needed. Finally, the target dimensions of the mask array are determined, such as adding a channel dimension to match the input requirements of the image processing library, and then using an array processing library such as numpy to expand the dimensions of the mask array.

[0069] In the above embodiment, by uniformly adjusting the size of the second motor shell image and the pixel mask, it is ensured that the data input into the model has consistent dimensions. Appropriately reducing the image size can reduce the computational load of the model and improve the training efficiency, while maintaining sufficient details helps the model to capture the features of rust spots. By normalizing the pixel values of the image data to a specific range, it helps to eliminate the brightness differences between different images caused by factors such as lighting and shooting conditions, so that the model can focus more on the rust spot features in the image. By expanding the dimensions of the mask array of the pixel mask, it can be consistent with the second motor shell image in data structure.

[0070] Further, using the second motor shell image and the pixel mask after data augmentation to train a U-Net model to obtain a rust spot recognition model, comprising:

[0071] Using the U-Net model as a base model and adding a global average pooling layer, a fully connected layer and an output layer of the base model to obtain an initial model;

[0072] Setting the optimizer, loss function and evaluation index corresponding to the initial model;

[0073] Using the second motor shell image as the input of the initial model and the pixel mask corresponding to the rust spot region as the output of the initial model to train the model;

[0074] The initial model is iteratively trained in combination with an optimizer and a loss function until the output of the initial model meets the evaluation indicators, obtaining the rust spot recognition model.

[0075] In the above embodiment, by introducing the U-Net model as the basis and customizing it in combination with the global average pooling layer, the fully connected layer and the output layer, the global average pooling layer helps to extract the global features of the image, and the fully connected layer can map these features to the final output space, improving the generalization ability of the model. Set appropriate optimizer and loss function, the optimizer can guide the update direction of the model parameters, accelerate the convergence process; while the loss function measures the difference between the model prediction and the true label, providing a clear target for the optimization of the model. A large amount of new data generated by data augmentation technology not only increases the diversity of training samples, but also reduces the risk of overfitting of the model to specific samples. The data enhanced data is used as input, so that the model can learn more robust and generalizable feature representation, thereby improving the accuracy and reliability of rust spot detection. Setting clear evaluation indicators can objectively reflect the performance of the model on the test set, so that the output of the model gradually meets the requirements of the evaluation indicators, and finally obtains a rust spot detection model with excellent performance.

[0076] In the rust spot recognition model constructed according to the newly added rust spot area and the corresponding motor shell image, first, the image is labeled to generate a mask of the rust spot area, and each pixel is labeled as rust spot or non-rust spot. For example, each pixel of a color image contains three color channels of red, green and blue (RGB), and the value of each channel is usually between 0 and 255. Normalize the value of each channel to the range of 0 to 1. The mask is usually binary, i.e. containing values 0 and 1, where 1 represents rust spot and 0 represents background. First, set the data path and call the function to read the image and mask, read each image and corresponding mask by traversing the image directory, and return the image and mask array. Then, pre-process the data, adjust the image and mask size, normalize the image and mask array, and expand the mask array dimension, and return the pre-processed image and mask array. Next, create an object and set the data augmentation parameters. Iterate the image and mask by batch, and perform data augmentation on the image and mask, including rotation, flipping, cropping, etc. Return the augmented image and mask. Divide the data into training set and validation set, and return the training set and validation set data. Next, build the model, use ResNet50 as the base model, add a global average pooling layer, a fully connected layer and an output layer, then compile the model, set the optimizer, loss function and evaluation metrics. Set the batch size and number of training rounds, train the model, and provide the training data generator and validation data generator. Train the U-Net model using the pre-processed image and mask data. The input of the model is the original image, and the output is the corresponding rust spot mask. Select cross-entropy loss as the loss function, and use Adam optimizer for iterative training to obtain the rust spot recognition model.

[0077] In step S210, the rust spot type of the newly added rust spot area is identified based on the rust spot recognition model, and the area corresponding to the newly added rust spot area is calculated.

[0078] Specifically, step S210 includes:

[0079] Obtain a real-time motor shell image, segment the real-time motor shell image through the rust spot recognition model, and identify and extract the newly added rust spot area;

[0080] Perform morphological operations on the newly added rust spot area, including dilation and corrosion, to eliminate noise points, fill holes and retain the main structure of the rust spot;

[0081] Convert the real-time motor shell image from RGB color space to Lab color space, and distinguish red rust spots and white rust spots based on the corresponding color components;

[0082] Use a convolutional neural network to extract features from the real-time motor shell image in Lab color space, to obtain color features, texture features and shape features of the newly added rust spot area;

[0083] The color features, texture features and shape features of the added rust area are combined to construct a multi-label classification model for simultaneously outputting red rust probability and white rust probability;

[0084] The red rust probability corresponding to each red rust spot and the white rust probability corresponding to each white rust spot are output according to the multi-label classification model;

[0085] The red rust area corresponding to each red rust spot and the white rust area corresponding to each white rust spot are calculated by pixel counting method;

[0086] The actual rust area of each added rust area is obtained by weighting the red rust probability, red rust area, white rust probability and white rust area.

[0087] Further, as shown in Figure 3 the whole rust area is traversed to determine whether there is any untreated rust area, and if not, the process is ended, and if so, the process is continued; for the detected rust area, a sub-image containing rust spots is extracted, and the extracted rust sub-image is resized. The resized rust sub-image is normalized and converted to a unified standard format. The normalized rust sub-image is input into the classification model, the classification model predicts the rust type, and the trained classification model is used to predict the input rust sub-image to determine which type of rust it belongs to, and the proportion or degree of red rust in the rust area is calculated. If the "red rust rate" is greater than 0.5, the area of red rust is more serious, and the red rust area record of the area is updated. The proportion or degree of white rust in the rust area is calculated to determine whether it is greater than 0.5, and if the white rust rate is greater than 0.5, the white rust area record of the area is updated.

[0088] In the above embodiment, the trained rust spot recognition model is used to segment the real-time motor shell image, which can accurately identify and extract the rust area. Through morphological operations such as dilation and erosion on the rust area, noise points can be effectively eliminated, holes can be filled, and the main structure of the rust spot can be preserved, improving the accuracy and reliability of rust spot recognition. The image is converted from RGB color space to Lab color space, and color components are used to distinguish red rust spots and white rust spots. Lab color space is more consistent with human eye color perception and can more accurately describe the color features of rust spots. A multi-label classification model that can simultaneously output red rust probability and white rust probability is constructed, which enables the model to identify multiple types of rust spots simultaneously, improving the accuracy and comprehensiveness of rust spot type differentiation. According to the red rust probability and white rust probability of each rust area output by the multi-label classification model, the actual area of each rust area is calculated by weighting the red rust area and white rust area, which considers the diversity of rust spot types and makes the area calculation more accurate.

[0089] According to another aspect of the embodiments of the present application, as shown in Figure 4 The motor shell rust spot identification device comprises:

[0090] The image acquisition module 401 is configured to acquire a first motor shell image before a salt spray test and a second motor shell image after the salt spray test.

[0091] The image registration module 403 is configured to perform image registration on the first motor shell image and the second motor shell image to register a rust spot area in the first motor shell image and a rust spot area in the second motor shell image to obtain an effective motor shell image.

[0092] The difference detection module 405 is configured to perform difference detection on the effective motor shell images before and after the salt spray test, and extract a newly added rust spot area based on the difference detection result.

[0093] The model construction module 407 is configured to construct a rust spot identification model according to the newly added rust spot area and a corresponding motor shell image.

[0094] The rust spot identification module 409 is configured to identify a rust spot type of the newly added rust spot area based on the rust spot identification model, and calculate an area corresponding to the newly added rust spot area.

[0095] It should be noted that the image preprocessing module 401 in this embodiment can be used to perform step S202 in the embodiments of the present application, the image registration module 403 in this embodiment can be used to perform step S204 in the embodiments of the present application, the difference detection module 405 in this embodiment can be used to perform step S206 in the embodiments of the present application, the model construction module 407 in this embodiment can be used to perform step S208 in the embodiments of the present application, and the rust spot identification module 409 in this embodiment can be used to perform step S210 in the embodiments of the present application.

[0096] It should be noted that the above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the above disclosed content. It should be noted that the above modules as part of the device can run in the hardware environment as shown in Figure 1 It can be implemented by software or hardware.

[0097] Optionally, the motor shell rust spot identification device further comprises an image preprocessing module, and the image preprocessing comprises the following specific steps: respectively performing denoising on the collected first motor shell image and second motor shell image; respectively performing contrast enhancement on the denoised first motor shell image and second motor shell image; and respectively performing grayscale processing on the contrast-enhanced first motor shell image and second motor shell image.

[0098] Optionally, the image acquisition module 403 is specifically configured to: perform feature point detection on the first motor shell image and the second motor shell image respectively to obtain a first detection result and a second detection result through a scale-invariant feature transform algorithm, extract a corresponding first descriptor based on the first detection result, and extract a corresponding second descriptor based on the second detection result; combine the first detection result and the first descriptor, the second detection result and the second descriptor, and perform matching on the first motor shell image and the second motor shell image through a brute force matcher to obtain a similar feature point pair; calculate a transformation matrix based on the similar feature point pair, and transform the first motor shell image into a coordinate system corresponding to the second motor shell image based on the transformation matrix; and perform transformation on the first motor shell image and the second motor shell image respectively based on the transformation matrix to achieve image registration to obtain an effective motor shell image, wherein the effective motor shell image includes a first effective motor shell image before the salt spray test and a second effective motor shell image after the salt spray test.

[0099] Optionally, the model construction module 405 is specifically configured to: extract a first pixel feature corresponding to the first effective motor shell image and a second pixel feature corresponding to the second effective motor shell image; perform pixel subtraction on the first effective motor shell image and the second effective motor shell image based on the first pixel feature and the second pixel feature to obtain a difference image; segment the difference image according to a preset threshold to obtain a corresponding binary image; extract a new rust stain area based on the binary image, and label the new rust stain area.

[0100] Optionally, the difference detection module 407 is specifically configured to: generate a pixel mask corresponding to the new rust stain area; set a data path and traverse the second motor shell image, read a pixel mask corresponding to each second motor shell image, and return in an array form; pre-process the second motor shell image and the pixel mask; set a data enhancement parameter and configure a data enhancement logic based on the data enhancement parameter; iteratively run the data enhancement logic to perform data enhancement on the second motor shell image and the pixel mask, and return the second motor shell image and the pixel mask after data enhancement; and train a U-Net model using the second motor shell image and the pixel mask after data enhancement to obtain a rust stain recognition model.

[0101] Optionally, the rust spot identification module 409 is specifically configured to: acquire a real-time motor shell image, perform segmentation on the real-time motor shell image through a rust spot identification model, and identify and extract a new rust spot area; perform morphological operation on the new rust spot area, the morphological operation including inflation and corrosion, eliminating noise points, filling holes, and retaining the main structure of the rust spot; convert the real-time motor shell image from an RGB color space to a Lab color space, and distinguish red rust spots and white rust spots based on corresponding color components; perform feature extraction on the real-time motor shell image in the Lab color space by using a convolutional neural network, to obtain color features, texture features, and shape features of the new rust spot area; construct a multi-label classification model for simultaneously outputting a red rust spot probability and a white rust spot probability, in combination with the color features, the texture features, and the shape features of the new rust spot area; output, according to the multi-label classification model, a red rust spot probability corresponding to each red rust spot and a white rust spot probability corresponding to each white rust spot; calculate, by using a pixel counting method, a red rust spot area corresponding to each red rust spot and a white rust spot area corresponding to each white rust spot; and obtain an actual rust spot area of each new rust spot area by weighting the red rust spot probability, the red rust spot area, the white rust spot probability, and the white rust spot area.

[0102] According to another aspect of the embodiments of the present application, the present application provides an electronic device, as shown in the accompanying drawings, comprising a memory 502, a processor 504, a communication interface 506, and a communication bus 508, the memory 502 stores a computer program executable on the processor 504, the memory 502 and the processor 504 communicate through the communication interface 506 and the communication bus 508, and the processor 502 executes the computer program to implement the steps of the above method. Figure 5

[0103] The memory and the processor in the above electronic device communicate through the communication bus and the communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0104] The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0105] ​The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0106] According to another aspect of the embodiments of this application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the above embodiments.

[0107] Optionally, in embodiments of this application, the computer-readable medium is configured to store program code for the processor to perform the following steps:

[0108] Step S202: Obtain an image of the first motor housing before the salt spray test and an image of the second motor housing after the salt spray test;

[0109] Step S204: Perform image registration on the first motor housing image and the second motor housing image to register the rust area in the first motor housing image and the rust area in the second motor housing image to obtain an effective motor housing image;

[0110] Step S206: Perform difference detection on the effective motor housing images before and after the salt spray test, and extract the newly added rust spots based on the difference detection results;

[0111] Step S208: Construct a rust spot recognition model based on the newly added rust spot areas and the corresponding motor housing images;

[0112] Step S210: Identify the type of newly added rust spot area based on the rust spot recognition model, and calculate the area corresponding to the type of newly added rust spot area.

[0113] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0114] In specific implementation, the embodiments of this application can be referred to the above embodiments and have corresponding technical effects.

[0115] It can be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP Devices), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.

[0116] For a software implementation, the techniques described herein can be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes can be stored in memory units and executed by processors. The memory unit can be implemented within the processor or external to the processor.

[0117] It can be clearly understood by a person skilled in the art that, for the purpose of convenience and conciseness, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be described here in detail.

[0118] It can be clearly understood by a person skilled in the art that, for the purpose of convenience and conciseness, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be described here in detail.

[0119] In the embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the division of the above-described device embodiments is only a logical function division, and other division manners can be adopted during actual implementation. For example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the modules or the components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0120] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0121] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0122] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products, and the computer software products are stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, and various program codes that can be stored in the medium. It should be noted that in this paper, terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or devices. Without more limitations, the elements defined by the statement "including a" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0123] The above is only a specific embodiment of the present application, so that those skilled in the art can understand or implement the present application. Various modifications of these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A method for identifying rust spots on an electric motor housing, characterized in that, include: Acquire images of the first motor housing before the salt spray test and the second motor housing after the salt spray test; Image registration is performed on the first motor housing image and the second motor housing image to register the rust area in the first motor housing image and the rust area in the second motor housing image to obtain an effective motor housing image; Difference detection was performed on the effective motor housing images before and after the salt spray test, and newly added rust spots were extracted based on the difference detection results; A rust spot recognition model is constructed based on the newly added rust spot areas and the corresponding motor housing images; Based on the rust spot recognition model, the rust spot type of the newly added rust spot area is identified, and the area corresponding to the newly added rust spot area is calculated; The step of identifying the rust type of the newly added rust area based on the rust recognition model and calculating the area corresponding to the newly added rust area includes: acquiring a real-time motor housing image, segmenting the real-time motor housing image using the rust recognition model, and identifying and extracting the newly added rust area; Morphological operations are performed on the newly added rust spots, including expansion and corrosion, to eliminate noise, fill holes, and preserve the main structure of the rust spots. The real-time motor housing image is converted from the RGB color space to the Lab color space, and red rust spots and white rust spots are distinguished based on the corresponding color components. The probability of red rust spots corresponding to the red rust spots and the probability of white rust spots corresponding to the white rust spots in the newly added rust spot areas are determined, and the areas of red rust spots corresponding to the red rust spots and the areas of white rust spots corresponding to the white rust spots are calculated. The actual rust spot area of ​​each newly added rust spot area is obtained by weighting the red rust spot probability, the red rust spot area, the white rust spot probability, and the white rust spot area.

2. The method for identifying rust spots on a motor housing according to claim 1, characterized in that, Image registration is performed on the first motor housing image and the second motor housing image to register the rust spots in the first motor housing image and the rust spots in the second motor housing image to obtain a valid motor housing image, including: The first motor housing image and the second motor housing image are respectively subjected to feature point detection by the scale-invariant feature transform algorithm to obtain the first detection result and the second detection result. The first descriptor is extracted based on the first detection result and the second descriptor is extracted based on the second detection result. By combining the first detection result and the first descriptor, the second detection result and the second descriptor, and using a brute-force matcher, similar feature point pairs are obtained by matching the first motor housing image and the second motor housing image. Based on the similar feature point pairs, a transformation matrix is ​​calculated, and the first motor housing image is transformed to the coordinate system corresponding to the second motor housing image based on the transformation matrix; Based on the transformation matrix, the first motor housing image and the second motor housing image are transformed respectively to achieve image registration and obtain an effective motor housing image. The effective motor housing image includes a first effective motor housing image before the salt spray test and a second effective motor housing image after the salt spray test.

3. The method for identifying rust spots on a motor housing according to claim 2, characterized in that, Difference detection was performed on the effective motor housing images before and after the salt spray test. Based on the difference detection results, newly added rust spots were extracted, including: Extract the first pixel feature corresponding to the first valid motor housing image and the second pixel feature corresponding to the second valid motor housing image; Based on the first pixel features and the second pixel features, the first effective motor housing image and the second effective motor housing image are subtracted by pixels to obtain a difference image; The difference image is segmented according to a preset threshold to obtain the corresponding binarized image; The newly added rust spots are extracted from the binarized image and then labeled.

4. The method for identifying rust spots on a motor housing according to claim 1, characterized in that, A rust spot recognition model is constructed based on the newly added rust spot areas and the corresponding motor housing images, including: Generate the pixel mask corresponding to the newly added rust spot area; Set the data path and traverse the second motor housing image, read the pixel mask corresponding to each second motor housing image and return it in array form; The second motor housing image and the pixel mask are preprocessed; Set data augmentation parameters and configure data augmentation logic based on the data augmentation parameters; The data augmentation logic is iteratively run to augment the second motor housing image and the pixel mask, and the augmented second motor housing image and pixel mask are returned. The U-Net model is trained using the data-enhanced image of the second motor housing and the pixel mask to obtain a rust spot recognition model.

5. The method for identifying rust spots on a motor housing according to claim 4, characterized in that, Preprocessing of the second motor housing image and the pixel mask includes: The size of the second motor housing image and the pixel mask are adjusted to the target size using an image scaling method. A normalization factor is calculated using the maximum and minimum pixel values ​​in the pixel array corresponding to the second motor housing image, and each pixel value in the pixel value array corresponding to the second motor housing image is normalized using the normalization factor. The dimensions of the mask array corresponding to the pixel mask are expanded to reach the target dimension.

6. The method for identifying rust spots on a motor housing according to claim 4, characterized in that, A rust spot recognition model is obtained by training a U-Net model using the data-augmented image of the second motor housing and a pixel mask, including: The U-Net model is used as the base model, and the global average pooling layer, fully connected layer and output layer of the base model are added to obtain the initial model; Set the optimizer, loss function, and evaluation metric corresponding to the initial model; The second motor housing image is used as the input to the initial model, and the pixel mask corresponding to the rust spot area is used as the output of the initial model for model training. The initial model is iteratively trained by combining the optimizer and the loss function until the output of the initial model meets the evaluation index, thus obtaining the rust spot recognition model.

7. The method for identifying rust spots on a motor housing according to claim 1, characterized in that, The step of determining the probability of red rust spots corresponding to the red rust spots and the probability of white rust spots corresponding to the white rust spots in the newly added rust spot area, and calculating the area of ​​red rust spots corresponding to the red rust spots and the area of ​​white rust spots corresponding to the white rust spots, includes: A convolutional neural network was used to extract features from the real-time motor housing image in the Lab color space to obtain the color features, texture features, and shape features of the newly added rust spots. A multi-label classification model is constructed by combining the color features, texture features, and shape features of the newly added rust spot region to simultaneously output the probabilities of red rust spots and white rust spots. The multi-label classification model outputs the probability of red rust spots corresponding to each red rust spot and the probability of white rust spots corresponding to each white rust spot. The area of ​​the red rust spot corresponding to the red rust spot and the area of ​​the white rust spot corresponding to the white rust spot are calculated by pixel counting method.

8. A device for identifying rust spots on a motor housing, used to implement the method for identifying rust spots on a motor housing as described in any one of claims 1 to 7, characterized in that, include: The image acquisition module is used to acquire images of the first motor housing before the salt spray test and images of the second motor housing after the salt spray test. The image registration module is used to perform image registration on the first motor housing image and the second motor housing image, so as to register the rust area in the first motor housing image and the rust area in the second motor housing image to obtain an effective motor housing image; The difference detection module is used to perform difference detection on the effective motor housing images before and after the salt spray test, and extract the newly added rust spot areas based on the difference detection results; A model is constructed based on the newly added rust spots and the corresponding motor housing image to build a rust spot recognition model. The rust spot recognition module is used to identify the rust spot type of the newly added rust spot area based on the rust spot recognition model, and to calculate the area corresponding to the newly added rust spot area.

9. An electronic device, characterized in that, The device includes a memory, a processor, a communication interface, and a communication bus. The memory stores a computer program that can run on the processor. The memory and the processor communicate with each other through the communication bus and the communication interface. The device is characterized in that when the processor executes the computer program, it implements the motor housing rust spot identification method according to any one of claims 1 to 7.

10. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the motor housing rust spot identification method according to any one of claims 1 to 7.

Images