Motor shell rusty spot recognition method, device and equipment and computer readable medium

Abstract

The invention relates to a motor shell rusty spot recognition method, device and equipment and a computer readable medium. The method comprises the following steps: acquiring a first motor shell image before a salt spray test and a second motor shell image after the salt spray test; performing image registration on the first motor shell image and the second motor shell image to perform registration on a rusty spot area in the first motor shell image and a rusty spot area in the second motor shell image to obtain an effective motor shell image; performing difference detection on the effective motor shell images before and after the salt spray test, and extracting a newly added rusty spot area based on a difference detection result; constructing a rusty spot recognition model according to the newly added rusty spot area and the corresponding motor shell image; based on the rusty spot recognition model, the rusty spot type of the newly-added rusty spot area is recognized, and the area corresponding to the newly-added rusty spot area is calculated. According to the invention, the problem of low detection efficiency and accuracy caused by manual motor shell rusty spot recognition detection is solved.

Description

Technical Field

[0001] The present application relates to the technical field of rust spot recognition, and in particular to a method, device, equipment and computer-readable medium for recognizing rust spots on a motor housing. Background Art

[0002] In the field of motor manufacturing and maintenance, the corrosion resistance of the motor housing is a crucial performance indicator, which is directly related to the service life and reliability of the motor. Especially in harsh conditions such as marine environment and chemical industry, the motor housing is exposed to humid air containing salt for a long time, which is prone to salt spray erosion and rust. Rust not only affects the appearance of the motor, but more seriously, it will gradually erode the housing material, resulting in a decrease in structural strength, and ultimately affecting the normal operation of the motor. Therefore, accurately and quickly identifying and quantifying rust on the motor housing is of great significance for evaluating the motor status and formulating maintenance plans.

[0003] At present, the detection and identification of rust spots on motor housings mainly rely on manual visual inspection. This method requires the inspector to rely on experience to observe the housing surface with the naked eye to determine whether rust spots exist and roughly estimate their area and type (such as red rust spots, white rust spots, etc.). Although this method is simple to implement, it has low detection efficiency and low detection accuracy when faced with large-scale and high-frequency detection needs.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The present application provides a method, device, equipment and computer-readable medium for identifying rust spots on a motor housing to solve the above-mentioned technical problem of "low detection efficiency and accuracy of manual detection of rust spots on a motor housing".

[0006] According to one aspect of an embodiment of the present application, the present application provides a method for identifying rust spots on a motor case, comprising: obtaining a first motor case image before a salt spray test and a second motor case image after the salt spray test; performing image registration on the first motor case image and the second motor case image to register the rust spot area in the first motor case image with the rust spot area in the second motor case image to obtain a valid motor case image; performing difference detection on the valid motor case images before and after the salt spray test, and extracting a newly added rust spot area based on the difference detection result; constructing a rust spot recognition model based on the newly added rust spot area and the corresponding motor case image; identifying the rust spot type of the newly added rust spot area based on the rust spot recognition model, and calculating the area corresponding to the newly added rust spot area.

[0007] Optionally, image registration is performed on the first motor housing image and the second motor housing image to register the rust spot area in the first motor housing image with the rust spot area in the second motor housing image to obtain a valid motor housing image, including: performing feature point detection on the first motor housing image and the second motor housing image respectively through a scale-invariant feature transformation algorithm to obtain a first detection result and a second detection result, extracting a corresponding first descriptor based on the first detection result, and extracting a corresponding second descriptor based on the second detection result; matching the first motor housing image and the second motor housing image through a brute force matcher in combination with the first detection result and the first descriptor, the second detection result and the second descriptor to obtain a similar feature point pair; calculating a transformation matrix based on the similar feature point pair, and transforming the first motor housing image to a coordinate system corresponding to the second motor housing image based on the transformation matrix; transforming the first motor housing image and the second motor housing image respectively based on the transformation matrix to achieve image registration to obtain a valid motor housing image, and the valid motor housing image includes a first effective motor housing image before a salt spray test and a second effective motor housing image after a salt spray test.

[0008] Optionally, a difference detection is performed on the effective motor housing images before and after the salt spray test, and a newly added rust spot area is extracted based on the difference detection result, including: extracting a first pixel feature corresponding to the first effective motor housing image and a second pixel feature corresponding to the second effective motor housing image; performing pixel subtraction on the first effective motor housing image and the second effective motor housing image based on the first pixel feature and the second pixel feature to obtain a difference image; segmenting the difference image according to a preset threshold to obtain a corresponding binary image; extracting a newly added rust spot area based on the binary image, and marking the newly added rust spot area.

[0009] Optionally, a rust spot recognition model is constructed according to the newly added rust spot area and the corresponding motor housing image, including: generating a pixel mask corresponding to the newly added rust spot area; setting a data path and traversing the second motor housing image, reading the pixel mask corresponding to each second motor housing image and returning it in the form of an array; preprocessing the second motor housing image and the pixel mask; setting data enhancement parameters and configuring data enhancement logic based on the data enhancement parameters; iteratively running the data enhancement logic to perform data enhancement on the second motor housing image and the pixel mask, and returning the data enhanced second motor housing image and pixel mask; using the data enhanced second motor housing image and the pixel mask to train a U-Net model to obtain a rust spot recognition model.

[0010] Optionally, the size of the second motor housing image and the pixel mask is adjusted to a target size by an image scaling method; a normalization factor is calculated by the maximum pixel value and the minimum pixel value in the pixel array corresponding to the second motor housing image, and each pixel value in the pixel value array corresponding to the second motor housing image is normalized by the normalization factor; and the mask array corresponding to the pixel mask is dimensionally expanded so that the dimension of the mask array reaches the target dimension.

[0011] Optionally, the second motor housing image and pixel mask after data enhancement are used to train a U-Net model to obtain a rust spot recognition model, including: taking the U-Net model as the basic model, and adding the global average pooling layer, fully connected layer and output layer of the basic model to obtain an initial model; setting an optimizer, loss function and evaluation index corresponding to the initial model; taking the second motor housing image as the input of the initial model and the pixel mask corresponding to the rust spot area as the output of the initial model for model training; and iteratively training the initial model in combination with the optimizer and the loss function until the output of the initial model meets the evaluation index to obtain a rust spot recognition model.

[0012] Optionally, identifying the rust type of the newly added rust area based on the rust spot recognition model, and calculating the area corresponding to the newly added rust spot area, including: acquiring a real-time motor housing image, segmenting the real-time motor housing image through the rust spot recognition model, identifying and extracting the newly added rust spot area; performing morphological operations on the newly added rust spot area, the morphological operations including expansion and corrosion, eliminating noise, filling holes, and retaining the main structure of the rust spot; converting the real-time motor housing image from the RGB color space to the Lab color space, distinguishing red rust spots from white rust spots based on corresponding color components; using a convolutional neural network to map the real-time motor housing image in the Lab color space; The image is subjected to feature extraction to obtain color features, texture features and shape features of the newly added rust spot area; a multi-label classification model for simultaneously outputting the probability of red rust spots and the probability of white rust spots is constructed by combining the color features, the texture features and the shape features of the newly added rust spot area; the probability of red rust spots corresponding to each red rust spot and the probability of white rust spots corresponding to the white rust spot are output according to the multi-label classification model; the red rust spot area corresponding to the red rust spot and the white rust spot area corresponding to the white rust spot are calculated by a pixel counting method; the actual rust spot area of ​​each newly added rust spot area is obtained by weighting the red rust spot probability, the red rust spot area, the white rust spot probability and the white rust spot area.

[0013] Optionally, according to another aspect of an embodiment of the present application, the present application provides a device, including: an image preprocessing module, used to collect motor housing images before and after a salt spray test and preprocess the images to be registered; an image registration module, used to perform image registration on the images to be registered to obtain a valid motor housing image; a difference detection module, used to perform difference detection on the valid motor housing images before and after the salt spray test, and extract and label the newly added rust spot areas based on the difference detection results; a model building model, used to build a rust spot recognition model according to the newly added rust spot areas and the corresponding motor housing images; a rust spot recognition module, used to identify the rust spot type of the newly added rust spot area based on the rust spot recognition model, and calculate the area corresponding to the newly added rust spot area.

[0014] According to another aspect of an embodiment of the present application, the present application provides an electronic device, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the processor implements the steps of the above-mentioned motor case rust spot identification method when executing the computer program.

[0015] According to another aspect of the embodiments of the present application, the present application further provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the above-mentioned method for identifying rust spots on a motor housing.

[0016] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:

[0017] By performing image registration and difference detection on the click shell images before and after the salt spray test, manual intervention is greatly reduced, and rust spot detection is achieved quickly. It can process multiple images in parallel, significantly shortening the detection cycle. It supports real-time monitoring and can provide instant feedback on the rust spot situation of the motor housing, which helps to respond in time and take corresponding measures. The deep learning model is used for rust spot segmentation and classification, and its powerful capabilities in image processing and pattern recognition are used to achieve high-precision recognition of rust spot areas. It can accurately distinguish between red rust spots and white rust spots, and maintain a high recognition accuracy rate even in complex backgrounds or overlapping rust spots. By comparing the motor housing images before and after the salt spray test, the newly added rust spot area is extracted, which effectively avoids misjudgment and missed judgment and improves the consistency of the recognition results. Accurate rust spot detection results help to discover potential problems in a timely manner and avoid major failures, thereby optimizing the production process and improving overall operational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 A schematic diagram of the hardware environment of a method for identifying rust spots on a motor housing provided according to an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a method for identifying rust spots on a motor housing according to an embodiment of the present application;

[0022] Figure 3 A flowchart of updating the rust spot area of ​​a method for identifying rust spots on a motor housing provided according to an embodiment of the present application;

[0023] Figure 4 A block diagram of a device for identifying rust spots on a motor housing provided according to an embodiment of the present application;

[0024] Figure 5 A schematic diagram of an optional electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0026] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0027] In the related art, the manual method of identifying and detecting rust spots on motors has the problems of low detection efficiency and low detection accuracy.

[0028] In order to solve the problem mentioned in the background technology, according to one aspect of the embodiments of the present application, an embodiment of a method is provided.

[0029] Optionally, in the embodiment of the present application, the above method can be applied to Figure 1 In the hardware environment composed of the terminal 101 and the server 103 shown in FIG. Figure 1 As shown, the server 103 is connected to the terminal 101 via a network, and can be used to provide services for the terminal or a client installed on the terminal. A database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The above-mentioned network includes but is not limited to: a wide area network, a metropolitan area network or a local area network, and the terminal 101 includes but is not limited to a PC, a mobile phone, a tablet computer, etc.

[0030] A method for identifying rust spots on a motor housing in an embodiment of the present application can be executed by the server 103, or can be executed jointly by the server 103 and the terminal 101, such as Figure 2 As shown, the method may include the following steps:

[0031] Step S202, obtaining a first motor housing image before a salt spray test and a second motor housing image after a salt spray test;

[0032] Specifically, after step S202, the following steps are further included:

[0033] De-noising the first motor housing image and the second motor housing image respectively;

[0034] performing contrast enhancement on the denoised first motor housing image and the second motor housing image respectively;

[0035] The first motor housing image and the second motor housing image after contrast enhancement are respectively subjected to grayscale processing.

[0036] In the above embodiment, by performing a denoising step on the collected motor housing image, noise interference in the motor housing image, such as sensor noise, ambient light interference, etc., can be effectively removed, making the image clearer. Contrast enhancement can improve the visual effect of the image, making the details in the image more prominent, especially tiny features such as rust spots, thereby improving the sensitivity of rust spot detection. Converting a color image to a grayscale image not only simplifies the image data and reduces processing time, but also avoids the possible interference of color information on rust spot recognition, so that the recognition process is more focused on the texture and shape features of the image.

[0037] Step S204, performing image registration on the first motor housing image and the second motor housing image, so as to register the rust spot area in the first motor housing image with the rust spot area in the second motor housing image to obtain a valid motor housing image;

[0038] Specifically, step S204 includes:

[0039] Performing feature point detection on the first motor housing image and the second motor housing image respectively by a scale-invariant feature transformation algorithm to obtain a first detection result and a second detection result, extracting a corresponding first descriptor based on the first detection result, and extracting a corresponding second descriptor based on the second detection result;

[0040] Combine the first detection result and the first descriptor, the second detection result and the second descriptor to match the first motor housing image and the second motor housing image through a brute force matcher to obtain similar feature point pairs;

[0041] Calculating a transformation matrix based on similar feature point pairs, and transforming the first motor housing image into a coordinate system corresponding to the second motor housing image based on the transformation matrix;

[0042] The first motor housing image and the second motor housing image are transformed respectively based on the transformation matrix to achieve image registration to obtain an effective motor housing image, wherein the effective motor housing image includes a first effective motor housing image before the salt spray test and a second effective motor housing image after the salt spray test.

[0043] Furthermore, feature point detection is performed by using a scale-constructed extreme value detection algorithm (SIFT algorithm). First, a series of scale space images are generated by applying Gaussian blur filters of different scales to the first motor housing image and the second motor housing image. The difference image between adjacent scale space images is calculated, and local extreme value points are found in the difference image. The local extreme value points obtained are potential key points. For each candidate key point, Gaussian fitting is used to accurately locate the position of the candidate key point, and key points with unstable edge response are removed. The position of the key point is refined by fitting a cubic polynomial function, and key points with low contrast and key points with large edge effects are removed. Directions are assigned to the key points, and the gradient size and direction of the pixels around the key points are calculated, and the gradient direction is divided into multiple direction histograms. The direction with the largest peak value is selected as the main direction of the key point. If the peak value exceeds a certain threshold, multiple directions can also be assigned to the key point. A descriptor is calculated for each key point, which is a fixed-length vector based on the gradient information of the pixels around the key point. A rectangular area of ​​a preset pixel size is selected around the key point and divided into smaller sub-areas. The gradient size and direction are calculated for the pixels in each sub-region, and a gradient histogram is constructed. The obtained gradient histograms are merged into a vector. The nearest neighbor search is used to find the most similar descriptor. In this embodiment, the Euclidean distance is used to measure the similarity between descriptors. A ratio test is applied to reduce mismatching. If the ratio of the distance of the nearest neighbor to the distance of the second nearest neighbor exceeds the threshold, the match is successful.

[0044] Furthermore, the feature point matching process based on the Brute-Force algorithm is to first extract feature points and descriptors, and then detect feature points from two input images, including corner points, interest points, etc. A descriptor is calculated for each feature point. The descriptor is a vector used to represent the local image content around the feature point. For each feature point descriptor in the first image, the distance between it and all feature point descriptors in the second image is calculated to obtain a distance matrix between descriptors. This embodiment uses the Euclidean distance. For each feature point descriptor in the first image, find the feature point descriptor corresponding to the minimum distance in the distance matrix. The feature point descriptor corresponding to this minimum distance represents the best match of the original feature point. Set a cross-check to filter out some incorrect matches. Finally, each feature point is assigned an existing match item, and the assigned match item is used to estimate the geometric relationship between the two images to obtain a feature point pair.

[0045] Furthermore, the specific process of calculating the transformation matrix is ​​to first randomly select a minimum number of corresponding point pairs from the source image and the target image. For affine transformation, at least 3 pairs of non-collinear points are required. Set the number of iterations N and the inlier threshold t. In each iteration, randomly select a set of samples from the corresponding point pairs. Use the selected set of samples to estimate the affine transformation matrix A. Apply all remaining corresponding point pairs to the affine transformation matrix A and calculate the distance between the transformed point and the target point. If the distance is less than the threshold t, mark the corresponding point pair as an inlier. Repeat the above process N times. Select the affine transformation matrix obtained from the iteration with the most inliers as the final transformation matrix.

[0046] In the above embodiment, the key points in the image can be detected by the scale-invariant feature transformation algorithm, and the scale-invariant feature descriptor of the key points can be extracted. The descriptor has good robustness to image rotation, scale scaling, brightness change, etc., so it can ensure that the same feature points are accurately matched in images under different conditions (such as before and after the salt spray test). The best match is found by traversing all possible feature point pairs through the brute force matcher, but it can ensure that the most similar feature point pairs are found, thereby improving the accuracy of the registration. Based on the matched feature point pairs, the transformation matrix from one image to another can be calculated. The transformation matrix describes the geometric transformation relationship between images, such as translation, rotation, scaling, etc. Using the transformation matrix, the image before the salt spray test can be transformed into the same coordinate system as the image after the salt spray test, or vice versa. In this way, the same position in the two images corresponds to the same part of the motor housing. After the image registration, the changes such as rust spots can be more intuitively displayed in the image. Because the images have been accurately aligned, the pixel values ​​or texture features at the same position in the two images can be directly compared, so as to quickly and accurately detect changes such as rust spots. Accurate image registration can reduce false positives and negative negatives caused by image misalignment or deformation, and improve the reliability of rust detection.

[0047] Step S206, performing difference detection on the effective motor housing images before and after the salt spray test, and extracting the newly added rust spot area based on the difference detection result;

[0048] Specifically, step S206 includes:

[0049] Extracting a first pixel feature corresponding to the first effective motor housing image and a second pixel feature corresponding to the second effective motor housing image;

[0050] Subtracting pixels from the first effective motor housing image and the second effective motor housing image based on the first pixel feature and the second pixel feature to obtain a difference image;

[0051] Segment the difference image according to a preset threshold to obtain a corresponding binary image;

[0052] The newly added rust spots are extracted based on the binary image and marked.

[0053] In this embodiment, pixel features for describing the surface state of the motor housing are selected, including color features (RGB values), brightness features, and texture features. MATLAB image processing software is used to traverse each pixel in the first effective motor housing image and each pixel in the second effective motor housing image, extract the feature values ​​corresponding to the corresponding color features, brightness features, and other texture features, and save them in array form. The first effective motor housing image and the second effective motor housing image are spatially aligned, and the aligned first effective motor housing image and the second effective motor housing image are pixel-by-pixel subtracted to obtain a difference image. Each pixel value in the difference image represents the change amount of the corresponding position pixel before and after the salt spray test. A threshold is set according to a statistical method to distinguish the newly added rust spots. The pixel values ​​in the difference image are compared with the set threshold value, and the pixels greater than the threshold value are the changed areas, and the pixels less than or equal to the threshold value are the unchanged areas. The segmented image is converted into a binary image, in which the changed areas are represented by white and the unchanged areas are represented by black. In the binary image, the shape of the rust spot area is optimized by morphological operations, such as corrosion, expansion, and filling holes, so that it is more continuous and accurate. The contour detection algorithm is used to extract the contour of the rust area in the binary image. On the original first valid motor housing image, the second valid motor housing image or the difference image, a rectangular frame, a circular mark or a specific color is used to highlight the newly added rust area, and a text annotation is added.

[0054] In the above embodiment, pixel features are extracted from the first effective motor housing image before the salt spray test and the second effective motor housing image after the salt spray test. Pixel subtraction is performed based on the pixel features to directly generate a difference image, and the generated difference image can clearly show the newly added rust spot area on the motor housing after the salt spray test. The difference image is segmented by a preset threshold and converted into a binary image, so that the newly added rust spot area is more prominent in the image, which is convenient for subsequent extraction and annotation. On the basis of the binary image, the newly added rust spot area can be easily extracted and annotated in the effective motor housing image or the difference image, thereby improving the efficiency of rust spot recognition.

[0055] Step S208, constructing a rust spot recognition model according to the newly added rust spot area and the corresponding motor housing image;

[0056] Specifically, step S208 includes:

[0057] Generate a pixel mask corresponding to the newly added rust area;

[0058] Set the data path and traverse the second motor housing image, read the pixel mask corresponding to each second motor housing image and return it in array form;

[0059] Preprocessing the second motor housing image and the pixel mask;

[0060] Set data enhancement parameters and configure data enhancement logic based on the data enhancement parameters;

[0061] Iteratively run the data enhancement logic to perform data enhancement on the second motor housing image and the pixel mask, and return the data enhanced second motor housing image and the pixel mask;

[0062] The U-Net model is trained using the data-enhanced second motor housing image and pixel mask to obtain a rust spot recognition model.

[0063] In the above embodiment, by generating a pixel mask corresponding to the newly added rust area, the pixel mask accurately identifies the rust area in the image, and the model can learn the specific features and boundaries of the rust, thereby improving the detection accuracy. Set the data path and traverse the second motor housing image, read the pixel mask corresponding to each image, and return the data in the form of an array to facilitate subsequent programming processing and batch operation. Preprocessing the image and pixel mask helps to eliminate irrelevant information in the image, highlight the rust features, and improve the generalization ability and detection effect of the model. By setting reasonable data enhancement parameters, a large amount of new data similar to but different from the original data can be generated, thereby effectively avoiding overfitting and improving the robustness of the model. By iteratively running the data enhancement logic, a large amount of new training data can be generated in batches, which not only maintains the characteristics of the original data, but also increases diversity. The U-Net model is trained using the second motor housing image and pixel mask after data enhancement to obtain a rust recognition model, which can accurately identify and segment the rust area in the image.

[0064] Further, the second motor housing image and the pixel mask are preprocessed, including:

[0065] Adjusting the size of the second motor housing image and the pixel mask to a target size by an image scaling method;

[0066] Calculating a normalization factor by using a maximum pixel value and a minimum pixel value in a pixel array corresponding to the second motor housing image, and normalizing each pixel value in a pixel value array corresponding to the second motor housing image by using the normalization factor;

[0067] The mask array corresponding to the pixel mask is dimensionally expanded so that the dimension of the mask array reaches the target dimension.

[0068] Specifically, the second motor housing image to be processed is stored in the form of a digital matrix, in which each element represents the brightness or color value of a pixel. The pixel mask is a binary mask corresponding to the second motor housing image, which is used to identify a specific area in the image, such as the motor housing itself. Set the target width and height of the second motor housing image. Apply an image scaling algorithm to adjust the second motor housing image and its corresponding pixel mask to the target size. Optionally, the image scaling algorithm can use a bilinear interpolation method or a nearest neighbor interpolation method. For example, the target size is: width = 256 pixels, height = 256 pixels; the original image size of the second motor housing image is: width = 512 pixels, height = 512 pixels. Select a bilinear interpolation algorithm for scaling to adjust the original image size to the target size. Traverse the pixel array corresponding to the second motor housing image to find the maximum pixel value and the minimum pixel value. For each pixel value in the pixel array, use a normalization formula based on the maximum pixel value and the minimum pixel value to convert it to a range of 0 to 1. The pixel mask of this embodiment uses a binary mask, so no normalization is required. Finally, determine the target dimension of the mask array, such as increasing the channel dimension to match the input requirements of the image processing library, and then use an array processing library such as numpy to expand the dimension of the mask array.

[0069] In the above embodiment, by uniformly adjusting the size of the second motor housing image and the pixel mask, it is ensured that the data input into the model has a consistent dimension. Appropriately reducing the image size can reduce the amount of calculation of the model and improve training efficiency, while maintaining sufficient details helps the model capture the characteristics of rust spots. Scaling the pixel values ​​of the image data to a specific range through normalization helps to eliminate the brightness differences between different images caused by factors such as lighting and shooting conditions, so that the model can focus more on the rust features in the image. By expanding the dimension of the mask array of the pixel mask, it can be kept consistent with the second motor housing image in data structure.

[0070] Furthermore, the U-Net model is trained using the data-enhanced second motor housing image and pixel mask to obtain a rust spot recognition model, including:

[0071] The U-Net model is used as the base model, and the global average pooling layer, fully connected layer, and output layer of the base model are added to obtain the initial model;

[0072] Set the optimizer, loss function and evaluation index corresponding to the initial model;

[0073] The second motor housing image is used as the input of the initial model, and the pixel mask corresponding to the rust spot area is used as the output of the initial model for model training;

[0074] The initial model is iteratively trained with the optimizer and loss function until the output of the initial model meets the evaluation index, and the rust spot recognition model is obtained.

[0075] In the above embodiment, by introducing the U-Net model as the basis, and combining the global average pooling layer, the fully connected layer and the output layer for customization, the global average pooling layer helps to extract the global features of the image, and the fully connected layer can map these features to the final output space, thereby improving the generalization ability of the model. By setting a suitable optimizer and loss function, the optimizer can guide the update direction of the model parameters and accelerate the convergence process; while the loss function measures the difference between the model prediction and the true label, providing a clear goal for the optimization of the model. The large amount of new data generated by the data enhancement technology not only increases the diversity of the training samples, but also reduces the risk of overfitting the model to specific samples. The data after data enhancement is used as input, so that the model can learn a more robust and generalized feature representation, thereby improving the accuracy and reliability of rust detection. Setting clear evaluation indicators can objectively reflect the performance of the model on the test set, so that the output of the model gradually meets the requirements of the evaluation indicators, and finally obtains a rust detection model with excellent performance.

[0076] In the above-mentioned rust spot recognition model constructed based on the newly added rust spot area and the corresponding motor housing image, the image is first annotated to generate a mask of the rust spot area, and each pixel is marked as rust spot or non-rust spot. For example, each pixel of a color image contains three color channels: red, green, and blue (RGB), and the value of each channel is usually between 0 and 255. The value of each channel is normalized and converted to a range of 0 to 1. The mask is usually binary, that is, the values ​​contained are 0 and 1, where 1 represents rust spots and 0 represents background. First, set the data path and call the function to read the image and mask. By traversing the image directory, read each image and the corresponding mask, and return the image and mask array. Then, preprocess the data, adjust the image and mask size, normalize the image and mask array, and expand the mask array dimension, and return the preprocessed image and mask array. Next, create an object and set the data enhancement parameters. Iterate the image and mask in batches, perform data enhancement on the image and mask, and the data enhancement includes rotation, flipping, cropping, etc., and return the enhanced image and mask. Divide the data into training set and validation set, and return the training set and validation set data. Next, build the model, use ResNet50 as the base model, add a global average pooling layer, a fully connected layer, and an output layer, then compile the model, set the optimizer, loss function, and evaluation indicators. Set the batch size and number of training rounds, train the model, and provide a training data generator and a validation data generator. Use the preprocessed image and mask data to train the U-Net model. The input of the model is the original image, and the output is the corresponding rust mask. Select cross entropy loss as the loss function, and the Adam optimizer for iterative training to obtain the rust recognition model.

[0077] Step S210, identifying the rust type of the newly added rust area based on the rust identification model, and calculating the area corresponding to the newly added rust area.

[0078] Specifically, step S210 includes:

[0079] Acquire a real-time motor housing image, segment the real-time motor housing image using a rust spot recognition model, and identify and extract newly added rust spot areas;

[0080] Perform morphological operations on the newly added rust spots, including dilation and corrosion, eliminating noise points, filling holes, and retaining the main structure of the rust spots;

[0081] Convert the real-time motor housing image from RGB color space to Lab color space, and distinguish red rust spots from white rust spots based on the corresponding color components;

[0082] The convolutional neural network is used to extract features from the real-time motor housing image in the Lab color space to obtain the color features, texture features, and shape features of the newly added rust spot area.

[0083] Combining the color features, texture features, and shape features of the newly added rust spots, a multi-label classification model is constructed to simultaneously output the probability of red rust spots and the probability of white rust spots.

[0084] Output the red rust probability corresponding to each red rust spot and the white rust probability corresponding to each white rust spot according to the multi-label classification model;

[0085] The red rust area corresponding to the red rust spot and the white rust area corresponding to the white rust spot were calculated by the pixel counting method;

[0086] The actual rust area of ​​each newly added rust area is obtained by weighting the red rust probability, red rust area, white rust probability and white rust area.

[0087] Furthermore, if Figure 3 As shown, the entire rust area is traversed to determine whether there are untreated rust areas. If not, the process is terminated. If yes, the process is continued. For the detected rust area, the sub-image containing the rust is extracted, and the extracted rust sub-image is resized. The resized rust sub-image is normalized and converted into a unified standard format. The normalized rust sub-image is ready to be input into the classification model. The classification model predicts the rust type. The input rust sub-image is predicted by the trained classification model to determine which rust type it belongs to, and the proportion or degree of red rust in the rust area is calculated. If the "red rust rate" is greater than 0.5, the red rust in the area is more serious, and the red rust area record of the area is updated. The proportion or degree of white rust in the rust area is calculated to determine whether it is greater than 0.5. If the white rust rate is greater than 0.5, the white rust area record of the area is updated.

[0088] In the above embodiment, the real-time motor housing image is segmented using a trained rust spot recognition model, which can accurately identify and extract the rust spot area. By performing morphological operations such as expansion and corrosion on the rust spot area, noise points can be effectively eliminated and holes can be filled, while retaining the main structure of the rust spot, thereby improving the accuracy and reliability of rust spot recognition. The image is converted from the RGB color space to the Lab color space, and the color components are used to distinguish red rust spots from white rust spots. The Lab color space is more in line with the human eye's perception of color and can more accurately describe the color characteristics of rust spots. A multi-label classification model that can simultaneously output the probability of red rust spots and the probability of white rust spots is constructed, so that the model can simultaneously identify multiple types of rust spots, thereby improving the accuracy and comprehensiveness of distinguishing rust spot types. According to the red rust spot probability and white rust spot probability of each rust spot area output by the multi-label classification model, a weighted calculation is performed in combination with the red rust spot area and the white rust spot area to obtain the actual area of ​​each rust spot area, taking into account the diversity of rust spot types, so that the area calculation is more accurate.

[0089] According to another aspect of the embodiment of the present application, Figure 4 As shown, a device for identifying rust spots on a motor housing is provided, comprising:

[0090] An image acquisition module 401 is used to acquire a first motor housing image before a salt spray test and a second motor housing image after a salt spray test;

[0091] An image registration module 403 is used to perform image registration on the first motor housing image and the second motor housing image, so as to register the rust spot area in the first motor housing image with the rust spot area in the second motor housing image to obtain a valid motor housing image;

[0092] A difference detection module 405 is used to perform difference detection on the effective motor housing images before and after the salt spray test, and extract the newly added rust spot area based on the difference detection result;

[0093] Model building model 407, used to build a rust spot recognition model according to the newly added rust spot area and the corresponding motor housing image;

[0094] The rust spot recognition module 409 is used to recognize the rust spot type of the newly added rust spot area based on the rust spot recognition model, and calculate the area corresponding to the newly added rust spot area.

[0095] It should be noted that the image preprocessing module 401 in this embodiment can be used to execute step S202 in the embodiment of the present application, the image registration module 403 in this embodiment can be used to execute step S204 in the embodiment of the present application, the difference detection module 405 in this embodiment can be used to execute step S206 in the embodiment of the present application, the model building model 407 in this embodiment can be used to execute step S208 in the embodiment of the present application, and the rust spot recognition module 409 in this embodiment can be used to execute step S210 in the embodiment of the present application.

[0096] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the above modules as part of the device can be run in Figure 1 In the hardware environment shown, it can be implemented by software or by hardware.

[0097] Optionally, the click shell rust spot identification device also includes an image preprocessing module, and the image preprocessing includes specifically used for: denoising the collected first motor shell image and the second motor shell image respectively; contrast enhancing the denoised first motor shell image and the second motor shell image respectively; and grayscale processing the contrast enhanced first motor shell image and the second motor shell image respectively.

[0098] Optionally, the image acquisition module 403 is specifically used to: perform feature point detection on the first motor housing image and the second motor housing image respectively through a scale-invariant feature transformation algorithm to obtain a first detection result and a second detection result, extract a corresponding first descriptor based on the first detection result, and extract a corresponding second descriptor based on the second detection result; match the first motor housing image and the second motor housing image through a brute force matcher in combination with the first detection result and the first descriptor, the second detection result and the second descriptor to obtain similar feature point pairs; calculate a transformation matrix based on the similar feature point pairs, and transform the first motor housing image to a coordinate system corresponding to the second motor housing image based on the transformation matrix; transform the first motor housing image and the second motor housing image respectively based on the transformation matrix to achieve image registration to obtain a valid motor housing image, the valid motor housing image including a first effective motor housing image before the salt spray test and a second effective motor housing image after the salt spray test.

[0099] Optionally, the model building model 405 is specifically used to: extract a first pixel feature corresponding to the first effective motor housing image and a second pixel feature corresponding to the second effective motor housing image; perform pixel subtraction on the first effective motor housing image and the second effective motor housing image based on the first pixel feature and the second pixel feature to obtain a difference image; segment the difference image according to a preset threshold to obtain a corresponding binary image; extract a newly added rust spot area based on the binary image, and mark the newly added rust spot area.

[0100] Optionally, the difference detection module 407 is specifically used to: generate a pixel mask corresponding to the newly added rust spot area; set a data path and traverse the second motor housing image, read the pixel mask corresponding to each second motor housing image and return it in the form of an array; preprocess the second motor housing image and the pixel mask; set data enhancement parameters and configure data enhancement logic based on the data enhancement parameters; iteratively run the data enhancement logic to perform data enhancement on the second motor housing image and the pixel mask, and return the data enhanced second motor housing image and pixel mask; use the data enhanced second motor housing image and pixel mask to train the U-Net model to obtain a rust spot recognition model

[0101] Optionally, the rust spot recognition module 409 is specifically used to: obtain a real-time motor housing image, segment the real-time motor housing image through a rust spot recognition model, identify and extract newly added rust spot areas; perform morphological operations on the newly added rust spot areas, the morphological operations include expansion and corrosion, eliminating noise, filling holes, and retaining the main structure of the rust spot; convert the real-time motor housing image from the RGB color space to the Lab color space, and distinguish red rust spots from white rust spots based on corresponding color components; use a convolutional neural network to extract features of the real-time motor housing image in the Lab color space to obtain color features, texture features, and shape features of the newly added rust spot areas; construct a multi-label classification model for simultaneously outputting the probability of red rust spots and the probability of white rust spots in combination with the color features, texture features, and shape features of the newly added rust spot areas; output the probability of red rust spots corresponding to each red rust spot and the probability of white rust spots corresponding to the white rust spot according to the multi-label classification model; calculate the red rust spot area corresponding to the red rust spot and the white rust spot area corresponding to the white rust spot by a pixel counting method; and weight the red rust spot probability, the red rust spot area, the white rust spot probability, and the white rust spot area to obtain the actual rust spot area of ​​each newly added rust spot area.

[0102] According to another aspect of the embodiments of the present application, the present application provides an electronic device, such as Figure 5 As shown, it includes a memory 502, a processor 504, a communication interface 506 and a communication bus 508. The memory 502 stores a computer program that can be run on the processor 504. The memory 502 and the processor 504 communicate through the communication interface 506 and the communication bus 508. When the processor 502 executes the computer program, the steps of the above method are implemented.

[0103] The memory and processor in the above electronic device communicate through the communication bus and the communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0104] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0105] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0106] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above embodiments.

[0107] Optionally, in an embodiment of the present application, the computer-readable medium is configured to store program codes for the processor to execute the following steps:

[0108] Step S202, obtaining a first motor housing image before a salt spray test and a second motor housing image after a salt spray test;

[0109] Step S204, performing image registration on the first motor housing image and the second motor housing image, so as to register the rust spot area in the first motor housing image with the rust spot area in the second motor housing image to obtain a valid motor housing image;

[0110] Step S206, performing difference detection on the effective motor housing images before and after the salt spray test, and extracting the newly added rust spot area based on the difference detection result;

[0111] Step S208, constructing a rust spot recognition model according to the newly added rust spot area and the corresponding motor housing image;

[0112] Step S210, identifying the type of the newly added rust spot area based on the rust spot recognition model, and calculating the area corresponding to the type of the newly added rust spot area.

[0113] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0114] When the embodiments of the present application are specifically implemented, reference may be made to the above-mentioned embodiments, which have corresponding technical effects.

[0115] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0116] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0122] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0123] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for identifying rust spots on a motor housing, characterized in that: include: Acquire a first motor housing image before a salt spray test and a second motor housing image after a salt spray test; Performing image registration on the first motor housing image and the second motor housing image to register the rust spot area in the first motor housing image with the rust spot area in the second motor housing image to obtain a valid motor housing image; Performing difference detection on the effective motor housing images before and after the salt spray test, and extracting the newly added rust spot area based on the difference detection result; Constructing a rust spot recognition model according to the newly added rust spot area and the corresponding motor housing image; The rust spot type of the newly added rust spot area is identified based on the rust spot identification model, and the area corresponding to the newly added rust spot area is calculated.

2. The method for identifying rust spots on a motor housing according to claim 1, characterized in that: Performing image registration on the first motor housing image and the second motor housing image to register a rust spot region in the first motor housing image with a rust spot region in the second motor housing image to obtain a valid motor housing image, including: Performing feature point detection on the first motor housing image and the second motor housing image respectively by a scale-invariant feature transformation algorithm to obtain a first detection result and a second detection result, extracting a corresponding first descriptor based on the first detection result, and extracting a corresponding second descriptor based on the second detection result; Combine the first detection result and the first descriptor, the second detection result and the second descriptor to match the first motor housing image and the second motor housing image through a brute force matcher to obtain a similar feature point pair; Calculating a transformation matrix based on the similar feature point pairs, and transforming the first motor housing image into a coordinate system corresponding to the second motor housing image based on the transformation matrix; The first motor housing image and the second motor housing image are transformed respectively based on the transformation matrix to achieve image registration to obtain an effective motor housing image, wherein the effective motor housing image includes a first effective motor housing image before the salt spray test and a second effective motor housing image after the salt spray test.

3. The method for identifying rust spots on a motor housing according to claim 2, characterized in that: Performing difference detection on the effective motor housing images before and after the salt spray test, and extracting the newly added rust spot area based on the difference detection result, including: Extracting a first pixel feature corresponding to the first effective motor housing image and a second pixel feature corresponding to the second effective motor housing image; Subtracting the first effective motor housing image from the second effective motor housing image based on the first pixel feature and the second pixel feature to obtain a difference image; Segmenting the difference image according to a preset threshold to obtain a corresponding binary image; A newly added rust spot area is extracted based on the binary image, and the newly added rust spot area is marked.

4. The method for identifying rust spots on a motor housing according to claim 1, characterized in that: A rust spot recognition model is constructed according to the newly added rust spot area and the corresponding motor housing image, including: Generating a pixel mask corresponding to the newly added rust spot area; Set a data path and traverse the second motor housing image, read the pixel mask corresponding to each second motor housing image and return it in an array form; Preprocessing the second motor housing image and the pixel mask; Setting data enhancement parameters and configuring data enhancement logic based on the data enhancement parameters; Iteratively running the data enhancement logic to perform data enhancement on the second motor housing image and the pixel mask, and returning the data enhanced second motor housing image and pixel mask; The second motor housing image after data enhancement and the pixel mask are used to train the U-Net model to obtain a rust spot recognition model.

5. The method for identifying rust spots on a motor housing according to claim 4, characterized in that: Preprocessing the second motor housing image and the pixel mask includes: Adjusting the size of the second motor housing image and the pixel mask to a target size by an image scaling method; Calculating a normalization factor by using a maximum pixel value and a minimum pixel value in a pixel array corresponding to the second motor housing image, and normalizing each pixel value in a pixel value array corresponding to the second motor housing image by using the normalization factor; The mask array corresponding to the pixel mask is dimensionally expanded so that the dimension of the mask array reaches a target dimension.

6. The method for identifying rust spots on a motor housing according to claim 4, characterized in that: The U-Net model is trained using the data-enhanced second motor housing image and pixel mask to obtain a rust spot recognition model, including: The U-Net model is used as the basic model, and the global average pooling layer, the fully connected layer, and the output layer of the basic model are added to obtain the initial model; Setting the optimizer, loss function and evaluation index corresponding to the initial model; Using the second motor housing image as the input of the initial model and using the pixel mask corresponding to the rust spot area as the output of the initial model to perform model training; The initial model is iteratively trained in combination with the optimizer and the loss function until the output of the initial model meets the evaluation index, thereby obtaining a rust spot recognition model.

7. The method for identifying rust spots on a motor housing according to claim 1, characterized in that: Identifying the rust type of the newly added rust area based on the rust identification model, and calculating the area corresponding to the newly added rust area, including: Acquire a real-time motor housing image, segment the real-time motor housing image using the rust spot recognition model, and identify and extract newly added rust spot areas; Performing morphological operations on the newly added rust spot area, wherein the morphological operations include dilation and corrosion, eliminating noise points, filling holes, and retaining the main structure of the rust spot; Converting the real-time motor housing image from the RGB color space to the Lab color space, and distinguishing red rust spots from white rust spots based on corresponding color components; Using a convolutional neural network to extract features from the real-time motor housing image in the Lab color space, the color features, texture features, and shape features of the newly added rust spot area are obtained; Combining the color feature, the texture feature, and the shape feature of the newly added rust spot area, constructing a multi-label classification model for simultaneously outputting the probability of red rust spots and the probability of white rust spots; Outputting the red rust spot probability corresponding to each of the red rust spots and the white rust spot probability corresponding to the white rust spots according to the multi-label classification model; Calculate the red rust spot area corresponding to the red rust spot and the white rust spot area corresponding to the white rust spot by a pixel counting method; The actual rust spot area of ​​each newly added rust spot region is obtained by weighting the red rust spot probability, the red rust spot area, the white rust spot probability, and the white rust spot area.

8. A device for identifying rust spots on a motor housing, characterized in that: include: An image acquisition module, used to acquire a first motor housing image before a salt spray test and a second motor housing image after a salt spray test; an image registration module, configured to perform image registration on the first motor housing image and the second motor housing image, so as to register the rust spot region in the first motor housing image with the rust spot region in the second motor housing image to obtain a valid motor housing image; A difference detection module, used to perform difference detection on the effective motor housing images before and after the salt spray test, and extract the newly added rust spot area based on the difference detection result; A model building model is used to build a rust spot recognition model according to the newly added rust spot area and the corresponding motor housing image; The rust spot recognition module is used to identify the rust spot type of the newly added rust spot area based on the rust spot recognition model, and calculate the area corresponding to the newly added rust spot area.

9. An electronic device, characterized in that: It includes a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate through the communication bus and the communication interface. It is characterized in that when the processor executes the computer program, the motor housing rust spot identification method described in any one of claims 1 to 7 is implemented.

10. A computer readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the method for identifying rust spots on a motor housing as described in any one of claims 1 to 7.

Images