Microscopic defect processing method and device based on image recognition and computer equipment

Abstract

The present application relates to a kind of microdefect processing method, device and computer equipment based on image recognition, the method includes obtaining input image, pre-processing input image, obtain to be identified image, utilize the deep learning model of trained to be identified image and carry out segmentation identification, output identification result, according to the determination defect type of identification result, according to defect type, identification result is post-processed and counted, wherein, deep learning model is based on decoder-encoder network structure construction, encoder and decoder are fused in the way of splicing with the feature map of same size in both, the image of laser in situ alloying sample is identified by the deep learning model of trained in the present application, after identifying defect type, corresponding processing and statistics are carried out for defect type, the present application combines traditional digital image processing method and deep learning method, for the efficient marking and integrated identification and statistics of defect, identification precision is high, implementation mode is simpler.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method, apparatus and computer equipment for processing microscopic defects based on image recognition. Background Technology

[0002] Laser in-situ alloying technology is a method that uses mixed elemental powders or specific alloy powders as raw materials to directly alloy and simultaneously form high-density samples during the selective laser melting process. It can overcome the limitations of customized powder preparation processes. Compared with selective laser melting (SLM), which usually uses pre-alloyed powders as raw materials, laser in-situ alloying technology has extremely high flexibility in changing alloy composition and can be used in the fields of new material development and multi-material printing.

[0003] Due to the characteristics of the process, samples prepared by laser in-situ alloying not only exhibit defects such as cracks and pores common in traditional SLM samples, but also produce unique microstructural defects such as a large number of unmelted particles. These microstructural defects not only worsen the compositional uniformity but also significantly reduce the mechanical properties and corrosion fatigue performance of the samples, ultimately compromising the final performance of the 3D printed product.

[0004] Currently, image recognition methods are mainly divided into two categories: traditional digital image processing methods and deep learning methods based on convolutional neural networks.

[0005] Traditional digital image processing methods primarily classify images based on the grayscale value, size, and shape of pixels. However, in metallographic photographs of additively manufactured samples, pores, cracks, and unmelted areas often have similar grayscale values ​​and are frequently interconnected. Therefore, traditional digital image processing methods struggle to accurately separate different types of defects.

[0006] Deep learning methods can train deep neural networks based on convolutional layers to learn features from sample photos, but they often suffer from overfitting and underfitting, leading to misidentification and affecting the final recognition results. Furthermore, the process of preparing labels before training the model is time-consuming and labor-intensive.

[0007] Therefore, current methods for product image recognition processing suffer from low recognition accuracy and are quite complex. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to overcome the shortcomings of the prior art and provide a micro-defect processing method, apparatus and computer equipment based on image recognition, so as to solve the problems of low recognition accuracy and high complexity in the existing product image recognition processing methods.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a microscopic defect processing method based on image recognition, comprising:

[0010] The input image is acquired and preprocessed to obtain the image to be recognized.

[0011] The trained deep learning model is used to segment and identify the image to be identified, and the identification result is output.

[0012] The defect type is determined based on the identification results, and the identification results are then post-processed and statistically analyzed based on the defect type.

[0013] The deep learning model is built on a decoder-encoder network structure, where the encoder and decoder are fused by concatenating feature maps of the same size.

[0014] Furthermore, the preprocessing of the input image to obtain the image to be recognized includes:

[0015] The input image is converted to grayscale to obtain a grayscale image;

[0016] The grayscale image is subjected to scale recognition to determine the relationship between the true length of the target and the number of pixels;

[0017] Read the defect area of ​​the grayscale image, determine the area outside the defect area as the background area, delete the background in the background area and fill it with a standardized background.

[0018] Furthermore, the training method for the deep learning model includes:

[0019] A dataset was generated based on the original metallographic images of manufactured samples from different batches and with different compositions; each original image contained more than one type of defect.

[0020] The original photos in the dataset are labeled to obtain labeled photos;

[0021] Based on the original and labeled photos, an augmentation operation is performed to generate a sample set, which is then divided into a training set and a test set.

[0022] The neural network model is iteratively trained using a training set. After each training round, a test set is used to perform a test and the accuracy of the test set recognition is given. After training, the model parameters with the highest accuracy of the test set recognition are selected as the model parameters of the deep learning model, and the deep learning model is output.

[0023] Furthermore, the iterative training of the neural network model using the training set includes:

[0024] The encoder extracts texture features from the input images in the training set to obtain feature maps;

[0025] The decoder restores the feature map to its original resolution;

[0026] The feature maps of the same size in the encoder and decoder are fused by splicing to obtain the fusion result;

[0027] The weights of each convolutional kernel in the convolutional layer of the model are adjusted through iterative training to obtain the deep learning model.

[0028] Furthermore, the step of using a trained deep learning model to segment and identify the image to be identified, and outputting the identification result, includes:

[0029] The image to be recognized is cropped to obtain an image of the first preset size;

[0030] The image of the first preset size is identified, and the identification result is cropped to the second preset size;

[0031] The recognition results with the second preset size are fused together, and the fused result is used as the final recognition result;

[0032] Wherein, the second preset size is smaller than the first preset size.

[0033] Furthermore, the identification results include different types of defects; the step of determining the defect type based on the identification results and performing post-processing on the identification results based on the defect type includes:

[0034] The identification results are statistically analyzed for defect types, including cracks, holes, and unmelted areas.

[0035] Different post-processing steps are performed for different defect types, specifically including:

[0036] For crack identification results, small objects are removed to reduce interference;

[0037] For the identification results of holes and unmelted areas, a circular region opening operation with a preset diameter of pixels is performed, that is, erosion followed by expansion, to connect the unconnected identification areas, and then a flooding filling operation is performed.

[0038] Furthermore, the defect type statistics of the identification results include:

[0039] For crack defects, the percentage of pixels identified as cracks is statistically analyzed, and the result is used as the percentage of crack defects.

[0040] For defects such as holes and unmelted areas, the percentage of pixels identified as holes or unmelted areas is statistically analyzed, and the result is used as the percentage of defects for holes or unmelted areas. At the same time, each connected region in the statistical analysis results is treated as a separate defect, and the number of pixels contained in the defect is calculated as the area of ​​the defect. Then, based on the scale information read during the preprocessing of the input image, the pixel area is converted into the actual area, and the reduced diameter is calculated. Finally, the above identification statistics are summarized.

[0041] Furthermore, the statistical information of the identification results includes: the proportion of crack defects, the proportion of pore defects, the proportion of unmelted area defects, the number and average diameter of pore defects, and the number and average diameter of unmelted area defects.

[0042] This application provides an imaging device for microscopic defects, comprising:

[0043] The acquisition module is used to acquire an input image and preprocess the input image to obtain an image to be recognized;

[0044] The recognition module is used to segment and recognize the image to be recognized using a trained deep learning model and output the recognition result.

[0045] The processing module is used to determine the defect type based on the identification result, and to perform post-processing and statistics on the identification result based on the defect type;

[0046] The deep learning model is built on a decoder-encoder network structure, where the encoder and decoder are fused by concatenating feature maps of the same size.

[0047] This application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method provided in any of the above embodiments.

[0048] The beneficial effects that can be achieved by adopting the above technical solution in this invention include:

[0049] This invention provides a method, apparatus, and computer device for processing microscopic defects based on image recognition. This invention uses a trained deep learning model to recognize images of laser in-situ alloyed samples. After identifying the defect type, it performs corresponding processing based on the defect type. This application combines traditional digital image processing methods with deep learning methods for efficient labeling, integrated recognition, and statistics of defects. It has high recognition accuracy and is simpler to implement. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram illustrating the steps of the microscopic defect processing method based on image recognition of the present invention;

[0052] Figure 2 This is a schematic diagram of the deep learning model training process of the present invention;

[0053] Figure 3 This is a comparison diagram of the image to be identified and the recognition result provided by the present invention;

[0054] Figure 4 This is a schematic diagram of the image device for microscopic defects based on image recognition according to the present invention;

[0055] Figure 5 This is a schematic diagram of the hardware structure of the implementation environment for the microscopic defect processing method based on image recognition of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0057] The following describes a specific image recognition-based method, apparatus, and computer device for processing microscopic defects, provided in an embodiment of this application, with reference to the accompanying drawings.

[0058] like Figure 1 As shown, the microscopic defect processing method based on image recognition provided in this application embodiment includes:

[0059] S101, acquire the input image, preprocess the input image to obtain the image to be recognized;

[0060] Understandably, the input image is obtained by taking a picture of the surface of the alloyed product with a camera. The camera is positioned above the alloyed product to clearly capture its surface. After obtaining the input image, it is preprocessed before being input into the deep learning model to obtain the image to be recognized. Then, the image to be recognized is directly input into the deep learning model.

[0061] In some embodiments, the preprocessing of the input image to obtain the image to be recognized includes:

[0062] The input image is converted to grayscale to obtain a grayscale image;

[0063] The grayscale image is subjected to scale recognition to determine the relationship between the true length of the target and the number of pixels;

[0064] Read the defect area of ​​the grayscale image, determine the area outside the defect area as the background area, delete the background in the background area and fill it with a standardized background.

[0065] Specifically, image preprocessing mainly includes grayscale image conversion. If the input image is an RGB image, it is converted to a grayscale image; if the input image is a grayscale image, the operation is skipped, and the image contrast is automatically adjusted to make defects more prominent and easier to identify.

[0066] The ruler recognition process involves automatically reading the ruler area located in the lower right corner of the image and identifying the length of the horizontal line representing the number of pixels in the area. This establishes a relationship between the actual length and the number of pixels. After recognizing the ruler, the ruler area is replaced with a standard grayscale background to eliminate interference from the ruler area.

[0067] Background standardization specifically involves: after reading the defect area separately in the image using a grayscale thresholding algorithm, the remaining part is considered the background by default. After deleting the background, a standardized background is added in the original position. This can complete the background standardization of photos with different brightness, eliminate the adverse effects caused by different brightness, and improve the recognition accuracy of subsequent deep learning models.

[0068] It should be noted that factors such as background brightness and magnification of metallographic photographs of additively manufactured samples can affect the recognition performance of deep learning models. In this application, the operation of defect region extraction, background deletion, and addition of standard background can unify the background brightness and improve the model's recognition accuracy; automatically reading the image scale numbers and scale length can determine the image magnification and establish the relationship between pixels and true length, providing a basis for subsequent quantitative statistics.

[0069] S102, The trained deep learning model is used to segment and identify the image to be identified, and the identification result is output;

[0070] In some embodiments, the training method of the deep learning model includes:

[0071] A dataset was generated based on the original metallographic images of manufactured samples from different batches and with different compositions; each original image contained more than one type of defect.

[0072] The original photos in the dataset are labeled to obtain labeled photos;

[0073] Based on the original and labeled photos, an augmentation operation is performed to generate a sample set, which is then divided into a training set and a test set.

[0074] The neural network model is iteratively trained using a training set. After each training round, a test set is used to perform a test and the accuracy of the test set recognition is given. After training, the model parameters with the highest accuracy of the test set recognition are selected as the model parameters of the deep learning model, and the deep learning model is output.

[0075] Specifically, the training of the deep learning model in this application requires the preparation of data. The data required for training the model includes a dataset consisting of 30 typical metallographic photographs (preferably no less than 15) of manufactured samples from different batches and with different compositions. Each photograph contains one or more of the three defects: cracks, holes, and unmelted areas.

[0076] Then, the porosity and unmelted area defects in the dataset images were manually labeled, followed by labeling the remaining crack defects using a grayscale thresholding method. After labeling, both the original and labeled images were cropped in the same way and expanded by rotation, symmetry, and other operations. Each original metallographic image was expanded into approximately 160 small images of 336 pixels × 336 pixels, forming the dataset.

[0077] This application utilizes digital image processing methods to provide accurate labels for deep learning training. Since crack defects are irregular in shape and widely distributed, manual labeling is time-consuming and labor-intensive. This invention employs a grayscale thresholding method to uniformly extract cracks, holes, and unmelted areas. Based on this, only holes and unmelted areas are manually labeled, significantly reducing the workload.

[0078] Eighty percent of the photos in the dataset were used as the training set, and the remaining 20% ​​were used as the test set to verify the training effect. The training model used the Stochastic Gradient Descent with Momentum (SGDM) solver, with cross-entropy as the loss function. The batch size was 16 small-sized photos, and the training run consisted of 50 epochs. The initial learning rate was 0.001, and it was adjusted to half its initial value every 10 epochs. During training, a test set was used after each epoch to test the model and obtain the recognition accuracy. After training, the model parameters with the highest recognition accuracy on the test set were selected as the final model and saved.

[0079] This application uses digital image processing methods to post-process the recognition results of the deep learning model, resulting in better recognition outcomes. It can identify and statistically analyze the three types of defects unique to laser in-situ alloyed samples—cracks, voids, and lack of fusion—and provide their respective proportions. For voids and lack of fusion with spherical structures, the application can also provide the number and diameter of the defects based on the recognition results.

[0080] In some embodiments, the iterative training of the neural network model using a training set includes:

[0081] The encoder extracts texture features from the input images in the training set to obtain feature maps;

[0082] The decoder restores the feature map to its original resolution;

[0083] The feature maps of the same size in the encoder and decoder are fused by splicing to obtain the fusion result;

[0084] The weights of each convolutional kernel in the convolutional layer of the model are adjusted through iterative training to obtain the deep learning model.

[0085] Specifically, such as Figure 2 As shown, the deep learning model is based on the classic U-Net architecture. The left side represents the structural encoder, which downsamples the input image to extract high-level texture features. The input image is then upsampled by the transposed convolutional layer in the decoder on the right to restore the original image resolution. In the model diagram, the horizontal connections between the encoder and decoder represent skip connections, indicating that feature maps of the same size from both are fused in a concatenated manner, allowing the decoder to better recover the recognition features. All convolutional layers and transposed convolutional layers used are 3x3 unpadded convolutions. The convolutional information for each layer is labeled in the diagram; the three numbers represent the width, height, and number of convolutional kernels in the output image of that layer, respectively. Finally, a Softmax layer determines the classification corresponding to each pixel and outputs the classification result.

[0086] In some embodiments, the step of segmenting and recognizing the image to be recognized using a trained deep learning model and outputting the recognition result includes:

[0087] The image to be recognized is cropped to obtain an image of the first preset size;

[0088] The image of the first preset size is identified, and the identification result is cropped to the second preset size;

[0089] The recognition results with the second preset size are fused together, and the fused result is used as the final recognition result;

[0090] Wherein, the second preset size is smaller than the first preset size.

[0091] Specifically, the deep learning model takes a 336-pixel × 336-pixel image as input. During recognition, the original image is first cropped into smaller 336-pixel × 336-pixel images, which are then recognized separately. The recognition results are then stitched together to complete the recognition of the entire image. Since boundary pixels are lost in each convolution, the accuracy of image edge recognition decreases. To avoid misidentification caused by small image boundaries, this invention employs an Overlap strategy. That is, a 336-pixel × 336-pixel image is used for recognition, but the recognition result only saves the central portion of a second preset size (300-pixel × 300-pixel), discarding areas that may cause boundary misidentification.

[0092] S103, determine the defect type based on the identification result, and perform post-processing and statistics on the identification result based on the defect type;

[0093] The deep learning model is built on a decoder-encoder network structure, where the encoder and decoder are fused by concatenating feature maps of the same size.

[0094] In this application, the identification results include different types of defects; the step of determining the defect type based on the identification results and performing post-processing on the identification results based on the defect type includes:

[0095] The identification results are statistically analyzed for defect types, including cracks, holes, and unmelted areas.

[0096] Different post-processing steps are performed for different defect types, specifically including:

[0097] For crack identification results, small objects are removed to reduce interference;

[0098] For the identification results of holes and unmelted areas, a circular region opening operation with a preset diameter of pixels is performed, that is, erosion followed by expansion, to connect the unconnected identification areas, and then a flooding filling operation is performed.

[0099] The step of performing defect type statistics on the identification results includes:

[0100] For crack defects, the percentage of pixels identified as cracks is statistically analyzed, and the result is used as the percentage of crack defects.

[0101] For defects such as holes and unmelted areas, the percentage of pixels identified as holes or unmelted areas is statistically analyzed, and the result is used as the percentage of defects for holes or unmelted areas. At the same time, each connected region in the statistical analysis results is treated as a separate defect, and the number of pixels contained in the defect is calculated as the area of ​​the defect. Then, based on the scale information read during the preprocessing of the input image, the pixel area is converted into the actual area, and the reduced diameter is calculated. Finally, the above identification statistics are summarized.

[0102] Specifically, the statistical information of the identification results includes: the proportion of crack defects, the proportion of pore defects, the proportion of unmelted area defects, the number and average diameter of pore defects, and the number and average diameter of unmelted area defects.

[0103] This application employs different post-processing methods for different types of defects.

[0104] For crack identification results, small objects are removed to reduce interference;

[0105] For the identification results of holes and unmelted areas, a circular region opening operation with a diameter of 5 pixels is performed, that is, erosion followed by expansion, to connect the unconnected identification areas, and then flooding and filling operations are performed.

[0106] Specifically, different summary and statistical methods are used for different types of defects:

[0107] For crack defects, the percentage of pixels identified as cracks is statistically analyzed, and the result is used as the percentage of crack defects.

[0108] For defects such as holes and unmelted areas, the percentage of pixels identified as holes or unmelted areas is statistically analyzed, and this result is used as the percentage of holes or unmelted area defects. Simultaneously, each connected region in the statistical analysis is treated as a separate defect, and the number of pixels contained in that defect is calculated as its area. Then, based on the scale information read during preprocessing, the pixel area is converted to the actual area, and the reduced diameter is calculated. Finally, the above statistical information is summarized, and the output information for each metallographic image includes: percentage of crack defects, percentage of hole defects, percentage of unmelted area defects, number and average diameter of hole defects, and number and average diameter of unmelted area defects.

[0109] The technical solution provided in this application enables rapid and accurate identification of various product defects. Specifically, it employs a combination of digital image processing and deep learning to correctly identify and segment multiple defects (including cracks, holes, and unmelted areas) that cannot be segmented by conventional image processing methods. For a standard-sized metallographic photograph of 2560 pixels × 1920 pixels, the processing time for each photograph is approximately 3 seconds.

[0110] This application enables quantitative statistical analysis of the identification results, establishing a relationship between image pixel dimensions and actual dimensions through an automatic scale reading method. Based on the correct identification of various defects, the proportion of each type of defect is statistically analyzed; for hole defects and unmelted area defects, the area and reduced diameter of each defect are statistically analyzed, ultimately providing quantitative analysis results for the image to be analyzed.

[0111] In addition, this application also enables batch processing and summarization. The background normalization algorithm and scale reading algorithm included in the image preprocessing method greatly enhance the robustness of the model, enabling correct identification and statistics for photos with different background brightness and magnification. A batch processing algorithm is also designed to process multiple photos at once and provide statistical results and summaries for each photo.

[0112] As a specific implementation method, this application uses the microscopic metallographic images of a laser-alloyed in-situ high-temperature alloy GH3230 sample as the experimental object. Based on the standard composition of GH3**, modifications were made by adding elements such as Co, Fe, Ni, W, and Mo. Samples were prepared using different printing processes, and after pre-grinding and polishing, they were observed and photographed under an optical microscope. The microscopic defects in the metallographic images of the samples included three types: cracks, pores, and lack of fusion.

[0113] First, a deep learning model was pre-trained. Specifically, the training data consisted of 30 typical metallographic images of GH3230 laser in-situ alloyed samples from different batches, with different compositions and preparation parameters. Each image contained one or more of the following defects: cracks, pores, and unmelted areas. Pores and unmelted areas in the dataset were manually labeled, and the remaining crack defects were labeled using a grayscale thresholding method. After labeling, both the original and labeled images were cropped in the same way, starting from the top left corner and cropping every 118 pixels to a size of 336 pixels × 336 pixels. The cropped images were then rotated by 90°, 180°, and 270°, with a symmetrical operation performed after each rotation. Finally, the small-sized images containing the magnification bar in the lower left corner were deleted to eliminate interference. Each original metallographic photograph can be expanded into approximately 168 smaller photographs of 336 pixels × 336 pixels after the above operations. Finally, the 30 original mirror photographs are expanded into 5040 smaller photographs of 336 pixels × 336 pixels to form a dataset.

[0114] 80% of the photos in the dataset were used as the training set, and the remaining 20% ​​were used as the test set to verify the training effect. The training model used the Stochastic Gradient Descent with Momentum (SGDM) solver, with cross-entropy as the loss function. The batch size was 16 small-sized photos, and the training epochs were 50. The initial learning rate was 0.001, and it was adjusted to half its initial value every 10 epochs. During training, a test set was used after each epoch to test the model and obtain the recognition accuracy. After training, the model parameters with the highest recognition accuracy on the test set were selected as the final model. The recognition accuracy was defined as:

[0115]

[0116] After 3 hours of training on an NVIDIA RTX 5000 single GPU, the best recognition accuracy on the training set was 97.63%, and the best recognition accuracy on the test set was 95.31%.

[0117] A metallographic photograph of a GH3230 laser in-situ alloyed sample outside the dataset was used as the image to be identified. A trained deep learning model was used for identification, and quantitative identification results were given.

[0118] Before input, the image needs to be preprocessed. The input image is an RGB image, which is converted into a grayscale image and its brightness is adjusted for uniformity. Pixels with a grayscale value below 150 are identified as defects to be identified. Other pixels in the image are regarded as background, and the grayscale value of the background pixels is uniformly modified to 175 to uniformize the brightness of the entire image. Finally, the contrast is adjusted to make the pixels in the defect area to be identified more obvious from the background pixels.

[0119] Ruler Recognition: Automatically reads the ruler area located in the lower right corner of the image and identifies the length of the horizontal lines representing the number of pixels in the area, establishing a relationship between the actual length and the number of pixels. The ruler number is read as 200μm, and the ruler line length is 212 pixels. After ruler recognition, the ruler area is replaced with a standard grayscale background to eliminate interference from the ruler area.

[0120] A trained deep learning model is used to perform semantic segmentation and recognition on preprocessed images. The input image size of the deep learning model is a first preset size of 336 pixels × 336 pixels. During recognition, the original image is first cropped into smaller 336-pixel × 336-pixel images and then recognized separately. The recognition result only saves the central part of the second preset size of 300 pixels × 300 pixels, discarding areas that may cause boundary misidentification. The recognition results are then stitched together to complete the recognition of the entire image.

[0121] Different post-processing methods are used for different types of defects. For crack identification results, small objects are deleted to reduce interference. For hole and unmelted area identification results, a circular region opening operation with a diameter of 5 pixels is performed, that is, erosion followed by expansion, to connect unconnected identification regions, and then flooding and filling operations are performed.

[0122] For crack defects, the percentage of pixels identified as cracks is statistically analyzed, and the result is used as the percentage of crack defects.

[0123] For defects such as holes and unmelted areas, the percentage of pixels identified as holes or unmelted areas is statistically analyzed, and this result is used as the percentage of holes or unmelted area defects. Simultaneously, each connected region in the statistical analysis is treated as a separate defect, and the number of pixels contained in that defect is calculated as its area. Then, based on the scale information read during preprocessing, the pixel area is converted to the actual area, and the reduced diameter is calculated. Finally, the above statistical information is summarized, and the output information for each metallographic image includes: percentage of crack defects, percentage of hole defects, percentage of unmelted area defects, number and average diameter of hole defects, and number and average diameter of unmelted area defects.

[0124] like Figure 3 As shown, this embodiment of the application uses three metallographic photographs of samples with different compositions and manufacturing processes to test the model's performance. The photographs contain defects such as cracks, holes, and unmelted particles for identification. The original image size is 2560 pixels × 1920 pixels. The recognition was performed using a standard-configuration office laptop, with each photograph taking approximately 3 seconds to process. The recognition results are shown in Table 1.

[0125]

[0126] Wherein, (a) is the image to be identified, (b) is the recognition result of (a), (c) is the image to be identified, (d) is the recognition result of (c), (e) is the image to be identified, and (f) is the recognition result of (e).

[0127] like Figure 4 As shown, this application embodiment provides a microscopic defect processing device based on image recognition, including:

[0128] The acquisition module 401 is used to acquire an input image and preprocess the input image to obtain an image to be recognized;

[0129] The recognition module 402 is used to segment and recognize the image to be recognized using a trained deep learning model and output the recognition result;

[0130] Processing module 403 is used to determine the defect type based on the identification result, and to perform post-processing and statistics on the identification result based on the defect type;

[0131] The deep learning model is built on a decoder-encoder network structure, where the encoder and decoder are fused by concatenating feature maps of the same size.

[0132] The working principle of the microscopic defect processing device based on image recognition provided in this application is as follows: the acquisition module 401 acquires an input image and preprocesses the input image to obtain an image to be recognized; the recognition module 402 uses a trained deep learning model to segment and recognize the image to be recognized and outputs the recognition result; the processing module 403 determines the defect type according to the recognition result and performs post-processing and statistics on the recognition result according to the defect type; wherein, the deep learning model is constructed based on a decoder-encoder network structure, and the encoder and decoder fuse feature maps of the same size in a concatenated manner.

[0133] This application provides a computer device, including a memory 1 and a processor 2, and may further include a network interface 3. The memory stores a computer program. The memory 1 may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The computer device stores an operating system 4. The memory 1 is an example of a computer-readable medium. When the computer program is executed by the processor, it causes the processor 2 to perform a microscopic defect processing method based on image recognition. Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0134] In one embodiment, the image recognition-based microscopic defect processing method provided in this application can be implemented as a computer program, which can be implemented in, for example... Figure 5 It runs on the computer device shown.

[0135] In some embodiments, when the computer program is executed by the processor, the processor performs the following steps: acquiring an input image, preprocessing the input image to obtain an image to be identified; segmenting and identifying the image to be identified using a trained deep learning model, and outputting the identification result; determining the defect type based on the identification result, and performing post-processing and statistics on the identification result based on the defect type; wherein the deep learning model is constructed based on a decoder-encoder network structure, and the encoder and decoder fuse feature maps of the same size in a concatenated manner.

[0136] This application also provides a computer storage medium, examples of which include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital optical disc (DVD) or other optical storage, magnetic tape storage or other magnetic storage devices, or any other non-transfer medium, which can be used to store information that can be accessed by a computing device.

[0137] In some embodiments, the present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, acquires an input image, preprocesses the input image to obtain an image to be identified, segments and identifies the image to be identified using a trained deep learning model, and outputs an identification result; determines the defect type based on the identification result, and performs post-processing and statistics on the identification result based on the defect type; wherein the deep learning model is constructed based on a decoder-encoder network structure, and the encoder and decoder fuse feature maps of the same size in a concatenated manner.

[0138] In summary, this invention provides a method, apparatus, and computer device for processing microscopic defects based on image recognition. The method includes acquiring an input image, preprocessing the input image to obtain an image to be recognized, segmenting and recognizing the image to be recognized using a trained deep learning model, outputting recognition results, determining the defect type based on the recognition results, and performing post-processing and statistics on the recognition results based on the defect type. The deep learning model is constructed based on a decoder-encoder network structure, where the encoder and decoder fuse feature maps of the same size by concatenation. This invention uses a trained deep learning model to recognize images of laser-alloyed samples. After identifying the defect type, corresponding processing is performed based on the defect type. This application combines traditional digital image processing methods with deep learning methods for efficient defect labeling and integrated recognition and statistics, achieving high recognition accuracy and a simpler implementation.

[0139] It is understood that the method embodiments provided above correspond to the device embodiments described above, and the specific details can be referred to each other, which will not be repeated here.

[0140] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0141] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus processor to produce a machine, such that the instructions, which execute via the computer or other programmable data processing apparatus processor, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction methods implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0144] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for processing microscopic defects based on image recognition, characterized in that, include: An input image is acquired, and the input image is preprocessed to obtain an image to be recognized; wherein, the preprocessing of the input image to obtain the image to be recognized includes: performing grayscale conversion on the input image to obtain a grayscale image; performing scale recognition on the grayscale image to determine the relationship between the true length of the target and the number of pixels; reading the defect area of ​​the grayscale image, determining the area outside the defect area as the background area, deleting the background in the background area and filling it with a standardized background; The image to be identified is segmented and identified using a trained deep learning model, and the identification result is output. The process includes: cropping the image to be identified to obtain an image of a first preset size; identifying the image of the first preset size and cropping the identification result to a second preset size; fusing the identification results with the second preset size and using the fused result as the final identification result; wherein the second preset size is smaller than the first preset size; and the identification result includes different types of defects. Based on the identification results, the defect type is determined, and the identification results are post-processed and statistically analyzed according to the defect type, including: performing defect type statistics on the identification results; the defect types include cracks, holes, and unmelted areas; performing corresponding post-processing for different defect types, specifically including: for the identification results of cracks, deleting small objects to reduce interference; for the identification results of holes and unmelted areas, performing a circular region opening operation with a preset diameter of pixels, that is, first eroding and then expanding, connecting the unconnected identification regions, and then performing a flooding and filling operation; The deep learning model is built on a decoder-encoder network structure, where the encoder and decoder fuse feature maps of the same size by concatenation. The defect type statistics of the identification results include: For crack defects, the percentage of pixels identified as cracks is statistically analyzed, and the result is used as the percentage of crack defects. For defects such as holes and unmelted areas, the percentage of pixels identified as holes or unmelted areas is statistically analyzed, and the result is used as the percentage of defects for holes or unmelted areas. At the same time, each connected region in the statistical analysis results is treated as a separate defect, and the number of pixels contained in the defect is calculated as the area of ​​the defect. Then, based on the scale information read during the preprocessing of the input image, the pixel area is converted into the true area, and the reduced diameter is calculated. The statistical information of the identification results is summarized. The statistical information of the identification results includes: the proportion of crack defects, the proportion of pore defects, the proportion of unmelted area defects, the number and average diameter of pore defects, and the number and average diameter of unmelted area defects.

2. The method according to claim 1, characterized in that, The training method for the deep learning model includes: A dataset was generated based on the original metallographic images of manufactured samples from different batches and with different compositions; each original image contained more than one type of defect. The original photos in the dataset are labeled to obtain labeled photos; Based on the original and labeled photos, an augmentation operation is performed to generate a sample set, which is then divided into a training set and a test set. The neural network model is iteratively trained using a training set. After each training round, a test set is used to perform a test and the accuracy of the test set recognition is given. After training, the model parameters with the highest accuracy of the test set recognition are selected as the model parameters of the deep learning model, and the deep learning model is output.

3. The method according to claim 2, characterized in that, The iterative training of the neural network model using a training set includes: The encoder extracts texture features from the input images in the training set to obtain feature maps; The decoder restores the feature map to its original resolution; The feature maps of the same size in the encoder and decoder are fused by splicing to obtain the fusion result; The weights of each convolutional kernel in the convolutional layer of the model are adjusted through iterative training to obtain the deep learning model.

4. A microscopic defect processing device based on image recognition, characterized in that, include: The acquisition module is used to acquire an input image and preprocess the input image to obtain an image to be recognized; Specifically, it is used to perform grayscale conversion on the input image to obtain a grayscale image; The grayscale image is subjected to scale recognition to determine the relationship between the true length of the target and the number of pixels; Read the defect area of ​​the grayscale image, determine the area outside the defect area as the background area, delete the background in the background area and fill it with a standardized background; The recognition module is used to segment and recognize the image to be recognized using a trained deep learning model and output the recognition result. Specifically, it is used to crop the image to be recognized to obtain an image of a first preset size; The image of the first preset size is identified, and the identification result is cropped to the second preset size; The recognition results with a second preset size are fused together, and the fused result is used as the final recognition result; wherein the second preset size is smaller than the first preset size; the recognition result includes different types of defects; The processing module is used to determine the defect type based on the identification result, and to perform post-processing and statistics on the identification result based on the defect type; specifically, it is used to perform defect type statistics on the identification result; the defect types include cracks, holes, and unmelted areas; and to perform corresponding post-processing for different defect types, specifically including: for the identification result of cracks, deleting small objects to reduce interference; for the identification results of holes and unmelted areas, performing a circular region opening operation with a preset diameter of pixels, that is, first eroding and then expanding, connecting the unconnected identification regions, and then performing a flooding and filling operation; specifically, for crack defects, it is used to statistically analyze the proportion of pixels identified as cracks, and the result is used as the crack defect proportion; For defects such as holes and unmelted areas, the percentage of pixels identified as holes or unmelted areas is statistically analyzed, and the result is used as the percentage of defects for holes or unmelted areas. At the same time, each connected region in the statistical analysis results is treated as a separate defect, and the number of pixels contained in the defect is calculated as the area of ​​the defect. Then, based on the scale information read during the preprocessing of the input image, the pixel area is converted into the true area, and the reduced diameter is calculated. The statistical information of the identification results is summarized. The deep learning model is built based on a decoder-encoder network structure, where the encoder and decoder are fused by splicing feature maps of the same size. The statistical information of the recognition results includes: the proportion of crack defects, the proportion of hole defects, the proportion of unmelted area defects, the number and average diameter of hole defects, and the number and average diameter of unmelted area defects.

5. A computer device, characterized in that, include: A memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 3.

Images