Microcrack intelligent identification method and system based on multi-modal nonlinear phased array fusion

Abstract

The invention belongs to the technical field of image recognition, and discloses a multi-modal nonlinear phased array fused micro-crack intelligent recognition method and system, and the method comprises the steps: obtaining micro-crack imaging pictures based on a nonlinear effect phased array and having different lengths and different angles, and carrying out the threshold segmentation of the micro-crack imaging pictures; carrying out noise addition processing on each segmented microcrack imaging picture; freely combining and classifying the expanded microcrack imaging pictures according to different lengths and different angles; the classification data set is converted into a PyTorch tensor, the converted training set is used for training a feature fusion network model, the converted test set is used for testing, and an intelligent microcrack recognition network model is obtained; and inputting a to-be-recognized microcrack image into the intelligent microcrack recognition network model to obtain an intelligent microcrack recognition result. According to the method, the characteristics of the microcrack imaging picture can be accurately extracted, so that the microcrack identification precision is improved.

Description

Technical Field

[0001] This application belongs to the field of image recognition, and more specifically, relates to a micro-crack intelligent recognition method and system based on multi-modal non-linear phased array fusion. Background Art

[0002] In recent years, safety issues caused by structural defects in materials have attracted much attention. Various structural defects will occur during the manufacturing and use of materials, among which cracks are the most dangerous. The initial crack size of metal structures is very small and tends to close without load, but during service, the cracks continue to expand, resulting in a sharp drop in the mechanical properties of the materials. In addition, the crack angle also affects the number of crack propagations and the expansion rate.

[0003] In the prior art, linear ultrasonic techniques are mostly used to detect the defect categories of materials, such as using linear ultrasonic techniques to diagnose whether there are cracks in materials. There are also methods using non-linear ultrasonic phased array imaging techniques to detect micro-cracks, but the above methods cannot achieve the imaging of micro-crack angles, and the imaging area will be larger than the actual size of the micro-cracks, and the accuracy of the identified micro-crack size is poor, which is difficult to meet the actual needs. Summary of the Invention

[0004] Aiming at the defects of the prior art, the purpose of this application is to provide a micro-crack intelligent recognition method and system based on multi-modal non-linear phased array fusion, aiming to solve the problems that the existing detection methods cannot achieve the imaging of micro-crack angles and the imaging area is larger than the actual size of the micro-cracks.

[0005] To achieve the above purpose, in the first aspect, this application provides a micro-crack intelligent recognition method based on multi-modal non-linear phased array fusion, including: S1 Obtain micro-crack imaging maps based on non-linear effect phased arrays with different lengths and different angles, perform threshold segmentation on the micro-crack imaging maps; perform noise addition processing on each segmented micro-crack imaging map to obtain an augmented data set; classify the augmented data set according to different lengths and different angles freely combined to obtain a classified data set; divide the classified data set into a training set and a test set; S2 Convert the classified data set into PyTorch tensors in the data preprocessing stage of the feature fusion network model, train the feature fusion network model using the converted training set, test the trained feature fusion network model using the converted test set to obtain a micro-crack intelligent recognition network model; S3 Input the micro-crack image to be recognized into the micro-crack intelligent recognition network model to obtain a micro-crack intelligent recognition result.

[0006] Further, in step S1, the micro-crack imaging map is a high-order harmonic and mixed-frequency harmonic non-linear imaging map, and the crack length of the micro-crack imaging mapl Satisfy: 0 < l < 1 mm, the length intervals of various cracks are ∆ l ; the crack angle is 0 < θ < 180°, and the angle intervals of various cracks are ∆θ.

[0007] Furthermore, in step S1, after classifying the expanded data set according to different lengths and angles in a free combination manner, categories are obtained.

[0008] Further, in step S1, the steps of performing threshold segmentation on the micro-crack imaging diagram are as follows: retain the pixel values greater than the preset pixel threshold in the micro-crack imaging diagram, and reset the remaining pixel values to the minimum pixel value in the crack imaging diagram.

[0009] Further, in step S2, the steps of training the feature fusion network model by using the training set include: performing weighted summation on the PyTorch tensors in the training set before inputting into the backbone network to obtain a first fusion matrix.

[0010] Further, in step S2, the steps of training the feature fusion network model by using the training set include: performing weighted summation on the extracted feature vectors after outputting from the backbone network to obtain a second fusion matrix.

[0011] Further, in step S2, the network model is a ResNet18 network.

[0012] In a second aspect, a micro-crack intelligent recognition system for multi-modal non-linear phased array fusion is also disclosed, which is used to implement the micro-crack intelligent recognition method as described above. The micro-crack intelligent recognition system includes: A data set processing module, which is used to obtain micro-crack imaging diagrams based on non-linear effect phased arrays with different lengths and different angles, perform threshold segmentation on the micro-crack imaging diagrams; perform noise addition processing on each segmented micro-crack imaging diagram to obtain an expanded data set; classify the expanded data set according to different lengths and different angles in a free combination manner to obtain a classified data set; divide the classified data set into a training set and a test set; A model training module, which is used to convert the classified data set into PyTorch tensors in the data preprocessing stage of the feature fusion network model; train the feature fusion network model by using the converted training set, test the trained feature fusion network model by using the converted test set to obtain a micro-crack intelligent recognition network model; A micro-crack recognition module, which is used to input the micro-crack image to be recognized into the micro-crack intelligent recognition network model to obtain a micro-crack intelligent recognition result.

[0013] In a third aspect, the present application provides a computer-readable storage medium storing a computer program, which, when running on a processor, causes the processor to execute the method described in any possible implementation manner of the first aspect.

[0014] In a fourth aspect, the present application provides a computer program product, which, when running on a processor, causes the processor to execute the method described in any possible implementation manner of the first aspect.

[0015] It can be understood that the beneficial effects of the above second to fourth aspects can be referred to the relevant descriptions in the first aspect above, and will not be elaborated here.

[0016] Generally speaking, compared with the prior art, the above technical solutions conceived by the present application have the following beneficial effects: (1) For the microcrack identification method provided by the present application, the nonlinear phased array imaging technology can achieve imaging of microcracks below millimeter level, and can provide higher imaging quality, anti-interference ability and system adaptability. The ResNet18 network of the present application can extract more size and angle information of microcracks through a large amount of training data. The combination of the two can achieve accurate prediction of the size and angle of microcracks.

[0017] (2) For the microcrack identification method provided by the present application, it focuses on the crack core area based on threshold segmentation, reduces the microcrack imaging area, reduces the redundant pixels of the input data, reduces the subsequent calculation amount, eliminates background interference, and enhances the proportion of effective information of the input data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of a multi-modal nonlinear phased array fusion-based microcrack intelligent identification method provided by an embodiment of the present application; Figure 2 is an imaging diagram of original sum frequency, difference frequency and second harmonic frequency in an embodiment of the present application; Figure 3 is an imaging diagram of sum frequency, difference frequency and second harmonic frequency after threshold segmentation processing in an embodiment of the present application; Figure 4 is a network structure diagram of pre-fusion identification of microcrack size based on the ResNet18 network in an embodiment of the present application; Figure 5 is a network structure diagram of post-fusion identification of microcrack size based on the ResNet18 network in an embodiment of the present application; Figure 6 is a bar chart of comparison of test set accuracy rates under two fusion methods in an embodiment of the present application; Figure 7is the confusion matrix of the test set length in the pre-fusion mode in the embodiments of the present application; Figure 8 is the confusion matrix of the test set angle in the pre-fusion mode in the embodiments of the present application; Figure 9 is the confusion matrix of the test set length in the post-fusion mode in the embodiments of the present application; Figure 10 is the confusion matrix of the test set angle in the post-fusion mode in the embodiments of the present application. Detailed implementation manners

[0019] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application.

[0020] The term "and / or" in this document is an association relationship describing associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The symbol " / " in this document represents an "or" relationship between associated objects. For example, A / B represents A or B.

[0021] The terms "first", "second", etc. in the description and claims of this application are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first response message and the second response message are used to distinguish different response messages, rather than to describe the specific order of the response messages.

[0022] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or more advantageous than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0023] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" refers to two or more. For example, a plurality of processing units refers to two or more processing units, etc.; a plurality of elements refers to two or more elements, etc.

[0024] The embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

[0025] The present application provides a micro-crack intelligent recognition method for multi-modal non-linear phased array fusion, as Figure 1 shown, the method includes the following steps: S1 Obtain microcrack imaging maps of phased arrays based on nonlinear effects with different lengths and different angles, perform threshold segmentation on the microcrack imaging maps; perform noise addition processing on each segmented microcrack imaging map to obtain an augmented dataset; classify the augmented dataset by freely combining different lengths and different angles to obtain a classified dataset; divide the classified dataset into a training set and a test set; S2 In the data preprocessing stage of the feature fusion network model, convert the classified dataset into PyTorch tensors, use the converted training set to train the feature fusion network model, and use the converted test set to test the trained feature fusion network model to obtain a microcrack intelligent recognition network model; S3 Input the microcrack image to be recognized into the microcrack intelligent recognition network model to obtain a microcrack intelligent recognition result.

[0026] In the foregoing step S1, the microcrack imaging map (hereinafter referred to as the sample map) is a high-order harmonic and mixed-frequency harmonic nonlinear imaging map. In this embodiment, it includes three types of modal maps of sum frequency, difference frequency, and second harmonic, and the crack length of the microcrack imaging map l satisfies: 0 < l < 1 mm, and the interval of each crack length is ∆ l ; the crack angle is 0 < θ < 180°, and the interval of each crack angle is ∆θ. Specifically, a finite element simulation software (such as Abaqus) can be used to control the crack parameters (length and angle) to collect ultrasonic echo signals of three frequencies of microcracks.

[0027] The specific parameters of the obtained sample map are: take the crack length to be 0.5 mm to 0.9 mm, the interval of each crack length is 0.1 mm, and the crack angle is 20° to 180°, with an interval of 20°. As Figure 2 shown, they are the original sum frequency (left), difference frequency (middle), and second harmonic (right) imaging maps respectively.

[0028] In step S1, the steps for performing threshold segmentation on the microcrack imaging map are: retain the pixel values greater than the preset pixel threshold in the microcrack imaging map, and reset the remaining pixel values to the minimum pixel value in the crack imaging map. As Figure 3 shown, for the sum frequency (left), difference frequency (middle), and second harmonic (right) imaging maps after threshold segmentation processing, the steps of threshold segmentation specifically include: process the pixel values of the obtained sample map. The pixel value ranges of the sum frequency and second harmonic pictures are -6 to 0, and the pixel value range of the difference frequency picture is -1 to 0. Retain the part of -3 to 0 in the sum frequency imaging map, and set the remaining part to -6; retain the part of -0.5 to 0 in the difference frequency imaging map, and set the remaining part to -1; retain the part of -2 to 0 in the second harmonic imaging map, and set the remaining part to -6, so as to achieve the purpose of reducing the microcrack imaging area.

[0029] In step S1, the steps for adding noise to each segmented microcrack imaging map are as follows: Add noise to the segmented pictures, add multiple types of noise to each picture, and expand the dataset to multiple times its original size. For example, add 11 different types of noise such as Gaussian noise, brightening and darkening, salt-and-pepper noise, and random noise, and expand the dataset to 11 times its original size.

[0030] In the aforementioned step S1, after freely combining and classifying the expanded microcrack imaging dataset according to different lengths and angles, microcrack imaging maps of l categories will be obtained. As described above, l is taken as 0.5 mm, ∆

[0031] is 0.1 mm, θ is 180°, and ∆θ is 20°, then a total of 45 categories of microcrack imaging maps can be output.

[0032] In step S2, the network model used is the ResNet18 network, and a feature fusion module is added to construct a feature fusion network model. Specifically, the ResNet18 network specifically includes the following structure: The first part includes a convolutional layer, an activation function layer, and a pooling layer; the second part to the ninth part are eight residual modules, each residual module includes two convolutional layers, an activation function layer, and a skip connection layer; the tenth part includes an average pooling layer, a fully connected layer, and an output layer; the activation function used in the activation function layer from the first part to the ninth part is the ReLu activation function; the tenth layer uses the Softmax activation function. The optimizer used for this network model is the Adam optimizer.

[0033] Set 3 parameters in the ResNet18 network, which respectively represent the weight values of the aforementioned 3 PyTorch tensors or feature vectors (corresponding to sum frequency, difference frequency, and second harmonic frequency) in feature fusion (i.e., w i , w j 、w k ). These parameters are continuously adjusted following the training of the ResNet18 network, and thus continuously achieve better fusion effects through the process of training the feature fusion network model.

[0034] In the foregoing step S2, multiple types of images (i.e., three types of microcrack imaging diagrams of sum frequency, difference frequency, and second harmonic) are subjected to feature fusion in the feature fusion network model. The feature fusion includes two fusion methods: the first is to perform weighted summation on the PyTorch tensors in the training set before inputting the backbone network (BackBone) to obtain the first fusion matrix; the second is to perform weighted summation on the extracted feature vectors after outputting the backbone network (BackBone) to obtain the second fusion matrix. As Figure 4 and 5 shown, they are respectively the network structure diagrams for identifying the microcrack size through pre-fusion and post-fusion.

[0035] In the test session, the trained feature fusion network model is tested using the test set. As Figure 6 shown, by fusing the microcrack imaging diagrams of sum frequency, difference frequency, and second harmonic, quantitative identification of the microcrack length and angle is achieved. The recognition accuracies of the pre-fusion for length and angle are 94.55% and 95.97% respectively. The length confusion matrix of the test set is as Figure 7 shown, and the angle confusion matrix is as Figure 8 shown; the recognition accuracies of the post-fusion for length and angle are 92.6% and 94.1% respectively. The length confusion matrix of the test set is as Figure 9 shown, and the angle confusion matrix is as Figure 10 shown.

[0036] In another embodiment, a microcrack intelligent recognition system based on multi-modal non-linear phased array fusion is also disclosed, which is used to implement the foregoing microcrack intelligent recognition method. The microcrack intelligent recognition system includes: A dataset processing module, which is used to obtain microcrack imaging diagrams based on non-linear effect phased arrays with different lengths and different angles, perform threshold segmentation on the microcrack imaging diagrams; perform noise addition processing on each segmented microcrack imaging diagram to obtain an augmented dataset; freely combine and classify the augmented dataset according to different lengths and different angles to obtain a classified dataset; divide the classified dataset into a training set and a test set; A model training and testing module, which is used to convert the training set and test set data into PyTorch tensors during the data preprocessing stage of the pre-constructed feature fusion network model, and use the converted training set to train the feature fusion network model, and use the converted test set to test the trained feature fusion network model until a microcrack intelligent recognition network model is obtained; A microcrack recognition module, which is used to input the microcrack image to be recognized into the microcrack intelligent recognition network model to obtain the microcrack intelligent recognition result.

[0037] It can be understood that the detailed function implementations of the above-mentioned various units / modules can be referred to the descriptions in the foregoing method embodiments, and will not be elaborated here.

[0038] It should be understood that the above device is used to execute the method in the above embodiment. For the corresponding program modules in the device, their implementation principles and technical effects are similar to those described in the above method. The working process of the device can refer to the corresponding process in the above method, and will not be elaborated here.

[0039] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device, which may include: a processor, a communications interface, a memory, and a communication bus. Among them, the processor, the communications interface, and the memory complete mutual communication through the communication bus. The processor can call the logical instructions in the memory to execute the method in the above embodiment.

[0040] In addition, when the logical instructions in the above memory are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0041] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program runs on the processor, the processor is enabled to execute the method in the above embodiment.

[0042] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on the processor, the processor is enabled to execute the method in the above embodiment.

[0043] It can be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0044] The method steps in the embodiments of the present application may be implemented in a hardware manner or by a processor executing software instructions. The software instructions may be composed of corresponding software modules, and the software modules may be stored in a random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), registers, hard disks, removable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in an ASIC.

[0045] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0046] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0047] It is easy for those skilled in the art to understand that the above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. The microcrack intelligent identification method based on multi-modal nonlinear phased array fusion is characterized by: include: S1 obtains microcrack imaging images based on nonlinear effect phased array of different lengths and angles, and performs threshold segmentation on the microcrack imaging images; Performing noise addition processing on each microcrack image after segmentation to obtain an expanded data set; freely combining and classifying the expanded data set according to different lengths and different angles to obtain a classified data set; dividing the classified data set into a training set and a test set; S2 converts the classification data set into a PyTorch tensor in the data preprocessing stage of the feature fusion network model, trains the feature fusion network model using the converted training set, and tests the trained feature fusion network model using the converted test set to obtain a microcrack intelligent recognition network model; S3 inputs the microcrack image to be identified into the microcrack intelligent identification network model to obtain the microcrack intelligent identification result.

2. The multi-modal nonlinear phased array fusion microcrack intelligent identification method according to claim 1, characterized in that: In step S1, the microcrack imaging image is a high-order harmonic and mixed harmonic nonlinear imaging image, and the crack length of the microcrack imaging image is l Satisfied: 0< l <1mm, the interval of each type of crack length is ∆ l ; The crack angle is 0<θ<180°, and the interval between various crack angles is ∆θ.

3. The multi-modal nonlinear phased array fusion microcrack intelligent identification method according to claim 2, characterized in that: In step S1, the expanded data set is classified according to different lengths and angles to obtain categories.

4. The multi-modal nonlinear phased array fusion microcrack intelligent identification method according to claim 1, characterized in that: In step S1, the step of performing threshold segmentation on the microcrack imaging image is: retaining pixel values ​​in the microcrack imaging image that are greater than a preset pixel threshold, and resetting the remaining pixel values ​​to the minimum pixel value in the crack imaging image.

5. The multi-modal nonlinear phased array fusion microcrack intelligent identification method according to claim 1, characterized in that: In step S2, the step of using the converted training set to train the feature fusion network model includes: performing weighted summation on the PyTorch tensors corresponding to the training set before inputting into the backbone network to obtain a first fusion matrix.

6. The multi-modal nonlinear phased array fusion microcrack intelligent identification method according to claim 1, characterized in that: In step S2, the step of training the feature fusion network model using the converted training set includes: performing weighted summation on the extracted feature vectors after outputting the backbone network to obtain a second fusion matrix.

7. The multi-modal nonlinear phased array fusion microcrack intelligent identification method according to claim 1, characterized in that: In step S2, the network model is a ResNet18 network.

8. The micro-crack intelligent identification system based on multi-modal nonlinear phased array fusion is characterized by: Used to implement the microcrack intelligent identification method according to any one of claims 1 to 7, the microcrack intelligent identification system comprises: A data set processing module is used to obtain microcrack imaging images based on nonlinear effect phased array of different lengths and different angles, and perform threshold segmentation on the microcrack imaging images; to perform noise addition processing on each microcrack imaging image after segmentation to obtain an extended data set; to classify the extended data set according to different lengths and different angles to obtain a classified data set; and to divide the classified data set into a training set and a test set; A model training module is used to convert the classification data set into a PyTorch tensor in the data preprocessing stage of the feature fusion network model; train the feature fusion network model using the converted training set, and test the trained feature fusion network model using the converted test set to obtain a microcrack intelligent identification network model; The microcrack identification module is used to input the microcrack image to be identified into the microcrack intelligent identification network model to obtain the microcrack intelligent identification result.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images