Multi-modal nonlinear phased array fusion micro-crack intelligent identification method and system

Abstract

This application belongs to the field of image recognition technology and discloses a method and system for intelligent microcrack recognition based on multimodal nonlinear phased array fusion. The method includes: acquiring microcrack images of different lengths and angles based on nonlinear effect phased arrays; performing threshold segmentation on the microcrack images; adding noise to each segmented microcrack image; classifying the expanded microcrack images by free combination according to different lengths and angles; converting the classification dataset into a PyTorch tensor; training a feature fusion network model using the converted training set and testing it using the converted test set to obtain a microcrack intelligent recognition network model; inputting the microcrack image to be recognized into the microcrack intelligent recognition network model to obtain the microcrack intelligent recognition result. This application can accurately extract the features of microcrack images, thereby improving the accuracy of microcrack recognition.

Description

Technical Field

[0001] This application belongs to the field of image recognition, and more specifically, relates to a method and system for intelligent microcrack recognition based on multimodal nonlinear phased array fusion. Background Technology

[0002] In recent years, safety issues caused by structural defects in materials have attracted much attention. Various structural defects can occur in materials during manufacturing and use, among which cracks are the most dangerous. The initial crack size of metal structures is very small and tends to close when not under load. However, during service, the cracks continue to propagate, causing a sharp drop in the mechanical properties of the material. In addition, the crack angle also affects the number of cracks and the propagation rate.

[0003] In existing technologies, linear ultrasonic technology is often used to detect the types of defects in materials, such as diagnosing the presence of cracks. There are also methods using nonlinear ultrasonic phased array imaging to detect microcracks, but these methods cannot achieve imaging of the microcrack angle, and the imaging area is larger than the actual size of the microcrack, resulting in poor accuracy in identifying the microcrack size and failing to meet practical needs. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this application is to provide a method and system for intelligent identification of microcracks using multimodal nonlinear phased array fusion, aiming to solve the problems that existing detection methods cannot achieve imaging of microcrack angles and that the imaging area is larger than the actual size of the microcrack.

[0005] To achieve the above objectives, in a first aspect, this application provides a microcrack intelligent identification method based on multimodal nonlinear phased array fusion, comprising:

[0006] S1 acquires microcrack imaging images based on nonlinear effect phased arrays of different lengths and angles, performs threshold segmentation on the microcrack imaging images; adds noise to each segmented microcrack imaging image to obtain an augmented dataset; classifies the augmented dataset by freely combining different lengths and angles to obtain a classified dataset; and divides the classified dataset into a training set and a test set.

[0007] S2 transforms the classification dataset into a PyTorch tensor during the data preprocessing stage of the feature fusion network model, trains the feature fusion network model using the transformed training set, and tests the trained feature fusion network model using the transformed test set to obtain a microcrack intelligent recognition network model.

[0008] S3 inputs the microcrack image to be identified into the microcrack intelligent recognition network model to obtain the microcrack intelligent recognition result.

[0009] Furthermore, in step S1, the microcrack imaging image is a high-order harmonic and mixed-frequency harmonic nonlinear imaging image, and the crack length of the microcrack imaging image is... l Satisfy: 0 < l <1mm, the length interval of various cracks is ∆ l The crack angle is 0 < θ < 180°, and the interval between various crack angles is ∆θ.

[0010] Furthermore, in step S1, the expanded dataset is freely combined and classified according to different lengths and angles to obtain... There are several categories.

[0011] Furthermore, in step S1, the step of threshold segmentation of the microcrack imaging image is as follows: retain the pixel values ​​in the microcrack imaging image that are greater than a preset pixel threshold, and reset the remaining pixel values ​​to the minimum pixel value in the crack imaging image.

[0012] Furthermore, in step S2, the step of training the feature fusion network model using the training set includes: performing a weighted summation of the PyTorch tensors in the training set before inputting them into the backbone network to obtain the first fusion matrix.

[0013] Furthermore, in step S2, the step of training the feature fusion network model using the training set includes: after outputting the backbone network, performing a weighted summation of the extracted feature vectors to obtain a second fusion matrix.

[0014] Furthermore, in step S2, the network model is a ResNet18 network.

[0015] Secondly, a multimodal nonlinear phased array fusion-based intelligent microcrack identification system is also disclosed, used to implement the aforementioned intelligent microcrack identification method. The intelligent microcrack identification system includes:

[0016] The dataset processing module is used to acquire microcrack imaging images based on nonlinear effect phased arrays of different lengths and angles, and to perform threshold segmentation on the microcrack imaging images; to perform noise addition processing on each segmented microcrack imaging image to obtain an expanded dataset; to classify the expanded dataset by freely combining different lengths and angles to obtain a classified dataset; and to divide the classified dataset into a training set and a test set.

[0017] The model training module is used to convert the classification dataset into a PyTorch tensor during the data preprocessing stage of the feature fusion network model; train the feature fusion network model using the converted training set; and test the trained feature fusion network model using the converted test set to obtain a microcrack intelligent recognition network model.

[0018] The microcrack recognition module is used to input the microcrack image to be recognized into the microcrack intelligent recognition network model to obtain the microcrack intelligent recognition result.

[0019] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in any possible implementation of the first aspect.

[0020] Fourthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in any possible implementation of the first aspect.

[0021] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0022] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art:

[0023] (1) The microcrack identification method provided in this application utilizes nonlinear phased array imaging technology to image microcracks smaller than millimeters, and offers higher imaging quality, anti-interference capabilities, and system adaptability. The ResNet18 network in this application can extract more information on the size and angle of microcracks through a large amount of training data. The combination of these two technologies enables accurate prediction of the size and angle of microcracks.

[0024] (2) The microcrack identification method provided in this application focuses on the core area of ​​the crack based on threshold segmentation, reduces the microcrack imaging area, reduces redundant pixels in the input data, reduces the amount of subsequent calculation, eliminates background interference, and enhances the effective information ratio of the input data. Attached Figure Description

[0025] Figure 1 This is a flowchart of the intelligent microcrack identification method based on multimodal nonlinear phased array fusion provided in the embodiments of this application;

[0026] Figure 2 These are the original sum frequency, difference frequency, and second harmonic imaging images from the embodiments of this application;

[0027] Figure 3 This is an image of the sum frequency, difference frequency, and second harmonic frequency after threshold segmentation in the embodiments of this application;

[0028] Figure 4 This is a network structure diagram of the pre-fusion identification of microcrack size based on the ResNet18 network in the embodiments of this application;

[0029] Figure 5This is a network structure diagram of the post-fusion identification of microcrack size based on the ResNet18 network in the embodiments of this application;

[0030] Figure 6 This is a bar chart comparing the test set accuracy under two fusion methods in the embodiments of this application;

[0031] Figure 7 This is the test set length confusion matrix in the pre-fusion method of this application embodiment;

[0032] Figure 8 This is the test set angle confusion matrix in the pre-fusion method of this application embodiment;

[0033] Figure 9 This is the test set length confusion matrix in the post-fusion method of this application embodiment;

[0034] Figure 10 This is the test set angle confusion matrix in the post-fusion mode of this application embodiment. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0036] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0037] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0038] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0039] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0040] The embodiments of this application are described below with reference to the accompanying drawings.

[0041] This application provides a method for intelligent microcrack identification based on the fusion of multimodal nonlinear phased arrays, such as... Figure 1 As shown, the method includes the following steps:

[0042] S1 acquires microcrack imaging images based on nonlinear effect phased arrays of different lengths and angles, performs threshold segmentation on the microcrack imaging images; adds noise to each segmented microcrack imaging image to obtain an augmented dataset; classifies the augmented dataset by freely combining different lengths and angles to obtain a classified dataset; and divides the classified dataset into training and testing sets.

[0043] In the data preprocessing stage of the feature fusion network model, S2 converts the classification dataset into a PyTorch tensor, uses the converted training set to train the feature fusion network model, and uses the converted test set to test the trained feature fusion network model, thus obtaining a microcrack intelligent recognition network model.

[0044] S3 inputs the microcrack image to be identified into the microcrack intelligent recognition network model to obtain the microcrack intelligent recognition result.

[0045] In step S1 above, the microcrack imaging image (hereinafter referred to as the sample image) is a nonlinear imaging image of high-order harmonics and mixing harmonics. In this embodiment, it includes three types of modal images: sum frequency, difference frequency, and second harmonic. The crack length of the microcrack imaging image is... l Satisfy: 0 < l <1mm, the length interval of various cracks is ∆ l The crack angle is 0 < θ < 180°, and the interval between various crack angles is ∆θ. Specifically, finite element simulation software (such as Abaqus) can be used to control the crack parameters (length and angle) to collect ultrasonic echo signals of three frequencies from the microcrack.

[0046] The specific parameters for the obtained sample images are as follows: crack lengths range from 0.5mm to 0.9mm, with crack length intervals of 0.1mm for each type, and crack angles range from 20° to 180°, with intervals of 20°. Figure 2 The images shown are the original sum frequency (left), difference frequency (middle), and second harmonic frequency (right).

[0047] In step S1, the threshold segmentation step for the microcrack imaging image is as follows: retain pixel values ​​in the microcrack imaging image that are greater than a preset pixel threshold, and reset the remaining pixel values ​​to the minimum pixel value in the crack imaging image. For example... Figure 3 As shown, the images of sum frequency (left), difference frequency (middle), and second harmonic (right) after threshold segmentation are displayed. The threshold segmentation steps specifically include: processing the pixel values ​​of the acquired sample images. The pixel value range of the sum frequency and second harmonic images is -6 to 0, and the pixel value range of the difference frequency image is -1 to 0. The sum frequency image retains the portion from -3 to 0 and sets the rest to -6; the difference frequency image retains the portion from -0.5 to 0 and sets the rest to -1; the second harmonic image retains the portion from -2 to 0 and sets the rest to -6, thereby achieving the purpose of reducing the imaging area of ​​microcracks.

[0048] In step S1, the noise addition process for each segmented microcrack image is as follows: noise is added to the segmented images, and multiple types of noise are added to each image, expanding the dataset to several times its original size. For example, 11 different types of noise are added, such as Gaussian noise, brightness adjustment, salt and pepper noise, and random noise, expanding the dataset to 11 times its original size.

[0049] In step S1 above, after freely combining and classifying the expanded microcrack imaging dataset according to different lengths and angles, the following results will be obtained: Microcrack imaging images for each category. As mentioned above. l Take 0.5mm, ∆ l With a thickness of 0.1 mm, an angle of θ of 180°, and a Δθ of 20°, a total of 45 categories of microcrack imaging images can be output.

[0050] In step S1, the sample images are divided into a training set and a test set. Specifically, a total of 540 sample images are collected, with the training set accounting for 80% (432 images) and the test set accounting for 20% (108 images). Then, in the data preprocessing stage of the pre-built feature fusion network model, the training and test set data are converted into PyTorch tensors.

[0051] In step S2, the network model used is ResNet18, with a feature fusion module added to construct a feature fusion network model. Specifically, the ResNet18 network includes the following structure: the first part includes a convolutional layer, an activation function layer, and a pooling layer; the second to ninth parts are eight residual modules, each consisting of two convolutional layers, an activation function layer, and a skip connection layer; the tenth part includes an average pooling layer, a fully connected layer, and an output layer; the activation function used in the first to ninth parts is the ReLU activation function; the tenth layer uses the Softmax activation function. The optimizer used in this network model is the Adam optimizer.

[0052] In the ResNet18 network, three parameters are set to represent the weights of the aforementioned three PyTorch tensors or feature vectors (corresponding to sum frequency, difference frequency, and second harmonic frequency) in feature fusion (i.e., w i , w j 、w k These parameters are continuously adjusted as the ResNet18 network is trained, thereby achieving better fusion results through the process of training the feature fusion network model.

[0053] In step S2 above, multiple types of images (i.e., sum frequency, difference frequency, and second harmonic frequency microcrack imaging images) are fused in the feature fusion network model. Feature fusion includes two methods: the first is to perform a weighted summation of the PyTorch tensors in the training set before inputting them into the backbone network to obtain a first fusion matrix; the second is to perform a weighted summation of the extracted feature vectors after outputting them into the backbone network to obtain a second fusion matrix. For example... Figure 4 and 5 The diagrams shown are network structures for identifying microcrack sizes through pre-fusion and post-fusion, respectively.

[0054] During the testing phase, the trained feature fusion network model is tested using a test set. For example... Figure 6 As shown, quantitative identification of microcrack length and angle is achieved by fusing sum-frequency, difference-frequency, and second-harmonic microcrack images. The accuracy rates of pre-fusion for length and angle identification are 94.55% and 95.97%, respectively. The length confusion matrix of the test set is shown in the figure. Figure 7 As shown, the angular confusion matrix is ​​as follows Figure 8 As shown; the post-fusion accuracy for length and angle recognition is 92.6% and 94.1%, respectively. The length confusion matrix of the test set is shown below. Figure 9 As shown, the angular confusion matrix is ​​as follows Figure 10 As shown.

[0055] In another embodiment, a microcrack intelligent identification system based on multimodal nonlinear phased array fusion is also disclosed for implementing the aforementioned microcrack intelligent identification method. This microcrack intelligent identification system includes:

[0056] The dataset processing module is used to acquire microcrack imaging images based on nonlinear effect phased arrays of different lengths and angles, and to perform threshold segmentation on the microcrack imaging images; to add noise to each segmented microcrack imaging image to obtain an augmented dataset; to freely combine and classify the augmented dataset according to different lengths and angles to obtain a classified dataset; and to divide the classified dataset into training and testing sets.

[0057] The model training and testing module is used to convert the training set and test set data into PyTorch tensors during the data preprocessing stage of the pre-built feature fusion network model, and to train the feature fusion network model using the converted training set and test the trained feature fusion network model using the converted test set, until the microcrack intelligent recognition network model is obtained.

[0058] The microcrack recognition module is used to input the microcrack image to be recognized into the microcrack intelligent recognition network model to obtain the microcrack intelligent recognition result.

[0059] It is understood that the detailed functional implementation of each of the above units / modules can be found in the description in the aforementioned method embodiments, and will not be repeated here.

[0060] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0061] Based on the methods in the above embodiments, this application provides an electronic device that may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor may invoke logical instructions in the memory to execute the methods in the above embodiments.

[0062] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0063] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0064] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0065] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0066] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0067] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0068] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0069] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A microcrack intelligent identification method based on multimodal nonlinear phased array fusion, characterized in that, include: S1 acquires microcrack imaging images based on nonlinear effect phased arrays of different lengths and angles, and performs threshold segmentation on the microcrack imaging images; Noise is added to each segmented microcrack image to obtain an augmented dataset; the augmented dataset is then freely combined and classified according to different lengths and angles to obtain a classified dataset; the classified dataset is then divided into a training set and a test set. S2 transforms the classification dataset into a PyTorch tensor during the data preprocessing stage of the feature fusion network model, trains the feature fusion network model using the transformed training set, and tests the trained feature fusion network model using the transformed test set to obtain a microcrack intelligent recognition network model. S3 inputs the microcrack image to be identified into the microcrack intelligent identification network model to obtain the microcrack intelligent identification result; In step S1, the microcrack imaging image is a high-order harmonic and mixed-frequency harmonic nonlinear imaging image, and the crack length of the microcrack imaging image is... l Satisfy: 0 < l <1mm, the length interval of various cracks is l ; Crack angle 0 < θ < 180°, and the interval between various crack angles is as follows: θ; In step S2, the step of training the feature fusion network model using the transformed training set includes: performing a weighted summation of the PyTorch tensors corresponding to the training set before inputting them into the backbone network to obtain a first fusion matrix; or, the step of training the feature fusion network model using the transformed training set includes: performing a weighted summation of the extracted feature vectors after outputting them into the backbone network to obtain a second fusion matrix.

2. The intelligent microcrack identification method based on multimodal nonlinear phased array fusion as described in claim 1, characterized in that, In step S1, the expanded dataset is freely combined and classified according to different lengths and angles to obtain... There are several categories.

3. The intelligent microcrack identification method based on multimodal nonlinear phased array fusion as described in claim 1, characterized in that, In step S1, the step of threshold segmentation of the microcrack imaging image is as follows: retain the pixel values ​​in the microcrack imaging image that are greater than the preset pixel threshold, and reset the remaining pixel values ​​to the minimum pixel value in the crack imaging image.

4. The intelligent microcrack identification method based on multimodal nonlinear phased array fusion as described in claim 1, characterized in that, In step S2, the network model is a ResNet18 network.

5. A microcrack intelligent identification system based on multimodal nonlinear phased array fusion, characterized in that, For implementing the intelligent microcrack identification method as described in any one of claims 1-4, the intelligent microcrack identification system comprises: The dataset processing module is used to acquire microcrack imaging images based on nonlinear effect phased arrays of different lengths and angles, and to perform threshold segmentation on the microcrack imaging images; to perform noise addition processing on each segmented microcrack imaging image to obtain an expanded dataset; to classify the expanded dataset by freely combining different lengths and angles to obtain a classified dataset; and to divide the classified dataset into a training set and a test set. The model training module is used to convert the classification dataset into a PyTorch tensor during the data preprocessing stage of the feature fusion network model; train the feature fusion network model using the converted training set; and test the trained feature fusion network model using the converted test set to obtain a microcrack intelligent recognition network model. The microcrack recognition module is used to input the microcrack image to be recognized into the microcrack intelligent recognition network model to obtain the microcrack intelligent recognition result.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on the processor, it causes the processor to perform the method as described in any one of claims 1-4.

7. A computer program product, characterized in that, When the computer program product is run on a processor, the processor causes the processor to perform the method as described in any one of claims 1-4.

Images