Defect identification and three-dimensional visualization method and system based on ultrasonic B scanning image

Abstract

The invention discloses a defect identification and three-dimensional visualization method and system based on an ultrasonic B scanning image, and belongs to the technical field of ultrasonic nondestructive testing. The method comprises the following steps: firstly, acquiring a color ultrasonic B scanning image sequence acquired for a to-be-identified workpiece and carrying out image preprocessing on the color ultrasonic B scanning image sequence; an image entropy sequence is calculated for the gray-scale B scanning image sequence obtained through preprocessing, a non-threshold recurrence plot is generated through phase-space reconstruction, a one-dimensional filter is obtained through binaryzation and row averaging, the one-dimensional filter is used for filtering the image entropy sequence, and a suspected defect image is extracted according to the difference value of the image entropy before and after filtering; and then, further identifying a defect feature region in each suspected defect image through a defect identification model for three-dimensional visual reconstruction. According to the method, ultrasonic B scanning image data of any complex workpiece can be rapidly processed, an accurate and reliable ultrasonic nondestructive defect detection result is obtained, and meanwhile defect recognition and three-dimensional visualization are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of ultrasonic non-destructive testing, and particularly relates to a method and system for defect recognition and three-dimensional visualization based on ultrasonic B-scan images. Background Art

[0002] Ultrasonic non-destructive testing technology detects and locates internal defects of materials through the propagation characteristics of ultrasonic waves in materials. When ultrasonic waves encounter different media, physical phenomena such as reflection and refraction will occur, which can help inspectors identify the internal defect conditions of materials. As an advanced ultrasonic testing method, phased array ultrasonic technology has been widely applied and recognized in industrial inspections due to its high resolution and flexibility.

[0003] However, traditional ultrasonic B-scan image defect detection methods have exposed some problems in practical applications. First, the detection efficiency is low. When faced with a large number of B-scan images, traditional methods need to process and analyze each image one by one, which not only consumes a large amount of time but also easily causes fatigue to inspectors, thus affecting the detection effect. Second, the detection results are greatly affected by the experience of operators. Since the judgment of defects depends to a large extent on the naked-eye observation and subjective judgment of operators, experienced operators may be able to identify defects more accurately, while inexperienced personnel are prone to missed detections and false detections. The existence of this subjectivity has a certain impact on the stability and reliability of the detection results. Finally, traditional methods lack three-dimensional visualization display of defects. On two-dimensional B-scan images, the shape and position of defects may be difficult to intuitively display, which poses a great challenge to users who need to deeply understand the defect characteristics.

[0004] Therefore, how to design an efficient and accurate defect recognition and three-dimensional visualization solution to achieve fast and reliable processing of ultrasonic B-scan image data, obtain accurate and reliable ultrasonic non-destructive testing results, reduce the detection difficulty, and improve the detection accuracy and stability has become a problem that must be solved in relevant production activities. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for defect recognition and three-dimensional visualization based on ultrasonic B-scan images to solve the problems of low efficiency, poor accuracy, and lack of three-dimensional visualization display in traditional methods.

[0006] The specific technical solutions adopted by the present invention are as follows:

[0007] In the first aspect, the present invention provides a method for defect recognition and three-dimensional visualization based on ultrasonic B-scan images, which includes:

[0008] S1, obtaining a color ultrasonic B-scan image sequence obtained for a workpiece to be identified, and performing image preprocessing including lossless grayscale conversion on the image sequence to obtain a grayscale B-scan image sequence;

[0009] S2. For the grayscale B-scan image sequence, the image entropy of each grayscale B-scan image is calculated to form an image entropy sequence. The image entropy sequence is expanded and then reconstructed in phase space to obtain a reconstruction vector that is the same as the color ultrasound B-scan image sequence. The similarity of all the reconstruction vectors is calculated pairwise and a two-dimensional threshold-free recursive graph is generated with the similarity as an element. The threshold-free recursive graph is binarized and the element mean of each row is calculated to characterize the pattern mutation probability density. The element means of all rows are normalized to form a one-dimensional filter with the same dimension as the color ultrasound B-scan image sequence. The image entropy sequence is filtered using the one-dimensional filter, and the difference between the original image entropy value and the filtered value of each grayscale B-scan image is calculated. All grayscale B-scan images with a difference greater than 0 are extracted as suspected defect images.

[0010] S3, inputting all suspected defect images into a defect recognition model pre-trained based on a neural network, identifying the defect feature area in each suspected defect image, and if the defect feature area is not identified, the suspected defect image will be eliminated;

[0011] S4. Mapping all identified defect feature areas to the three-dimensional model space of the workpiece to be identified according to their respective suspected defect images through three-dimensional visualization technology to complete the three-dimensional visualization of the defects in the workpiece to be identified.

[0012] As a preferred embodiment of the first aspect, for each color ultrasound B-scan image, when losslessly graying it into a gray-scale B-scan image, the gray value of each pixel is 255 minus one third of the green component value and two thirds of the blue component value.

[0013] As a preferred embodiment of the first aspect mentioned above, the specific processing method for performing the image preprocessing on each color ultrasonic B-scan image is: first, the color ultrasonic B-scan image is losslessly grayscaled, and then the grayscale ultrasonic B-scan image is median filtered to complete image denoising, and then the noise is further eliminated and the target is segmented through an opening operation, and finally, the region of interest where defects need to be identified is cropped, and the cropped grayscale B-scan image is output.

[0014] As a preferred embodiment of the first aspect, a method for generating a threshold-free recursive graph based on an image entropy sequence is:

[0015] According to the color ultrasound B-scan image sequence n, the pre-optimized optimal delay time τ, and the optimal embedding dimension m, first perform tail padding on the n-dimensional image entropy sequence to obtain an extended image entropy sequence of n+(m - 1)τ dimensions. Then, starting from the first n image entropies of the extended image entropy sequence as starting points, sample m image entropies at intervals of τ from each starting point to form row vectors. Next, construct an n×n threshold-free recurrence graph, where the element value at the i-th row and j-th column is the similarity between the i-th and j-th starting point row vectors.

[0016] As a preference of the first aspect above, when performing binarization on the threshold-free recurrence graph, if the element value in the threshold-free recurrence graph exceeds the threshold, set it as a black dot, otherwise set it as a white dot, and the threshold should be such that the proportion of black dots in the binarized recurrence graph is 5%.

[0017] As a preference of the first aspect above, when filtering the image entropy sequence using a one-dimensional filter, when filtering the image entropy sequence with a one-dimensional filter, obtain a filtered image entropy sequence identical to the color ultrasound B-scan image sequence by multiplying each element of the element mean sequence corresponding to the one-dimensional filter with the image entropy sequence element by element.

[0018] As a preference of the first aspect above, the defect recognition model is based on YOLOv5 or YOLOv8 as the base model and is obtained through supervised training with defect annotation data.

[0019] As a preference of the first aspect above, the method for mapping the defect feature region to the three-dimensional model space of the workpiece to be recognized through three-dimensional visualization technology is as follows: Generate two-dimensional background frames with the same number of images as in the color ultrasound B-scan image sequence. For each recognized defect feature region, extract a defect image block from the corresponding color ultrasound B-scan image or grayscale B-scan image using the outer bounding rectangle of the defect feature region, and then map and overlay the defect image block on the corresponding two-dimensional background frame. After completing the mapping and overlay of all recognized defect feature regions, obtain a three-dimensional visualization model of the workpiece to be recognized containing defects.

[0020] As a preference of the first aspect above, when performing three-dimensional visualization of defects in the three-dimensional model space of the workpiece to be recognized, identify isolated three-dimensional defect regions through the bounding box algorithm and calculate their shapes and positions as information for visualization queries.

[0021] In the second aspect, the present invention provides a defect recognition and three-dimensional visualization system based on ultrasound B-scan images, which includes:

[0022] A B-scan image preprocessing module for obtaining a color ultrasound B-scan image sequence obtained for the workpiece to be recognized and performing image preprocessing including lossless grayscaling on it to obtain a grayscale B-scan image sequence;

[0023] A defective image screening module, which is used to calculate the image entropy of each grayscale B-scan image in a sequence of grayscale B-scan images and form an image entropy sequence, perform phase space reconstruction after expanding the image entropy sequence to obtain reconstruction vectors identical to the sequence of color ultrasound B-scan images, calculate the similarity between every two of all the reconstruction vectors and generate a two-dimensional threshold-free recurrence plot with the similarities as elements, binarize the threshold-free recurrence plot and characterize the mode mutation probability density by calculating the element mean of each row, normalize the element means of all rows to form a one-dimensional filter with the same dimension as the sequence of color ultrasound B-scan images, filter the image entropy sequence using the one-dimensional filter, and calculate the difference between the original value and the filtered value of the image entropy of each grayscale B-scan image, and extract all grayscale B-scan images with differences greater than 0 as suspected defective images;

[0024] A defect feature recognition module, which is used to input all suspected defective images into a defect recognition model pre-trained based on a neural network, identify the defective feature regions in each suspected defective image, and eliminate the suspected defective image if no defective feature region is identified;

[0025] A three-dimensional visualization module, which is used to map all identified defective feature regions to the three-dimensional model space of the workpiece to be identified according to the suspected defective images where they are located through three-dimensional visualization technology to complete the three-dimensional visualization of the defects in the workpiece to be identified.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0027] (1) The present invention combines an information-lossless grayscale algorithm, and is equipped with a median filter, an opening operation, and an ROI extraction algorithm to design a preprocessing scheme suitable for typical B-scan images, which can realize the lossless reduction and denoising of the original B-scan image data information.

[0028] (2) The present invention uses image entropy to represent the information content of B-scan images, introduces the recurrence plot method into the task of screening defective images, and designs a screening filter for segments of the defective image entropy sequence according to the recurrence plot structure, which can realize reliable and accurate screening of suspected defective B-scan sequences without relying on a large number of experiments.

[0029] (3) The present invention further trains a defect recognition model based on a neural network, and couples the defect recognition model with the screening method based on the image entropy sequence and the recurrence plot method, which can not only realize efficient preliminary screening of suspected defective images, but also make up for the defect of insufficient filtering rate by using the accurate recognition ability of the neural network.

[0030] (4) The present invention can further perform three-dimensional visualization of the defect characteristics of the workpiece, visualize the three-dimensional characteristics of the defect, and accurately perform automated quantitative detection of its shape and position. This method itself has strong scalability and can adapt to workpieces of various shapes and materials. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0032] Figure 1 is a schematic diagram of the steps of a method for defect recognition and three-dimensional visualization based on ultrasonic B-scan images;

[0033] Figure 2 is a curve graph of RGB channel components - gray values;

[0034] Figure 3 is a flowchart of the steps for image preprocessing of a color ultrasonic B-scan image;

[0035] Figure 4 is a flowchart for extracting suspected defect images;

[0036] Figure 5 is an exemplary original image entropy sequence;

[0037] Figure 6 is Figure 5 the threshold-free recurrence graph corresponding to the original image entropy sequence in;

[0038] Figure 7 is Figure 6 the binary recurrence graph corresponding to the threshold-free recurrence graph in;

[0039] Figure 8 is an exemplary schematic diagram of a normalized filter;

[0040] Figure 9 is an exemplary schematic diagram of the entropy sequence of the filtered image;

[0041] Figure 10 is a comparison schematic diagram of the suspected defect images obtained by screening and the defect image frames actually manually marked;

[0042] Figure 11 is an exemplary defect feature region recognition result;

[0043] Figure 12 is a schematic diagram of the data construction method for the three-dimensional visualization image sequence;

[0044] Figure 13 is a logical schematic diagram of the volume reconstruction process in the embodiment;

[0045] Figure 14 It is a three-dimensional visualization effect diagram for exemplary local defect identification;

[0046] Figure 15 Schematic diagram of the module composition of a defect identification and three-dimensional visualization system based on ultrasonic B-scan images. Specific embodiments

[0047] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the specific embodiments of the present invention in detail with reference to the accompanying drawings. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined correspondingly without conflict.

[0048] In a preferred embodiment of the present invention, a method for defect identification and three-dimensional visualization based on ultrasonic B-scan images is provided. As Figure 1 shown, it includes the following steps:

[0049] S1. Obtain a sequence of color ultrasonic B-scan images of the workpiece to be identified, and perform image preprocessing including lossless grayscale conversion on it to obtain a sequence of grayscale B-scan images.

[0050] It should be noted that the above lossless grayscale conversion of the present invention is a grayscale algorithm that can achieve lossless full mapping of information in RGB color ultrasonic B-scan images. To better understand the principle of this algorithm, the specific derivation process and implementation of the algorithm will be described in detail below.

[0051] The correspondence between the RGB components of a color ultrasonic B-scan image and the display color and echo amplitude is shown in Table 1:

[0052] Table 1

[0053]

[0054] According to Table 1, the curves of the RGB channel components and the grayscale value x can be drawn as Figure 2 shown. Referring to the curve relationship shown in Figure 2 shown, the expression of the RGB channel components with the converted grayscale value as the independent variable can be obtained:

[0055]

[0056]

[0057]

[0058] By observing the variation rules of the three channel components, it is obvious that only the weighted average algorithm may be able to achieve lossless grayscale conversion. The formula of the weighted average algorithm is as follows:

[0059]

[0060] In the formula: , , are the weighted coefficients of the R, G, and B components respectively.

[0061] Thus, the relational expression between the grayscale value and the weighted average coefficient can be obtained:

[0062]

[0063] According to the property that the sequence of expected grayscale values to be obtained is continuous and differentiable on its domain, the above expression is differentiated to obtain the following transformed expression:

[0064]

[0065] Solving the above transformed expression, the weighted coefficients are obtained as:

[0066]

[0067] Thus, for each color ultrasound B-scan image, when it is losslessly grayscaled into a grayscale B-scan image by using the above lossless grayscale conversion, the calculation formula for the grayscale value x of each pixel is as follows:

[0068]

[0069] In the formula: G is the green component value, and B is the blue component value.

[0070] In addition, it should be noted that the specific operations of the above image preprocessing need to be selected according to the actual image quality. In the embodiments of the present invention, a combined processing method of lossless grayscale conversion, median filtering, morphological processing (such as opening operation or closing operation), and region of interest (ROI) recognition can be adopted. Thus, as Figure 3 shown, in the embodiments of the present invention, the specific processing method for performing image preprocessing on each color ultrasound B-scan image is:

[0071] First, perform lossless grayscale conversion on the color ultrasound B-scan image, then perform median filtering on the grayscale ultrasound B-scan image to complete image denoising, further eliminate noise and segment the target through opening operation, and finally crop the region of interest (ROI) where the defect needs to be recognized, and output the cropped grayscale B-scan image. The ROI here needs to be determined according to the actual detection requirements and is not limited herein.

[0072] After each color ultrasound B-scan image in the color ultrasound B-scan image sequence is converted into a grayscale B-scan image, a grayscale B-scan image sequence can be formed. This grayscale B-scan image sequence will subsequently screen the preliminary suspected defect images through step S2, and then further identify the workpiece curve using a defect recognition model trained based on a neural network.

[0073] S2. For the grayscale B-scan image sequence obtained after image preprocessing, calculate the image entropy of each grayscale B-scan image therein and form an image entropy sequence. After expanding the image entropy sequence, perform phase space reconstruction to obtain the same reconstruction vectors as the color ultrasound B-scan image sequence. Calculate the similarity between all pairs of the reconstruction vectors and generate a two-dimensional threshold-free recurrence plot with the similarities as elements. After binarizing the threshold-free recurrence plot, characterize the pattern mutation probability density by calculating the element mean of each row. After normalizing the element means of all rows, form a one-dimensional filter with the same dimension as the color ultrasound B-scan image sequence. Use the one-dimensional filter to filter the image entropy sequence, and calculate the difference between the original value and the filtered value of the image entropy of each grayscale B-scan image. Extract all grayscale B-scan images with differences greater than 0 as suspected defect images.

[0074] The process for extracting the suspected defect images shown in S2 above is shown in Figure 4 and is described in detail below for the specific implementation of each link of the extraction process in the embodiments of the present invention.

[0075] S21. Calculation of the image entropy sequence

[0076] In the embodiments of the present invention, for the grayscale B-scan image sequence obtained after image preprocessing, calculate the image entropy H of each grayscale B-scan image therein. The calculation formula of the image entropy H is as follows:

[0077]

[0078] Where: is the proportion of pixels with grayscale value in the grayscale B-scan image to all pixels.

[0079] Thus, the image entropy H of all grayscale B-scan images in the grayscale B-scan image sequence can be used to form an image entropy sequence. Suppose the number of grayscale B-scan images in the grayscale B-scan image sequence is n, then the image entropy sequence is a one-dimensional sequence with the number of elements being n.

[0080] The image entropy sequence can be regarded as a one-dimensional signal. Figure 5 Exemplarily shown is the image entropy sequence obtained by converting the detected color ultrasound B-scan image sequence for a copper ring workpiece in an embodiment of the present invention, where the number of elements n = 860. It has been verified that this signal is a discrete signal, and its corresponding dynamic system is a non-stationary deterministic system. Its properties can be studied in a high-dimensional space, so phase space reconstruction and recurrence analysis are required.

[0081] S22. Generate a threshold-free recurrence plot based on the image entropy sequence:

[0082] First, according to the color ultrasound B-scan image sequence n, the optimal delay time τ, and the optimal embedding dimension m, first perform tail padding on the n-dimensional image entropy sequence to obtain an extended image entropy sequence with n+(m - 1)τ dimensions. It should be noted that when performing tail padding on the n-dimensional image entropy sequence, (m - 1)τ elements can be taken from the head of the image entropy sequence and filled to the tail to obtain an extended image entropy sequence with n+(m - 1)τ dimensions.

[0083] In addition, it should be noted that the above optimal delay time τ and optimal embedding dimension m need to be optimized in advance according to the final effect. In the embodiment of the present invention, the mutual information method can be used to solve the optimal delay time τ, and on this basis, the Cao's algorithm is further used to solve the optimal embedding dimension m.

[0084] Then, starting from the first n image entropies of the extended image entropy sequence in turn, m image entropies are sampled at intervals of τ from each starting point to form a row vector. Specifically, suppose the image entropy sequence is , then according to the optimal delay time τ and the optimal embedding dimension m, a -dimensional reconstruction matrix can be constructed, where the row vector of the i-th row in the reconstruction matrix is the reconstruction vector , which is a row vector formed by sampling m image entropies at intervals of τ starting from the i-th starting point of the extended image entropy sequence. It is represented by the formula as follows:

[0085]

[0086] In the formula represents the -th image entropy in the extended image entropy sequence, and i = 1, 2, ……, n.

[0087] Finally, after the phase space reconstruction is completed, the recurrence plot can be used to study its properties. The recurrence plot (RP) is essentially a method for observing the phase space of any dimension in a two-dimensional space. It calculates the similarity between two different reconstructed vectors and . In this embodiment, the similarity between the reconstructed vectors and is calculated using their Euclidean distance. Thus, when the similarities of all pairs of reconstructed vectors are calculated, an n×n two-dimensional recurrence plot without thresholds can be generated with these n×n similarities as elements. The element value at the i-th row and j-th column of the n×n recurrence plot without thresholds is the similarity between the i-th and j-th starting point row vectors .

[0088] In the embodiment of the present invention, based on the image entropy sequence of the length n of the copper ring workpiece shown in Figure 5 , the formed n×n recurrence plot without thresholds is as shown in Figure 6 .

[0089] S23. Perform binarization on the recurrence plot without thresholds

[0090] The recurrence plot without thresholds can be used to observe and analyze the phase space of any dimension after reconstruction at the two-dimensional level. However, due to problems such as large differences in the absolute values of elements, it is difficult to use invariant criteria to analyze the recurrence plot without thresholds of different types of signals, and the recurrence plot needs to be binarized. However, for the binarized recurrence plot of a certain signal sequence, the selection of the threshold ε is very crucial, which determines the performance of the final binarized recurrence plot. The selection of the threshold greatly affects the display effect of the binarized recurrence plot, and further affects the analysis and identification of its system characteristics. Based on previous research findings, for signals from different sources, the methods for selecting the optimal threshold vary greatly, and the values of the optimal thresholds also vary greatly, which is due to the different characteristics of different dynamic systems themselves. Therefore, in the embodiment of the present invention, in combination with the requirements and characteristics of the defect inspection system, with the goal of making the recurrence plot structure clear, 5% of the black dot ratio is selected to determine the optimal threshold ε. That is, if the element value in the recurrence plot without thresholds exceeds the threshold ε, it is set as a black dot, otherwise it is set as a white dot, and the threshold ε should make the black dot ratio in the binarized recurrence plot 5%.

[0091] In the embodiment of the present invention, based on the recurrence plot without thresholds shown in Figure 6 , the finally obtained n×n dimensional binarized recurrence plot is as shown in Figure 7 .

[0092] S24. Filter screening

[0093] Suppose the original image entropy sequence has a total of elements, then the dimension of the binary recurrence plot is still , and the element in the i-th row and j-th column is denoted as . For the -th row in the recurrence plot, it represents whether the similarity between the -th image entropy and the reconstruction of other image entropies reaches the threshold, and to a certain extent, it also represents the probability of pattern mutation of the -th image entropy compared with other image entropies. Therefore, combining the feature that the recurrence plot is symmetric about the diagonal, the row distribution density of the recurrence points can be used to characterize the pattern mutation probability density. The row distribution density can be obtained by calculating the element mean of a row of elements in the binary recurrence plot. The element mean of the i-th row in the binary recurrence plot can be used to characterize the pattern mutation probability density of the i-th image entropy in the image entropy sequence , and the calculation formula:

[0094]

[0095] Thus, n row vectors in the binary recurrence plot can be used to calculate n element means . The vector composed of is the filter :

[0096]

[0097] To normalize the numerical range of the filter, the distribution density elements in the filter are normalized as follows:

[0098]

[0099] The normalized element means of all n rows constitute a one-dimensional filter composed of n elements, and its expression is:

[0100]

[0101] In the embodiment of the present invention, Figure 5 the 860 filter weights (i.e., the elements in the filter) of the one-dimensional filter corresponding to the 860-dimensional image entropy sequence shown in Figure 8 are shown as

[0102] S25. Calculation of the image entropy sequence after filtering

[0103] Filter the image entropy sequence using a one-dimensional filter, calculate the difference between the original value and the filtered value of the image entropy for each grayscale B-scan image, and extract all grayscale B-scan images with differences greater than 0 as suspected defect images.

[0104] In an embodiment of the present invention, when filtering the image entropy sequence using a one-dimensional filter, when filtering the image entropy sequence with the one-dimensional filter, by multiplying the corresponding element mean sequence of the one-dimensional filter with the image entropy sequence element by element, a filtered image entropy sequence identical to the color ultrasound B-scan image sequence is obtained. For the original image entropy sequence, assuming the i-th image entropy among them , the corresponding i-th element needs to be extracted from the one-dimensional filter , and its filtered image entropy is calculated by multiplication :

[0105]

[0106] Thus, for all n filtered image entropies , the change in the image entropy value caused by system mutation can be obtained using the difference between the original image entropy sequence and the filtered image entropy sequence (which can be considered as the change brought by "defect feature + environmental interference"). Specifically, the difference between the original value of the image entropy of each grayscale B-scan image and the filtered value can be calculated. The expression of this difference calculation formula is: and the filtered value is:

[0107]

[0108] Considering that the image entropy change amount sequence most directly reflects the system mutation caused by defect features, the image entropy value change amount, that is, the difference is used as a criterion, and it is considered that B-scan images with differences greater than 0 are suspected defect images, thereby realizing the screening of suspected defect images.

[0109] In an embodiment of the present invention, Figure 5 the filtered image entropy sequence corresponding to the 860-dimensional image entropy sequence shown is as Figure 9 shown. Based on this filtered image entropy sequence, the suspected defect images screened and the actually manually labeled defect image frames are as Figure 10 shown. From the Figure 10 comparison, it can be seen that the suspected defect images screened by the present invention completely cover the manually labeled defect image frames, which can greatly reduce the computational amount required for subsequent neural network defect recognition.

[0110] To better demonstrate the screening effect of the method based on image entropy sequence and recurrence plot method for suspected defect images shown in step S2 of the present invention, it is compared with some other existing image defect screening methods (including threshold method, mean method, maximum method, variance method) below, and the effect is shown in Table 2:

[0111] Table 2

[0112]

[0113] It can be seen that the screening method based on image entropy sequence and recurrence plot method used in step S2 of the present invention has the highest coverage rate in all defect intervals, and at the same time has the highest comprehensive coverage rate, and there is no missed detection in the middle of the interval; in terms of the filtering rate, the performance of the recurrence method is slightly inferior to other methods, about 50%. Therefore, in the present invention, a defect recognition model based on neural network training is used to further screen the suspected defect images.

[0114] S3. Input all suspected defect images into a defect recognition model pre-trained based on a neural network, identify the defect feature regions in each suspected defect image, and if no defect feature region is identified, the suspected defect image will be removed.

[0115] It should be noted that the neural network model used in the above defect recognition model is not limited, as long as it can achieve a good defect recognition effect. In the embodiment of the present invention, the defect recognition model can use YOLOv5 or YOLOv8 as the basic model and be supervised and trained through defect annotation data. Thus, by training the neural network model for specific workpiece data to be recognized, a neural network model with a certain generality and the ability to recognize defect features in B-scan images can be obtained. Coupled with the above screening method based on image entropy sequence and recurrence plot method, it can not only realize the efficient preliminary screening of suspected defect images, but also make up for the deficiency of the filtering rate by using the accurate recognition ability of the neural network.

[0116] In the embodiment of the present invention, Figure 11 The recognition result of the defect feature region output by the defect recognition model for a certain frame of suspected defect image is further shown. The result label includes the defect type and the confidence level. For example, hole: 0.66 means that a defect of the type of hole is recognized, and its confidence level is 0.66.

[0117] S4. Map all the recognized defect feature regions to the three-dimensional model space of the workpiece to be recognized according to the suspected defect images where they are located, and complete the three-dimensional visualization of the defects in the workpiece to be recognized.

[0118] It should be noted that the above three-dimensional visualization technology can use a three-dimensional visualization engine (such as VTK, MATLAB, etc.). By designing an algorithm, the B-scan image is mapped into a three-dimensional physical space, and according to the visualization requirements as needed, different transparencies and colors are assigned according to its pixel values to achieve the effect of facilitating the observation of the shape and position of the defect.

[0119] In the embodiment of the present invention, the method of mapping the defect feature area to the three-dimensional model space of the workpiece to be recognized through the three-dimensional visualization technology can be specifically implemented in the following manner:

[0120] Generate two-dimensional background frames with the same number as the number of images in the color ultrasonic B-scan image sequence. For each recognized defect feature area, use the outer bounding rectangle of the defect feature area to extract the defect image block from the corresponding color ultrasonic B-scan image or grayscale B-scan image, and then map and superimpose the defect image block on the corresponding two-dimensional background frame. After completing the mapping and superimposition of all recognized defect feature areas, a three-dimensional visualization image sequence is obtained, and this three-dimensional visualization image sequence is the three-dimensional visualization model of the workpiece to be recognized containing the defect.

[0121] In addition, when performing three-dimensional visualization of the defect in the three-dimensional model space of the workpiece to be recognized, the isolated three-dimensional defect area can be further recognized through the bounding box algorithm, and its shape and position are calculated as the information for visualization query.

[0122] In the embodiment of the present invention, as Figure 12 shown, a data construction method for the three-dimensional visualization image sequence is further provided, and the specific approach is as follows:

[0123] (1) Data reading

[0124] Use the imread command of OpenCV to read the suspected defect image sequence containing only defect features that have undergone image preprocessing and defect feature recognition, save it in the three-dimensional image sequence space, and automatically fill the data corresponding to the remaining images in the matrix with appropriate pixel values according to the set image serial numbers;

[0125] (2) Volume reconstruction

[0126] Create a three-dimensional voxel space of the vtkImageImport class according to the actual size and accuracy requirements of the workpiece to be measured, and then use the voxel nearest neighbor method to perform coordinate transformation on the matrix according to the coordinate mapping rule and write the calculated pixel values into the voxels at the corresponding positions. The logic of the volume reconstruction process is as Figure 13As shown, it can be summarized as follows: traverse the voxel numbers, find the corresponding image numbers according to the coordinate mapping rule, and determine whether the image is a defective feature image; if so, find the corresponding pixels in the corresponding defective feature image and write the gray value into the voxel value; if not, it is considered that there is no defect at this position, and the set background gray value is directly written into the voxel value.

[0127] In the embodiment of the present invention, since the workpiece to be measured is a copper ring and its three-dimensional model is annular, the present invention designs a coordinate mapping rule for ring-shaped parts:

[0128] If the coordinates of a certain voxel are , and the absolute angular position of its corresponding B-scan image is , and the coordinates in the corresponding image sequence space are , then the mapping formula is as follows:

[0129]

[0130] (3) Rendering

[0131] Use the vtkVolumeMapper class to convert the three-dimensional voxels into three-dimensional entities, set the threshold according to the gray value of the voxels, complete the setting of color and transparency, and use the ray casting method for image rendering and display.

[0132] (4) Post-processing

[0133] Use the vtkRender class to complete the display and interaction of the three-dimensional model after the visualization operation, and then use the MarchingCubes method and the AABB bounding box to perform relevant calculations on the defective features, and the information such as the position and size of the defect equivalent can be obtained.

[0134] In the embodiment of the present invention, in the constructed three-dimensional model space, the bounding box algorithm is used to automatically calculate the shape and position of the isolated defective features. An exemplary three-dimensional visualization effect diagram of local defect recognition is as Figure 14 shown.

[0135] It should be noted that although the above embodiments take the copper ring workpiece as an example for defect recognition and three-dimensional visualization, the present invention is not limited to such workpieces and can theoretically be extended to any workpiece.

[0136] Thus, based on the defect recognition and three-dimensional visualization method based on the ultrasonic B-scan image sequence shown in the above S1~S4, the present invention can quickly process the ultrasonic B-scan image data of any complex workpiece, obtain accurate and reliable ultrasonic non-destructive defect detection results, and at the same time realize defect recognition and three-dimensional visualization, thereby reducing the detection difficulty and improving the detection accuracy.

[0137] In addition, based on the same inventive concept, as Figure 15 shown, in another embodiment of the present invention, a defect recognition and three-dimensional visualization system based on ultrasonic B-scan images is further provided, which includes: 1) a B-scan image preprocessing module for grayscale conversion and preprocessing of the original color image to achieve image reduction and noise reduction; 2) a defective image screening module for achieving high-speed screening of suspected defective images, reducing the amount of data to be processed in subsequent steps, and improving data processing efficiency; 3) a defect feature recognition module for automatically identifying and marking the positions of defect features in the B-scan image; 4) a three-dimensional visualization module for visualizing defects in three-dimensional space and automatically calculating the shapes and positions of defects in three-dimensional space. The functions and cooperation relationships of the four modules are as follows:

[0138] The B-scan image preprocessing module is used to obtain a sequence of color ultrasonic B-scan images of the workpiece to be recognized, and perform image preprocessing including lossless grayscale conversion on it to obtain a sequence of grayscale B-scan images.

[0139] The defective image screening module is used to calculate the image entropy of each grayscale B-scan image in the sequence of grayscale B-scan images and form an image entropy sequence, perform phase space reconstruction after expanding the image entropy sequence to obtain reconstruction vectors identical to the sequence of color ultrasonic B-scan images, calculate the similarity between all pairs of reconstruction vectors and generate a two-dimensional threshold-free recurrence plot with the similarity as elements, binarize the threshold-free recurrence plot and characterize the mode mutation probability density by calculating the element mean of each row, normalize the element means of all rows to form a one-dimensional filter with the same dimension as the sequence of color ultrasonic B-scan images, filter the image entropy sequence using the one-dimensional filter, and calculate the difference between the original value and the filtered value of the image entropy of each grayscale B-scan image, and extract all grayscale B-scan images with differences greater than 0 as suspected defective images.

[0140] The defect feature recognition module is used to input all suspected defective images into a defect recognition model pre-trained based on a neural network to identify the defect feature regions in each suspected defective image, and if no defect feature region is recognized, the suspected defective image will be excluded.

[0141] The three-dimensional visualization module is used to map all recognized defect feature regions to the three-dimensional model space of the workpiece to be recognized according to the suspected defective images where they are located through three-dimensional visualization technology to complete the three-dimensional visualization of the defects in the workpiece to be recognized.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that it is still possible to modify the specific implementation manners of the present invention or perform equivalent replacements for some technical features; without departing from the spirit of the technical solutions of the present invention, they should all be covered within the scope of the technical solutions claimed by the present invention.

Claims

1. A defect recognition and three-dimensional visualization method based on ultrasonic B-scan images, characterized in that: include: S1, obtaining a color ultrasonic B-scan image sequence obtained for a workpiece to be identified, and performing image preprocessing including lossless grayscale conversion on the image sequence to obtain a grayscale B-scan image sequence; S2. For the grayscale B-scan image sequence, the image entropy of each grayscale B-scan image is calculated to form an image entropy sequence. The image entropy sequence is expanded and then reconstructed in phase space to obtain a reconstruction vector that is the same as the color ultrasound B-scan image sequence. The similarity of all the reconstruction vectors is calculated pairwise and a two-dimensional threshold-free recursive graph is generated with the similarity as an element. The threshold-free recursive graph is binarized and the element mean of each row is calculated to characterize the pattern mutation probability density. The element means of all rows are normalized to form a one-dimensional filter with the same dimension as the color ultrasound B-scan image sequence. The image entropy sequence is filtered using the one-dimensional filter, and the difference between the original image entropy value and the filtered value of each grayscale B-scan image is calculated. All grayscale B-scan images with a difference greater than 0 are extracted as suspected defect images. S3, inputting all suspected defect images into a defect recognition model pre-trained based on a neural network, identifying the defect feature area in each suspected defect image, and if the defect feature area is not identified, the suspected defect image will be eliminated; S4. Mapping all identified defect feature areas to the three-dimensional model space of the workpiece to be identified according to their respective suspected defect images through three-dimensional visualization technology to complete the three-dimensional visualization of the defects in the workpiece to be identified.

2. The defect recognition and three-dimensional visualization method based on ultrasonic B-scan images according to claim 1, characterized in that: For each color ultrasound B-scan image, when it is losslessly gray-scaled into a gray-scale B-scan image, the gray value of each pixel is 255 and one-third of the green component value and two-thirds of the blue component value are subtracted.

3. The defect recognition and three-dimensional visualization method based on ultrasonic B-scan images according to claim 1, characterized in that: The specific processing method for performing the image preprocessing on each color ultrasonic B-scan image is as follows: first, the color ultrasonic B-scan image is losslessly grayed, and then the grayed ultrasonic B-scan image is subjected to median filtering to complete image denoising, and then the noise is further eliminated and the target is segmented through an opening operation, and finally, the region of interest where defects need to be identified is cropped, and the cropped grayed B-scan image is output.

4. The defect recognition and three-dimensional visualization method based on ultrasonic B-scan images according to claim 1, characterized in that: The method for generating a threshold-free recurrence graph based on the image entropy sequence is: According to the color ultrasound B-scan image sequence n, the pre-optimized optimal delay time τ and the optimal embedding dimension m, the n-dimensional image entropy sequence is firstly tail-filled to obtain an n+(m-1)τ-dimensional expanded image entropy sequence, and then the first n image entropies of the expanded image entropy sequence are sequentially taken as the starting point, and m image entropies are sampled from each starting point at an interval of τ to form a row vector; Reconstruct an n×n threshold-free recursive graph, in which the element value of the i-th row and j-th column is the similarity between the row vectors of the i-th and j-th starting points.

5. The defect recognition and three-dimensional visualization method based on ultrasonic B-scan images according to claim 1, characterized in that: When the threshold-free recursive graph is binarized, if the value of an element in the threshold-free recursive graph exceeds the threshold, it is set as a black point, otherwise it is set as a white point, and the threshold should make the proportion of black points in the binarized recursive graph 5%.

6. The defect recognition and three-dimensional visualization method based on ultrasonic B-scan images according to claim 1, characterized in that: When the image entropy sequence is filtered by a one-dimensional filter, the filtered image entropy sequence that is the same as the color ultrasound B-scan image sequence is obtained by multiplying the element mean sequence corresponding to the one-dimensional filter by the image entropy sequence element by element.

7. The defect recognition and three-dimensional visualization method based on ultrasonic B-scan images according to claim 1, characterized in that: The defect recognition model is based on YOLOv5 or YOLOv8 and is obtained through supervised training with defect annotation data.

8. The defect recognition and three-dimensional visualization method based on ultrasonic B-scan images according to claim 1, characterized in that: The method for mapping the defect feature area to the three-dimensional model space of the workpiece to be identified by three-dimensional visualization technology is as follows: generating two-dimensional background frames with the same number as the images in the color ultrasonic B-scan image sequence, for each identified defect feature area, using the outer rectangular frame of the defect feature area to extract the defect image block from the corresponding color ultrasonic B-scan image or grayscale B-scan image, and then mapping the defect image block to be superimposed on the corresponding two-dimensional background frame. After completing the mapping and superposition of all identified defect feature areas, a three-dimensional visualization model of the workpiece to be identified containing defects is obtained.

9. The defect recognition and three-dimensional visualization method based on ultrasonic B-scan images according to claim 1, characterized in that: When three-dimensionally visualizing defects in the three-dimensional model space of the workpiece to be identified, the bounding box algorithm is used to identify isolated three-dimensional defect areas, and their shapes and positions are calculated as information for visualization queries.

10. A defect recognition and three-dimensional visualization system based on ultrasonic B-scan images, characterized in that: include: A B-scan image preprocessing module is used to obtain a color ultrasonic B-scan image sequence obtained for the workpiece to be identified, and perform image preprocessing including lossless grayscale conversion on the image sequence to obtain a grayscale B-scan image sequence; A defective image screening module is used to calculate the image entropy of each grayscale B-scan image in the grayscale B-scan image sequence and form an image entropy sequence, expand the image entropy sequence and then reconstruct the phase space to obtain a reconstruction vector that is the same as the color ultrasound B-scan image sequence, calculate the similarity of all the reconstruction vectors in pairs and generate a two-dimensional threshold-free recursive graph with the similarity as the element, binarize the threshold-free recursive graph and characterize the pattern mutation probability density by calculating the element mean of each row, normalize the element means of all rows to form a one-dimensional filter with the same dimension as the color ultrasound B-scan image sequence, filter the image entropy sequence using the one-dimensional filter, calculate the difference between the original image entropy value and the filtered value of each grayscale B-scan image, and extract all grayscale B-scan images with a difference greater than 0 as suspected defective images; The defect feature recognition module is used to input all suspected defect images into a defect recognition model pre-trained based on a neural network, identify the defect feature area in each suspected defect image, and remove the suspected defect image if the defect feature area is not recognized; The three-dimensional visualization module is used to map all identified defect feature areas according to their respective suspected defect images into the three-dimensional model space of the workpiece to be identified through three-dimensional visualization technology, so as to complete the three-dimensional visualization of the defects in the workpiece to be identified.

Images