Method for Determining Defect Type Based on Gray Value Distribution of X-ray Images

Abstract

The present application discloses a method for determining the type of defect based on the gray value distribution of X-ray images. Input the to-be-detected X-ray image Pic1, establish the OUV pixel coordinates, and read the pixel coordinates p of any pixel point within the region of interest n (u x , v y ); Read the pixel points with the gray value of Cr to determine the image patch Kr, perform selective partial erosion and re-assignment operations on the image patch Krm, and input it into the trained adversarial network model for discrimination and output the result information. By extracting the disorder characteristics of the boundary of the defect image patch, and at the same time adding the ratio of the pixel area of the defect image patch to the total area and the consistency evaluation of the pixel gray values around the defect image patch, and then fusing the adversarial network model for discrimination, the authenticity of the defect and the accuracy of the curve type can be accurately discriminated to a great extent.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to the technical field of defect recognition in X-ray detection pictures of power equipment. Specifically, it relates to a method for determining defect types based on the gray value distribution of X-ray pictures. Background Art

[0002] The power transmission stability of power equipment is related to the normality of the entire power grid. In the entire power grid, components that are prone to defects leading to power failures mainly include strain clamps, GIS housings, etc.; as an important part of power transmission, they play a crucial role in the entire power grid. Once an accident occurs and power outage maintenance is required, a large amount of manpower, material resources, financial resources, and time need to be invested. Solving (or reducing) the impact of strain clamps has significant social significance for improving the safety, stability, and economy of the line.

[0003] Currently, the detection of power component defects mainly includes ultrasonic, X-ray, infrared thermal imaging, and ultraviolet imaging. Crimping defects occur during construction under power outage conditions such as new construction, expansion, reconstruction, and maintenance of lines. Only ultrasonic or X-ray detection methods can be relied on. The ultrasonic detection method can only detect the crimping defects of the steel anchor anti-slip groove, while X-ray can detect all crimping defects and internal crack defects caused by structural fatigue.

[0004] Currently, the determination of power structure defects mainly relies on manual data image review and defect determination, and there are the following problems:

[0005] The professional skills and experience requirements for manual X-ray image interpretation personnel are very high, and long-term professional training is required. The timeliness of manual completion of analysis, film evaluation, and issuance of analysis results is poor, which has a serious impact on energizing the line on time. Even because the results cannot be given in time, the line operates with serious and critical defects. Manual interpretation is prone to misjudgment, missed judgment, and wrong judgment, affecting the accuracy of the detection results.

[0006] Therefore, an intelligent defect determination method is needed to solve the problems of long time for manual determination and being severely restricted by experience. Summary of the Invention

[0007] In order to solve the problems of low discrimination efficiency and large differences in discrimination results among different personnel when manually checking the defects in X-ray pictures of power equipment one by one, the present application provides a method for determining defect types based on the gray value distribution of X-ray pictures, which integrates the artificially set determination logic into the defect discrimination process by using the high efficiency of software calculation, so as to achieve effective compatibility in both high efficiency and accuracy of discrimination.

[0008] As one of the representatives of intelligence and technology, currently, using computer software to achieve deep learning discrimination is undoubtedly the most efficient and intelligent option. However, current deep learning networks, neural network systems based on convolutional operations, or autoencoder neural networks based on multiple layers of neurons, or pre-training in the form of multi-layer autoencoder neural networks, are all improved designs aimed at different objects and purposes to obtain results that are more in line with expectations or closer to the principles of human judgment. The essence of deep learning is that computer software recognizes digital signal data at a speed unimaginable to humans and performs calculations according to established, complex, and optional rules, thereby achieving the result of artificial intelligence judgment. In the determination of defects in X-ray photographs of power equipment, the applicant found that due to the concentration of structural components with defects in power equipment and the limited types of defects, the determination of defects is particularly special. The images formed by the defects have specificity or uniqueness compared to the structural components themselves. Through the R & D team of the applicant's analysis of a huge number of power equipment defect samples at the pixel level, the analyzed defects include steel core fracture, steel core strand breakage, anti-slip groove underpressure, anti-slip groove underpressure, insufficient insertion depth of the steel core, insufficient crimping length of the steel core, fitting damage, fitting bending, insufficient crimping length of aluminum stranded wire, and over-crimping of the steel anchor pipe. The conclusion of the analysis reveals a characteristic: in the X-ray images of power equipment, the distribution of gray values has regularity, the fluctuation range of the gray value distribution of the same component is small, and it has unity; no matter which type of defect appears on which power structure, the number of adjacent structural component image patches involved around the image patch formed by the defect is mostly between 1 and 3. For image patches of types such as fracture and crack, they often only exist in the same structural component. Specifically, the biggest difference between the defect image patch and the power structure component image patch is the irregularity of the patch edge formed on the X-ray image, that is, the edge of the X-ray image patch formed by the power equipment structural component has high regularity and is usually composed of straight lines or smooth curves; while the image patch formed by the defect does not have this characteristic, but instead has great disorder. The invention team completed the present invention based on this characteristic.

[0009] Specifically, to achieve the above object, the technical solution adopted in this application is as follows:

[0010] A method for determining the type of defect based on the distribution of gray values of X-ray pictures, comprising the following steps:

[0011] Step STP100, input the X-ray picture Pic to be detected 1 , determine the preset region of interest, establish the OUV pixel coordinate with the upper left corner of the region of interest as the origin O, and read the pixel coordinate p of any pixel point within the region of interest n (u x , v y );

[0012] Step STP200: Read the pixel points with the gray value Cr, obtain the set Mr of pixel points with the gray value Cr, and determine the image patch Kr formed by covering all the pixel points with the gray value Cr by reading the pixel coordinates of any pixel point with the gray value Cr; where r ∈ [0, 255].

[0013] Step STP300: Sort the image patches Kr in ascending or descending order of the r value in sequence, mark them as Krm, and perform selective partial erosion on the image patches Krm to obtain the X-ray image Pic composed of the eroded image patches Krm 2 ; where the serial number m ≥ 1.

[0014] Step STP400: Calculate the ratio β of the pixel area Srm of each image patch Krm after image erosion to the pixel area of the X-ray image Pic 1 If the ratio β > 0.2, then mark the image patch Krm as 0; otherwise, mark it as 1.

[0015] Step STP500: Reassign the gray value of the image patch Krm marked as 0 to 0 or 255 to obtain the X-ray image Pic 3 And input it into the trained adversarial network model for discrimination and output the result information.

[0016] As a preferred method, specifically, in order to improve the boundary clarity of different components or different imaging parts in the X-ray image, the steps of performing selective partial erosion on the image patch Krm in Step STP300 are as follows: Erode the image patch Krm with an odd serial number m outward by t pixels. If the serial number m of other adjacent image patches Krm is even, then the image patch Krm with an even serial number m is eroded inward by t pixels; if the serial number m of the adjacent image patch Krm is still odd, then the number of pixels eroded in this adjacent part is 0, where t ≥ 1.

[0017] Preferably, the adversarial network model in Step STP500 includes a convolutional neural module and a conditional determination module, and the determination module includes a boundary extraction unit, an assignment unit, a curve fitting unit, and a slope statistics unit;

[0018] The defect determination steps in the image patch Krm are as follows:

[0019] Step STP501: Input the image patch Krm marked as 1 into the conditional determination module, and extract the boundary Bm of each image patch Krm with a gray value not equal to 0 or 255 through the boundary extraction unit; if multiple boundaries Bm are obtained, then

[0020] Step STP502, in the OUV pixel coordinates, the OUV pixel coordinates are gridded with a length unit of 10 - 100 pixels by an assignment unit, and the intersections Qz of multiple boundaries Bm and the grid are obtained, and the pixel coordinates of each intersection Qz are recorded;

[0021] Step STP503, the curve fitting unit establishes a fitting curve with the intersection Qz as the fitting point;

[0022] Step STP504, the slope statistics unit calculates the slope kz at each intersection Qz and statistically obtains a slope set;

[0023] Step STP505, draw a disordered broken line of the slope set according to the slope kz, obtain the span of the slope set and the number of times the direction of the slope kz changes, and output the conclusion of whether the discriminated image patch Krm is defective;

[0024] Step STP506, if the output result in Step STP505 is 'no', then perform the discrimination of the next image patch Krm+1; if the output result in Step STP505 is 'yes', then input the image patch Krm into the convolutional neural module for defect type discrimination and output the defect type.

[0025] Further preferably, the determination process of outputting whether the discriminated image patch Krm is defective is as follows:

[0026] Step STP5051, calculate the span of the slope set, and the span of the slope set is the sum of the absolute values of the slope kzmin and kzmax;

[0027] Step STP5052, calculate the number of times the direction of the slope kz changes within the span of the slope set;

[0028] Step STP5053, calculate the number of times the direction of the slope kz changes per unit span δ = the number of times the direction of the slope kz changes / the span of the slope set;

[0029] Step STP5054, when δ≥100, it is determined that the image patch Krm is defective; when 99≥δ≥40, it is determined that the image patch Krm is a suspected defect and Step STP5055 is executed; when 39≥δ, it is determined that the image patch Krm is non-defective;

[0030] Step STP5055, read and compare the gray values of at least two pixel points A1 and A2 on different sides or in the same direction of the boundary Bm of the image patch Krm. If the difference in the gray values of A1A2 is less than 15, it is determined to be defective; if the difference in the gray values of A1A2 is greater than 15, it is determined to be non-defective.

[0031] Beneficial effects:

[0032] By extracting the disorder characteristics of the boundaries of defective image patches, and adding the ratio of the pixel area of the defective image patches to the total area, as well as the consistency evaluation of the pixel gray values around the defective image patches, and then fusing the discriminative ability of the adversarial network model, the authenticity of the defects and the accuracy of the curve types can be accurately discriminated to a great extent.

[0033] The present invention combines the intelligence and high efficiency of the deep learning model, and has a targeted condition determination module for the X-ray images of power equipment defects, which can make up for the problem of weak pertinence of the adversarial network model in specific image fields. By implementing the determination of the condition determination module, the non-defective image patches in the X-ray images can be eliminated, the time consumed by useless judgments can be reduced, and the misjudgments and false judgments introduced by the non-defective image patches can be eliminated, thereby improving the overall accuracy of the determination conclusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0035] Figure 1 It is an X-ray picture of a steel core fracture / crack defect.

[0036] Figure 2 It is Figure 1 The picture after corrosion and re-assignment to 0.

[0037] Figure 3 It is Figure 2 The partial boundary diagram of

[0038] Figure 4 It is Figure 3 The enlarged view of area V in

[0039] Figure 5 It is a partial schematic diagram of the slope Kz. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Usually, the components of the embodiments of the present application described and illustrated in the drawings here can be arranged and designed in various different configurations.

[0041] Accordingly, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts fall within the scope of protection of the present application.

[0042] It should be noted that like reference numerals and letters denote like items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0043] Preferred Embodiment:

[0044] This embodiment takes the crimping of a steel core as an example to illustrate the method for determining the defect type based on the gray value distribution of X-ray images provided by the present application, including the following steps:

[0045] Step STP100, input the X-ray image Pic to be detected 1 , as Figure 1 shown, determine the preset region of interest, establish the OUV pixel coordinates with the upper left corner of the region of interest as the origin O, and read the pixel coordinates p n (u x , v y ) of any pixel point within the region of interest; where x is the value of the pixel point sorted as n on the pixel abscissa U, y is the value of the pixel point sorted as n on the pixel ordinate V, and the sorting method of n is defined by the system. As one of the optional methods, it can be arranged row by row and column by column in sequence starting from the first row and the first column, and then from the second row and the first column from left to right until the last column of the last row, and all pixel points within the region of interest are read. After completing this step, the input X-ray image Pic 1 is obtained, which consists of a total of n pixel points, and the position of each pixel point is defined by pixel coordinates. Thus, the next step can be carried out.

[0046] Step STP200, read the pixel points with the gray value Cr, obtain the set Mr of pixel points with the gray value Cr, and determine the image patch Kr formed by covering all pixel points with the gray value Cr by reading the pixel coordinates of any pixel point with the gray value Cr; where r ∈ [0, 255]; any detected X-ray image Pic 1 is composed of gray values from 0 to 255. The set of pixel points with the same gray value is extracted separately. Thus, an X-ray image Pic 1It will be segmented into multiple image patches Kr according to different gray values. It should be noted that since the gray value span of the pixel points involved in a picture may be very large, and even pixel points with any gray value exist, the maximum number of the obtained pixel point set Mr and the image patches Kr can reach 255, that is, r = 255 at this time. In this case, a large number of image patches Kr are in a discrete state, and as a whole, they reflect a certain specific shape of the patches. However, the pixel point set Mr that makes up the image patch Kr is composed of multiple discrete or partially discrete pixel points. Therefore, this will bring a huge obstacle to discrimination. As a general case, when the same structural member is imaged under X-ray irradiation, theoretically, the same gray value should be obtained, such as Cr = 189. However, for the actually obtained image, it may be alternately formed by Cr = 187 - 192 to form the complete image of the structural member. Therefore, in order to clean these inevitable image noises, the X-ray picture Pic 1 needs to be further processed to obtain a clear target image.

[0047] Step STP300: Arrange the image patches Kr in ascending or descending order of the r value, mark them as Krm, and perform selective partial erosion on the image patches Krm to obtain the X-ray picture Pic composed of the eroded image patches Krm 2 ; where the serial number m ≥ 1. In order to improve the boundary clarity of different components or different imaging parts in the X-ray image, the steps of selective partial erosion of the image patches Krm in step STP300 are as follows: Erode the image patches Krm with odd serial numbers m outward by t pixels. If the serial numbers m of the other adjacent image patches Krm are even, then the image patches Krm with even serial numbers m are eroded inward by t pixels; if the serial number m of the adjacent image patch Krm is still odd, the number of eroded pixels in this adjacent part is 0, where t ≥ 1. The X-ray picture Pic 1 As Figure 1 shown, after the erosion operation, see Figure 2 the X-ray picture Pic shown 2 ; After the erosion operation in this step, the boundary in the gradient region can be made clearer. At the same time, it can make the image patches with relatively small differences dissolve into each other, and merge multiple intersecting patches into the same patch.

[0048] Step STP400: Calculate the pixel area Srm of each image patch Krm after image erosion and the X-ray picture Pic 1The pixel area ratio β. If the ratio β > 0.2, the image patch Krm is marked as 0; otherwise, it is marked as 1. Based on the defect characteristics of power equipment, such as steel core fracture, steel core strand breakage, anti-slip groove underpressure, anti-slip groove undervoltage, insufficient steel core insertion depth, insufficient steel core crimping length, fitting damage, fitting bending, insufficient aluminum stranded wire crimping length, over-crimping of steel anchor pipe, etc., the proportion of the imaging of these defects in the entire image pixel area usually does not exceed 5%, and the general defect imaging is between 0.5% - 3%. Therefore, through this step, most of the content that needs to be discriminated or is unnecessary to be discriminated can be eliminated. On the one hand, it can improve the discrimination efficiency and reduce the operation content by at least one order of magnitude; on the other hand, it can improve the discrimination accuracy, increase the accuracy of defect acquisition, and reduce the possibility of missed judgment.

[0049] Step STP500, re-assign the gray value of the image patch Krm marked as 0 to 0 or 255 to obtain the X-ray picture Pic 3 And input it into the trained adversarial network model for discrimination and output the result information.

[0050] In this embodiment, the adversarial network model in step STP500 includes a convolutional neural module and a conditional determination module. The determination module includes a boundary extraction unit, an assignment unit, a curve fitting unit, and a slope statistics unit; the adversarial network model adopts the adversarial network model described in the patent application of the applicant with the publication number CN111027631A and the application number CN201911286070.7.

[0051] The defect determination steps in the image patch Krm are as follows:

[0052] Step STP501, input the image patch Krm marked as 1 into the conditional determination module, and extract the boundary Bm of each image patch Krm whose gray value is not equal to 0 or 255 through the boundary extraction unit; if multiple boundaries Bm are obtained, then

[0053] Step STP502, in the OUV pixel coordinates, grid the OUV pixel coordinates with a length unit of 10 - 100 pixels through the assignment unit, obtain the intersection points Qz of multiple boundaries Bm and the grid, and record the pixel coordinates of each intersection point Qz.

[0054] Step STP503, the curve fitting unit establishes a fitting curve with the intersection points Qz as the fitting points.

[0055] Step STP504, the slope statistics unit calculates the slope kz at each intersection point Qz and statistically obtains the slope set.

[0056] Step STP505, draw the disorder discount of the slope set according to the slope kz, obtain the span of the slope set and the number of times the direction of the slope kz changes, and output the conclusion of whether the discriminated image patch Krm is defective; as Figures 3 - 4 shown.

[0057] Step STP506, if the output result in Step STP505 is 'no', then perform the discrimination of the next image patch Krm+1; if the output result in Step STP505 is 'yes', then input the image patch Krm into the convolutional neural module for defect type discrimination and output the defect type.

[0058] Further preferably, the determination process of outputting whether the discriminated image patch Krm is defective is as follows:

[0059] Step STP5051, calculate the span of the slope set, and the span of the slope set is the sum of the absolute values of the slope kzmin and kzmax;

[0060] Step STP5052, calculate the number of times the direction of the slope kz changes within the span of the slope set;

[0061] Step STP5053, calculate the number of times the direction of the slope kz changes per unit span δ = the number of times the direction of the slope kz changes / the span of the slope set; as Figure 5 shown, within the range of the slope k1-k11, the span is 0.89 + 0.43 = 1.32, and the number of times the slope changes is 8967 times, that is, δ = 8967 / 1.32 = 6793.18.

[0062] Step STP5054, when δ≥100, then determine that the image patch Krm is defective; when 99≥δ≥40, then determine that the image patch Krm is a suspected defect and execute Step STP5055; when 39≥δ, then determine that the image patch Krm is non-defective;

[0063] Step STP5055, read and compare the gray values of at least two pixel points A1 and A2 on different sides or in the same orientation of the boundary Bm of the image patch Krm. If the difference between the gray values of A1A2 is less than 15, then determine it as defective; if the difference between the gray values of A1A2 is greater than 15, then determine it as non-defective.

[0064] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. Method for determining defect type based on gray value distribution of X-ray images, Characterized in that: It includes the following steps: Step STP100, input the X-ray image Pic to be detected 1 , determine the preset region of interest, establish the OUV pixel coordinates with the upper left corner of the region of interest as the origin O, and read the pixel coordinates p of any pixel point within the region of interest n (u x , v y ); Step STP200, read the pixel points with gray value Cr, obtain the set Mr of pixel points with gray value Cr, and determine the image patch Kr formed by covering all pixel points with gray value Cr by reading the pixel coordinates of any pixel point with gray value Cr; where r ∈ [0, 255]; Step STP300, sort the image patches Kr in ascending or descending order of the r value, label them as Krm, and perform selective partial erosion on the image patches Krm to obtain an X-ray picture Pic composed of the eroded image patches Krm 2 ; where the serial number m ≥ 1; Step STP400, calculate the ratio β of the pixel area Srm of each image patch Krm after image erosion to the pixel area of the X-ray picture Pic 1 If the ratio β > 0.2, mark the image patch Krm as 0; otherwise, mark it as 1; Step STP500, reassign the gray value of the image patch Krm marked as 0 to 0 or 255 to obtain the X-ray picture Pic 3 And input it into the trained adversarial network model for discrimination and output the result information; the adversarial network model includes a convolutional neural module and a conditional determination module, and the determination module includes a boundary extraction unit, an assignment unit, a curve fitting unit, and a slope statistics unit; The defect determination steps in the image patch Krm are as follows: Step STP501, input the image patch Krm marked as 1 into the condition determination module, and extract the boundary Bm of each image patch Krm whose gray value is not equal to 0 or 255 through the boundary extraction unit; if multiple obtained boundaries Bm are obtained, then Step STP502, in the OUV pixel coordinates, grid the OUV pixel coordinates with a length unit of 10 - 100 pixels through the assignment unit, obtain the intersection points Qz of multiple boundaries Bm and the grid, and record the pixel coordinates of each intersection point Qz; Step STP503, the curve fitting unit establishes a fitting curve with the intersection points Qz as the fitting points; Step STP504, the slope statistics unit calculates the slope kz at each intersection point Qz and statistically obtains the slope set; Step STP505, draw a disordered broken line of the slope set according to the slope kz, obtain the span of the slope set and the number of times of slope kz direction change, and output the conclusion of whether the discriminated image patch Krm is a defect; Step STP506, if the output result in Step STP505 is 'no', then perform the discrimination of the next image patch Krm+1; if the output result in Step STP505 is 'yes', then input the image patch Krm into the convolutional neural module for defect type discrimination and output the defect type.

2. The method for determining defect type based on gray value distribution of X-ray images according to claim 1, Characterized in that: The steps of selectively partially corroding the image patch Krm in Step STP300 are as follows: corrode the image patch Krm with an odd serial number m outward by t pixels, and if the serial number m of the other image patch Krm adjacent to the image patch Krm is even, then the image patch Krm with an even serial number m is corroded inward by t pixels; if the serial number m of the adjacent image patch Krm is still odd, then the number of pixels corroded in this adjacent part is 0, where t ≥ 1.

3. The method for determining defect type based on gray value distribution of X-ray images according to claim 1, Characterized in that: The determination process of outputting whether the discriminated image patch Krm is a defect is as follows: Step STP5051, calculate the span of the slope set, and the span of the slope set is the sum of the absolute values of the slope kzmin and kzmax; Step STP5052, calculate the number of times of slope kz direction change within the span of the slope set; Step STP5053, calculate the number of times of slope kz direction change per unit span δ = number of times of slope kz direction change / span of the slope set; Step STP5054: When δ≥100, it is determined that the image patch Krm is a defect; when 99≥δ≥40, it is determined that the image patch Krm is a suspected defect and step STP5055 is executed; when 39≥δ, it is determined that the image patch Krm is a non-defect. Step STP5055: Read and compare the gray values of at least two pixel points A1 and A2 that are not on the same side or in the same orientation of the boundary Bm of the image patch Krm. If the difference between the gray values of A1 and A2 is less than 15, it is determined to be a defect; if the difference between the gray values of A1 and A2 is greater than 15, it is determined to be a non-defect.

Images