A hull plate crack identification method based on deep learning

Abstract

The present application relates to the technical field of ship detection, and particularly relates to a hull plate crack identification method based on deep learning. The method comprises the following steps: acquiring actual image data of a hull plate; marking the image data with cracks through a CNN-based identification positioning model; dividing the image data into obvious crack image data and non-obvious crack image data according to crack sizes; performing risk early warning on the obvious crack image data; inputting the non-obvious crack image data into a tracking identification model to judge crack changes; if the crack changes are greater than a change threshold, performing risk early warning; if the crack changes are less than the change threshold, judging whether there is a crack convergence risk; if there is a convergence risk, performing risk early warning. The present application has an automatic crack identification method for hull plate cracks that cannot be directly observed by human eyes, and has different processing methods for different hull plate cracks, thereby saving a large amount of manpower and resources.

Description

Technical Field

[0001] This invention relates to the field of ship hull inspection technology, and in particular to a method for identifying cracks in ship hull plates based on deep learning. Background Technology

[0002] While minor cracks in the hull plating may initially appear to have a minor impact on the hull, this impact is actually potential, long-term, and can gradually worsen. However, during operation, any crack that forces the ship to a repair point significantly increases the distance traveled and affects its route, thereby substantially increasing operating costs. This method primarily involves two aspects: first, identifying and determining the most obvious cracks in the hull plating; and second, further processing the identified cracks by tracking and monitoring less obvious cracks over a period of time to assess their variation and develop appropriate treatment methods. Summary of the Invention

[0003] This invention discloses a deep learning-based method for identifying cracks in ship hull plates. The specific method is as follows:

[0004] Obtain actual image data of the hull hull;

[0005] Cracked image data was labeled using a CNN-based recognition and localization model.

[0006] For image data with cracks, calculate the crack size and divide it into image data with obvious cracks and image data with inconspicuous cracks based on the crack size.

[0007] For image data with obvious cracks, risk warnings should be issued;

[0008] For image data with indistinct cracks, the data is input into a tracking and recognition model to determine the changes in the cracks.

[0009] If the change in cracks exceeds the change threshold, a risk warning will be issued.

[0010] If the change in cracks is less than the change threshold, determine whether there is a risk of crack convergence. If there is a risk of convergence, issue a risk warning.

[0011] Furthermore, the specific method for calculating the crack size of cracked image data is as follows:

[0012] Convert image data with obvious cracks into black and white images;

[0013] Iterate through each pixel in the image and count the number of white pixels;

[0014] The crack size is calculated using the formula S = n * Sn, where S is the crack area, n is the number of white pixels in the crack region, and Sn is the actual area of ​​the pixel.

[0015] Furthermore, based on crack size, image data is divided into images of obvious cracks and images of inconspicuous cracks, as follows:

[0016] Set the size threshold ρ';

[0017] According to the formula Calculate the crack classification index; where S' is the total area of ​​the image;

[0018] When ρ≥ρ', it is determined to be image data with obvious cracks; otherwise, it is determined to be image data with indistinct cracks.

[0019] Furthermore, for image data with indistinct cracks, the specific method for determining crack changes is as follows:

[0020] After edge detection processing, the crack is fitted into a curve;

[0021] The formula for calculating the change in crack length is as follows:

[0022]

[0023] In the formula, the original crack length is l, and the crack length after time change is l';

[0024] The formula for calculating the change in crack height is as follows:

[0025]

[0026] In the formula, the initial crack height is h, and the crack height after time change is h';

[0027] The overall amplitude of the cracks over a long period of time is:

[0028] δ=θ·m l +(1-θ)n l

[0029] In the formula, θ is the variation range parameter, which is determined based on the properties of the crack itself and the surrounding environment.

[0030] Furthermore, the variation range parameter θ is determined through fuzzy recommendation, with the specific formula as follows:

[0031] θ=a1 / x1+a2 / x2+a3 / x3+a4 / x4+…+an / xn

[0032] In the formula, x1 represents the factors of the crack and water flow direction before the tracking period; x2 represents the factors of the water flow and the ship's direction of travel; x3 represents the temperature and humidity factors in the sea area; x4 represents the factors of the crack's own structure; and xn represents the factors added artificially under different conditions.

[0033] In the formula, the denominator represents the different elements of the influencing factors, and the numerator represents the membership degree of that element, which is determined by an expert system.

[0034] Furthermore, to determine whether there is a risk of crack convergence, the specific methods are as follows:

[0035] After edge detection processing, the tiny cracks are fitted into a curve, and the relevant parameters are calculated.

[0036] Locate adjacent, inconspicuous cracks based on the markings;

[0037] Based on whether inconspicuous cracks intersect, the degree of crack distribution within the region, and the degree of crack parallelism, a decision tree is used to classify whether there is a risk of crack convergence.

[0038] Furthermore, the distribution and parallelism of cracks within the region are calculated using the following methods:

[0039] Calculate the degree of crack distribution:

[0040]

[0041] In the formula, S1 is the area of ​​the region, and S2 is the area of ​​the crack;

[0042] Let the straight-line distance between any two cracks be l, and the length and width of the region be l. m and l n ;

[0043] When θ s <5%, When the condition is met, it is judged to be sparse;

[0044] When 5% < θ s <20% At that time, it was judged as general;

[0045] When θ s >20%, At that time, it was judged to be dense;

[0046] Let θ be the angle between different cracks. l ;

[0047] When θ l <5° indicates a high degree of parallelism;

[0048] When 5° < θ s The degree of parallelism is generally judged.

[0049] Due to the adoption of the above technical solutions, the present invention has the following beneficial effects:

[0050] 1. There is an automatic crack identification method for hull plates that cannot be directly observed by the human eye. Different treatment methods are available for different hull plate cracks, which can save a lot of manpower and resources.

[0051] 2. This invention does not require a large amount of manpower and resources. Based on the ship's hardware conditions, after the corresponding software and hardware are installed, only a relatively small number of personnel are needed to operate it, and no special training is required.

[0052] 3. This invention is highly editable. Different parameters can be set to suit different environments and conditions.

[0053] 4. The present invention is easy to maintain and update. It does not require a lot of time, and maintenance and updates will not affect the use of this device.

[0054] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0055] The accompanying drawings of this invention are described below.

[0056] Figure 1 This is a schematic diagram of the overall process of Example 1.

[0057] Figure 2 This is a flowchart illustrating the identification and positioning model of Example 1.

[0058] Figure 3 This is a schematic diagram of the tracking and recognition model process in Example 1.

[0059] Figure 4 This is a schematic diagram of the decision tree in Example 1.

[0060] Figure 5 This is a schematic diagram of the device structure in Example 2. Detailed Implementation

[0061] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0062] Example 1:

[0063] A deep learning-based method for identifying cracks in ship hull plates, such as... Figure 1 As shown, the specific steps are as follows:

[0064] S1. Obtain actual image data of the hull plating.

[0065] In step S1, the actual image data is stored in a database and divided by time and location. The database is cleaned periodically every 15 days, starting from the first data collection. The labels are numbered from 1 to 15, and each day's data collection is considered complete after the first collection.

[0066] The location is first divided into different areas, and then underwater cameras or robots determine the approximate area for image data transmission, using classifications such as I, II, and III. This is represented as:

[0067]

[0068] The specific label is determined based on the specific region n and the specific time m.

[0069] S2. Cracked image data is labeled using a CNN-based recognition and localization model.

[0070] The labeled real-time data is used as input into the recognition and localization model to obtain output images with different classification results. Figure 2 The specific steps for identifying the localization model are as follows:

[0071] S21. Collect an image dataset of hull plate cracks from the open-source dataset on the Kaggle website.

[0072] S22. The Selective Search algorithm is used to generate candidate regions, generating a series of candidate regions that may contain the target.

[0073] S23. The image is segmented into many small regions using the Felzenszwalb method.

[0074] S24. Obtain the candidate region set R = {r1, r2, ..., r} using segmentation. n}

[0075] S25. Initialize the set Used to store similarity results of neighboring regions.

[0076] S26. Traverse adjacent regions (r) in R i ,r j ), calculate the similarity S(r) i ,r j ), S=S∪S(r) i ,r j )

[0077] S27. If S is not an empty set, then S(r) i ,r j = max(S), r t =r i ∪rj Remove region r from S. i and r j Collect all relevant similarity results and calculate the new region r. t The set of similarities S with the surrounding region t , will S t Add the new region r to set S. t Place it into set R.

[0078] S28. After the traversal is complete, extract the bounding boxes of all regions in R, which are the regions where objects may exist.

[0079] S29. Convolutional and pooling layers of a CNN are used to extract features from the image. These layers can automatically learn local features in the image and combine these features into a higher-level feature representation. Based on a dataset with ground truth bounding box labels, a selective search method is used to generate candidate regions. The candidate regions are then scaled to a size of 227x227. The preprocessed candidate regions are then fed into the CNN model for forward propagation to extract fixed-length feature vectors.

[0080] A classifier is used to classify the feature vectors of candidate regions to determine whether they contain the target and the target's category. An SVM classifier is trained based on the extracted feature vectors and the true class labels.

[0081] The feature vectors of the candidate regions are input into the trained SVM classifier for classification prediction to obtain the category to which each candidate region belongs.

[0082] Use a regressor to adjust the bounding box of the candidate region to more accurately locate the target.

[0083] The feature vectors of the candidate regions and the offset values ​​between the corresponding bounding boxes and the ground truth bounding boxes are used as the inputs and labels of the regression model.

[0084] A linear regression model is trained based on the training data.

[0085] The feature vector of the candidate region is input into the trained regression model to obtain the predicted bounding box offset value, which is then added to the original bounding box position to obtain the corrected bounding box position.

[0086] Different actions are taken based on the classification results. If the crack is obvious, the area is identified by labeling for investigation, and maintenance personnel are notified to make appropriate decisions. If the crack is not obvious, the output image is used as input to the tracking and recognition module.

[0087] S3. For image data with cracks, calculate the crack size and divide it into image data with obvious cracks and image data with inconspicuous cracks based on the crack size. The specific steps are as follows:

[0088] S31. Binarize the processed image, that is, convert the image into an image containing only black and white colors.

[0089] S32. In a binarized image, crack regions are typically represented by white pixels. By traversing every pixel in the image and counting the number of white pixels, the number of pixels in the crack region can be obtained.

[0090] S33. By determining the actual size of each pixel through the resolution and physical size of the image acquisition device, the area of ​​the crack region can be obtained by directly multiplying the number of pixels in the crack region by the actual area of ​​each pixel.

[0091] S34. The calculation formula is: S=n*Sn, where S is the area of ​​the crack region, n is the number of pixels in the crack region, and Sn is the actual area of ​​the pixel.

[0092] S35. The judgment method is: S' is the total area of ​​the image. Set a threshold ρ'. When ρ≥ρ', the crack is considered obvious; otherwise, it is not obvious.

[0093] S4. For image data with obvious cracks, issue a risk warning; for image data with indistinct cracks, input the data into the tracking and recognition model, such as... Figure 3 As shown, the crack changes are judged; if the crack change is greater than the change threshold, a risk warning is issued; if the crack change is less than the change threshold, it is judged whether there is a risk of crack convergence, and if there is a risk of convergence, a risk warning is issued.

[0094] In step S4, for cracks that are not obvious, images of the same crack at different time periods within the cycle are input into the tracking and recognition model. Images of small cracks in the classification are further processed, and the spatial location of the cracks is aligned across multiple time periods using STN, with a focus on tracking changes in the collected inconspicuous crack damage. The specific steps are as follows:

[0095] S41. Input the crack image that needs further processing;

[0096] Convert the data into time series data;

[0097] Data normalization restricts data to the range (0,1);

[0098] The data is divided into training, validation, and test sets according to a set ratio.

[0099] S42. Receive the original image or data as input, extract the feature information of the input data through the convolutional neural network (CNN) structure, map the extracted feature information to the output space of the transformation parameters, and output the predicted transformation parameters.

[0100] In this embodiment, eight convolutional layers are used, each with a 3×3 convolutional kernel, three max pooling layers, and two fully connected layers. The output layer uses the SoftMax function to predict eight real values ​​as parameters of the transformation matrix.

[0101] The H matrix has eight degrees of freedom:

[0102]

[0103] These correspond to rotation, translation, and tilt offset, respectively, where the rotation term is derived from the submatrix. This indicates that the translation offset is determined by the vector. This indicates that the tilt offset is determined by the vector [a] 31 a 32 ]express;

[0104] The Spatial Transformation Layer (STL) mainly consists of two parts: a mesh generator and a sampler.

[0105] Based on the transformation matrix output by the local network, the coordinates of the output feature map are mapped to the coordinate space of the input feature map through matrix multiplication, thus obtaining the coordinates of each point on the output feature map on the input feature map.

[0106] For a point (x', y') on the output feature map, we want to find the point (x, y) on the input feature map through the transformation matrix H.

[0107]

[0108] in, It is the rotation factor. It is a translation factor, [a 31 a 32 ] is the tilt factor.

[0109] Based on the coordinates obtained from parametric grid sampling, bilinear interpolation is used to calculate the pixel values ​​at non-integer coordinates. The corresponding pixel values ​​are then extracted from the input feature map to form the output feature map.

[0110] Determine the interpolation point (x, y);

[0111] Find four points adjacent to the interpolation point, denoted as (x0,y0), (x1,y0), (x0,y1), (x1,y1);

[0112] Calculate the horizontal interpolation factor and vertical interpolation factor

[0113] f(x,y)=(1-α)(1-β)·f(x0,y0)+α(1-β)·f(x1,y0)+(1-α)β·f(x0,y1)+αβ·f(x1,y1);

[0114] Here, f(x0,y0), f(x0,y0), f(x0,y0), and f(x0,y0) are the values ​​of the four adjacent points, respectively.

[0115] S43. In the output image, cracks with small changes can be observed first, which can save some manpower and resources, while cracks with large changes can be located and processed directly to reduce safety hazards.

[0116] The changes in cracks are distinguished by different colors. After edge detection processing, the cracks are fitted into curves, and relevant parameters are calculated.

[0117] The magnitude of the crack length change, denoted as l for the original crack length and l' for the crack length after time, is then...

[0118] Let the initial crack height be h, and the crack height after time be h'. Then, the range of height change is...

[0119] The total amplitude of the crack under long-term variation is δ = θ·m l +(1-θ)n l θ is determined based on the properties of the crack itself;

[0120] A threshold δ' is set. When the overall amplitude exceeds the threshold, it indicates that the crack is likely to degenerate into a significant crack and needs to be dealt with as soon as possible.

[0121] S44. After edge detection processing, the minute cracks are fitted into a curve, and the relevant parameters are calculated.

[0122] The adjacent tiny cracks are located based on the markings;

[0123] Analyze n (n>2) cracks;

[0124] Let the area of ​​the region be S1 and the area of ​​the crack be S2, then the distribution degree

[0125] Let the straight-line distance between any two cracks be l, and the length and width of the region be l. m and l n ;

[0126]

[0127] Determine the extent of crack distribution based on the table above.

[0128] Let θ be the angle between different cracks. l .

[0129] <![CDATA[θ l ]]> Description of parallelism <![CDATA[θ l <5°]]> Larger <![CDATA[5°<θ s ]]> General or small

[0130] The smaller the angle between the cracks, the closer they are to being parallel, the slower they intersect, and the lower the risk of convergence.

[0131] Make a judgment based on the following decision tree, such as Figure 4 As shown.

[0132] Example 2:

[0133] A deep learning-based device for identifying cracks in ship hull plates, such as... Figure 5 As shown, it includes:

[0134] Data Acquisition Module: This module connects to the underwater camera or robot's camera module, receiving and processing real-time images. It primarily connects to the wireless image transmission module via a WiFi module, responsible for transmitting the acquired image data from the camera to the image processing platform or storage device. The Wi-Fi module for camera image transmission needs to meet requirements such as long distance, high definition, low latency, and strong anti-interference capabilities. For example, the Feirui Intelligent CV5200 module uses advanced LR-Wi-Fi technology, which can significantly improve communication distance while ensuring data transmission rate. It features high-definition image quality, stability, reliability, and flexibility, ensuring data stability and reliability even in complex environments.

[0135] Crack Identification Module: This module integrates advanced digital signal processors, such as the TMS320C series chip, possessing powerful mathematical computing capabilities and a built-in floating-point unit (FPU). It can efficiently perform correlation calculations and inference tasks based on the Long Short-Term Memory (LSTM) model. By performing real-time analysis and deep learning processing on the acquired image data, this module responds promptly to the transmitted real-time data and makes judgments based on a preset threshold θ. When the error exceeds the set threshold, it sends maintenance commands to the control system via the communication interface, achieving rapid and accurate crack location and management.

[0136] Interface Communication Module: This module uses the ESP8266 communication chip, supports the 2.4GHz Wi-Fi 802.11b / g / n standard, and has powerful Wi-Fi connectivity, providing high-speed data transmission rates and energy-saving design, up to 150Mbps. Through its efficient transmission speed, processing results can be transmitted to the host computer or other remote devices in real time, enabling users to manage efficiently and quickly.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying cracks in the hull plating of a ship based on deep learning, characterized in that, The specific method is as follows: Obtain actual image data of the hull hull; Cracked image data was labeled using a CNN-based recognition and localization model. For image data with cracks, calculate the crack size and divide it into image data with obvious cracks and image data with inconspicuous cracks based on the crack size. For image data with obvious cracks, risk warnings should be issued; For image data with indistinct cracks, the data is input into a tracking and recognition model to determine the changes in the cracks. If the change in cracks exceeds the change threshold, a risk warning will be issued. If the change in cracks is less than the change threshold, determine whether there is a risk of crack convergence. If there is a risk of convergence, issue a risk warning. For image data with indistinct cracks, the specific method for determining crack changes is as follows: After edge detection processing, the crack is fitted into a curve; The formula for calculating the change in crack length is as follows: In the formula, the original crack length is l, and the crack length after time change is l ' ; The formula for calculating the change in crack height is as follows: In the formula, the initial crack height is h, and the crack height after time change is h0. ' ; The overall amplitude of the cracks over a long period of time is: δ=θ·m l +(1-θ)n l In the formula, θ is the variation range parameter, which is determined based on the properties of the crack itself and the surrounding environment.

2. The deep learning-based method for identifying cracks in ship hull plates as described in claim 1, characterized in that, The specific method for calculating the crack size of cracked image data is as follows: Convert image data with obvious cracks into black and white images; Iterate through each pixel in the image and count the number of white pixels; The crack size is calculated using the formula S = n * Sn, where S is the crack area, n is the number of white pixels in the crack region, and Sn is the actual area of ​​the pixel.

3. The deep learning-based method for identifying cracks in ship hull plates as described in claim 2, characterized in that, Based on crack size, image data is divided into images of obvious cracks and images of inconspicuous cracks, as follows: Set size threshold ρ ' ; According to the formula Calculate the crack classification index; where S ' The total area of ​​the image; When ω≥ω ' If the crack is not obvious, the image data is considered as having obvious cracks; otherwise, it is considered as having indistinct cracks.

4. The deep learning-based method for identifying cracks in ship hull plates as described in claim 1, characterized in that, The specific methods for determining whether there is a risk of crack convergence are as follows: After edge detection processing, the tiny cracks are fitted into a curve, and the relevant parameters are calculated. Locate adjacent, inconspicuous cracks based on the markings; Based on whether inconspicuous cracks intersect, the degree of crack distribution within the region, and the degree of crack parallelism, a decision tree is used to classify whether there is a risk of crack convergence.

5. The deep learning-based method for identifying cracks in ship hull plates as described in claim 4, characterized in that, The distribution and parallelism of cracks within the area are calculated using the following methods: Calculate the degree of crack distribution: In the formula, S1 is the area of ​​the region, and S2 is the area of ​​the crack; Let the straight-line distance between any two cracks be l, and the length and width of the region be l. m and l n ; When θ s <5%, When the condition is met, it is judged to be sparse; When 5% < θ s <20% At that time, it was judged as general; When θ s >20%, At that time, it was judged to be dense; Let θ be the angle between different cracks. l ; When θ l <5° indicates a high degree of parallelism; When 5° < θ s The degree of parallelism is generally judged.

Images