Deep learning-based hull planking crack identification method

Abstract

The invention relates to the technical field of hull detection, in particular to a hull planking crack identification method based on deep learning. The method comprises the following steps: acquiring actual image data of a hull plate; through a CNN-based recognition positioning model, crack image data is marked; dividing into obvious crack image data and non-obvious crack image data according to the crack size; carrying out risk early warning on the obvious crack image data; unobvious crack image data are input into the tracking identification model, and crack changes of the unobvious crack image data are judged; if the crack change is greater than a change threshold value, risk early warning is carried out; and if the crack change is smaller than the change threshold, judging whether a crack convergence risk exists, and if the convergence risk exists, carrying out risk early warning. An automatic crack identification method is provided for the hull plate which cannot be directly observed by human eyes, different processing methods are provided for different hull plate cracks, and a large amount of manpower and material resources can be saved.

Description

Technical Field

[0001] The present invention relates to the technical field of hull detection, and in particular to a method for identifying cracks in hull outer plates based on deep learning. Background Art

[0002] The tiny cracks in the hull plating may seem to have little impact on the hull in the early stage, but in fact this impact is potential, long-term, and may gradually intensify. However, when the ship is working, if any cracks will drive the ship to the maintenance point, it will greatly increase the ship's travel distance and affect the ship's travel route, thereby greatly increasing the ship's travel cost. The main work of this method is two points. One is to identify the cracks in the hull plating and determine the obvious cracks; the other is to further process the identified cracks, track and identify the inconspicuous cracks over a period of time, and judge their change range so as to have corresponding processing methods. Summary of the invention

[0003] The present invention discloses a method for identifying cracks in hull outer plates based on deep learning, and the specific method is as follows:

[0004] Acquire actual image data of the hull outer plate;

[0005] Through the CNN-based recognition and positioning model, the crack image data is marked;

[0006] For image data with cracks, the crack size is calculated and the image data is divided into image data with obvious cracks and image data with inconspicuous cracks according to the crack size;

[0007] For obvious crack image data, risk warning is issued;

[0008] For the image data of inconspicuous cracks, input them into the tracking and recognition model to determine the crack changes;

[0009] If the crack change is greater than the change threshold, a risk warning is issued;

[0010] If the crack change is less than the change threshold, determine whether there is a risk of crack convergence. If there is a risk of convergence, issue a risk warning.

[0011] Furthermore, for the crack image data, the crack size is calculated as follows:

[0012] Convert image data with obvious cracks into black and white images;

[0013] Traverse each pixel in the image and count the number of white pixels;

[0014] The crack size is calculated using the formula S=n*Sn; where S is the crack size area, n is the number of white pixels in the crack area, and Sn is the actual pixel area.

[0015] Furthermore, the image data is divided into obvious crack image data and inconspicuous crack image data according to the crack size, and the specific method is as follows:

[0016] Set the size threshold ρ';

[0017] According to the formula Calculate the crack classification index; where S' is the total area of ​​the image;

[0018] When ρ≥ρ', it is determined as obvious crack image data, otherwise it is determined as inconspicuous crack image data.

[0019] Furthermore, for the image data of inconspicuous cracks, the crack changes are judged, and the specific method is as follows:

[0020] The cracks are fitted into a curve after edge detection;

[0021] Calculate the length variation of the crack using the following formula:

[0022]

[0023] In the formula, the original crack length is l, and the crack length after time change is l';

[0024] Calculate the height variation of the crack using the following formula:

[0025]

[0026] In the formula, the initial crack height is h, and the crack height after time change is h';

[0027] The overall amplitude of the crack under long-term variation is:

[0028] δ=θ·m l +(1-θ)n l

[0029] Where θ is the variation amplitude parameter, which is determined according to the properties of the crack itself and the surrounding environment.

[0030] Furthermore, the variation parameter θ is determined by fuzzy recommendation, and the specific formula is as follows:

[0031] θ=a1 / x1+a2 / x2+a3 / x3+a4 / x4+…+an / xn

[0032] In the formula, x1 is the factor of the crack and the direction of the water flow before the tracking period; x2 is the factor of the water flow and the direction of the ship; x3 is the temperature and humidity factor in the sea area; x4 is the factor of the crack's own structure; xn is the factor added artificially in different situations;

[0033] In the formula, the denominator is the different elements of the influencing factors, and the numerator is the membership degree corresponding to the element, which is determined by the expert system.

[0034] Furthermore, to determine whether there is a risk of crack convergence, the specific method is as follows:

[0035] After edge detection, tiny cracks are fitted into curves and related parameters are calculated;

[0036] Locate adjacent inconspicuous cracks based on the markings;

[0037] Based on whether the inconspicuous cracks intersect, the degree of crack distribution in the area, and the degree of crack parallelism, a decision tree is used to classify whether there is a risk of crack convergence.

[0038] Furthermore, the distribution degree of cracks in the region and the degree of parallelism of cracks are calculated as follows:

[0039] Calculate the crack distribution:

[0040]

[0041] Where S1 is the area of ​​the region, S2 is the crack area;

[0042] Let the straight-line distance between any two cracks be l, and the length and width of the area be l m and l n ;

[0043] When θ s <5%, When , it is judged as sparse;

[0044] When 5% < θ s <20%, When , it is judged as general;

[0045] When θ s >20%, When , it is judged as dense;

[0046] The angle between different cracks is θ l ;

[0047] When θ l <5°, the degree of parallelism is judged to be large;

[0048] When 5°<θ s , judging the degree of parallelism is average.

[0049] Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0050] 1. There is an automatic crack recognition method for the hull plate that cannot be directly observed by the human eye. There are different treatment methods for different hull plate cracks, which can save a lot of manpower and material resources.

[0051] 2. The present invention does not require a large amount of manpower and material resources. Based on the hardware conditions of the ship itself, after the corresponding software and hardware are installed, only a relatively small number of personnel are needed to operate it, and no special training is required.

[0052] 3. The present invention has strong editability. Different parameters can be set to suit operators in different environments and under different conditions.

[0053] 4. The present invention is easy to maintain and update. It does not need to consume a lot of time, and maintenance and updating will not affect the use of the device.

[0054] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings of the present invention are as follows.

[0056] Figure 1 This is a schematic diagram of the overall process of Example 1.

[0057] Figure 2 This is a flow chart of the identification and positioning model of Example 1.

[0058] Figure 3 This is a schematic diagram of the tracking and recognition model flow of Example 1.

[0059] Figure 4 Schematic diagram of the decision tree of Example 1.

[0060] Figure 5 This is a schematic diagram of the device structure of Example 2. DETAILED DESCRIPTION

[0061] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0062] Embodiment 1:

[0063] A method for identifying cracks in hull plates based on deep learning, such as Figure 1 As shown, the specific steps are as follows:

[0064] S1. Acquire actual image data of the hull outer plate.

[0065] In step S1, the actual image data is stored in the database and divided by time and location. The database regularly cleans up the data, with a cycle of 15 days, starting from the first data collection, and the labels are from 1 to 15. After the collection of each day is completed, it is regarded as the collected data of one day.

[0066] The location is first divided into different areas, and then the underwater camera or robot determines the approximate area when the image data is transmitted, and is divided into Ⅰ, Ⅱ, Ⅲ, etc. It is expressed as:

[0067]

[0068] The specific label is determined according to the specific area n and the specific time m.

[0069] S2. The crack image data is marked through the CNN-based recognition and positioning model.

[0070] The labeled real-time data is used as input into the recognition and positioning model to obtain output images with different classification results. Figure 2 To identify the process of positioning the model, the specific steps are as follows:

[0071] S21. Collect image datasets of hull shell cracks from the open source datasets on the Kaggle website.

[0072] S22. Use the Selective Search algorithm to generate candidate regions, generating a series of candidate regions that may contain the target.

[0073] S23. Use the Felzenszwalb method to divide the image into many small areas.

[0074] S24, using segmentation to obtain a candidate region set R = {r1, r2, ..., r n}.

[0075] S25. Initialize the collection Used to store the similarity results of adjacent areas.

[0076] S26, traverse the adjacent region pairs in R (r i ,r j ), calculate the similarity S(r i ,r j ), S = S ∪ S (r i ,r j )

[0077] S27. If S is not an empty set, S(r i ,r j )=max(S),r t =r i ∪rj . Remove region r from S i and r j All related similarity results and calculate the new region r t The similarity set S with the surrounding area t , S t Put it into the set S and put the new area r t Put it into the set R.

[0078] S28. After the traversal is completed, the bounding boxes of all regions in R are extracted, that is, the regions where objects may exist.

[0079] S29. Use the convolutional and pooling layers of CNN to extract features from the image. These layers can automatically learn local features in the image and combine these features into higher-level feature representations. Based on the dataset with true bounding box labeling information, the candidate region is generated using the selective search method. The candidate region is scaled to a size of 227x227. The preprocessed candidate region is input into the CNN model for forward propagation to extract a fixed-length feature vector.

[0080] The feature vector of the candidate region is classified using a classifier to determine whether it contains an object and the category of the object. Based on the extracted feature vector and the true category label, an SVM classifier is trained.

[0081] The feature vector of the candidate region is input into the trained SVM classifier for classification prediction to obtain the category to which each candidate region belongs.

[0082] Use a regressor to adjust the bounding box of the candidate region to more accurately locate the object.

[0083] The feature vector of the candidate region and the offset value between the corresponding bounding box and the true bounding box are used as the input and label of the regression model.

[0084] Based on the training data, train a linear regression model.

[0085] The feature vector of the candidate region is input into the trained regression model to obtain the predicted bounding box offset value, which is added to the original bounding box position to obtain the corrected bounding box position.

[0086] Different processing is performed according to the classification results. If the crack is obvious, the area is determined by labeling for investigation, and the maintenance personnel are notified to make corresponding decisions. If the crack is not obvious, the output image is used as input to the tracking and recognition module.

[0087] S3. For the image data with cracks, calculate the crack size and divide it into image data with obvious cracks and image data with inconspicuous cracks according to the crack size. The specific steps are as follows:

[0088] S31, performing binarization processing on the processed image, that is, converting the image into an image containing only black and white colors.

[0089] S32. In the binary image, the crack region is usually represented by white pixels. The number of pixels in the crack region can be obtained by traversing each pixel in the image and counting the number of white pixels.

[0090] S33. The actual size of each pixel is determined by the resolution and physical size of the image acquisition device. Then, the number of pixels in the crack area can be directly multiplied by the actual area of ​​each pixel to obtain the area of ​​the crack area.

[0091] S34. The calculation formula is: S=n*Sn, where S is the area of ​​the crack region, n is the number of pixels in the crack region, and Sn is the actual area of ​​the pixel.

[0092] S35, the judgment method is: S' is the total area of ​​the image, A threshold ρ' is set. When ρ ≥ ρ', the crack is judged to be obvious, otherwise it is not obvious.

[0093] S4. For obvious crack image data, risk warning is performed; for inconspicuous crack image data, it is input into the tracking and recognition model, such as Figure 3 As shown, the crack change is judged; if the crack change is greater than the change threshold, a risk warning is issued; if the crack change is less than the change threshold, it is judged whether there is a risk of crack convergence. If there is a risk of convergence, a risk warning is issued.

[0094] In step S4, if the crack is not obvious, the images of the same crack at different time periods within the cycle are input into the tracking and recognition model. The images of small cracks in the classification are further processed, and the spatial position of the cracks is aligned in the multi-time images through STN, focusing on tracking the changes of the collected inconspicuous crack damage. The specific steps are as follows:

[0095] S41, inputting a crack image that needs to be further processed;

[0096] Convert data into time series data;

[0097] Data normalization, limiting the data to the range of (0,1);

[0098] The data is divided into training set, validation set and test set in proportion.

[0099] S42, receiving the original image or data as input, extracting feature information of the input data through a convolutional neural network (CNN) structure, mapping the extracted feature information to an output space of transformation parameters, and outputting predicted transformation parameters.

[0100] In this embodiment, eight convolutional layers are used, each convolutional layer uses a 3×3 convolution kernel, three maximum pooling layers and two full connections, and the output layer predicts eight real values ​​as parameters of the transformation matrix through the SoftMax function.

[0101] The H matrix has eight degrees of freedom:

[0102]

[0103] Corresponding to rotation, translation and tilt offset respectively, where the rotation term is represented by the submatrix Represented by the vector The tilt offset is represented by the vector [a 31 a 32 ]express;

[0104] The spatial transformation layer STL mainly consists of two parts: grid generator and sampler.

[0105] According to the transformation matrix output by the local network, the coordinates of the output feature map are mapped to the coordinate space of the input feature map through matrix multiplication to obtain the coordinates of each point on the output feature map on the input feature map.

[0106] For a point (x', y') on the output feature map, we want to find (x, y) on the input feature map through the transformation matrix H.

[0107]

[0108] in, is the rotation factor, is the translation factor, [a 31 a 32 ] is the tilt factor.

[0109] According to the coordinates obtained by parameterized grid sampling, bilinear interpolation method is used to calculate the pixel values ​​at non-integer coordinates. The corresponding pixel values ​​are extracted from the input feature map to form the output feature map.

[0110] Determine the interpolation point (x,y);

[0111] Find four points adjacent to the interpolation point, recorded as (x0,y0), (x1,y0), (x0,y1), (x1,y1);

[0112] Calculate horizontal interpolation factors and the vertical interpolation factor

[0113] f(x,y)=(1-α)(1-β)·f(x0,y0)+α(1-β)·f(x1,y0)+(1-α)β·f(x0,y1)+αβ·f(x1,y1);

[0114] Among them, f(x0,y0), f(x0,y0), f(x0,y0), and f(x0,y0) are the values ​​of four adjacent points respectively.

[0115] S43. In the output image, cracks with small changes can be observed first, which can save some manpower and material resources, while cracks with large changes can be directly located and processed to reduce safety hazards.

[0116] The changes in cracks are distinguished by different colors, and the cracks are fitted into curves after edge detection processing, and the relevant parameters are calculated.

[0117] The length change amplitude of the crack, let the original crack length be l, and the crack length after time change be l', then the length change amplitude

[0118] The height variation of the crack is: the original crack height is h, and the height of the crack after time change is h'. Then the height variation is

[0119] The overall amplitude of the crack under long-term changes is δ = θ·m l +(1-θ)n l ,θ is determined according to the properties of the crack itself;

[0120] A threshold δ' is set. When the overall amplitude exceeds the threshold, it means that the probability of the crack transforming into an obvious crack is high and needs to be processed as soon as possible.

[0121] S44, fitting the tiny cracks into a curve after edge detection processing, and calculating relevant parameters;

[0122] Locate adjacent tiny cracks based on the markings;

[0123] Analyze n (n>2) cracks;

[0124] The area of ​​the region is S1, the area of ​​the crack is S2, then the distribution degree

[0125] Let the straight-line distance between any two cracks be l, and the length and width of the area be l m and l n ;

[0126]

[0127] Determine the degree of crack distribution according to the above table.

[0128] The angle between different cracks is θ l .

[0129] <![CDATA[θ l ]]> Description of the degree of parallelism <![CDATA[θ l <5°]]> Larger <![CDATA[5°<θ s ]]> Normal or small

[0130] When the angle between the cracks is smaller, we can assume that the two are closer to parallel, the speed at which they intersect will be slower, and the risk of convergence will be smaller.

[0131] Make a decision based on the following decision tree: Figure 4 shown.

[0132] Embodiment 2:

[0133] A hull shell crack recognition device based on deep learning, such as Figure 5 As shown, including:

[0134] Data acquisition module: This module is connected to the camera module of the underwater camera or robot to receive and process the real-time images taken in real time. It is mainly connected to the wireless image transmission module through the WiFi module, and is responsible for transmitting the collected image data from the camera to the image processing platform or storage device. The Wi-Fi module for camera image transmission needs to meet the requirements of long distance, high definition, low latency and strong anti-interference ability. For example, the Feirui Intelligent CV5200 module uses advanced LR-Wi-Fi technology, which can significantly improve the communication distance while ensuring the data transmission rate. It has the characteristics of high-definition image quality, stability and reliability, flexibility and convenience, and can ensure the stability and reliability of data in complex environments.

[0135] Crack identification module: This module integrates advanced digital signal processors, such as TMS320C series chips, with powerful mathematical computing capabilities and built-in floating-point unit FPU, which can efficiently perform correlation calculations and reasoning tasks based on the long short-term memory network LSTM model. Through real-time analysis and deep learning processing of the collected image data, this module responds to the transmitted real-time data in a timely manner and makes judgments based on the preset threshold θ. When the error exceeds the set threshold, the communication interface is used to send maintenance instructions to the control system, realizing rapid and accurate positioning and management of cracks.

[0136] Interface communication module: This module uses the ESP8266 communication chip, supports 2.4GHz Wi-Fi 802.11b / g / n standards, has strong Wi-Fi connection capabilities, provides high-speed data transmission rate and energy-saving design, up to 150Mbps. Through its efficient transmission speed, the processing results can be transmitted to the host computer or other remote devices in real time, allowing users to manage efficiently and quickly.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying cracks in hull plates based on deep learning, characterized in that: The specific method is as follows: Acquire actual image data of the hull outer plate; Through the CNN-based recognition and positioning model, the crack image data is marked; For image data with cracks, the crack size is calculated and the image data is divided into image data with obvious cracks and image data with inconspicuous cracks according to the crack size; For obvious crack image data, risk warning is issued; For the image data of inconspicuous cracks, input them into the tracking and recognition model to determine the crack changes; If the crack change is greater than the change threshold, a risk warning is issued; If the crack change is less than the change threshold, determine whether there is a risk of crack convergence. If there is a risk of convergence, issue a risk warning.

2. The method for identifying cracks in hull outer plate based on deep learning according to claim 1, characterized in that: For crack image data, the specific method for calculating the crack size is as follows: Convert image data with obvious cracks into black and white images; Traverse each pixel in the image and count the number of white pixels; The crack size is calculated using the formula S=n*Sn; where S is the crack size area, n is the number of white pixels in the crack area, and Sn is the actual pixel area.

3. The method for identifying cracks in hull outer plates based on deep learning according to claim 2, characterized in that: According to the crack size, it is divided into obvious crack image data and inconspicuous crack image data. The specific method is as follows: Set the size threshold ρ'; According to the formula Calculate the crack classification index; where S' is the total area of ​​the image; When ρ≥ρ', it is determined as obvious crack image data, otherwise it is determined as inconspicuous crack image data.

4. The method for identifying cracks in hull outer plate based on deep learning according to claim 1, characterized in that: For the image data of inconspicuous cracks, the specific method for judging the crack changes is as follows: The cracks are fitted into a curve after edge detection; Calculate the length variation of the crack using the following formula: In the formula, the original crack length is l, and the crack length after time change is l'; Calculate the height variation of the crack using the following formula: In the formula, the initial crack height is h, and the crack height after time change is h'; The overall amplitude of the crack under long-term variation is: δ=θ·m l +(1-θ)n l Where θ is the variation amplitude parameter, which is determined according to the properties of the crack itself and the surrounding environment.

5. The method for identifying cracks in hull outer plate based on deep learning according to claim 4, characterized in that: The variation parameter θ is determined by fuzzy recommendation, and the specific formula is as follows: θ=a1 / x1+a2 / x2+a3 / x3+a4 / x4+…+an / xn In the formula, x1 is the factor of the crack and the direction of the water flow before the tracking period; x2 is the factor of the water flow and the direction of the ship; x3 is the temperature and humidity factor in the sea area; x4 is the factor of the crack's own structure; xn is the factor added artificially in different situations; In the formula, the denominator is the different elements of the influencing factors, and the numerator is the membership degree corresponding to the element, which is determined by the expert system.

6. The method for identifying cracks in hull outer plates based on deep learning according to claim 1, characterized in that: The specific method for judging whether there is a risk of crack convergence is as follows: After edge detection, tiny cracks are fitted into curves and related parameters are calculated; Locate adjacent inconspicuous cracks based on the markings; Based on whether the inconspicuous cracks intersect, the degree of crack distribution in the area, and the degree of crack parallelism, a decision tree is used to classify whether there is a risk of crack convergence.

7. The method for identifying cracks in hull outer plate based on deep learning according to claim 7, characterized in that: The specific calculation method of the degree of crack distribution and crack parallelism in the region is as follows: Calculate the crack distribution: Where S1 is the area of ​​the region, S2 is the crack area; Let the straight-line distance between any two cracks be l, and the length and width of the area be l m and l n ; When θ s <5%, When , it is judged as sparse; When 5% < θ s <20%, When , it is judged as general; When θ s >20%, When , it is judged as dense; The angle between different cracks is θ l ; When θ l <5°, the degree of parallelism is judged to be large; When 5°<θ s , judging the degree of parallelism is average.

Images