Crack identification method and device based on partition convolution
By splitting the large-size images and processing them independently in the convolutional neural network, the results of each split tensor are spliced, and the problem of low recognition efficiency of short cracks in large-size images is solved, achieving efficient and accurate crack identification, suitable for health monitoring of building structures.
Patent Information
- Application Number
- CN202411073744.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-12
AI Technical Summary
When identifying short cracks in large-size images, the existing convolutional neural network model has problems such as high computational cost, high processing complexity, long recognition time and low recognition accuracy. Especially in structures such as aqueducts, the recognition effect of short cracks is achieved. Not ideal.
Using partition convolution-based crack recognition method, tensor splitting of the original image, input split tensors to pre-trained convolutional neural network model for calculation, and splicing the output grayscale matrix to obtain cracks Predict the results, reduce the number of image format conversions, expand the segmentation area, and ensure that the crack features are not divided into too small parts.
It improves the efficiency and accuracy of crack recognition, overcomes the problem of slow processing of large images, ensures the reliability and accuracy of crack recognition, and provides an effective tool for health monitoring of building structures.
Smart Images

Figure CN118941923A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition, and in particular to a crack recognition method and device based on partitioned convolution. Background Art
[0002] With the development of deep learning technology, crack image recognition based on convolutional neural network technology has brought revolutionary progress to the health monitoring of building structures. However, there are still challenges in identifying short cracks in large-size images. The reason is that the processing of large images increases the computing cost, processing complexity and image recognition time, and the recognition of short cracks is limited by the resolution and receptive field of the network. Taking a certain aqueduct as an example, crack recognition is performed based on high-resolution aqueduct images taken by drones. The pixel size of a single image is large, and the crack prediction time exceeds 60 seconds. At the same time, since the width of aqueduct cracks is mostly a few millimeters and the length is mostly tens of centimeters, the crack area accounts for a very small proportion of pixels in a single image. Various convolutional neural network models are used for crack prediction, such as VGGNet (full name Visual Geometry Group Network), ResNet (full name Residual Network), U-Net and YOLO (full name You Only Look Once), but the prediction results are not ideal.
[0003] In order to solve the above problems, two main methods are currently used to identify short cracks in large-size images. The first method is to improve the generalization ability of the convolutional neural network model. When training the model, large-size image training samples are introduced to enhance the model's ability to process images of different sizes. By increasing the depth of the convolution layer or increasing the size of the convolution kernel, a wider range of crack feature information can be captured. Multi-scale image processing technology is applied to run the neural network to consider multiple different sizes of images to improve adaptability to complex scenes. However, the disadvantage of this method is that a large number of large-size images need to be annotated when training the model. The time to produce the training data set increases exponentially, and the model needs to be retrained, which further increases the time cost. In addition, the structure of the model needs to be changed, which easily leads to problems such as excessive calculation and overfitting. The main steps of the second method are: converting the large-size image into a tensor for segmentation, converting the segmented tensor into a small-size image, and inputting each small-size image into a convolutional neural network for recognition. During recognition, the small-size image needs to be read as a small tensor before crack recognition is performed. Finally, the small-size crack prediction binary image obtained by small-size image recognition is spliced. The defect of this method is that there are multiple conversions between images and tensors, the detection time is long, the efficiency is low, and when segmenting, the image cannot be non-integer segmented, which limits the optional segmentation size. At the same time, the cracks will be divided into multiple parts, resulting in the model being unable to recognize complete cracks, which may lead to false detection or missed detection, and the crack recognition accuracy is low. Summary of the invention
[0004] The present application provides a crack recognition method and device based on partitioned convolution, which directly splits the image tensor and splices the grayscale matrix, reducing the number of image format conversions and improving the efficiency of crack recognition. When performing tensor splitting, it is convenient to expand the segmentation area to ensure that features such as cracks will not be divided into too small parts, and the crack recognition accuracy is high.
[0005] On the one hand, an embodiment of the present application provides a crack identification method based on partitioned convolution, comprising:
[0006] Input the original image containing cracks into the image segmentation module, split the tensor according to the tensor splitting size, and output Q split tensors of the same size, Q>1;
[0007] Inputting each of the split tensors into a pre-trained convolutional neural network model, performing independent calculations on each of the split tensors, and outputting a grayscale matrix corresponding to each of the split tensors, wherein the convolutional neural network model is used to predict cracks in the original image;
[0008] All output grayscale matrices are spliced to obtain a crack prediction grayscale matrix corresponding to the original image;
[0009] The original image and the crack prediction grayscale matrix are fused and visualized to obtain a crack distribution map.
[0010] On the other hand, the embodiment of the present application further provides a crack identification device based on partitioned convolution, including a processor and a memory:
[0011] The memory is used to store a crack identification program based on partition convolution;
[0012] The processor is used to read the partitioned convolution-based crack identification program and perform the partitioned convolution-based crack identification method as described in the above embodiment.
[0013] Compared with the related art, the crack recognition method and device based on partitioned convolution in the embodiment of the present application directly perform intelligent splitting of the image tensor and splicing of the grayscale matrix, thereby reducing the number of image format conversions and improving the efficiency of crack recognition. Partitioned convolution of the split tensor further overcomes the problem of slow processing speed of large images and further improves the efficiency of crack recognition. When performing tensor splitting, it can take into account the receptive field of the convolutional neural network model to ensure that each split area is fully feature extracted and analyzed, and expand the segmented area to ensure that features such as cracks will not be segmented into too small parts. The crack recognition has high accuracy and reliability, overcomes the difficulty of identifying short cracks based on large-size images, and provides an effective tool for construction project monitoring.
[0014] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.
[0016] Figure 1 is a flow chart of a crack identification method based on partition convolution according to an embodiment of the present application;
[0017] Figure 2 is a schematic diagram of the tensor splitting result of a specific example of this application;
[0018] Figure 3 It is a schematic diagram of a crack identification device based on partitioned convolution according to an embodiment of the present application. DETAILED DESCRIPTION
[0019] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.
[0020] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.
[0021] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.
[0022] When processing images, convolutional neural networks rely on their internal convolution kernels to extract image features. In order to improve the efficiency and effect of feature extraction at the same time, multiple layers of convolution kernels with smaller receptive fields are generally selected. However, the receptive field of each layer of convolution kernels is limited, and increasing the number of convolution kernel layers cannot guarantee the effect of feature extraction. Moreover, when training convolutional neural network models, in order to improve training efficiency, the training images used are generally small, and the generalization ability for large-size images is weak. Therefore, when processing large-size images, tiny crack features are not easy to be extracted. Using large-size training convolutional neural network models is very time-consuming. There is an urgent need for a method that can use existing convolutional neural network models to identify short cracks in large-size images.
[0023] The present application embodiment provides a crack identification method based on partition convolution, including steps S1000-S4000, such as Figure 1 As shown,
[0024] S1000: Input the original image containing the cracks into the image segmentation module, perform tensor splitting according to the tensor splitting size, and output Q split tensors of the same size, Q>1;
[0025] S2000: inputting each of the split tensors into a pre-trained convolutional neural network model, performing independent calculations on each of the split tensors, and outputting a grayscale matrix corresponding to each of the split tensors, wherein the convolutional neural network model is used to predict cracks in the original image;
[0026] S3000: splicing all output grayscale matrices to obtain a crack prediction grayscale matrix corresponding to the original image;
[0027] S4000: fusing and visualizing the original image and the crack prediction grayscale matrix to obtain a crack distribution map.
[0028] In this embodiment, the image segmentation module can be implemented using the Python-based open source machine learning library PyTorch. When executing step S1000, the image segmentation module needs to convert the original image into a large-size tensor before performing tensor splitting. For example, the original image can be an RGB image, and the image segmentation module can convert the original image into a third-order tensor (L 0 ,W 0 ,3), the third-order tensor (L 0 ,W 0 ,3) is a large-size tensor, where L 0 Represents the pixel length of the original image, W 0 represents the pixel width of the original image, and 3 represents the number of color channels of the original image, namely red, green, and blue channels. When splitting a tensor, it is necessary to split the tensor according to the tensor splitting size. Adjacent split tensors may or may not overlap, as long as Q split tensors of the same size can cover the large-size tensor and ensure that no tensor information is missed.
[0029] In this embodiment, the original image may be an image of a building structure acquired by an onboard imaging device carried by a drone, an onboard imaging device, manual photography, or a building monitoring camera. The building structure may be any one of a masonry structure, a concrete structure, a steel structure, a foundation, a wall coating, a floor, a ceiling, and the like. The building structure is affected by factors such as temperature changes, humidity changes, material shrinkage, aging, uneven settlement, or prestress, resulting in short cracks on its surface. Although these short cracks are very small, they need to be regularly evaluated and monitored to ensure that they do not expand over time and affect the safety of the building structure.
[0030] In this embodiment, the selection of the tensor splitting size needs to be carefully designed. The tensor splitting size needs to be as close as possible to the training image size of the convolutional neural network model to match the receptive field of the convolutional neural network model. At the same time, it is necessary to ensure that each split tensor is not too small and contains enough information to ensure that the crack features in the original image will not be divided into too small parts and difficult to be recognized by the convolutional neural network model.
[0031] In this embodiment, the convolutional neural network model performs crack prediction and grayscale calculation on each split tensor of the input, generates a grayscale matrix, and implements partitioned convolution for crack identification. The convolutional neural network model can be any one of VGGNet, ResNet, U-Net, DenseNet, GoogLeNet and YOLO. The above convolutional neural network models are all exemplary descriptions and are not intended to be limitations of this application. They will not be described one by one in this article.
[0032] In this embodiment, each element in the grayscale matrix corresponding to each of the split tensors not only represents a pixel point, but also includes a grayscale value. The grayscale value ranges from 0 to 255. According to the grayscale value of each pixel point, it can be determined whether the pixel point is a crack. The grayscale value range of each element in the crack prediction grayscale matrix obtained after splicing in step S3000 is also 0-255.
[0033] In this embodiment, when executing step S4000, steps S4100-S4400 may be included:
[0034] S4100: Set grayscale threshold G T ;
[0035] S4200: The gray value in the crack prediction gray matrix is less than or equal to G T The pixel points with gray values greater than G in the crack prediction gray matrix are set to 0. T The pixel point is set to 1 to obtain a crack prediction binary matrix, where 1 represents that the pixel point is a crack and 0 represents that the pixel point is not a crack;
[0036] S4300: converting the crack prediction binary matrix into a crack prediction binary image;
[0037] S4400: Align the same pixels of the crack prediction binary image and the original image to obtain the crack distribution map.
[0038] In this embodiment, when executing steps S4100-S4300, the open source machine learning library matplotlib based on Python can be used to implement it, and the grayscale threshold G can be set. T The grayscale value of the crack prediction grayscale matrix is 127, and the elements with grayscale values of 0-127 in the crack prediction grayscale matrix are set to 0, indicating that there is no crack, and the elements with grayscale values of 128-255 in the crack prediction grayscale matrix are set to 1, indicating that there is a crack.
[0039] The crack recognition method based on partitioned convolution in this embodiment directly splits the image tensor and splices the grayscale matrix, which reduces the number of image format conversions and improves the efficiency of crack recognition. When splitting the tensor, it can take into account the receptive field of the convolutional neural network model and expand the segmentation area, ensuring that features such as cracks will not be divided into too small parts, and the crack recognition accuracy is high.
[0040] In an exemplary embodiment, the original image contains short cracks, wherein the short cracks satisfy at least one of the following conditions: the crack pixel width is less than one pixel, the crack pixel length is less than ten pixels, the crack pixel width is less than 1 / 100 of the original image pixel width, and the crack pixel length is less than 1 / 100 of the original image pixel length.
[0041] In this embodiment, the original image may be an image of an aqueduct taken by an unmanned aerial vehicle carrying an onboard imaging device. An aqueduct is a water conservancy project mainly used to transport water. An aqueduct spans a river, a valley or other obstacles. The cracks on the aqueduct are usually a few millimeters wide and tens of centimeters long. The presence of these cracks will affect the durability of the aqueduct and needs to be monitored to prevent short cracks from developing into larger cracks that affect the safety of the aqueduct structure.
[0042] In an exemplary embodiment, before step S1000, step S0000 may also be included:
[0043] S0000: Obtain the training picture size of the convolutional neural network model, and perform a tensor splitting test on the original image based on the training picture size to determine the tensor splitting size, wherein the size of the original image is larger than the training picture size.
[0044] In this embodiment, before performing tensor splitting on the original image, the tensor splitting size can be determined through a tensor splitting test. The determined tensor splitting size can ensure that the segmentation range is expanded when performing tensor splitting, ensuring that features such as cracks are not split into too small parts and increase the difficulty of crack identification, thereby ensuring that each split tensor retains enough crack information to avoid the loss of crack information.
[0045] In this embodiment, when performing a tensor splitting test, several original images can be selected as test images. For each test image, several different tensor splitting test sizes are used to implement the tensor splitting test, and the split test small tensors are input into the convolutional neural network model to obtain a test small grayscale matrix. The test small grayscale matrix is then spliced to obtain a test grayscale matrix, and the test grayscale matrix corresponding to each tensor splitting test size is evaluated to select a suitable tensor splitting size.
[0046] In an exemplary embodiment, step S0000 may include steps S0100-S0400, wherein step S0400 may include steps S0410-S0420:
[0047] S0100: Obtain the pixel length l and pixel width w corresponding to the training image size of the convolutional neural network model;
[0048] S0200: Obtain multiple of the original images of the target area, and select test images from the multiple original images according to a predetermined ratio;
[0049] S0300: Perform tensor splitting and grayscale calculation on each of the test images according to the set first tensor splitting size Bl*Bw to obtain the first grayscale matrix M1 i , where i = 1...I, I represents the number of the test images, and B≥3;
[0050] S0400: Loop through the following steps S0410 - S0420 to perform one or more rounds of tests until the current average similarity S is greater than the preset similarity threshold S T , then set the tensor splitting size to the current round's second tensor splitting size Al*Aw, 1≤A<B; or, until A = B and S is still not greater than S T , then set the tensor splitting size to Bl*Bw:
[0051] S0410: Perform tensor splitting and grayscale calculation on each of the test images according to the set current round's second tensor splitting size Al*Aw to obtain the current second grayscale matrix M2 i , calculate M2 i and M1 i 's current similarity S i ; A = 1 during the first round of testing;
[0052] S0420: Calculate the current average similarity Judge the relationship between S and S T , if S≤S T , set A to A + d and perform the next round of testing, where d is the step size and is a number greater than 0.
[0053] In this embodiment, both the first tensor splitting size and the second tensor splitting size are pixel sizes. When performing step S0200, the predetermined ratio can be set to 2%. For example, if there are 500 original images, 10 images are selected as test images.
[0054] In this embodiment, the tensor splitting size determined after the tensor splitting test satisfies the following relationship: the training picture size l*w ≤ the tensor splitting size ≤ the first tensor splitting size Bl*Bw. A and B can take decimal and integer values. The initial value of A is 1, the minimum value of B is 3, and a step size d can be set for A to achieve the gradual approach of the value A to the value B during the test process.
[0055] In this embodiment, when executing step S0300, when setting the first tensor split size, the value of B should not be too large or too small. If the value of B is too large, that is, the first tensor split size is too large compared to the training image size, it will cause feature extraction challenges and overfitting risks when the convolutional neural network model is predicted; if the value of B is too small, that is, the first tensor split size is close to the training image size, it may cause the final tensor split size to be too small and the crack features to be cut into too small parts and difficult to identify. Accordingly, cutting the crack features into too small parts will cause the gray matrix M2 predicted by the convolutional neural network to be too large. i The grayscale matrix M1 obtained under the first tensor split size i In comparison, the similarity is very low, which leads to an increase in the number of tests and the test time.
[0056] In this embodiment, when executing step S0410, the current similarity S is calculated. i The method can use cosine similarity, Euclidean distance similarity, Manhattan distance similarity and other similarity calculation methods. The above similarity calculation methods are all exemplary descriptions and are not intended to be limiting of the present application. They will not be described in detail herein.
[0057] In an exemplary embodiment, the “performing tensor splitting and grayscale calculation on each of the test images” in step S0300 and step S0410 may include:
[0058] The following steps S0310-S0340 are performed for each of the test images:
[0059] S0310: Input the test image into the image segmentation module and convert it into a third-order tensor (L, W, C);
[0060] S0320: performing tensor splitting on the third-order tensor (L, W, C) according to the tensor splitting size set for the test image to obtain multiple tensors;
[0061] S0330: Input each of the tensors into the convolutional neural network model, and output a grayscale matrix corresponding to each of the tensors;
[0062] S0340: splicing all output grayscale matrices to obtain a grayscale matrix corresponding to the test image;
[0063] In this embodiment, L, W, and C represent the pixel length, pixel width, and number of color channels of the test image, respectively.
[0064] In an exemplary embodiment, step S2000 may include: performing the following steps S2100-S2200 on each of the split tensors respectively:
[0065] S2100: Input the split tensor into the convolutional neural network model;
[0066] S2200: Perform convolution, activation and pooling calculations based on the convolutional neural network model, extract crack features and calculate grayscale, and output a grayscale matrix corresponding to the split tensor.
[0067] In this embodiment, by performing convolution, activation and pooling calculations on each split tensor, it helps to reduce the computational burden and improve the ability to recognize subtle features and short crack features.
[0068] In an exemplary embodiment, the step S3000 and the step S0340 of “joining all output grayscale matrices” may include steps S3100-S3300:
[0069] S3100: performing edge processing on each of the grayscale matrices: multiplying the grayscale values of a first preset number of pixel points located at the leftmost and rightmost sides of the grayscale matrix by a magnification coefficient, and multiplying the grayscale values of a second preset number of pixel points located at the top and bottom sides of the grayscale matrix by the magnification coefficient;
[0070] S3200: Arranging all grayscale matrices after the edge processing according to a position relationship to obtain a first splicing matrix, wherein the position relationship is a position relationship between the split tensor and the original image;
[0071] S3300: performing stitching processing on the overlapped part of the first stitching matrix, wherein each overlapping pixel point at the overlapped part includes grayscale values G corresponding to a plurality of grayscale matrices. j ;
[0072] In this embodiment, j=1...J, where J represents the position of G at each overlapping pixel point. j The number of, 2≤J≤4.
[0073] In this embodiment, by executing step S3100, the calculation weight of the edge area is expanded. Even if a crack is split into two adjacent split tensors during splitting, the crack located at the split edge can be effectively detected, thereby reducing the edge effect. The first preset number and the second preset number can be set proportionally according to the number of pixels in the width direction and the length direction of the grayscale matrix, respectively.
[0074] In this embodiment, by executing step S3200, all grayscale matrices are spliced according to the positional relationship, and the spliced first spliced matrix corresponds to the original image size. After executing step S3200 and obtaining the first spliced matrix, it is necessary to traverse each pixel point in the first spliced matrix, extract all overlapping pixel points, and perform splicing processing on each overlapping pixel point.
[0075] In an exemplary embodiment, the step S3300 of “stitching the overlapped portion of the first stitching matrix” may include: performing steps S3310 to S3330 on each of the overlapping pixels at the overlapped portion:
[0076] S3310: Calculate the grayscale average value corresponding to the overlapping pixel point
[0077] S3320: According to G j And G calculate the variance value of the overlapping pixel
[0078] S3330: Determine the variance value σ and the variance threshold σ T If σ<σ T , then the grayscale average G is used as the grayscale value of the overlapping pixel. If σ≥σ T , then delete the gray value G corresponding to the overlapping pixel j , use the cubic spline interpolation method to interpolate and obtain the grayscale value of the overlapping pixel.
[0079] In this embodiment, when the variance value σ is less than the variance threshold σ T When , it indicates that the J grayscale values G of the overlapping pixel point j The discreteness is low, and the gray value has good stability, consistency and stability. j Can represent the grayscale of the overlapping pixel, using J grayscale values G j The average value G is used as the grayscale of the overlapping pixel; when the variance σ is greater than or equal to the variance threshold σ T When , it indicates that the J grayscale values G of the overlapping pixel point j The discreteness is high, there are extreme values, and the risk of inaccurate grayscale values is high. Delete the J grayscale values G corresponding to the overlapping pixel points. j , the cubic spline interpolation method is used to interpolate the grayscale of the overlapping pixels, which reduces the possibility of abnormal points appearing in the crack prediction results.
[0080] In an exemplary embodiment, the step S3330 of "using a cubic spline interpolation method to interpolate to obtain the grayscale value of the overlapping pixel point" may include steps S3331-S3332:
[0081] S3331: Select an extension direction with the overlapping pixel point as the center, and select the grayscale values G of the nearest N pixels at one end of the extension direction. -n , select the grayscale values G of the nearest N pixels at the other end of the extension direction +n , where n=1...N;
[0082] S3332: Based on G -n and G +n Perform cubic spline interpolation to obtain the grayscale value of the overlapping pixel;
[0083] In this embodiment, the extending direction includes the up-down direction or the left-right direction along the overlapping pixel point.
[0084] In this embodiment, the grayscale values of overlapping pixels are obtained by the cubic spline interpolation method, information is exchanged between different split blocks, and global features are integrated to achieve a smooth transition between the grayscale values of overlapping pixels and the grayscale values of pixels in the upper and lower directions or the left and right directions, thereby ensuring a natural transition and effectively reducing the discontinuity and visual breaks that may occur during the stitching process.
[0085] In order to illustrate the crack identification method based on partitioned convolution according to the embodiment of the present application, a specific example is used to describe it in detail below.
[0086] The drone's onboard imaging device obtains original images of the aqueduct in RGB format, with 1,000 images. Based on steps S1-S9, crack identification of the aqueduct is performed. The cracks on the aqueduct are all short cracks, generally a few millimeters wide and tens of centimeters long.
[0087] S1: Get the training image size of the pre-trained convolutional neural network model. The pixel size of the training image size is (400, 300). Select 2% of the original image as the test image, that is, use 20 test images for tensor splitting test. After the test, determine the tensor splitting size to be (800, 600);
[0088] S2: Input an original image of the aqueduct into the image segmentation module, convert the third-order tensor (5080, 3960, 3), 3 represents the three channels of red, green and blue, that is, the original image is converted into three large tensors with pixel sizes of (5080, 3960), and the three large tensors are split into split tensors of the same size according to (800, 600), and the position relationship between each split tensor and the large tensor is saved. Each split tensor corresponds to three channels;
[0089] The split result is as follows Figure 2 As shown in the figure, the split tensor is split into 7 rows and 7 columns, with a total of 49 split tensors. The diagonal filled part is the overlapping part. The horizontal number is the X-axis pixel range of the split tensor, and the vertical number is the Y-axis pixel range of the split tensor. Figure 2This is just one example of tensor splitting. In actual tensor splitting, there can also be overlapping parts between adjacent rows, overlapping parts between adjacent columns, overlapping parts between any two adjacent columns, and overlapping parts between any two adjacent rows, as long as all the split tensors obtained can fully cover the large tensor of the original image conversion.
[0090] S3: Input the data of the three channels corresponding to each split tensor into the convolutional neural network model for independent calculation, and output the grayscale matrix corresponding to each split tensor, with a total of 49 grayscale matrices;
[0091] S4: performing the following edge processing on each gray matrix: multiplying the gray values of the eight pixels located at the top and bottom of the gray matrix by a magnification factor of 1.05, and multiplying the gray values of the eight pixels located at the leftmost and rightmost sides of the gray matrix by a magnification factor of 1.05;
[0092] S5: splicing the 49 grayscale matrices after edge processing according to the position relationship saved in step S2 to obtain a first splicing matrix;
[0093] S6: traverse the pixel points in the first stitching matrix and extract all overlapping pixel points;
[0094] S7: For each overlapping pixel, calculate the variance value σ based on multiple gray values and compare it with the variance threshold σ T Compare; if σ<σ T , the average value of multiple gray values is used as the gray value of the overlapping pixel; if σ≥σ T , then delete the multiple gray values of the overlapping pixel points, use the cubic spline interpolation method to interpolate the gray values of the overlapping pixel points, and finally obtain the crack prediction gray matrix after processing all the overlapping pixel points;
[0095] S8: In the crack prediction grayscale matrix, set the pixel with a grayscale value of 0-127 to 0, indicating that it is not a crack; set the pixel with a grayscale value of 128-255 to 1, indicating that the pixel is a crack, and obtain a crack prediction binary matrix;
[0096] S9: converting the crack prediction binary matrix into a crack prediction binary image;
[0097] S10: Align the same pixels of the crack prediction binary image and the original image to obtain the crack distribution map.
[0098] When performing this specific example, the time consumed by the partitioned convolution-based crack recognition method of the present application was counted, and the average time consumed for crack recognition on an original aqueduct image was 5 seconds per image. In addition, the second method (i.e., the image stitching method) mentioned in the background technology of the present application was tested, and the average time consumed for crack recognition on an original aqueduct image was 8.2 seconds per image. The original aqueduct image was not split in any way, and the overall image recognition was performed directly, which consumed 62.4 seconds per image. As shown in Table 1, it can be seen that the partitioned convolution-based crack recognition method of the present application is less time-consuming and faster, thereby improving the efficiency of identifying short and small cracks based on large images.
[0099] Table 1 Crack recognition time of the original image of the aqueduct
[0100] method Time consumed (seconds / photo) Speed up (%) Overall image recognition 62.4 Image stitching 8.2 661 Crack Identification Method Based on Partitioned Convolution 5 1148
[0101] The present application also provides a crack identification device based on partition convolution, such as Figure 3 As shown, including a processor and a memory,
[0102] The memory is used to store a crack identification program based on partition convolution;
[0103] The processor is used to read the partitioned convolution-based crack identification program and perform the partitioned convolution-based crack identification method as described in the above embodiment.
[0104] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
Claims
1. A crack identification method based on partitioned convolution, It is characterized in that include: Input the original image containing cracks into the image segmentation module, split the tensor according to the tensor splitting size, and output Q split tensors of the same size, Q>1; Inputting each of the split tensors into a pre-trained convolutional neural network model, performing independent calculations on each of the split tensors, and outputting a grayscale matrix corresponding to each of the split tensors, wherein the convolutional neural network model is used to predict cracks in the original image; All output grayscale matrices are spliced to obtain a crack prediction grayscale matrix corresponding to the original image; The original image and the crack prediction grayscale matrix are fused and visualized to obtain a crack distribution map.
2. The crack identification method based on partition convolution according to claim 1, Features: The original image contains short cracks; Among them, the short cracks meet at least one of the following conditions: the crack pixel width is less than one pixel, the crack pixel length is less than ten pixels, the crack pixel width is less than 1 / 100 of the original image pixel width, and the crack pixel length is less than 1 / 100 of the original image pixel length.
3. The crack identification method based on partition convolution according to claim 1, It is characterized in that Before inputting the original image containing the cracks into the image segmentation module and performing tensor splitting according to the tensor splitting size, the method further includes: The training picture size of the convolutional neural network model is obtained, and a tensor splitting test is performed on the original image based on the training picture size to determine the tensor splitting size, wherein the size of the original image is larger than the training picture size.
4. The crack identification method based on partition convolution as claimed in claim 3, It is characterized in that The obtaining of the training picture size of the convolutional neural network model, and performing a tensor splitting test on the original image based on the training picture size to determine the tensor splitting size includes: Obtain the pixel length l and pixel width w corresponding to the training image size of the convolutional neural network model; Acquire a plurality of the original images of the target area, and select a test image from the plurality of the original images according to a predetermined ratio; According to the set first tensor splitting size Bl*Bw, tensor splitting and grayscale calculation are performed on each of the test images to obtain the first grayscale matrix M1 i , where i=1...I, I represents the number of the test images, B≥3; Execute the following steps cyclically for one or more rounds of testing until the current average similarity S is greater than the preset similarity threshold S T , then set the tensor splitting size to the second tensor splitting size Al*Aw in the current round, where 1≤A<B; or, until A = B, S is still not greater than S T , then set the tensor splitting size to Bl*Bw: According to the set current round second tensor splitting size Al*Aw, tensor splitting and grayscale calculation are performed on each of the test images to obtain the current second grayscale matrix M2 i , calculate M2 i With M1 i The current similarity S i ; In the first round of testing, A=1; Calculate the current average similarity Judging S and S T If S≤S T , set A to A+d and proceed to the next round of testing, where d is the step size and is a number greater than 0.
5. The crack identification method based on partition convolution according to claim 4, It is characterized in that The performing tensor splitting and grayscale calculation on each of the test images includes: For each of the test images, perform the following steps: Input the test image into the image segmentation module and convert it into a third-order tensor (L, W, C); According to the tensor splitting size set for the test image, the third-order tensor (L, W, C) is tensor split to obtain a plurality of tensors; Input each of the tensors into the convolutional neural network model, and output a grayscale matrix corresponding to each of the tensors; All the output grayscale matrices are spliced to obtain the grayscale matrix corresponding to the test image; Wherein, L, W, and C represent the pixel length, pixel width, and number of color channels of the test image, respectively.
6. The crack identification method based on partition convolution according to claim 1, It is characterized in that The method of inputting each of the split tensors into a pre-trained convolutional neural network model, performing independent calculations on each of the split tensors, and outputting a grayscale matrix corresponding to each of the split tensors, includes: For each of the split tensors, perform the following steps: Input the split tensor into the convolutional neural network model; Based on the convolutional neural network model, convolution, activation and pooling calculations are performed, and crack feature extraction and grayscale calculations are performed, and a grayscale matrix corresponding to the split tensor is output.
7. The crack identification method based on partition convolution according to claim 1 or 5, It is characterized in that The step of splicing all output grayscale matrices includes: Performing edge processing on each of the grayscale matrices: multiplying the grayscale values of a first preset number of pixel points located at the leftmost and rightmost sides of the grayscale matrix by a magnification coefficient, and multiplying the grayscale values of a second preset number of pixel points located at the uppermost and lowermost sides of the grayscale matrix by the magnification coefficient; Arranging all grayscale matrices after the edge processing according to a position relationship to obtain a first splicing matrix, wherein the position relationship is a position relationship between the split tensor and the original image; The overlapping parts of the first stitching matrix are stitched together, and each overlapping pixel point at the overlapping part includes grayscale values G corresponding to a plurality of grayscale matrices. j ; Wherein, j=1...J, J represents the G at each overlapping pixel point. j The number of, 2≤J≤4.
8. The crack identification method based on partition convolution according to claim 7, It is characterized in that The step of performing stitching processing on the overlapped parts of the first stitching matrix includes: For each of the overlapping pixels at the overlapping position, the following steps are performed: Calculate the grayscale average value corresponding to the overlapping pixel According to G j And G calculate the variance value of the overlapping pixel Determine the variance value σ and the variance threshold σ T If σ<σ T , then the grayscale average G is used as the grayscale value of the overlapping pixel. If σ≥σ T , then delete the gray value G corresponding to the overlapping pixel j , use the cubic spline interpolation method to interpolate and obtain the grayscale value of the overlapping pixel.
9. The crack identification method based on partition convolution according to claim 8, It is characterized in that The method of interpolating using a cubic spline interpolation method to obtain the grayscale value of the overlapping pixel point includes: An extension direction is selected with the overlapping pixel point as the center, and the grayscale values G of the nearest N pixels are selected at one end of the extension direction. -n , select the grayscale values G of the nearest N pixels at the other end of the extension direction +n , where n=1...N; Based on G -n and G +n Perform cubic spline interpolation to obtain the grayscale value of the overlapping pixel; The extending direction includes the up-down direction or the left-right direction along the overlapping pixel point.
10. A crack identification device based on partition convolution, comprising a processor and a memory, Features: The memory is used to store a crack identification program based on partition convolution; The processor is used to read the partitioned convolution-based crack identification program and perform the partitioned convolution-based crack identification method as described in any one of claims 1-9.
