Slit detection method, system and storage medium based on YOLOv7 optimization
By optimizing the structure and fusing features of the YOLOv7 model, the problems of time-consuming and labor-intensive manual detection and the need for professional expertise in specialized models in existing technologies have been solved, achieving high-precision road crack detection and segmentation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-27
- Publication Date
- 2026-03-20
AI Technical Summary
In existing technologies, manual inspection of road cracks is time-consuming and labor-intensive, while professional model inspection requires specialized knowledge and is difficult to promote. Furthermore, the output of the existing YOLOv7 model requires professional interpretation, which limits its application.
The YOLOv7 model was structurally optimized by introducing Conv convolutional networks and Concat concatenation networks to enhance feature fusion. The ELAN network was replaced with Mycontact-4 and Mycontact-6 networks, and a segmentation head was used for crack detection and segmentation.
It improves the model's ability to understand targets, enhances the accuracy and robustness of detection, reduces sensitivity to noise and environmental changes, and achieves high-precision remote monitoring.
Smart Images
Figure CN117237312B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of concrete crack monitoring and computer technology, and particularly relates to a crack detection method and system based on YOLOv7 optimization and a storage medium. BACKGROUND
[0002] Manual detection and model detection are the main means of current road crack detection. The former is time-consuming and inefficient, and the latter requires high professional knowledge and is difficult to promote.
[0003] Taking the commonly used YOLO network as an example, the current YOLOv7 model is often used in road detection, and is a relatively stable and excellent model. However, the output of the YOLOv7 model is professional detection data, which needs to be interpreted by professional personnel, and the demand for professional personnel limits its development in road detection. SUMMARY
[0004] In order to overcome the defects that there is no reliable and effective structural health monitoring method in the prior art, the present application provides a crack detection method based on YOLOv7 optimization, which can realize remote high-precision monitoring of road cracks, and achieves more stable and excellent performance in target detection and instance segmentation.
[0005] The crack detection method based on YOLOv7 optimization provided by the present application comprises the following steps:
[0006] S1, the structure of the traditional YOLOv7 model is optimized as a basic model; and a concrete damage image labeled with a crack category is obtained as a learning sample;
[0007] S2, the basic model is subjected to machine learning on the learning sample, and a converged basic model is obtained as a target detection model; when it is necessary to detect concrete cracks, a concrete image is input into the target detection model, and the target detection model outputs a detection result, which includes a crack category;
[0008] Compared with the traditional YOLOv7 model, the basic model further comprises a fifteenth Conv convolutional network, an eighth Concat splicing network, a sixteenth Conv convolutional network, a ninth Concat splicing network, a seventeenth Conv convolutional network and a tenth Concat splicing network;
[0009] The input of the fifteenth Conv convolutional network is connected with the output of the second ELAN network, the output of the fifteenth Conv convolutional network is connected with the input of the eighth Concat splicing network, and the input of the eighth Concat splicing network is also connected with the output of the second ELAN-W network; the input end of the REP network in the first output network is respectively connected with the output of the eighth Concat splicing network and the output of the second ELAN-W network;
[0010] The input of the sixteenth Conv convolutional network is connected with the output of the third ELAN network, the output of the sixteenth Conv convolutional network is connected with the input of the ninth Concat splicing network, and the input of the ninth Concat splicing network is also connected with the output of the third ELAN-W network; the input end of the REP network in the second output network is respectively connected with the output of the ninth Concat splicing network and the output of the third ELAN-W network;
[0011] The input of the seventeenth Conv convolutional network is connected with the output of the fourth ELAN network, the output of the seventeenth Conv convolutional network is connected with the input of the tenth Concat splicing network, and the input of the tenth Concat splicing network is also connected with the output of the fourth ELAN-W network; the input end of the REP network in the third output network is respectively connected with the output of the tenth Concat splicing network and the output of the fourth ELAN-W network.
[0012] Preferably, the manner of obtaining the basic model by optimizing the structure of the traditional YOLOv7 model further comprises:
[0013] The first ELAN network, the second ELAN network, the third ELAN network and the fourth ELAN network in the traditional YOLOv7 model are all replaced by the Mycontact-4 network, and the first ELAN-W network, the second ELAN-W network, the third ELAN-W network and the fourth ELAN-W network are all replaced by the Mycontact-6 network;
[0014] The Mycontact-4 network comprises a first Conv convolutional network, a second Conv convolutional network, a third Conv convolutional network, a fourth Conv convolutional network, a fifth Conv convolutional network, a sixth Conv convolutional network, a fifth Concat concatenation network and a seventh Conv convolutional network; the second Conv convolutional network, the third Conv convolutional network, the fourth Conv convolutional network, the fifth Conv convolutional network and the sixth Conv convolutional network are sequentially connected, and an input end of the second Conv convolutional network is connected with an input end of the first Conv convolutional network; an output end of the first Conv convolutional network, an output end of the second Conv convolutional network, an output end of the fourth Conv convolutional network and an output end of the sixth Conv convolutional network are all connected with an input end of the fifth Concat concatenation network, and an output end of the fifth Concat concatenation network is connected with an input end of the seventh Conv convolutional network; the input end of the first Conv convolutional network serves as an input end of the Mycontact-4 network, and an output end of the seventh Conv convolutional network serves as an output end of the Mycontact-4 network;
[0015] The Mycontact-6 network comprises an eighth Conv convolutional network, a ninth Conv convolutional network, a tenth Conv convolutional network, an eleventh Conv convolutional network, a twelfth Conv convolutional network, a thirteenth Conv convolutional network, a sixth Concat concatenation network and a fourteenth Conv convolutional network; the ninth Conv convolutional network, the tenth Conv convolutional network, the eleventh Conv convolutional network, the twelfth Conv convolutional network and the thirteenth Conv convolutional network are sequentially connected, and an input end of the ninth Conv convolutional network is connected with an input end of the eighth Conv convolutional network; an output end of the eighth Conv convolutional network, an output end of the ninth Conv convolutional network, an output end of the tenth Conv convolutional network, an output end of the eleventh Conv convolutional network, an output end of the twelfth Conv convolutional network and an output end of the thirteenth Conv convolutional network are all connected with an input end of the sixth Concat concatenation network, and an output end of the sixth Concat concatenation network is connected with an input end of the fourteenth Conv convolutional network; an input end of the eighth Conv convolutional network serves as an input end of the Mycontact-6 network, and an output end of the fourteenth Conv convolutional network serves as an output end of the Mycontact-6 network.
[0016] Preferably, a data structure of output data of the fifteenth Conv convolutional network is same as a data structure of output data of the second Mycontact-6 network, a data structure of output data of the sixteenth Conv convolutional network is same as a data structure of output data of the third Mycontact-6 network, and a data structure of output data of the seventeenth Conv convolutional network is same as a data structure of output data of the fourth Mycontact-6 network.
[0017] Preferably, it further comprises a step S3: after inputting the concrete image into the target detection model, transmitting the detection result of the target detection model and the input of the target detection model to a set segmentation head, the segmentation head performing instance segmentation on the cracks in the concrete image; the segmentation head outputs a segmented image of the cracks and labels the crack size and trend.
[0018] The application provides a crack detection system, which comprises an image acquisition module, a target detection model, a segmentation head and a crack display module.
[0019] The image acquisition module is used for acquiring a concrete image to be recognized; the target detection model is used for recognizing cracks on the concrete image; the segmentation head is used for performing instance segmentation on the cracks in the concrete image; and the segmentation head sends the segmented cracks to the crack display module for display.
[0020] The application provides a crack detection system, which comprises a memory and a processor, the memory stores a computer program, and the processor is connected with the memory and used for executing the computer program to realize the crack detection method based on the YOLOv7 optimization.
[0021] The application provides a storage medium, which stores a computer program, and the computer program is used for realizing the crack detection method based on the YOLOv7 optimization when executed.
[0022] The application further provides a crack detection system, which comprises a memory and a processor, the memory stores a computer program, and the processor is connected with the memory and used for executing the computer program to realize the crack detection method based on the YOLOv7 optimization.
[0023] The application further provides a storage medium, which stores a computer program, and the computer program is used for realizing the crack detection method based on the YOLOv7 optimization when executed.
[0024] The application has the following advantages:
[0025] (1) In the application, the Conv convolutional network and the Cancat splicing network are added to fuse the shallow features in front of the model and the deep features behind the model; since the feature maps in the YOLO gradually decrease from the shallow layer to the deep layer, each layer of features contains different levels of semantic information, by fusing the features of different layers, the network can capture low-level and high-level semantic information at the same time, and the understanding ability of the target is improved.
[0026] (2) The application increases feature fusion on the basis of the traditional YOLOv7, so that the model obtains more rich semantic information from different convolution layers, which helps to improve the understanding of more detailed target shape, texture and the like of the model, thereby improving the accuracy of the detector.
[0027] (3) In the optimization process of the application, for each output network, a weighted feature fusion module composed of a Conv convolution network and a Cancat splicing network is set at the front end thereof, and the over-reliance on specific level features can be reduced through weighted feature fusion, so that the network is more robust for target detection under noise, occlusion and different environmental conditions.
[0028] (4) In the application, the weighted feature fusion of the feature map is realized through the setting of the Mycontact-4 network and the Mycontact-6 network, and more useful feature information can be extracted. In the application, the Mycontact-4 network and the Mycontact-6 network cooperate to fuse the shallow features and the deep features of the image. The features extracted by the shallow network, i.e. the Mycontact-4 network, are closer to the input, the receptive field is smaller, and the overlapping area is also smaller, so more detailed and pixel-level information can be captured. The features extracted by the deep network, i.e. the Mycontact-6 network, are closer to the output, the receptive field is increased, and the overlapping area is increased, so that the overall information of the image is obtained. Through the fusion of the two parts of features, the dependence of the model on a single feature can be reduced, and the stability and accuracy of the model can be improved. Feature fusion can also reduce the sensitivity of the model to noise and outliers, and improve the robustness of the model. In addition, since the two features come from different levels, there may be redundant or complementary relationship between them. Through feature fusion, they can be integrated into more rich and comprehensive feature representation, thereby enhancing the expression ability of the model. BRIEF DESCRIPTION OF DRAWINGS
[0029] Figure 1 It is a flow chart of the gap detection method based on YOLOv7 optimization;
[0030] Figure 2 It is a structure diagram of the traditional YOLOv7 model;
[0031] Figure 3 It is a structure diagram of the model YOLOv7-F;
[0032] Figure 4 It is a structure diagram of the model YOLOv7-WF;
[0033] Figure 5(a) is a structure diagram of the Mycontact-4 network;
[0034] Figure 5(b) is a structure diagram of the Mycontact-6 network;
[0035] Fig. 6(a) is a trend chart of the performance of each model on target detection in the embodiment;
[0036] Fig. 6(b) is a comparison chart of the performance of each model on target detection in the embodiment;
[0037] Fig. 7(a) is a trend chart of the performance of each model on instance segmentation in the embodiment;
[0038] Fig. 7(b) is a comparison chart of the performance of each model on instance segmentation in the embodiment;
[0039] Figure 8 Fig. 8 is a display chart of segmentation results in the embodiment. DETAILED DESCRIPTION
[0040] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0041] For ease of description, in the present embodiment, 2 n0 The layer feature data is denoted as data P0xQ0x2 n0 , that is, P0xQ0x2 n0 represents the features in 2 n0 dimensions of the image with size P0xQ0, and can also be understood as image data composed of 2 n0 feature images with size P0xQ0. n0 is an arbitrary positive integer. It is worth noting that in the present embodiment, (P / r)x(Q / r)xR represents the features in R dimensions of the image with size (P / r)x(Q / r), that is, the image described by R feature images with size (P / r)x(Q / r), and P, Q and R are all arbitrary positive integers. For the convenience of drawing, in the drawings of the present embodiment, (P / r)x(Q / r)xR represents P0xQ0x2 n0 is expressed as , and n0 and r are both positive integers.
[0042] Referring to Figure 1 The gap detection method based on YOLOv7 optimization provided by the present application first optimizes the traditional YOLOv7 model, and then performs gap detection and segmentation in combination with the optimized model.
[0043] The traditional YOLOv7 model
[0044] By Figure 2As shown, the traditional YOLOv7 model includes a backbone network, a neck network, and a head network.
[0045] The backbone network includes a first CBS-1 network, a first CBS-2 network, a second CBS-1 network, a second CBS-2 network, a first ELAN network, a first MP-1 network, a second ELAN network, a second MP-1 network, a third ELAN network, and a third MP-1 network connected in sequence.
[0046] The neck network includes a fourth ELAN network, an SPPCSPC network, a first CBS-3 network, a first UPsample network, a first Concat network, a first ELAN-W network, a second CBS-3 network, a second UPsample network, a second Concat network, a second ELAN-W network, a first MP-2 network, a third Concat network, a third ELAN-W network, a second MP-2 network, a fourth Concat network, and a fourth ELAN-W network connected in sequence; the neck network further includes a third CBS-3 network and a fourth CBS-3 network.
[0047] An input end of the third CBS-3 network is connected to an output end of the third ELAN network, and an output end of the third CBS-3 network is connected to an input end of the first Concat network; the first Concat network is configured to concatenate the output of the third ELAN network and the output of the first UPsample network.
[0048] An input end of the fourth CBS-3 network is connected to an output end of the second ELAN network, and an output end of the fourth CBS-3 network is connected to an input end of the second Concat network; the second Concat network is configured to concatenate the output of the second ELAN network and the output of the second UPsample network.
[0049] The head network includes a first output network, a second output network, and a third output network; the first output network, the second output network, and the third output network are the same in structure and each include a REP network and a CBM network connected in sequence; an input of the REP network is an input of the output network, an output of the REP network is connected to an input of the CBM network, and an output of the CBM network is an output of the output network.
[0050] An input of the first output network is connected to an output of the second ELAN-W network, an input of the second output network is connected to an output of the third ELAN-W network, and an input of the third output network is connected to an output of the fourth ELAN-W network.
[0051] The YOLOv7 input size is a P×Q image, the first CBS-1 network extracts image features to obtain data P×Q×2 k , k is any positive integer; the first CBS-2 network converts the data P×Q×2 k into data (P / 2)×(Q / 2)×2 k+1 , the second CBS-1 network convolves the data (P / 2)×(Q / 2)×2 k+1 , and the convolution result is converted into data (P / 4)×(Q / 4)×2 k+2 by the second CBS-2 network; the data (P / 4)×(Q / 4)×2 k+2 is converted into data (P / 4)×(Q / 4)×2 k+3 by the first ELAN network.
[0052] The data (P / 4)×(Q / 4)×2 k+3 is converted into data (P / 8)×(Q / 8)×2 k+3 by the first MP-1 network, and the data (P / 8)×(Q / 8)×2 k+3 is converted into data (P / 8)×(Q / 8)×2 k+4 by the second ELAN network.
[0053] The data (P / 8)×(Q / 8)×2 k+4 is converted into data (P / 16)×(Q / 16)×2 k+4 by the second MP-1 network, and the data (P / 16)×(Q / 16)×2 k+4 is converted into data (P / 16)×(Q / 16)×2 k+5 by the third ELAN network.
[0054] The data (P / 16)×(Q / 16)×2 k+5 is converted into data (P / 32)×(Q / 32)×2 k+5 by the third MP-1 network, and the data (P / 32)×(Q / 32)×2 k+5 is processed by the fourth ELAN network, and the fourth ELAN network outputs data (P / 32)×(Q / 32)×2 k+5 .
[0055] The data (P / 32)×(Q / 32)×2 k+5 output by the fourth ELAN network is converted into data (P / 32)×(Q / 32)×2 k+4 by the SPPCSPC, and the data (P / 32)×(Q / 32)×2 k+4 is converted into data (P / 32)×(Q / 32)×2 k+3, data (P / 32) x (Q / 32) x 2 k+3 to data (P / 16) x (Q / 16) x 2 via a first Upsample network k+3 . A third CBS-3 network converts data (P / 16) x (Q / 16) x 2 output by the third ELAN network k+5 to data (P / 16) x (Q / 16) x 2 k+3 ; a first Concat network concatenates data (P / 16) x (Q / 16) x 2 output by the first Upsample k +3 and data (P / 16) x (Q / 16) x 2 output by the third CBS-3 network k+3 to obtain and output data (P / 16) x (Q / 16) x 2 k+4 .
[0056] Data (P / 16) x (Q / 16) x 2 output by the first Concat network k+4 is converted to data (P / 16) x (Q / 16) x 2 via a first ELAN-W network k+3 , data (P / 16) x (Q / 16) x 2 k+3 is converted to data (P / 16) x (Q / 16) x 2 via a second CBS-3 network k+2 , data (P / 16) x (Q / 16) x 2 k+2 is converted to (P / 8) x (Q / 8) x 2 via a second Upsample network k+2 . A fourth CBS-3 network converts data (P / 8) x (Q / 8) x 2 output by the second ELAN network k+4 to data (P / 8) x (Q / 8) x 2 k+2 ; a second Concat network concatenates data (P / 8) x (Q / 8) x 2 output by the second Upsample k+2 and data (P / 8) x (Q / 8) x 2 output by the fourth CBS-3 network k+2 to obtain and output data (P / 8) x (Q / 8) x 2 k+3 .
[0057] Data (P / 8) x (Q / 8) x 2 output by the second Concat network k+3 is converted to data (P / 8) x (Q / 8) x 2 via a second ELAN-W network k+2 , data (P / 8) x (Q / 8) x 2 k+2 is converted to data (P / 16) x (Q / 16) x 2 via a first MP-2 network k+3; the third Concat concatenation network concatenates the data (P / 16) x (Q / 16) x 2 output by the first MP-2 network k+3 and the data (P / 16) x (Q / 16) x 2 output by the first ELAN-W k+3 to output data (P / 16) x (Q / 16) x 2 k +4 .
[0058] The data (P / 16) x (Q / 16) x 2 output by the third Concat concatenation network k+4 is converted by the third ELAN-W network into data (P / 16) x (Q / 16) x 2 k+3 , the data (P / 16) x (Q / 16) x 2 k+3 is converted by the second MP-2 network into data (P / 32) x (Q / 32) x 2 k+4 ; the fourth Concat concatenation network concatenates the data (P / 32) x (Q / 32) x 2 output by the second MP-2 network k+4 and the data (P / 32) x (Q / 32) x 2 output by the SPPCSPC network k+4 to output data (P / 32) x (Q / 32) x 2 k+5 .
[0059] The data (P / 32) x (Q / 32) x 2 output by the fourth Concat concatenation network k+5 is converted by the fourth ELAN-W network into data (P / 32) x (Q / 32) x 2 k+4 .
[0060] The output of the second ELAN-W network is input to the first output network, and after being processed by the REP network and the CBM network of the first output network in turn, the first output network outputs data (P / 8) x (Q / 8) x m, m being a set value.
[0061] The third ELAN-W network outputs data (P / 16) x (Q / 16) x 2 k+3 , and the output of the third ELAN-W network is input to the second output network, and after being processed by the REP network and the CBM network of the second output network in turn, the second output network outputs data (P / 16) x (Q / 16) x m.
[0062] The fourth ELAN-W network outputs data (P / 32) x (Q / 32) x 2 k+4 , and the output of the fourth ELAN-W network is input to the third output network, and after being processed by the REP network and the CBM network of the third output network in turn, the third output network outputs data (P / 32) x (Q / 32) x m.
[0063] The YOLOv7 model outputs detection results according to (P / 8)×(Q / 8)×m, (P / 16)×(Q / 16)×m and (P / 32)×(Q / 32)×m output by three output layers.
[0064] The YOLOv7 model is a model structure existing in the prior art, wherein the backbone network, the neck network and the head network are all defined in the YOLOv7 model and belong to the technical common sense in the art.
[0065] In the drawings of the present application, the input of each model is set as 512×512×3 image data, that is, the input of the model is composed of three feature images with a size of 512×512; P=Q=n, n is an arbitrary positive integer; m=(number of categories+3)×3; for example, in the embodiment, the crack categories are divided into three categories of horizontal cracks, vertical cracks and fatigue cracks, and m=24.
[0066] The first YOLOv7 optimization model is denoted as YOLOv7-F
[0067] Referring to Figure 3 Compared with the traditional YOLOv7 model, the model YOLOv7-F proposed in the embodiment further includes a fifteenth Conv convolutional network, an eighth Concat splicing network, a sixteenth Conv convolutional network, a ninth Concat splicing network, a seventeenth Conv convolutional network and a tenth Concat splicing network; the model YOLOv7-F further optimizes the input of the REP network in the first output network, the second output network and the third output network relative to the model YOLOv7-M.
[0068] Specifically, the input of the fifteenth Conv convolutional network is connected with the output of the second ELAN network, the output of the fifteenth Conv convolutional network is connected with the input of the eighth Concat splicing network, and the input of the eighth Concat splicing network is further connected with the output of the second ELAN-W network. The input end of the REP network in the first output network is respectively connected with the output of the eighth Concat splicing network and the output of the second ELAN-W network.
[0069] The input of the sixteenth Conv convolutional network is connected with the output of the third ELAN network, the output of the sixteenth Conv convolutional network is connected with the input of the ninth Concat splicing network, and the input of the ninth Concat splicing network is further connected with the output of the third ELAN-W network. The input end of the REP network in the second output network is respectively connected with the output of the ninth Concat splicing network and the output of the third ELAN-W network.
[0070] The output of the fourth ELAN network is connected to the input of the seventeenth Convolutional network, the output of the seventeenth Convolutional network is connected to the input of the tenth Concatenation network, and the output of the fourth ELAN-W network is also connected to the input of the tenth Concatenation network. The input of the REP network in the third output network is connected to the output of the tenth Concatenation network and the output of the fourth ELAN-W network, respectively.
[0071] The fifteenth Convolutional network converts the data (P / 8) x (Q / 8) x 2 k+4 output by the second ELAN network into data (P / 8) x (Q / 8) x 2 k+2 , and the data (P / 8) x (Q / 8) x 2 k+2 output by the fifteenth Convolutional network and the data (P / 8) x (Q / 8) x 2 k+2 output by the second ELAN-W network are output after dimension concatenation by the eighth Concatenation network; the input of the REP network in the first output network is connected to the data (P / 8) x (Q / 8) x 2 k+3 and the data (P / 8) x (Q / 8) x 2 k+2 output by the second ELAN-W network, and the REP network in the first output network adjusts the dimensions of the input data, and the adjusted data is output to the connected CBM network for prediction to obtain the crack category and position, number, etc.
[0072] The sixteenth Convolutional network converts the data (P / 16) x (Q / 16) x 2 k+5 output by the third ELAN network into data (P / 16) x (Q / 16) x 2 k+3 , and the data (P / 16) x (Q / 16) x 2 k+3 output by the sixteenth Convolutional network and the data (P / 16) x (Q / 16) x 2 k+3 output by the third ELAN-W network are output after dimension concatenation by the ninth Concatenation network; the input of the REP network in the second output network is connected to the data (P / 16) x (Q / 16) x 2 k+4 and the data (P / 16) x (Q / 16) x 2 k+3 output by the third ELAN-W network, and the REP network in the second output network adjusts the dimensions of the input data, and the adjusted data is output to the connected CBM network for prediction to obtain the crack category and position, number, etc.
[0073] The seventeenth Convolutional network converts the data (P / 32) x (Q / 32) x 2 k+5converted into data (P / 32) x (Q / 32) x 2 k+4 data (P / 32) x (Q / 32) x 2 output by the seventeenth Conv convolutional network k+4 data (P / 32) x (Q / 32) x 2 output by the fourth ELAN-W network k+4 After dimension splicing by the tenth Concat splicing network, the output is obtained; the input end of the REP network in the third output network is connected to the data (P / 32) x (Q / 32) x 2 output by the tenth Concat splicing network k+5 data (P / 32) x (Q / 32) x 2 output by the fourth ELAN-W network k+4 In the third output network, the REP network adjusts the dimensions of the input data, and the adjusted data is output to the connected CBM network for prediction to obtain the crack category and position, number, etc.
[0074] It is worth noting that in the first output network, the second output network and the third output network, only the input of the REP network is optimized relative to the existing YOLOv7. In the model YOLOv7-MF, the REP network structure and the CBM network structure directly adopt the REP network structure and the CBM network structure in the traditional YOLOv7. The REP is used to adjust the dimensions of the input data to the specified data dimensions, and the CBM outputs the image data of the specified size and the crack classification result; the image sizes output by the CBM networks of the three output networks are different, thereby realizing crack classification under different sizes.
[0075] The REP network is used for splicing processing of the dimensions of the input data. In the present embodiment, only the data input of the REP network is increased, and the structure and working principle of the REP network are both prior art. The REP network uses its own setting to splice the input data, and the REP input data dimension is expanded to better propagate and utilize additional information, which helps to improve the positioning of the object boundary and shape of the model. Because the REP module can introduce more context information, the accuracy of object detection is improved.
[0076] The REP network improves the expression of features by introducing residual connection and feature fusion to better capture the details and context of the object.
[0077] The structure and working principle of the CBM network also adopt prior art. The CBM network helps to improve the context perception ability of object detection, and in crack recognition, it can better understand the relationship between different parts of the image, including the relationship between the crack and the surrounding environment. The application of the CBM network can introduce more context information to help the model better understand the position and shape of the crack.
[0078] The CBM can make the model focus on the region of interest by introducing an attention mechanism, and improve the performance of target detection through different scale context information.
[0079] The second YOLOv7 optimization model is denoted as YOLOv7-WF
[0080] As shown in Figure 4 YOLOv7-WF, the first ELAN network, the second ELAN network, the third ELAN network and the fourth ELAN network are all replaced by Mycontact-4 network, and the first ELAN-W network, the second ELAN-W network, the third ELAN-W network and the fourth ELAN-W network are all replaced by Mycontact-6 network.
[0081] As shown in FIG. 5(a), the Mycontact-4 network includes a first Conv convolutional network, a second Conv convolutional network, a third Conv convolutional network, a fourth Conv convolutional network, a fifth Conv convolutional network, a sixth Conv convolutional network, a fifth Concat splicing network and a seventh Conv convolutional network.
[0082] The second Conv convolutional network, the third Conv convolutional network, the fourth Conv convolutional network, the fifth Conv convolutional network and the sixth Conv convolutional network are sequentially connected, and the input end of the second Conv convolutional network is connected to the input end of the first Conv convolutional network; the output end of the first Conv convolutional network, the output end of the second Conv convolutional network, the output end of the fourth Conv convolutional network and the output end of the sixth Conv convolutional network are all connected to the input end of the fifth Concat splicing network, and the output end of the fifth Concat splicing network is connected to the input end of the seventh Conv convolutional network.
[0083] The input end of the first Conv convolutional network is the input end of the Mycontact-4 network, and the output end of the seventh Conv convolutional network is the output end of the Mycontact-4 network.
[0084] In the Mycontact-4 network, the first Conv convolutional network and the second Conv convolutional network perform convolutional processing on the input h-dimensional data x(h), and the output of the second Conv convolutional network is sequentially subjected to convolutional processing by the third Conv convolutional network, the fourth Conv convolutional network, the fifth Conv convolutional network and the sixth Conv convolutional network. The output of the first Conv convolutional network after adding a weight W1 generates convolutional data x1(h), the output of the second Conv convolutional network after adding a weight W2 generates convolutional data x2(h), the output of the fourth Conv convolutional network after adding a weight W3 generates convolutional data x3(h), and the output of the sixth Conv convolutional network after adding a weight W4 generates convolutional data x4(h). The data dimensions of the convolutional data x1(h), x2(h), x3(h) and x4(h) are all h. The fifth Concat concatenation network obtains the convolutional data x1(h), x2(h), x3(h) and x4(h) for dimension concatenation and outputs 4h-dimensional data x'(4h), and the seventh Conv convolutional network performs convolutional processing on the data x'(4h) and outputs 2h-dimensional data x''(2h), which is the output of the Mycontact-4 network. As can be seen, the data dimension of the output data x''(2h) of the Mycontact-4 network is twice the data dimension of the input data x(h) of the Mycontact-4 network. Obviously, the Mycontact-4 network has the same feature that the output is doubled in data dimension relative to the input as the ELAN network, and therefore the Mycontact-6 network can directly replace the ELAN network.
[0085] As shown in FIG. 5(b), the Mycontact-6 network includes an eighth Conv convolutional network, a ninth Conv convolutional network, a tenth Conv convolutional network, an eleventh Conv convolutional network, a twelfth Conv convolutional network, a thirteenth Conv convolutional network, a sixth Concat concatenation network and a fourteenth Conv convolutional network.
[0086] The ninth Conv convolutional network, the tenth Conv convolutional network, the eleventh Conv convolutional network, the twelfth Conv convolutional network and the thirteenth Conv convolutional network are sequentially connected, and the input end of the ninth Conv convolutional network is connected to the input end of the eighth Conv convolutional network. The output end of the eighth Conv convolutional network, the output end of the ninth Conv convolutional network, the output end of the tenth Conv convolutional network, the output end of the eleventh Conv convolutional network, the output end of the twelfth Conv convolutional network and the output end of the thirteenth Conv convolutional network are all connected to the input end of the sixth Concat concatenation network, and the output end of the sixth Concat concatenation network is connected to the input end of the fourteenth Conv convolutional network.
[0087] The input end of the eighth Conv convolutional network is the input end of the Mycontact-6 network, and the output end of the fourteenth Conv convolutional network is the output end of the Mycontact-6 network.
[0088] In the Mycontact-6 network, the eighth Conv convolutional network and the ninth Conv convolutional network both perform convolutional processing on the input h-dimensional data x(h), and the output of the ninth Conv convolutional network is sequentially subjected to convolutional processing by the tenth Conv convolutional network, the eleventh Conv convolutional network, the twelfth Conv convolutional network and the thirteenth Conv convolutional network. The output of the eighth Conv convolutional network generates convolutional data x5(h) after adding a weight W5, the output of the ninth Conv convolutional network generates convolutional data x6(h) after adding a weight W6, the output of the tenth Conv convolutional network generates convolutional data x7(h) after adding a weight W7, the output of the eleventh Conv convolutional network generates convolutional data x8(h) after adding a weight W8, the output of the twelfth Conv convolutional network generates convolutional data x9(h) after adding a weight W9, and the output of the thirteenth Conv convolutional network generates convolutional data x10(h) after adding a weight W10. The data dimensions of the convolutional data x5(h), x6(h), x7(h), x8(h), x9(h) and x10(h) are all h. The sixth Concat concatenation network obtains the convolutional data x5(h), x6(h), x7(h), x8(h), x9(h) and x10(h) to perform dimension concatenation and then outputs 6h-dimensional data x'(6h), and the fourteenth Conv convolutional network performs convolutional processing on the data x'(6h) to output h / 2-dimensional data x''(h / 2), which is the output of the Mycontact-6 network. As can be seen, the data dimension of the output data x''(h / 2) of the Mycontact-6 network is 1 / 2 of the data dimension of the input data x(h) of the Mycontact-6 network. Obviously, the Mycontact-6 network has the same feature that the output is halved in data dimension relative to the input as the ELAN-W network, and therefore the Mycontact-6 network can directly replace the ELAN network.
[0089] The model YOLOv7-F and the model YOLOv7-WF provided by the application are verified in combination with specific embodiments.
[0090] In this embodiment, YOLOv7, YOLOv5, Cascade Mask R-CNN, Condinst, SOLOv2, and Sparseinst are selected as comparison models to verify the performance of YOLOv7-F and YOLOv7-WF in object detection, and further verify their performance in instance segmentation.
[0091] In this embodiment, manually annotated concrete damage images are used as training samples. All concrete damage images are real-world photographs. In this embodiment, 1371 manually annotated concrete damage images are divided into training and validation sets. The training set is used to perform machine learning on the YOLOv7-F and YOLOv7-WF models provided in this invention, as well as comparison models YOLOv7, YOLOv5, Cascade Mask R-CNN, and Condinst. Then, the converged model is tested on the validation set, and the model's performance in detecting gaps in the concrete damage images on the validation set is statistically analyzed. In this embodiment, precision, recall, and mean AP are used. 50 The performance of each model is evaluated using a combination of three indicators.
[0092] In this embodiment, the training set contains 960 concrete damage images, and the number and type of cracks in each concrete damage image are manually labeled. The 960 concrete damage images contain a total of 357 longitudinal cracks, 320 transverse cracks, and 283 fatigue cracks. The validation set contains 411 concrete damage images, and the number and type of cracks in each concrete damage image are manually labeled. The 411 concrete damage images contain a total of 137 longitudinal cracks, 123 transverse cracks, and 151 fatigue cracks.
[0093] In this embodiment, the specific statistics of the training set and the test set are shown in Table 1 below.
[0094] Table 1: Sample Statistics
[0095]
[0096] In this embodiment, after each model is trained, a segmentation head is set at the output of each model. This segmentation head is an existing structure used to segment the detected cracks into instances to visually display the location, size and direction of the cracks.
[0097] In this embodiment, the segmentation head can adopt any existing structure. The segmentation head does not require training; it only needs to cooperate with a trained target detection model (i.e., the YOLOv7-F and YOLOv7-WF models provided in this invention, or any comparative model). The detection results of the target detection model are input into the segmentation head, which can then visually display the location, size, and orientation of the cracks. Specifically, as shown... Figure 8 As shown.
[0098] In this embodiment, the validation results of various models on the validation set are shown in Table 2, Figure 6(a), Figure 6(b), Figure 7(a) and Figure 7(b).
[0099] Table 2 Comparison of target detection performance and instance segmentation performance of each model
[0100]
[0101] As shown in Figures 6(a) and 6(b), in object detection, models YOLOv7-F, YOLOv7-WF, and the comparative model YOLOv7 all outperform the other comparative models in terms of average accuracy. Models YOLOv7-F and YOLOv7-WF significantly outperform the comparative model YOLOv7 in precision and recall, and both models achieve over 87% in precision, recall, and average accuracy. This demonstrates that models YOLOv7-F and YOLOv7-WF exhibit better stability compared to any of the comparative models. The proposed models YOLOv7-F and YOLOv7-WF represent an improvement in object detection, demonstrating better performance in precision, recall, and average accuracy compared to existing models.
[0102] As shown in Figures 7(a) and 7(b), models YOLOv7-F and YOLOv7-WF outperform any existing comparative model in terms of precision, recall, and mean precision for instance segmentation. Compared to the most accurate comparative model, YOLOv5, models YOLOv7-F and YOLOv7-WF show a slight improvement in precision, while achieving significant improvements in recall and mean precision. While the comparative model YOLOv7 exhibits relatively stable performance in precision, recall, and mean precision, all around 70%, the proposed models YOLOv7-F and YOLOv7-WF achieve precision, recall, and mean precision all exceeding 80%. Therefore, models YOLOv7-F and YOLOv7-WF not only maintain stable performance but also achieve significant improvements in instance segmentation.
[0103] Of course, the present application is not limited to the details of the above-described exemplary embodiments but comprises the same or similar structures which can be realized in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and not restrictive, the scope of the present application being defined by the appended claims rather than the above description, and it is intended that all changes which come within the meaning and range of equivalency of the claims are embraced therein. Any reference signs in the claims should not be construed as limiting the claims to the figures in which the reference signs are used.
[0104] Furthermore, it should be understood that although the description is made on the embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, and the skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be combined appropriately to form other embodiments which can be understood by the skilled in the art.
[0105] The technologies, shapes, and structural parts not described in detail in the present application are well-known technologies.
Claims
1. A gap detection method based on YOLOv7 optimization, characterized in that, Includes the following steps: S1. Optimize the structure of the traditional YOLOv7 model as the base model; obtain concrete damage images labeled with crack categories as learning samples; S2. The base model performs machine learning on the learning samples to obtain the converged base model as the target detection model. When it is necessary to detect concrete cracks, the concrete image is input into the target detection model, and the target detection model outputs the detection results, which include the crack category. Compared to the traditional YOLOv7 model, the basic model also includes the fifteenth Conv convolutional network, the eighth Concat concatenation network, the sixteenth Conv convolutional network, the ninth Concat concatenation network, the seventeenth Conv convolutional network, and the tenth Concat concatenation network; The input of the fifteenth Conv convolutional network is connected to the output of the second ELAN network, and the output of the fifteenth Conv convolutional network is connected to the input of the eighth Concat splicing network. The input of the eighth Concat splicing network is also connected to the output of the second ELAN-W network. The input of the REP network in the first output network is connected to the output of the eighth Concat splicing network and the output of the second ELAN-W network, respectively. The input of the sixteenth Conv convolutional network is connected to the output of the third ELAN network, the output of the sixteenth Conv convolutional network is connected to the input of the ninth Concat splicing network, and the input of the ninth Concat splicing network is also connected to the output of the third ELAN-W network; the input of the REP network in the second output network is connected to the output of the ninth Concat splicing network and the output of the third ELAN-W network respectively. The input of the seventeenth Conv convolutional network is connected to the output of the fourth ELAN network, and the output of the seventeenth Conv convolutional network is connected to the input of the tenth Concat splicing network. The input of the tenth Concat splicing network is also connected to the output of the fourth ELAN-W network. The input of the REP network in the third output network is connected to the output of the tenth Concat splicing network and the output of the fourth ELAN-W network, respectively.
2. The gap detection method based on YOLOv7 optimization as described in claim 1, characterized in that, Other methods for structural optimization of traditional YOLOv7 models to obtain the base model include: Replace the first ELAN network, second ELAN network, third ELAN network, and fourth ELAN network in the traditional YOLOv7 model with the Mycontact-4 network, and replace the first ELAN-W network, second ELAN-W network, third ELAN-W network, and fourth ELAN-W network with the Mycontact-6 network. The Mycontact-4 network comprises a first Conv convolutional network, a second Conv convolutional network, a third Conv convolutional network, a fourth Conv convolutional network, a fifth Conv convolutional network, a sixth Conv convolutional network, a fifth Concat concatenation network, and a seventh Conv convolutional network. The second, third, fourth, fifth, and sixth Conv convolutional networks are sequentially connected, with the input of the second Conv convolutional network connected to the input of the first Conv convolutional network. The outputs of the first, second, fourth, and sixth Conv convolutional networks are all connected to the input of the fifth Concat concatenation network, and the output of the fifth Concat concatenation network is connected to the input of the seventh Conv convolutional network. The input of the first Conv convolutional network serves as the input of the Mycontact-4 network, and the output of the seventh Conv convolutional network serves as the output of the Mycontact-4 network. The Mycontact-6 network includes an eighth, ninth, tenth, eleventh, twelfth, and thirteenth Convolutional Network (Conv), a sixth Concat concatenation network, and a fourteenth Conv. The ninth, tenth, eleventh, twelfth, and thirteenth Conv networks are sequentially connected, with the input of the ninth Conv network connected to the input of the eighth Conv network. The eighth Conv... The outputs of the convolutional networks, the ninth Conv convolutional network, the tenth Conv convolutional network, the eleventh Conv convolutional network, the twelfth Conv convolutional network, and the thirteenth Conv convolutional network are all connected to the input of the sixth Concat convolutional network. The output of the sixth Concat convolutional network is connected to the input of the fourteenth Conv convolutional network. The input of the eighth Conv convolutional network serves as the input of the Mycontact-6 network, and the output of the fourteenth Conv convolutional network serves as the output of the Mycontact-6 network.
3. The gap detection method based on YOLOv7 optimization as described in claim 2, characterized in that, The data structure of the output data of the fifteenth Conv convolutional network is the same as that of the second Mycontact-6 network. The data structure of the output data of the sixteenth Conv convolutional network is the same as that of the third Mycontact-6 network. The data structure of the output data of the seventeenth Conv convolutional network is the same as that of the fourth Mycontact-6 network.
4. The gap detection method based on YOLOv7 optimization as described in claim 1, characterized in that, It also includes step S3: after inputting the concrete image into the target detection model, the detection results of the target detection model and the input of the target detection model are transmitted to the set segmentation head, and the segmentation head performs instance segmentation on the cracks in the concrete image; The segmentation head outputs segmented images of the cracks and labels the crack size and trend.
5. A gap detection system employing the gap detection method based on YOLOv7 optimization as described in claim 4, characterized in that, It includes an image acquisition module, an object detection model, a segmentation head, and a crack display module; The image acquisition module is used to acquire the concrete image to be identified; the target detection model is used to identify cracks on the concrete image; the segmentation head is used to segment the cracks in the concrete image; and the segmentation head sends the segmented cracks to the crack display module for display.
6. A gap detection system, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor is connected to the memory to execute the computer program to implement the gap detection method based on YOLOv7 optimization as described in any one of claims 1-4.
7. A storage medium, characterized in that, The device contains a computer program that, when executed, implements the gap detection method based on YOLOv7 optimization as described in any one of claims 1-4.