A method and device for evaluating inference results of a crack semantic segmentation model

Abstract

The application provides a kind of crack semantic segmentation model inference result evaluation method and device, belong to concrete crack detection technical field.The method comprises: S101, using crack semantic segmentation model and labeling software respectively obtains the inference result and real result of crack image;S102, the crack of inference result is compared with real result, whether the row pixel is segmented correctly is judged;S103, when row pixel is segmented correctly, the next row pixel is traversed;When row pixel is segmented incorrectly, all column pixel points of the row are traversed, and compared with real result, which column pixel of the row is segmented incorrectly is judged;S104, repeat step S102-step S103, until all rows of crack are traversed;S105, determine the pixel distribution of all rows of crack segmentation error.Using the application, the evaluation accuracy of model inference result can be improved, and the evaluation efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of concrete crack detection, in particular to a crack semantic segmentation model inference result evaluation method and device. BACKGROUND

[0002] In recent years, with the rapid development of intelligent algorithms, the use of deep learning technology to identify concrete cracks has become a research hotspot in this field. Among them, model inference is one of the important links of deep learning algorithm, which can reflect the generalization ability and robustness of the model. For the crack target detection model, the inference result is in the form of rectangular frame label of predicted crack, which can be directly compared with the real crack. However, for the crack segmentation model, the inference result is a binary image of the predicted crack, which cannot be directly compared with the real crack. Even if the optimal F1 value, pixel accuracy, average intersection over union, recall rate and other parameters are used as evaluation indicators, only an overall evaluation result can be given, and the crack inference effect of each part cannot be finely evaluated. In addition, because the crack pixels in the crack image generally account for a small proportion, and the background pixels account for a large proportion, the above evaluation indicators also have errors caused by class imbalance. Therefore, it is necessary to find a fine evaluation method for the inference result of the crack semantic segmentation model. SUMMARY

[0003] The crack semantic segmentation model inference result evaluation method and device provided by the embodiments of the present application can improve the evaluation accuracy of the model inference result and improve the calculation efficiency. The technical solution is as follows:

[0004] On the one hand, a crack semantic segmentation model inference result evaluation method is provided, which is applied to an electronic device, and the method comprises the following steps:

[0005] S101, obtaining the inference result and the real result of the crack image by using a crack semantic segmentation model and a labeling software respectively;

[0006] S102, comparing the row of the crack in the inference result with the real result to determine whether the row of pixels is segmented correctly;

[0007] S103, when the row of pixels is segmented correctly, the next row of pixels is traversed in a loop; when the row of pixels is segmented incorrectly, all columns of pixels in the row are traversed in a loop, and compared with the real result to determine which column of pixels in the row is segmented incorrectly;

[0008] S104, repeating steps S102 to S103 until all rows of cracks are traversed;

[0009] S105, determining the distribution of pixels segmented incorrectly in all rows of cracks.

[0010] Further, the obtaining of the inference result and the real result of the crack image by using the crack semantic segmentation model and the labeling software respectively comprises:

[0011] The crack semantic segmentation model trained by the crack data set performs inference on the crack image; wherein, the model inference result is a predicted crack binary image;

[0012] The crack pixels in the predicted crack binary image are converted into a first preset value, and the background pixels are converted into a second preset value;

[0013] The crack image is polygonally labeled by using the labeling software to generate a crack binary mask image, and the crack pixels in the generated crack binary mask image are converted into a first preset value, and the background pixels are converted into a second preset value.

[0014] Further, when the included angle between the crack and the horizontal direction is less than a preset threshold, the row where the crack is located is replaced by the column where the crack is located, that is, the column where the crack is located is compared with the real result.

[0015] Further, the comparison of the row where the crack is located in the inference result with the real result to determine whether the pixels in the row are segmented correctly comprises:

[0016] The comparison of the row where the crack is located in the predicted crack binary image with the real result is to compare the encoding of the pixels in the whole row, that is, to determine whether the encoding of the i-th row of pixels P i in the predicted crack binary image is equal to the encoding of the i-th row of pixels T i in the real result;

[0017] If they are equal, the i-th row of pixels is segmented correctly;

[0018] If they are not equal, the i-th row of pixels is segmented incorrectly.

[0019] Further, the pixel segmentation error means that the background pixel point is misrecognized as a crack pixel point, or the crack pixel point is misrecognized as a background pixel point.

[0020] Further, when the row of pixels is segmented incorrectly, all the column pixels in the row are cyclically traversed and compared with the real result to determine which column of pixels in the row is segmented incorrectly, comprising:

[0021] When the row of pixels is segmented incorrectly, it is determined whether the j-th column of pixels P ij in the i-th row of the predicted crack binary image is equal to the j-th column of pixels T ij in the i-th row of the real result;

[0022] If they are equal, the i-th row of pixels is segmented correctly

[0023] If they are not equal, the i-th row of pixels is segmented incorrectly.

[0024] Further, the repeating step S102-step S103 until all rows of the crack are traversed to completion includes:

[0025] Repeating step S102-step S103, all rows of the crack are traversed and compared until the traversal is completed.

[0026] Further, the determination of the pixel distribution of the row segmentation error of all rows of the crack includes:

[0027] The pixel distribution of the row segmentation error of all rows of the crack is described in a statistical method, and the distribution characteristics of the pixel points of each row segmentation error are displayed in a point density map.

[0028] Further, the statistical method includes: the upper and lower limit interval, the mean and the coefficient of variation of the row segmentation error of all rows of the crack, and the number of rows corresponding to the segmentation error pixel points.

[0029] In one aspect, an evaluation device for inference results of a crack semantic segmentation model is provided, comprising:

[0030] An obtaining module is configured to obtain inference results and true results of a crack image using a crack semantic segmentation model and a labeling software, respectively.

[0031] A first judging module is configured to compare the row of the crack in the inference results with the true results, and determine whether the row of pixels is segmented correctly.

[0032] A second judging module is configured to loop through the next row of pixels when the row of pixels is segmented correctly, and loop through all columns of pixels in the row and compare them with the true results when the row of pixels is segmented incorrectly, to determine which column of pixels in the row is segmented incorrectly.

[0033] A traversal module is configured to repeat step S102-step S103 until all rows of the crack are traversed to completion.

[0034] A determination module is configured to determine the pixel distribution of the row segmentation error of all rows of the crack.

[0035] In one aspect, an electronic device is provided, comprising a processor and a memory, the memory having at least one instruction stored therein, the at least one instruction being loaded and executed by the processor to implement the above-mentioned evaluation method for inference results of a crack semantic segmentation model.

[0036] In one aspect, a computer-readable storage medium is provided, the storage medium having at least one instruction stored therein, the at least one instruction being loaded and executed by a processor to implement the above-mentioned evaluation method for inference results of a crack semantic segmentation model.

[0037] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0038] 1) The crack row pixel of the model inference result and the real result is analyzed and compared in detail, so that the evaluation accuracy of the model inference result is improved, and the evaluation efficiency is improved;

[0039] 2) The distribution characteristics of the error pixel points of each row of the crack row of the inference result are intuitively displayed in the form of point density map visualization, and the method can intuitively reflect the robustness of the model. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical scheme in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 The flowchart of the evaluation method of the crack semantic segmentation model inference result provided by the embodiment of the present application is shown.

[0042] Figure 2 The detailed flowchart of the evaluation method of the crack semantic segmentation model inference result provided by the embodiment of the present application is shown.

[0043] Figure 3 The original drawing of the crack image provided by the embodiment of the present application is shown.

[0044] Figure 4 The result diagram of the crack semantic segmentation model inference provided by the embodiment of the present application is shown.

[0045] Figure 5 The real result diagram provided by the embodiment of the present application is shown.

[0046] Figure 6 The crack image row error pixel density visualization diagram provided by the embodiment of the present application is shown.

[0047] Figure 7 The structure diagram of an electronic device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail with reference to the drawings.

[0049] As Figure 1As shown, the embodiment of the present application provides an evaluation method for the inference result of the crack semantic segmentation model, which is more accurate and efficient, and can be implemented by an electronic device, which can be a terminal or a server, and the method comprises the following steps:

[0050] In S101, pre-processing: the inference result and the true result of the crack image are obtained by using the crack semantic segmentation model and the labeling software respectively; specifically, the following steps can be included:

[0051] A1, the crack semantic segmentation model trained by the crack data set is used to infer the crack image; wherein the model inference result is a predicted crack binary image;

[0052] In this embodiment, the crack semantic segmentation model is a DeepLabv3+ segmentation model trained by the crack data set, but it is not limited to this model.

[0053] A2, the crack pixels in the predicted crack binary image are converted into a first preset value, and the background pixels are converted into a second preset value;

[0054] In this embodiment, the first preset value is 1, and the second preset value is 0, i.e., the crack pixels are 1 and the background pixels are 0.

[0055] A3, the crack image is polygon labeled by using the labeling software, a crack binary mask image is generated, and the crack pixels in the generated crack binary mask image are converted into the first preset value and the background pixels are converted into the second preset value.

[0056] In S102, the pixels in the whole row are compared quickly: the crack in the inference result is compared with the true result to quickly judge whether the pixels in the row are segmented correctly; for example, Figure 2 As shown, the following steps can be included:

[0057] B1, the crack in the predicted crack binary image is compared with the true result, which is to compare the encoding of the pixels in the whole row, i.e., to judge whether the encoding of the i-th row of pixels P i in the predicted crack binary image is equal to the encoding of the i-th row of pixels T i in the true result;

[0058] B2, if they are equal, the i-th row of pixels is segmented correctly;

[0059] B3, if they are not equal, the i-th row of pixels is segmented incorrectly.

[0060] In this embodiment, the crack in the predicted crack binary image is compared with the true result, which is to compare the encoding of the pixels in the whole row, rather than to traverse and compare each pixel one by one, which can greatly improve the calculation efficiency.

[0061] In this embodiment, the row where the crack is located can also be the column where the crack is located. When the crack is approximately horizontal (specifically, the angle between the crack and the horizontal direction is less than a preset threshold), the column where the crack is located should be compared with the true value. In order to briefly and conveniently summarize this embodiment, the row where the crack is located is uniformly described in the following.

[0062] S103, when the row pixel segmentation is correct, the next row of pixels is looped; when the row pixel segmentation is incorrect, all columns of pixels in the row are looped and compared with the true result to accurately determine which column of pixels in the row is incorrectly segmented; as shown in Figure 2 , which can specifically include the following steps:

[0063] C1, when the row pixel segmentation is incorrect, it is judged whether the jth column of pixels P ij in the ith row of the predicted crack binary image is equal to the jth column of pixels T ij in the ith row of the true result.

[0064] C2, if equal, the jth column of pixels in the ith row is correctly segmented

[0065] C3, if not equal, the jth column of pixels in the ith row is incorrectly segmented.

[0066] In this embodiment, the pixel segmentation error means that the background pixel point is misidentified as a crack pixel point, or the crack pixel point is misidentified as a background pixel point.

[0067] S104, repeat steps S102-S103 until all rows of cracks are traversed.

[0068] In this embodiment, steps S102-S103 are repeated to compare all rows with crack pixels until the traversal is completed, which can greatly improve the calculation efficiency.

[0069] S105, determine the pixel distribution of all rows of cracks with segmentation errors.

[0070] In this embodiment, the pixel distribution of all rows of cracks with segmentation errors is described by a metrological statistical method, and the distribution characteristics of the pixel points with segmentation errors in each row are intuitively displayed in a point density map.

[0071] In this embodiment, the metrological statistical method includes but is not limited to the upper and lower limit intervals, the mean and the coefficient of variation of the row segmentation errors of all rows of cracks, and the number of rows corresponding to the pixel points with segmentation errors.

[0072] In order to verify the effectiveness of the evaluation method of the crack semantic segmentation model inference result according to the embodiments of the application, a specific embodiment is used for illustration, as shown in Figure 3 , which can specifically include the following steps: Figure 3The crack image resolution shown is 640*640, and the inference result and the true result of the crack image are obtained by using a crack semantic segmentation model based on DeepLabv3+ and annotation software, respectively, as shown in Figure 4 、 Figure 5 The evaluation method of the inference result of the crack semantic segmentation model described in the embodiment of the present application obtains the upper and lower limit intervals of the row segmentation error of all rows of the crack as [0, 4] pixels, the mean value is 1.19 pixels, and the coefficient of variation is 0.81. The number of rows corresponding to 0 pixels, 1 pixel, 2 pixels, 3 pixels, and 4 pixels of segmentation error pixels is 135, 214, 122, 43, and 9 rows, respectively, as shown in Figure 6 The crack segmentation error point density map visualizes the distribution characteristics of the segmentation error pixel points of each row of the crack. The evaluation result shows that the crack semantic segmentation model has high accuracy, small error, and good robustness in the inference result of the crack image.

[0073] In summary, the evaluation method of the inference result of the crack semantic segmentation model described in the embodiment of the present application has at least the following effects:

[0074] 1) The crack pixels in the model inference result and the true result are analyzed and compared in detail, which can improve the evaluation accuracy of the model inference result while improving the evaluation efficiency;

[0075] 2) The distribution characteristics of the segmentation error pixel points of each row of the crack in the inference result are intuitively displayed in the form of point density map visualization, which can intuitively reflect the robustness of the model.

[0076] The present application also provides a specific embodiment of an evaluation device for the inference result of a crack semantic segmentation model. Since the evaluation device for the inference result of the crack semantic segmentation model provided by the present application corresponds to the specific embodiment of the evaluation method for the inference result of the crack semantic segmentation model described above, the evaluation device for the inference result of the crack semantic segmentation model can achieve the purpose of the present application by executing the process steps in the specific embodiment of the method described above. Therefore, the explanations and descriptions in the specific embodiment of the method described above are also applicable to the specific embodiment of the evaluation device for the inference result of the crack semantic segmentation model provided by the present application, and will not be repeated in the following specific embodiments of the present application.

[0077] The embodiment of the present application also provides an evaluation device for the inference result of a crack semantic segmentation model, comprising:

[0078] An obtaining module is configured to obtain the inference result and the true result of the crack image by using a crack semantic segmentation model and annotation software, respectively;

[0079] The first judging module is used for comparing the row of the crack in the inference result with the real result to judge whether the row of pixels is segmented correctly or not.

[0080] The second judging module is used for cyclically traversing the next row of pixels when the row of pixels is segmented correctly, and cyclically traversing all column pixels of the row and comparing with the real result to judge which column of pixels of the row is segmented incorrectly when the row of pixels is segmented incorrectly.

[0081] The traversing module is used for repeating steps S102-S103 until all rows of the crack are traversed.

[0082] The determining module is used for determining the distribution of the pixels segmented incorrectly in all rows of the crack.

[0083] The crack semantic segmentation model inference result evaluation device provided in the embodiment has at least the following beneficial effects:

[0084] 1) The model inference result and the real result are finely analyzed and compared in the row of the crack, so that the evaluation accuracy of the model inference result can be improved, and the evaluation efficiency can be improved.

[0085] 2) The row segmented incorrectly pixel point distribution characteristics of the row of the crack in the inference result are intuitively displayed in the point density map visualization mode, and the method can intuitively reflect the robustness of the model.

[0086] Figure 7 is a structural schematic diagram of an electronic device 600 provided in the embodiment, the electronic device 600 can have relatively large differences due to different configurations or performances, and can include one or more processors (central processing units, CPUs) 601 and one or more memories 602, wherein the memory 602 stores at least one instruction, the at least one instruction is loaded and executed by the processor 601 to realize the crack semantic segmentation model inference result evaluation method described above.

[0087] In the exemplary embodiment, a computer readable storage medium, for example, a memory including instructions, is also provided, and the above-mentioned instructions can be executed by the processor in the terminal to complete the crack semantic segmentation model inference result evaluation method. For example, the computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0088] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete related hardware, and the program can be stored in a computer readable storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0089] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for evaluating inference results of a crack semantic segmentation model, characterized in that, The method comprises the following steps: S101, obtaining inference results and true results of a crack image by using a crack semantic segmentation model and annotation software, respectively; S102, comparing the row where the crack is located in the inference results with the true results to determine whether the row of pixels is segmented correctly; S103, when the row of pixels is segmented correctly, the next row of pixels is looped through; when the row of pixels is segmented incorrectly, all columns of pixels in the row are looped through and compared with the true results to determine which column of pixels in the row is segmented incorrectly; S104, repeating steps S102-S103 until all rows of cracks are traversed; S105, determining the distribution of pixels segmented incorrectly in all rows of cracks; The method comprises the following steps: The crack semantic segmentation model trained by the crack data set performs inference on the crack image; the model inference result is a predicted crack binary image; The crack pixels in the predicted crack binary image are converted into a first preset value, and the background pixels are converted into a second preset value; The crack image is polygonally annotated by using the annotation software to generate a crack binary mask image, and the crack pixels in the generated crack binary mask image are converted into a first preset value, and the background pixels are converted into a second preset value; The method comprises the following steps: The line where the crack in the predicted crack binarization image is located is compared with the true result, which is to compare the encoding of the entire row of pixels, that is, to judge whether the encoding of the i-th row of pixels P i of the predicted crack binarization image is equal to the encoding of the i-th row of pixels T i of the true result; If they are equal, the i-th row of pixels is segmented correctly; If they are not equal, the i-th row of pixels is segmented incorrectly.

2. The method of claim 1, wherein the method further comprises: When the included angle between the crack and the horizontal direction is less than a preset threshold, the row where the crack is located is replaced by the column where the crack is located, that is, the column where the crack is located is compared with the true results.

3. The method of claim 1, wherein the method further comprises: The pixel segmentation error means that the background pixel points are misrecognized as crack pixel points, or the crack pixel points are misrecognized as background pixel points.

4. The method of claim 1, wherein the method further comprises: The method comprises the following steps: When the row pixel is partitioned incorrectly, it is determined whether the jth column pixel P of the ith row in the prediction crack binary image is equal to the jth column pixel T of the ith row of the true result ij ij whether the jth column pixel P of the ith row in the prediction crack binary image is equal to the jth column pixel T of the ith row of the true result​ If they are equal, the i-th row of pixels is segmented correctly If they are not equal, the i-th row of pixels is segmented incorrectly.

5. The method of claim 1, wherein the method further comprises: The method comprises the following steps: Steps S102-S103 are repeated until all rows of cracks are traversed.

6. The method of claim 1, wherein the method further comprises: The method comprises the following steps: The distribution of pixels segmented incorrectly in all rows of cracks is described by a metrological statistical method, and the distribution characteristics of pixel points segmented incorrectly in each row are displayed in a visual manner of a point density map.

7. The method of claim 6, wherein the method further comprises: The metrological statistical method comprises the upper and lower limit intervals, the mean value and the coefficient of variation of the row segmented incorrectly in all rows of cracks, and the number of rows corresponding to the pixel points segmented incorrectly.

8. An apparatus for evaluating inference results of a crack semantic segmentation model, characterized in that, The method comprises the following steps: An obtaining module is configured to obtain inference results and true results of a crack image by using a crack semantic segmentation model and annotation software, respectively; A first determining module is configured to compare the row where the crack is located in the inference results with the true results to determine whether the row of pixels is segmented correctly; The second judging module is configured to, when the row of pixels is correctly segmented, loop through a next row of pixels; and when the row of pixels is incorrectly segmented, loop through all the pixels in the column of the row and compare with the real result to determine which column of pixels in the row is incorrectly segmented; The iterating module is configured to repeat the step S102 and the step S103 until all the rows of the crack are iterated. The determining module is configured to determine the distribution of the incorrectly segmented pixels of all the rows of the crack. The obtaining module is configured to: infer the crack image by using the crack semantic segmentation model trained by the crack data set; the model inference result is a predicted crack binary image; convert the crack pixels in the predicted crack binary image into a first preset value and convert the background pixels into a second preset value; generate a crack binary mask image by polygon labeling of the crack image by using a labeling software, and convert the crack pixels in the generated crack binary mask image into the first preset value and convert the background pixels into the second preset value. The first judging module is used for comparing the row where the crack in the predicted crack binary image is located with the true result, i.e. comparing the encoding of the i-th row pixel P i of the predicted crack binary image with the encoding of the i-th row pixel T i of the true result, to determine whether they are equal; if yes, the i-th row pixel is segmented correctly; if not, the i-th row pixel is segmented incorrectly.

Images