Road engineering quality detection method and system based on image recognition

Abstract

The invention provides a highway engineering quality detection method and system based on image recognition. The method comprises the following steps: acquiring a pavement image of a target highway; performing crack identification on the pavement image to obtain a crack boundary of each surface crack in the pavement image; determining spatio-temporal evolution characteristics of each surface crack; performing trend extrapolation on each surface crack based on all crack boundaries and the spatio-temporal evolution characteristics to obtain expansion confidence of each surface crack on each potential expansion point; and determining the risk level of each surface crack according to all the extension confidence coefficients, and outputting a crack risk assessment result of the target road. By adopting the scheme of the invention, the surface crack in the pavement image can be accurately positioned, and then the expansion speed and direction of the surface crack are dynamically tracked and the risk level is evaluated according to the characteristics of the surface crack, so that the safety of highway engineering is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision engineering detection, and more specifically, to a highway engineering quality detection method and system based on image recognition. Background Art

[0002] Highway engineering quality inspection is an important part of ensuring that highway engineering construction meets design standards and safety requirements, including inspections of cracks, displacement, settlement, etc. Among them, surface cracks are the early manifestation of highway diseases. If they can be detected and repaired in the early stage, the losses caused by road diseases can be significantly reduced. Therefore, the identification and tracking of highway surface cracks is an important part of ensuring that highway engineering construction meets design standards and safety requirements; computer vision engineering inspection refers to the use of computer vision technology to detect, analyze and evaluate the quality and status of engineering projects. By combining computer vision with engineering inspection, engineering images are collected, processed and analyzed to automatically identify and detect potential problems in engineering structures, such as cracks, deformation, damage, wear, etc.

[0003] At present, the detection of surface cracks at the on-site end of computer vision engineering inspection operations in my country is mainly carried out by detecting crack edges through operators such as Canny and Sobel, extracting their geometric features, and combining morphological operations such as corrosion and expansion to remove noise and connect broken edges. Improve the continuity of crack detection, thereby achieving accurate identification of cracks on the highway surface. However, most existing methods focus on crack detection in a single period of time, and the prediction of crack expansion trends is relatively weak. It is impossible to predict the expansion speed and direction of cracks, and then evaluate the risk level. Therefore, how to improve the accuracy of risk level assessment of surface cracks in road surface images to ensure the safety and durability of highway projects has become a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides a highway engineering quality detection method and system based on image recognition, which can improve the accuracy of risk level assessment of surface cracks in the road surface image, thereby ensuring the safety and durability of the highway engineering.

[0005] In a first aspect, the present application provides a highway engineering quality detection method based on image recognition, comprising: Collecting road surface images of the target highway; Performing crack recognition on the road surface image to obtain connected domains of each surface crack in the road surface image, and then performing edge detection on the surface cracks according to all connected domains to obtain crack boundaries of each surface crack in the road surface image; Acquire crack images of each surface crack at each detection time point in the quality inspection stage, and determine the spatiotemporal evolution characteristics of each surface crack according to edge information of each crack image; Determine all potential extension points of each surface crack through gradient change information of the surface crack in each crack image, perform trend extrapolation on each surface crack based on all crack boundaries and the spatiotemporal evolution characteristics, and obtain the extension confidence of each surface crack at each potential extension point; The risk level of each surface crack is determined according to all the extended confidences, and then the crack risk assessment result of the target highway is output.

[0006] In some embodiments, performing crack recognition on the road surface image to obtain a connected domain of each surface crack in the road surface image specifically includes: Performing grayscale processing on the road surface image to obtain a grayscale highway image; Obtaining a crack distribution map on the road surface image by performing semantic segmentation on the grayscale road image; Connected domain detection is performed on the crack distribution map to obtain connected domains of each surface crack in the road surface image.

[0007] In some embodiments, performing edge detection on the surface cracks according to all connected domains to obtain crack boundaries of each surface crack in the road surface image specifically includes: Selecting a surface crack in the road surface image as a selected surface crack, and performing smoothing on a connected domain of the selected surface crack to obtain a smoothed connected domain of the selected surface crack; Determine the edge gradient of each pixel in the smooth connected domain; Performing double threshold detection on the smooth connected domain according to all edge gradients to obtain crack boundaries of the selected surface cracks; Continue to determine the crack boundaries for the remaining surface cracks.

[0008] In some embodiments, determining the spatiotemporal evolution characteristics of each surface crack according to the edge information of each crack image specifically includes: Selecting a surface crack in the road surface image as a selected surface crack, performing edge tracking on the selected surface crack according to each crack image, and obtaining an edge information sequence of the selected surface crack; Determine the spatiotemporal distribution domain of the selected surface crack edge based on all edge information in the edge information sequence; Extracting features from the spatiotemporal distribution domain to obtain time-speed trend features and spatial trend features of the edge of the selected surface crack; The spatiotemporal evolution characteristics of the selected surface cracks are determined based on the speed trend characteristics and the spatial trend characteristics, and the spatiotemporal evolution characteristics of the remaining surface cracks are further determined.

[0009] In some embodiments, determining all potential extension points of each surface crack through gradient change information of the surface crack in each crack image specifically includes: Selecting a surface crack in the road surface image as a selected surface crack, and determining a plurality of abnormal change points on the edge of the selected surface crack according to all gradient change information; Determine the gradient direction of each abnormal change point; Determine the gradient similarity of the corresponding abnormal change point based on all pixel points in the gradient direction of each abnormal change point; All potential extension points of the selected surface cracks are determined based on all gradient similarities, and all potential extension points of the remaining surface cracks are further determined.

[0010] In some embodiments, performing trend extrapolation on each surface crack based on all crack boundaries and the spatiotemporal evolution characteristics to obtain the expansion confidence of each surface crack at each potential expansion point specifically includes: Perform feature extraction on all crack boundaries to obtain contour boundary features of all surface cracks; Selecting a surface crack in the road surface image as a selected surface crack, and determining an extension direction constraint of the selected surface crack at each potential extension point through the contour boundary feature and the spatial trend in the spatiotemporal evolution feature; Determining the expansion velocity constraint of the selected surface crack during the expansion process according to the velocity trend in the spatiotemporal evolution characteristics; Based on all the extension direction constraints and the extension speed constraints, the selected surface cracks are trend predicted to obtain the extension confidence of the selected surface cracks at each potential extension point, and then the extension confidence of the remaining surface cracks at each potential extension point is obtained.

[0011] In some embodiments, determining the risk level of each surface crack according to all the extended confidences may be achieved by using the following steps: Selecting a surface crack in the road surface image as a selected surface crack, and extrapolating a preliminary prediction curve of the selected surface crack through all extended confidence levels; Performing boundary correction on the preliminary prediction curve according to the boundary conditions of the target highway to obtain an edge extrapolation curve of the selected surface crack on the edge extension; The risk level of the selected surface cracks on the regional surface image is determined by the edge extrapolation curve, and the risk levels of the remaining surface cracks are further determined.

[0012] In a second aspect, the present application provides a highway engineering quality detection system based on image recognition, comprising: A collection module, used for collecting road surface images of a target highway; A processing module, configured to perform crack recognition on the pavement image to obtain connected domains of various surface cracks in the pavement image, and then perform edge detection on the surface cracks according to all connected domains to obtain crack boundaries of various surface cracks in the pavement image; The processing module is used to obtain crack images of each surface crack at each detection time point in the quality inspection stage, and determine the spatiotemporal evolution characteristics of each surface crack according to edge information of each crack image; The processing module is used to determine all potential extension points of each surface crack through the gradient change information of the surface crack in each crack image, and to perform trend extrapolation on each surface crack based on all crack boundaries and the spatiotemporal evolution characteristics to obtain the extension confidence of each surface crack at each potential extension point; The execution module is used to determine the risk level of each surface crack according to all the extended confidences, and then output the crack risk assessment result of the target highway.

[0013] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned highway engineering quality detection method based on image recognition.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned highway engineering quality inspection method based on image recognition is implemented.

[0015] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects: In the highway engineering quality detection method and system based on image recognition provided by the present application, first, a pavement image of a target highway is collected; crack recognition is performed on the pavement image to obtain a connected domain of each surface crack in the pavement image, and then edge detection is performed on the surface cracks based on all connected domains to obtain crack boundaries of each surface crack in the pavement image; crack images of each surface crack at each detection time point in the quality inspection stage are obtained, and the spatiotemporal evolution characteristics of each surface crack are determined based on the edge information of each crack image; all potential extension points of each surface crack are determined through the gradient change information of the surface crack in each crack image, all potential extension points of each surface crack are determined through the gradient change information of the surface crack in each crack image, trend extrapolation is performed on each surface crack based on all crack boundaries and the spatiotemporal evolution characteristics to obtain the extension confidence of each surface crack at each potential extension point, the risk level of each surface crack is determined based on all extension confidences, and then the crack risk assessment result of the target highway is output.

[0016] It can be seen that the present application determines the risk level of the selected surface cracks on the pavement image through the edge extrapolation curve; firstly, the crack boundary is determined to obtain the edge contour representing the shape and structural characteristics of the surface cracks on the pavement image, thereby providing basic data for subsequent crack expansion tracking, helping to identify the direction and speed of crack expansion, and predicting the development trend of cracks; secondly, the spatiotemporal evolution characteristics are determined to obtain trend information on the development direction and speed of surface cracks, thereby being able to capture the dynamic changes of surface cracks, and predict the future development of cracks by combining the dimensions of time and space, thereby more effectively quantifying key factors such as the expansion rate, direction, and morphological changes of surface cracks, making the prediction of surface crack expansion more accurate and Comprehensive; finally, the expansion confidence of the selected surface cracks at each potential expansion point can be determined. The edge extrapolation curve of the selected surface cracks on the edge expansion can be determined according to all the expansion confidences. The edge extrapolation curve is used to show the predicted trajectory of the expansion trend of the surface cracks in a specified future time period, which helps to identify surface cracks with faster expansion speed and direction tending to important structural areas and evaluate the risk level of surface cracks, so as to give priority to these surface cracks and prevent potential structural damage; In summary, based on the above scheme, the surface cracks in the pavement image can be accurately located, and then the expansion speed and direction of the surface cracks can be dynamically tracked and the risk level can be evaluated according to the characteristics of the surface cracks, so as to ensure the safety of highway projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an exemplary flow chart of a highway engineering quality detection method based on image recognition according to some embodiments of the present application; Figure 2is an exemplary flow chart of determining spatiotemporal evolution characteristics according to some embodiments of the present application; Figure 3 is an exemplary flow chart of determining an extended confidence level according to some embodiments of the present application; Figure 4 is a schematic diagram of the structure of a highway engineering quality inspection system based on image recognition according to some embodiments of the present application; Figure 5 It is a structural schematic diagram of a computer device for implementing a highway engineering quality inspection method based on image recognition according to some embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0019] refer to Figure 1 , which is an exemplary flow chart of a highway engineering quality detection method based on image recognition according to some embodiments of the present application. The highway engineering quality detection method 100 based on image recognition mainly includes the following steps: In step 101, a road surface image of a target highway is collected.

[0020] It should be noted that, in the present application, the pavement image refers to a two-dimensional image of the surface of the target highway showing the characteristics of the highway such as the pavement material, cracks and damage; in specific implementation, for example, after the construction is completed, a drone with a high-resolution camera or other image acquisition equipment can be controlled to collect the surface image of the target highway to obtain the pavement image of the target highway.

[0021] In step 102, crack recognition is performed on the road surface image to obtain connected domains of various surface cracks in the road surface image, and then edge detection is performed on the surface cracks based on all connected domains to obtain crack boundaries of various surface cracks in the road surface image.

[0022] In some embodiments, crack recognition is performed on the road surface image to obtain the connected domain of each surface crack in the road surface image by using the following steps: Performing grayscale processing on the road surface image to obtain a grayscale highway image; Obtaining a crack distribution map on the road surface image by performing semantic segmentation on the grayscale road image; Connected domain detection is performed on the crack distribution map to obtain connected domains of each surface crack in the road surface image.

[0023] It should be noted that in this application, a grayscale highway image refers to a road surface image that contains only grayscale information; a crack distribution map refers to a binary image specifically used to represent the location and morphology of cracks on the target highway surface; and a connected domain refers to a continuous area composed of adjacent pixel points in a crack distribution map.

[0024] In addition, it should be noted that in the present application, the grayscale highway image can highlight the grayscale differences of the cracks by reducing the dimension of the image, which is convenient for subsequent crack identification; the connected domain represents a crack or part of a crack, and all independent crack areas in the crack distribution map can be identified through connected domain detection.

[0025] In specific implementation, first, the existing image grayscale processing method (for example, color channel weighted average method) can be used to grayscale the road image, and the processed image is used as the grayscale road image; then, a trained semantic segmentation model (for example, full convolutional network model) can be loaded, and the grayscale road image is used as the input of the semantic segmentation model. After running the semantic segmentation model, the output image of the semantic segmentation model is used as a probability map showing whether each pixel in the grayscale road image belongs to a crack area, and then the pixel points with probability values ​​higher than the preset probability threshold in the probability map are set to black (pixel value is 0), marked as crack areas, and the pixel points with probability values ​​lower than the preset probability threshold are set to white (pixel value is 255), marked as non-crack areas, to obtain a crack distribution map on the road image; finally, the pixel area composed of adjacent pixel points with pixel values ​​of 0 in the crack distribution map can be marked, and the corresponding area of ​​each connected pixel area in the grayscale road image in the above step is used as a connected domain of a surface crack on the road image, thereby obtaining the connected domain of each surface crack on the road image.

[0026] It should be noted that since the road surface image is obtained by photographing the target highway outdoors, when there are local contrast differences in the road surface image due to different lighting conditions, the existing local threshold processing method (for example, histogram analysis method) is used in this application to automatically set the probability threshold.

[0027] In some embodiments, edge detection of surface cracks is performed based on all connected domains to obtain crack boundaries of each surface crack in the road surface image by using the following steps: Selecting a surface crack in the road surface image as a selected surface crack, and performing smoothing on a connected domain of the selected surface crack to obtain a smoothed connected domain of the selected surface crack; Determine the edge gradient of each pixel in the smooth connected domain; Performing double threshold detection on the smooth connected domain according to all edge gradients to obtain crack boundaries of the selected surface cracks; Continue to determine the crack boundaries for the remaining surface cracks.

[0028] It should be noted that in the present application, the crack boundary refers to the edge contour of the structural features of the surface cracks in the pavement image; the smooth connected domain refers to a connected domain with smoother boundaries, reduced noise and more uniform shape; the edge gradient refers to the amplitude of the pixel value change of each pixel point in the image, which can be used to detect the edge position of the corresponding surface cracks in the pavement image.

[0029] In the specific implementation, first, a surface crack is selected in the road surface image as the selected surface crack. For each pixel point in the connected domain of the selected surface crack, the average value of the pixel point and the adjacent pixel points in pixel value is used as the new pixel value of the pixel point. The new pixel value of each pixel point in the connected domain can be obtained by the above method, and then the processed connected domain is used as the smooth connected domain of the selected surface crack; secondly, for each pixel point in the smooth connected domain, the existing edge detection operator (for example: Sobel operator) can be used to calculate the horizontal gradient of the pixel point in the horizontal direction and the vertical gradient in the vertical direction, and the square root of the sum of the squares of the horizontal gradient and the vertical gradient is used as the edge gradient of the pixel point. The edge gradient of each pixel point in the smooth connected domain can be obtained by the above method; then, 0.3 times the edge gradient with the largest value among all edge gradients is used as the high threshold, and 0.5 times the high threshold is used as the low threshold. For each pixel point in the smooth connected domain, the pixel value of the pixel point is compared with the high threshold and the low threshold. If the pixel value is greater than the high threshold, the pixel point is marked as a strong edge. If the pixel value is between the high threshold and the low threshold, the pixel point is marked as a weak edge. If the pixel value is less than the low threshold, the pixel point is marked as a non-edge. In the above manner, all pixel points in the smooth connected domain can be marked, and then all pixel points marked as weak edges are traversed. If there is a strong edge among the adjacent pixel points of the pixel point, the pixel point is marked as a strong edge. If there is no strong edge among the adjacent pixel points of the pixel point, the pixel point is marked as a weak edge. Then all pixel points marked as strong edges are connected, and the obtained contour is used as the crack boundary of the selected surface crack; finally, the crack boundaries of the remaining surface cracks are continued to be determined to obtain the crack boundaries of each surface crack on the road surface image; it should be noted that in the present application, adjacent pixel points refer to pixel points in the eight neighborhoods of pixel points in the connected domain.

[0030] In step 103, a crack image of each surface crack at each detection time point in the quality inspection stage is obtained, and the spatiotemporal evolution characteristics of each surface crack are determined according to the edge information of each crack image.

[0031] It should be noted that, in the present application, the crack image is a two-dimensional image containing surface cracks on the pavement image collected during the quality inspection stage of the target highway project; in specific implementation, one surface crack can be selected from all the surface cracks as the selected surface crack, and the position of the selected surface crack can be obtained in the crack distribution map of the above step, and then according to the position of the selected surface crack, the surface image of the specified area of ​​the target highway at each detection time point in the pre-collected quality inspection stage of the target highway project is intercepted to obtain the crack image of the selected surface crack at each detection time point in the quality inspection stage, and then continue to obtain the crack image of the remaining surface cracks at each detection time point in the quality inspection stage, wherein the detection time point is the time set for detecting the target highway in the quality inspection stage of the target highway project, which can be set to detect the target highway once every half a month. In other embodiments, it can also be set to other times, which are not specifically limited here.

[0032] It should be noted that, in the present application, the crack images are arranged adjacent to each other in the order of the various detection time points in the quality inspection stage of the target highway project.

[0033] In some embodiments, reference Figure 2 , which is an exemplary flow chart of determining the spatiotemporal evolution characteristics according to some embodiments of the present application. In the present application, the spatiotemporal evolution characteristics of each surface crack can be determined according to the edge information of each crack image by using the following steps: In step 1031, a surface crack is selected in the road surface image as a selected surface crack, and edge tracking is performed on the selected surface crack according to each crack image to obtain an edge information sequence of the selected surface crack; In step 1032, the spatiotemporal distribution domain of the selected surface crack edge is determined based on all edge information in the edge information sequence; In step 1033, feature extraction is performed on the temporal and spatial distribution domain to obtain the temporal and spatial trend features of the selected surface crack edge; In step 1034, the spatiotemporal evolution characteristics of the selected surface cracks are determined based on the speed trend characteristics and the spatial trend characteristics, and the spatiotemporal evolution characteristics of the remaining surface cracks are further determined.

[0034] It should be noted that, in the present application, the spatiotemporal distribution domain is the distribution domain of the changes in the selected surface cracks in the time and space dimensions; the speed trend characteristics are the characteristics of the changing law of the expansion speed of the surface cracks over time; the spatiotemporal trend characteristics are the characteristics of the changing law of the expansion direction of the surface cracks over time; the spatiotemporal evolution characteristics are used to characterize the trend characteristics of the surface cracks in the development direction and speed.

[0035] In a specific implementation, first, a surface crack can be selected in the pavement image as the selected surface crack, and the edge of the selected surface crack is tracked according to each crack image to obtain an edge information sequence of the selected surface crack; secondly, the time point of the engineering stage corresponding to the crack image is added to the coordinates of the edge point of the surface crack on each crack image in the edge information sequence to obtain a three-dimensional coordinate containing position information and time information, and then all the three-dimensional coordinates are mapped to the three-dimensional space coordinate system to obtain the spatiotemporal distribution domain of the edge of the selected surface crack; then, a time point is selected in the spatiotemporal distribution domain as the selected time point, the moving speed of the selected surface crack between the selected time point and the next time point is determined, and the curvature of the selected surface crack at the selected time point is calculated, and the moving speed of the selected surface crack between the remaining time points and the next time point in the spatiotemporal distribution domain and the curvature of the selected surface crack at the selected time point are continued to be determined. curvature, and then use the sequence composed of all moving speeds in time order as the speed trend feature of the selected surface cracks, and use the sequence composed of all curvatures in time order as the spatial trend feature of the selected surface cracks; finally, load an existing time series model (such as an autoregressive model), use the speed trend feature as the input of the time series model, and after executing the time series model, use the predicted sequence in the output result of the time series model as the speed trend of the selected surface cracks, and use the spatial trend feature as the input of the time series model, and after executing the time series model, use the predicted sequence in the output result of the time series model as the spatial trend of the selected surface cracks, and then use the set of the speed trend and the spatial trend as the spatiotemporal evolution feature of the selected surface cracks, continue to determine the spatiotemporal evolution features of the remaining surface cracks, and obtain the spatiotemporal evolution features of each surface crack.

[0036] It should be noted that, in the present application, the speed trend is a time series that describes the change of the expansion speed of the surface crack edge over time; the spatial trend is a time series that reflects the change of the main direction of the surface crack expansion over time.

[0037] In the above embodiment, determining the moving speed of the selected surface crack between the selected time point and the next time point can be achieved in the following manner, namely: for each edge point on the selected surface crack, the square root of the sum of the square of the horizontal position coordinate difference and the square of the vertical position coordinate difference between the edge point at the selected time point and the next time point is taken as the displacement of the edge point between the two time points, and then the ratio of the displacement to the time difference between the two time points is taken as the speed of the edge point between the two time points. The speed of each edge point on the selected surface crack between the selected time point and the next time point can be obtained in the above manner, and then the average value of all speeds is taken as the moving speed of the selected surface crack between the selected time point and the next time point.

[0038] Preferably, in some embodiments, edge tracking of the selected surface cracks is performed according to each crack image to obtain edge information sequences of the selected surface cracks, which can be achieved by using the following steps: For each crack image of the selected surface crack, determining gradient change information of the selected surface crack in the crack image; Determine the tracking starting point of the upper edge of the selected surface crack according to the gradient change information; Tracking the shape change of the edge of the selected surface crack from the tracking starting point to obtain edge information of the selected surface crack in the crack image, and then obtaining edge information of the selected surface crack in each crack image; The edge information sequence of the selected surface crack is determined according to all the edge information.

[0039] It should be noted that, in the present application, gradient change information refers to information reflecting the gradient change of the edge of the surface crack in the crack image; the tracking starting point refers to the initial point selected as the starting position for tracking the crack edge during the edge tracking process; the edge information sequence refers to a time series containing the edge information of the surface crack at different engineering stages, wherein the edge information refers to information reflecting the geometric shape and position of the edge of the surface crack.

[0040] In the specific implementation, first, for each crack image, the existing image binarization method (for example, the adaptive threshold method) can be used to binarize the crack image to obtain a binary image. For each pixel in the binary image, the existing edge detection operator (for example, the Sobel operator) can be used to calculate the horizontal gradient of the pixel in the horizontal direction and the vertical gradient in the vertical direction. The square root of the sum of the squares of the horizontal gradient and the vertical gradient is used as the gradient amplitude of the pixel, and the inverse tangent function value of the ratio of the horizontal gradient to the vertical gradient is used as the gradient direction of the pixel. The gradient amplitude and gradient direction of each pixel in the binary image can be obtained in the above manner, and the gradient amplitude and gradient direction of all black pixels are used as the gradient change information of the selected surface cracks in the crack image. The gradient change information of the selected surface crack in each crack image can be obtained in this way; secondly, the pixel point corresponding to the largest gradient amplitude in the gradient change information can be used as the tracking starting point of the upper edge of the selected surface crack in the crack image; then, the tracking starting point is used as an edge point of the selected crack, and the black pixel point closest to the edge point in the gradient direction is selected as the next edge point, and the edge points on the selected surface crack are moved forward point by point, and the coordinates, direction and length of each edge point are recorded, and the coordinates, directions and lengths of all edge points are used as the edge information of the selected surface crack in the crack image. The edge information of the selected surface crack in each crack image can be obtained in the above way; finally, all the edge information is stacked in sequence according to the arrangement order of the crack images to obtain the edge information sequence of the selected surface crack.

[0041] In step 104, all potential expansion points of each surface crack are determined through the gradient change information of the surface cracks in each crack image, and the trend of each surface crack is extrapolated based on all crack boundaries and the spatiotemporal evolution characteristics to obtain the expansion confidence of each surface crack at each potential expansion point.

[0042] In some embodiments, determining all potential extension points of each surface crack through the gradient change information of the surface crack in each crack image can be achieved by using the following steps: Selecting a surface crack in the road surface image as a selected surface crack, and determining a plurality of abnormal change points on the edge of the selected surface crack according to all gradient change information; Determine the gradient direction of each abnormal change point; Determine the gradient similarity of the corresponding abnormal change point based on all pixel points in the gradient direction of each abnormal change point; All potential extension points of the selected surface cracks are determined based on all gradient similarities, and all potential extension points of the remaining surface cracks are further determined.

[0043] It should be noted that, in the present application, an abnormal change point refers to a pixel point on the edge of a surface crack that exhibits a significant gradient change; a gradient direction refers to the direction in which the pixel intensity of the abnormal change point changes; gradient similarity is a parameter that describes the similarity in gradient change between a pixel point in the gradient direction of the abnormal change point and the abnormal change point; a potential extension point refers to a pixel point on the potential extension path of a surface crack.

[0044] In a specific implementation, first, a surface crack is selected in the pavement image as the selected surface crack. For each crack image of the selected surface crack, each pixel point on the edge of the selected surface crack in the crack image is traversed to obtain the gradient amplitude corresponding to the pixel point in the gradient change information on the edge of the surface crack in all crack images. If the average value of all gradient amplitudes is greater than a preset demarcation threshold, the pixel point is marked as an abnormal change point of the selected surface crack. In the above manner, it can be determined whether the pixel point on the edge of the selected surface crack in each crack image is an abnormal change point of the selected surface crack, and multiple abnormal change points on the edge of the selected surface crack are obtained; secondly, the gradient direction of each abnormal change point is obtained in the gradient change information of the selected surface crack; then, an abnormal change point is selected as the selected abnormal change point. For each pixel point in the gradient direction of the selected abnormal change point, the gradient amplitude and gradient direction of the pixel point are first obtained, and then 1 is subtracted from the value of the pixel point. The difference between the gradient amplitude and the gradient amplitude of the selected abnormal change point, the ratio of the difference obtained and the gradient amplitude of the selected abnormal change point is taken as the amplitude similarity, and the difference between the gradient direction of the pixel point and the gradient direction of the selected abnormal change point is subtracted from 1, the ratio of the difference obtained and the gradient direction of the selected abnormal change point is taken as the direction similarity, and the average value of the amplitude similarity and the direction similarity is taken as the gradient similarity between the pixel point and the selected abnormal change point. In the above manner, the gradient similarity between each pixel point in the gradient direction of the selected abnormal change point and the selected abnormal change point can be obtained, and the gradient similarity between each pixel point in the gradient direction of the remaining abnormal change points and the corresponding abnormal change point can be determined; finally, each pixel point with a gradient similarity greater than 0.8 is marked as a potential extension point of the selected surface crack, and all potential extension points of the selected surface crack are obtained, and all potential extension points of the remaining surface cracks are continued to be determined to obtain all potential extension points of each surface crack.

[0045] It should be noted that in the present application, the gradient direction reflects the direction in which the brightness changes most dramatically on the edge of the selected surface crack, that is, the possible extension direction of the selected surface crack; the demarcation threshold can be automatically set using an existing local threshold processing method (for example: histogram analysis method).

[0046] In some embodiments, reference Figure 3 , which is an exemplary flow chart of determining the expansion confidence according to some embodiments of the present application. In the present application, the trend of each surface crack is extrapolated based on all the crack boundaries and the spatiotemporal evolution characteristics to obtain the expansion confidence of each surface crack at each potential expansion point, which can be achieved by the following steps: In step 1041, feature extraction is performed on all crack boundaries to obtain contour boundary features of all surface cracks; In step 1042, a surface crack is selected in the road surface image as a selected surface crack, and an extension direction constraint of the selected surface crack at each potential extension point is determined by the contour boundary feature and the spatial trend in the spatiotemporal evolution feature; In step 1043, the expansion speed constraint of the selected surface crack during the expansion process is determined according to the time speed trend in the time-space evolution characteristics; In step 1044, a trend prediction is performed on the selected surface cracks based on all the extension direction constraints and the extension speed constraints to obtain the extension confidence of the selected surface cracks at each potential extension point, and then the extension confidence of the remaining surface cracks at each potential extension point is obtained.

[0047] It should be noted that, in the present application, the expansion confidence is a parameter that reflects the possibility that a potential expansion point on a surface crack in a specified future time period will become a pixel point on the surface crack during the surface crack expansion process; the contour boundary feature is a set of parameters that describe the morphological characteristics of the surface crack edge; the expansion direction constraint refers to the restriction condition on the expansion direction of the surface crack in the process of predicting the expansion of the surface crack; the expansion speed constraint refers to the restriction condition on the predicted expansion speed of the surface crack in a specified future time period.

[0048] In specific implementation, first, the fractal features of all crack boundaries can be analyzed using existing fractal feature analysis methods (for example, the box counting method) to obtain the fractal dimension of each crack boundary, and the set of fractal dimensions of all crack boundaries is used as the contour boundary feature of all surface cracks; secondly, a surface crack is selected in the pavement image as the selected surface crack, and for each potential expansion point on the selected surface crack, the gradient direction of the potential expansion point in the last crack image is obtained, and all predicted values ​​in a specified future time period are obtained in the spatial trend of the spatiotemporal evolution feature, and the fractal dimension of the selected surface gap in the contour boundary feature is multiplied by the mean of the gradient direction and all predicted values, and the obtained value is used as the expansion direction constraint of the selected surface crack at the potential expansion point. The expansion direction constraint of the selected surface crack at each potential expansion point can be obtained in the above manner; then, the speed trend of the spatiotemporal evolution feature can be used. All predicted values ​​in the specified future time period are obtained, and the maximum value of all predicted values ​​is used as the expansion speed constraint of the selected surface crack during the expansion process; finally, for each potential expansion point on the selected surface crack, the product of the expansion speed constraint and the cosine function value of the expansion direction constraint of the selected surface crack at the potential expansion point is used as the exponent of the natural exponential function, and the inverse of the sum of the obtained function value and 1 is used as the expansion confidence of the selected surface crack at the potential expansion point. The expansion confidence of the selected surface crack at each potential expansion point can be obtained by the above method, and the expansion confidence of the remaining surface cracks at each potential expansion point is further determined to obtain the expansion confidence of each surface crack at each potential expansion point. Among them, the fractal dimension is an indicator to measure the irregularity of the edge contour of the surface crack, which is helpful to predict the potential path of surface crack expansion. Surface cracks with high fractal dimensions will have more expansion directions and branches.

[0049] It should be noted that, in this application, the contour boundary features can help predict the expansion trend of surface cracks and provide a scientific basis for the quality inspection of the target highway.

[0050] In step 105, the risk level of each surface crack is determined according to all the extended confidences, and then the crack risk assessment result of the target highway is output.

[0051] It should be noted that, in the present application, the crack risk assessment result refers to a report recording the risk levels of all surface cracks on the target highway; in specific implementation, each surface crack on the target highway and its risk level can be recorded in a file, and the file can be output as the crack risk assessment result of the target highway.

[0052] In some embodiments, determining the risk level of each surface crack according to all the extended confidences may be achieved by using the following steps: Selecting a surface crack in the road surface image as a selected surface crack, and extrapolating a preliminary prediction curve of the selected surface crack through all extended confidence levels; Performing boundary correction on the preliminary prediction curve according to the boundary conditions of the target highway to obtain an edge extrapolation curve of the selected surface crack on the edge extension; The risk level of the selected surface cracks on the regional surface image is determined by the edge extrapolation curve, and the risk levels of the remaining surface cracks are further determined.

[0053] It should be noted that, in the present application, the risk level is a risk level for evaluating the potential threat of surface cracks to the overall safety of the target highway structure; the preliminary prediction curve is a preliminary estimated trajectory describing the expansion trend of the surface cracks in a specified future time period; the edge extrapolation curve refers to a predicted trajectory for showing the expansion trend of the surface cracks in a specified future time period, wherein the specified future time period refers to three years, and in other embodiments it may also be other times, which are not specifically limited here; In the specific implementation, first, a surface crack is selected in the pavement image as the selected surface crack, the crack boundary of the selected surface crack is obtained, and all potential extension points with extension confidence values ​​greater than 0.9 are used as future edge points, and then each future edge point is connected to the nearest edge point in the crack boundary to obtain the extension trajectory of the selected surface crack, and the extension trajectory is used as the preliminary prediction curve of the selected surface crack; then, the physical boundary conditions of the target highway are obtained, the part of the extension trajectory that exceeds the range of the physical boundary conditions in the preliminary prediction curve is removed, and the adjusted preliminary prediction curve is used as the edge extrapolation curve of the selected surface crack on the edge extension; finally, an existing risk assessment model in highway engineering quality inspection can be loaded, and the crack extension speed and extension direction in the edge extrapolation curve are used as risk factors affecting the durability of the highway engineering structure in the risk assessment model, and then the risk assessment model is run to integrate multiple risk factors to output a comprehensive risk level, and the comprehensive risk level is used as the risk level of the selected surface crack, and the risk level of the remaining surface cracks is continued to be determined to obtain the risk level of each surface crack.

[0054] In addition, in another aspect of the present application, in some embodiments, the present application provides a highway engineering quality detection system based on image recognition, referring to Figure 4 , which is a schematic diagram of the structure of a highway engineering quality detection system based on image recognition according to some embodiments of the present application. The highway engineering quality detection system based on image recognition 400 includes: an acquisition module 401, a processing module 402 and an execution module 403, which are respectively described as follows: The acquisition module 401 in this application is mainly used to acquire the road surface image of the target highway; Processing module 402, in the present application, is mainly used to identify cracks in the road surface image, obtain connected domains of each surface crack in the road surface image, and then perform edge detection on the surface cracks according to all connected domains to obtain crack boundaries of each surface crack in the road surface image; It should be noted that the processing module 402 in the present application is also used to obtain crack images of each surface crack at each detection time point in the quality inspection stage, and determine the spatiotemporal evolution characteristics of each surface crack according to the edge information of each crack image; In addition, it should be noted that the processing module 402 in the present application is also used to determine all potential extension points of each surface crack through the gradient change information of the surface crack in each crack image, and to perform trend extrapolation on each surface crack based on all crack boundaries and the spatiotemporal evolution characteristics to obtain the extension confidence of each surface crack at each potential extension point; Execution module 403, in the present application, the execution module 403 is mainly used to determine the risk level of each surface crack according to all the extended confidences, and then output the crack risk assessment result of the target highway.

[0055] In addition, the present application also provides a computer device, which includes a memory and a processor, the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned highway engineering quality detection method based on image recognition.

[0056] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for implementing a highway engineering quality detection method based on image recognition according to some embodiments of the present application. The highway engineering quality detection method based on image recognition in the above embodiment can be performed by Figure 5 The computer device 500 shown in the figure is implemented, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503 and at least one communication interface 504.

[0057] The processor 501 may be a general-purpose central processing unit (CPU) or an application specific integrated circuit (ASIC).

[0058] The communication bus 502 may be used to transmit information between the above-mentioned components.

[0059] The memory 503 may be a read only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read only memory (EEPROM), a compact disc read only memory (CD ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 503 may exist independently and be connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0060] The memory 503 is used to store the program code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the program code stored in the memory 503. The program code may include one or more software modules. The highway engineering quality detection method based on image recognition in the above embodiment can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0061] The communication interface 504 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0062] In a specific implementation, as an embodiment, a computer device may include multiple processors, each of which may be a single-core (single CPU) processor or a multi-core (multi CPU) processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0063] The above-mentioned computer device may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of computer device.

[0064] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned highway engineering quality detection method based on image recognition.

[0065] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0066] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A highway engineering quality detection method based on image recognition, characterized in that: The steps include: Collecting road surface images of the target highway; Performing crack recognition on the road surface image to obtain connected domains of each surface crack in the road surface image, and then performing edge detection on the surface cracks according to all connected domains to obtain crack boundaries of each surface crack in the road surface image; Acquire crack images of each surface crack at each detection time point in the quality inspection stage, and determine the spatiotemporal evolution characteristics of each surface crack according to edge information of each crack image; Determine all potential extension points of each surface crack through gradient change information of the surface crack in each crack image, perform trend extrapolation on each surface crack based on all crack boundaries and the spatiotemporal evolution characteristics, and obtain the extension confidence of each surface crack at each potential extension point; The risk level of each surface crack is determined according to all the extended confidences, and then the crack risk assessment result of the target highway is output.

2. The method according to claim 1, characterized in that Performing crack recognition on the road surface image to obtain a connected domain of each surface crack in the road surface image specifically includes: Performing grayscale processing on the road surface image to obtain a grayscale highway image; Obtaining a crack distribution map on the road surface image by performing semantic segmentation on the grayscale road image; Connected domain detection is performed on the crack distribution map to obtain connected domains of each surface crack in the road surface image.

3. The method according to claim 1, characterized in that Performing edge detection on the surface cracks according to all connected domains to obtain the crack boundaries of each surface crack in the road surface image specifically includes: Selecting a surface crack in the road surface image as a selected surface crack, and performing smoothing on a connected domain of the selected surface crack to obtain a smoothed connected domain of the selected surface crack; Determine the edge gradient of each pixel in the smooth connected domain; Performing double threshold detection on the smooth connected domain according to all edge gradients to obtain crack boundaries of the selected surface cracks; Continue to determine the crack boundaries for the remaining surface cracks.

4. The method according to claim 1, characterized in that Determining the spatiotemporal evolution characteristics of each surface crack according to the edge information of each crack image specifically includes: Selecting a surface crack in the road surface image as a selected surface crack, performing edge tracking on the selected surface crack according to each crack image, and obtaining an edge information sequence of the selected surface crack; Determine the spatiotemporal distribution domain of the selected surface crack edge based on all edge information in the edge information sequence; Extracting features from the spatiotemporal distribution domain to obtain time-speed trend features and spatial trend features of the edge of the selected surface crack; The spatiotemporal evolution characteristics of the selected surface cracks are determined based on the speed trend characteristics and the spatial trend characteristics, and the spatiotemporal evolution characteristics of the remaining surface cracks are further determined.

5. The method according to claim 1, characterized in that Determining all potential extension points of each surface crack through the gradient change information of the surface crack in each crack image specifically includes: Selecting a surface crack in the road surface image as a selected surface crack, and determining a plurality of abnormal change points on the edge of the selected surface crack according to all gradient change information; Determine the gradient direction of each abnormal change point; Determine the gradient similarity of the corresponding abnormal change point based on all pixel points in the gradient direction of each abnormal change point; All potential extension points of the selected surface cracks are determined based on all gradient similarities, and all potential extension points of the remaining surface cracks are further determined.

6. The method according to claim 1, characterized in that Based on all the crack boundaries and the spatiotemporal evolution characteristics, the trend of each surface crack is extrapolated to obtain the expansion confidence of each surface crack at each potential expansion point, which specifically includes: Perform feature extraction on all crack boundaries to obtain contour boundary features of all surface cracks; Selecting a surface crack in the road surface image as a selected surface crack, and determining an extension direction constraint of the selected surface crack at each potential extension point through the contour boundary feature and the spatial trend in the spatiotemporal evolution feature; Determining the expansion velocity constraint of the selected surface crack during the expansion process according to the velocity trend in the spatiotemporal evolution characteristics; Based on all the extension direction constraints and the extension speed constraints, the selected surface cracks are trend predicted to obtain the extension confidence of the selected surface cracks at each potential extension point, and then the extension confidence of the remaining surface cracks at each potential extension point is obtained.

7. The method according to claim 1, characterized in that Determining the risk level of each surface crack according to all the extended confidences can be achieved by the following steps: Selecting a surface crack in the road surface image as a selected surface crack, and extrapolating a preliminary prediction curve of the selected surface crack through all extended confidence levels; Performing boundary correction on the preliminary prediction curve according to the boundary conditions of the target highway to obtain an edge extrapolation curve of the selected surface crack on the edge extension; The risk level of the selected surface cracks on the regional surface image is determined by the edge extrapolation curve, and the risk levels of the remaining surface cracks are further determined.

8. A highway engineering quality detection system based on image recognition, characterized in that: include: A collection module, used for collecting road surface images of a target highway; A processing module, configured to perform crack recognition on the pavement image to obtain connected domains of various surface cracks in the pavement image, and then perform edge detection on the surface cracks according to all connected domains to obtain crack boundaries of various surface cracks in the pavement image; The processing module is used to obtain crack images of each surface crack at each detection time point in the quality inspection stage, and determine the spatiotemporal evolution characteristics of each surface crack according to edge information of each crack image; The processing module is used to determine all potential extension points of each surface crack through the gradient change information of the surface crack in each crack image, and to perform trend extrapolation on each surface crack based on all crack boundaries and the spatiotemporal evolution characteristics to obtain the extension confidence of each surface crack at each potential extension point; The execution module is used to determine the risk level of each surface crack according to all the extended confidences, and then output the crack risk assessment result of the target highway.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the highway engineering quality detection method based on image recognition described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the highway engineering quality detection method based on image recognition as described in any one of claims 1 to 7 are implemented.

Images