A method for identifying defects in asphalt pavement paving based on video images

Abstract

This invention belongs to the field of asphalt pavement paving defect detection technology. Specifically, it discloses a video image-based asphalt pavement paving defect recognition method. By continuously acquiring video images of the paving operation surface, the initial material area and the movement area are extracted separately. The two are then combined with morphological optimization and edge contour localization to lock the paving operation area in the current frame, significantly improving the boundary positioning accuracy of the operation area and providing a reliable analysis domain for subsequent defect detection. At the same time, within the identified paving operation area, a gridded local feature field based on image appearance features is constructed, and the consistency of grid cell features along the paver's travel direction is analyzed. This identifies suspected defect areas, and the suspected defect areas are screened out as real defect areas through spatial position verification and morphological gradient feature verification. This method can suppress non-defect visual interference to the greatest extent and improve the reliability of defect detection.

Description

Technical Field

[0001] This invention belongs to the field of asphalt pavement paving defect detection technology, and specifically discloses a method for identifying asphalt pavement paving defects based on video images. Background Technology

[0002] Asphalt pavement paving is a crucial step in road construction, and its quality directly determines the pavement's service performance. During high-speed, continuous paving, surface texture defects and other quality problems often occur due to factors such as material segregation and uneven temperature. If these defects are not detected and addressed promptly, they will become hidden dangers for early pavement damage, seriously affecting road safety.

[0003] With the development of computer vision technology, image and video-based paving defect monitoring has become an effective means of construction management. Existing methods mainly analyze the collected road surface images and extract appearance features such as color and texture to identify surface anomalies.

[0004] However, the above methods have the following limitations in practical applications: 1. The paving surface is constantly updated as the paver moves forward, and its position, shape, and coverage in the image are constantly changing dynamically. Existing technologies mostly rely on fixed thresholds or static templates to extract the working area, which is difficult to effectively cope with complex imaging conditions such as fluctuations in on-site lighting, shadows from mechanical structures, and reflective interference, resulting in positioning deviations and blurred boundaries of the paving area, introducing significant errors for subsequent defect analysis.

[0005] 2. The construction site environment is complex. In addition to real road surface defects, the video footage contains a large number of interfering factors, such as momentary shadows and splashed water stains. These interferences can also create visual anomalies in local areas of the image. If initial screening is based solely on visual features, these interferences are easily misjudged as real defects, reducing the reliability of defect detection. Summary of the Invention

[0006] To solve the above-mentioned technical problems, or at least partially solve them, the present invention provides a method for identifying defects in asphalt pavement paving based on video images.

[0007] The objective of this invention can be achieved through the following technical solution: a method for identifying defects in asphalt pavement paving based on video images, comprising the following steps: continuously acquiring video images of the paving operation surface through a video acquisition device, extracting the newly paved material area based on block feature matching, and extracting the dynamic area of ​​the paving front based on multi-frame difference.

[0008] By integrating the newly paved material area with the dynamic area at the paving front, the paving operation area in the current frame image is identified.

[0009] The identified paving area is divided into grid cells, and a local feature vector containing color and texture information is extracted for each grid cell. The feature consistency of the local feature vector of each grid cell with the average feature vector of the upstream paved area is calculated according to the paving direction, and a feature consistency coefficient distribution map is generated.

[0010] Based on the feature consistency coefficient distribution map, through neighborhood comparison and spatial clustering, abnormal areas with feature consistency lower than that of the neighborhood are identified as suspected defect areas.

[0011] For suspected defective areas, spatial location verification and morphological gradient verification are performed sequentially to screen out the actual defective areas.

[0012] The contours of the actual defect areas are extracted and geometrically measured to output the defect features.

[0013] Combining all the above technical solutions, the positive effects of this invention are as follows: 1. This invention continuously acquires video images of the paving operation surface, extracts the newly paved material area and the dynamic area of ​​the paving front edge, and further integrates the two with morphological optimization and edge contour positioning to lock the paving operation area in the current frame in real time. This achieves adaptive tracking of the dynamic paving operation area, can accurately distinguish between the real paving surface and background interference, improves the boundary positioning accuracy of the operation area, and provides a reliable analysis domain for subsequent defect detection.

[0014] 2. This invention constructs a gridded local feature vector based on image appearance features within the identified paving operation area, and analyzes the propagation continuity of the local feature vector of each grid unit along the paving direction. This identifies suspected defect areas, and filters out real defect areas through spatial location verification and morphological gradient verification. This can suppress non-defect visual interference to the greatest extent and improve the reliability of defect detection. Attached Figure Description

[0015] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0016] Figure 1 This is a diagram illustrating the implementation steps of the method of the present invention.

[0017] Figure 2 This is a flowchart illustrating the implementation of spatial position verification in this invention.

[0018] Figure 3 This is a flowchart illustrating the implementation process of morphological gradient verification in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See Figure 1 As shown, the present invention proposes a method for identifying defects in asphalt pavement paving based on video images, including the following steps: S1, continuously acquiring video images of the paving operation surface through a video acquisition device, extracting the newly paved material area based on block feature matching, and extracting the dynamic area of ​​the paving front based on multi-frame difference.

[0021] During asphalt pavement paving, the paver continuously moves forward, laying fresh asphalt mixture onto the roadbed surface, forming a dynamically expanding work area over time. The image sequence acquired by the video acquisition device not only includes the currently effective paving area but also inevitably contains non-work background interference components such as hardened old pavement. Since asphalt pavement paving defects only affect the paving work area, it is necessary to define the paving work area in each frame of the image to provide spatial constraints for subsequent defect detection.

[0022] To achieve the above objectives, this invention extracts the newly laid material area and the dynamic area at the paving front, respectively. The newly laid material area reflects the static characteristics of newly laid asphalt mixture and is used to identify candidate surfaces where material has been laid but has not yet been affected by subsequent disturbances. The dynamic area at the paving front reflects the dynamic changes caused by the paver's movement and material flow, and is used to capture active areas where paving is currently taking place. By integrating the above static appearance characteristics and dynamic behavior information, the paving operation area can be locked.

[0023] In one specific embodiment, the extraction process for the newly laid material area is as follows: the acquired continuous video frame images are segmented into blocks, and each frame image is divided into several sub-regions.

[0024] Considering that fresh asphalt mixtures have relatively stable visual characteristics after paving, including color and texture features, the color feature refers to the uniform dark black or dark brown color of the aggregate due to the asphalt binder coating it, and the texture feature refers to the medium-scale, isotropic granular texture formed by the random distribution of coarse and fine aggregates on the surface. Therefore, for each sub-region, its color histogram and texture spectrum are extracted as color and texture features respectively.

[0025] The feature distance between the color histogram and texture spectrum of each sub-region and the pre-established feature template of the newly paved asphalt mixture is calculated. The feature template of the newly paved asphalt mixture is formed by collecting multiple sets of known quality qualified paved surface image samples under standard lighting and paving conditions, dividing them into several local sub-regions, extracting the color histogram and texture spectrum of each sub-region, and calculating the mean vector of all sample features to form a feature template that characterizes the appearance properties of the newly mixed asphalt mixture.

[0026] Since individual sub-regions may be misjudged, but the actual paved surface usually occupies a large continuous area, sub-regions with a feature distance less than the tolerance threshold (e.g., 0.3) are aggregated as the new paved material area.

[0027] An example of the above operation is the feature distance calculation process as follows: First, for each sub-region obtained by dividing the current video frame, the color histogram and texture spectrum (such as the statistical histogram based on the local binary mode LBP) with the same preset dimension are extracted simultaneously to form the original local feature vector, and the original feature vector is normalized, for example, by using L2 norm normalization.

[0028] At the same time, the characteristic templates for newly paved asphalt mixtures are also constructed using a normalization method to ensure that they are in the same characteristic measurement space.

[0029] Finally, Euclidean distance is used as the feature distance to measure the difference between the normalized representation vector and the normalized feature template of the current sub-region. The smaller the Euclidean distance, the closer the sub-region is to the apparent characteristics of the newly laid asphalt mixture in terms of color distribution and texture structure, and the more likely it is to belong to the newly laid material area.

[0030] In a further specific implementation, the dynamic region at the paving front is extracted as follows: Considering the dynamic characteristics of asphalt paving operations, while the background area is relatively static, inter-frame difference can highlight the motion area related to paving. However, single-frame difference is easily affected by instantaneous interference such as sudden changes in lighting and splashing water droplets, resulting in false responses. Therefore, this invention adopts a three-frame difference method, extracting three temporally adjacent frames from continuous video images, which are respectively denoted as the previous frame, the current frame, and the next frame.

[0031] The absolute inter-frame difference between the current frame and the frames before and after is calculated, resulting in a forward difference map reflecting the change from the previous moment to the current moment and a reverse difference map reflecting the change from the current moment to the next moment. Since the actual paving motion has temporal continuity, both maps show high difference values. However, transient interference usually only occurs in one frame. Based on this, a pixel-by-pixel minimum value operation is performed on the forward difference map and the reverse difference map to generate a joint difference map. In the joint difference map, since the continuous motion region has a high response in both maps, its minimum value remains high and is preserved. Meanwhile, the transient interference has a low response in at least one map, so its minimum value is reduced and it is effectively suppressed.

[0032] Considering that difference maps often have edge breaks, internal holes and isolated noise, making them difficult to use directly for region analysis, while real paving motion in images is represented as regions with a certain area, complete shape and spatial connectivity, median filtering and morphological closing operations are applied to the joint difference map in sequence. Median filtering is used to eliminate isolated noise, and morphological closing operations are used to fill small holes and enhance region connectivity.

[0033] The processed image is labeled with 8-connected components to obtain a set of unconnected candidate regions.

[0034] The candidate regions are filtered as follows: i) Candidate regions located in the lower half of the image are retained. This is because the paving operation surface is always located in the lower half of the image field of view, while the upper half is mostly the sky, the top of the equipment, or the background in the distance, and its movement is unrelated to paving.

[0035] ii) Eliminate elongated areas with a height-to-width ratio greater than the preset ratio (e.g., the height is more than twice the width). This is because the actual paving movement area is usually horizontally distributed and in the shape of blocks or strips with a width greater than the height; while elongated areas are considered interference.

[0036] iii) Candidate regions with spatial overlap in adjacent time frames are retained. This is because the paver moves at a steady speed, and the resulting motion region has temporal continuity and spatial translation consistency. If a region appears only in one frame, it is very likely a transient artifact. Its authenticity can be further verified by checking its overlap in the preceding and following frames.

[0037] Finally, candidate regions that meet the above three constraints will be merged to form the dynamic paving front region.

[0038] S2. Merge the newly paved material area and the dynamic area of ​​the paving front to identify the paving operation area in the current frame image. The specific implementation process is as follows: S21. Perform an intersection operation on the newly paved material area and the dynamic area of ​​the paving front of the current frame image to obtain a fused area. This fused area represents a set of pixels that simultaneously satisfy the appearance characteristics of newly paved asphalt mixture and dynamic changes.

[0039] S22. Given that the paving surface is a large area that is physically connected and unbroken, morphological closing operations are performed on the fused region to fill the internal holes, and then connected component analysis is performed to retain the connected component with the largest area as the main paving region of the current frame in order to eliminate isolated interference blocks.

[0040] S23. Since the boundary between the paving area and the background ideally appears as a sudden change in grayscale or texture in the image, the image gradient magnitude directly quantifies the severity of this change. Therefore, the true boundary should theoretically correspond to a local maximum of the gradient magnitude. Based on this, a search is performed within a limited distance along the outward normal direction, starting from the edge pixels of the paving main area. However, uneven lighting and slight water stains at the construction site can cause low-contrast gradient areas in the image. These areas may also produce local fluctuations in gradient magnitude, but they are not true physical boundaries. Therefore, an edge intensity limit is introduced, and the pixel with the largest image gradient magnitude along this path that exceeds the local edge intensity limit is identified as a candidate boundary point.

[0041] S24. Connect all candidate boundary points in sequence according to the polar angle of the line connecting them to the geometric center of the paving main area to form the initial boundary polyline.

[0042] S25. Perform endpoint connection and smoothing filtering on the boundary polylines to eliminate jagged noise and generate closed, smooth polygons as the geometric boundaries of the paving area.

[0043] In the above implementation process, the introduced edge intensity limit can suppress low-intensity pseudo edges. Specifically, it can be determined in the following way: starting from the edge point of the paving main area, along the search path in the direction of its outer normal, take a local neighborhood window containing the current pixel, calculate the statistical distribution of the gradient magnitude of all pixels in the window, and set a certain higher quantile value (e.g., the 80th percentile) as the edge intensity limit of the path.

[0044] After the paving operation area is locked through steps S1 and S2, since paving defects are usually manifested in the discontinuity of appearance characteristics such as color and texture, in order to accurately identify these defects, it is necessary to perform appearance consistency analysis along the paving direction. Therefore, the following S3 is set.

[0045] S3. Divide the identified paving area into grid cells, and extract local feature vectors containing color and texture information for each grid cell. Calculate the feature consistency between the local feature vector of each grid cell and the average feature vector of the upstream paved area based on the paving direction, and generate a feature consistency coefficient distribution map.

[0046] As one way to implement the present invention, S3 is implemented as follows: S31, in order to support the subsequent quantitative analysis of appearance consistency, by using a square grid of fixed size within the locked current frame paving operation area, the operation area is divided into multiple non-overlapping grid units, so that each grid unit has the same scale, which is beneficial for subsequent unified comparison of appearance features between different grid units.

[0047] S32. Calculate the gray-level co-occurrence matrix of the gray-level image of each grid cell, and extract the contrast and energy features from the matrix to form a texture statistics sub-vector. The energy feature reflects the uniformity of the gray-level distribution of the image and the coarseness of the texture. The larger the value, the more uniform the texture.

[0048] S33. Since the LAB color space is closer to the visual perception of color, each grid cell is converted from the RGB color space to the LAB color space, and the mean and standard deviation of the pixel values ​​of its L, A, and B channels are calculated to form a color matrix vector.

[0049] S34. Concatenate the texture statistics subvector and the color moment subvector in the order of texture first and then color to form the local feature vector of the grid cell.

[0050] S35. Based on the direction of travel of the paver, define a unidirectional spatial propagation direction from the paved area to the paving front, which facilitates the subsequent analysis of the feature consistency of the local feature vectors of the grid cells along this direction.

[0051] S36. Considering the material continuity of asphalt mixture during continuous paving, the apparent characteristics of the current location are usually highly similar to those of its upstream neighboring areas. Therefore, for any grid cell located at the current propagation position, all cells in the two adjacent grid rows that are closest in spatial distance upstream of it in the reverse propagation direction are selected to form a reference cell set to characterize the normal apparent state of the grid cell.

[0052] S37. Calculate the Mahalanobis distance between the local feature vector of the current grid cell and the arithmetic mean of the local feature vectors of all cells in the reference cell set, as the difference between the two. This is because the Mahalanobis distance takes into account the correlation between the dimensions of the feature vectors and can more accurately measure the deviation of the current feature from the multivariate normal distribution formed by the features of the upstream reference set. Since the Mahalanobis distance is negatively correlated with feature consistency, in order to obtain positive feature consistency, the difference is reverse normalized. Specifically, the Mahalanobis distance is first normalized and mapped to the interval [0, 1]. Then, the value 1 is subtracted from the normalization result to obtain the feature consistency coefficient of the corresponding grid cell. The closer the coefficient is to 1, the more consistent its apparent features are with the upstream region; otherwise, it indicates that there may be paving defects.

[0053] S38. Traverse all grid cells within the current frame paving operation area to generate a feature consistency coefficient distribution map that corresponds one-to-one with the spatial grid cells.

[0054] S4. Based on the feature consistency coefficient distribution map, through neighborhood comparison and spatial clustering, abnormal areas with feature consistency lower than that of the neighborhood are identified as suspected defect areas.

[0055] After generating the feature consistency coefficient distribution map in step S3, apparent anomaly areas can be identified based on this distribution. However, considering that on-site interference factors may also cause a temporary decrease in the consistency coefficient, such anomalies do not necessarily correspond to actual defects. Therefore, this invention defines the initially identified low consistency areas as suspected defect areas.

[0056] As a preferred embodiment of the present invention, the above steps are implemented as follows: S41, taking the current grid cell to be analyzed as the center, define a local neighborhood window that covers all adjacent grid cells within a fixed radius around it.

[0057] S42. The median value of the feature consistency coefficient of all grid cells within the local neighborhood window is used as the background consistency level.

[0058] S43. Given that asphalt pavers have apparent continuity under normal working conditions, the characteristic consistency at any location should be highly consistent with its local neighborhood. Therefore, calculate the absolute deviation between the characteristic consistency coefficient of the current grid cell and the background consistency level. If the absolute deviation is greater than the allowable deviation, where the allowable deviation can be determined by multiplying the median absolute deviation of all characteristic consistency coefficients in the local neighborhood by an empirical coefficient (e.g., 1.5 times), it indicates that the apparent consistency of the current grid cell deviates from the normal fluctuation of its local neighborhood. At this time, the grid cell is determined to be an initial anomalous cell.

[0059] S44. Perform spatial connectivity-based clustering analysis on the marked initial abnormal units, and aggregate spatially adjacent abnormal units into one or more independent connected regions. Each connected region is a suspected defect region.

[0060] S5. For suspected defective areas, perform spatial location verification and morphological gradient verification in sequence to screen out the real defective areas.

[0061] After locating suspected defect areas, these areas need to be further verified to improve the accuracy of defect identification. Based on this, the present invention distinguishes real defects by setting up dual verification of spatial location and morphological gradient.

[0062] Specifically, the principle of spatial location verification is as follows: After the asphalt mixture is laid by the screed of the paver, it needs to undergo a short period of compaction and cooling stabilization. The material in the area near the front edge of the paver is not yet stable and is easily affected by uneven material distribution. The area near the side or tail edge is easily affected by equipment structure shadows, edge effects or interference from the old pavement. Therefore, only the internal area far from the boundary is suitable as the basis for defect identification.

[0063] See Figure 2 As shown, under the above implementation principle, the spatial position verification process is as follows: For the paving operation area identified in the current frame, a morphological erosion operation is performed using a structuring element of a preset size, which is equivalent to shrinking a fixed distance inward from the original boundary of the paving operation area to generate an effective detection sub-region located inside the operation area.

[0064] For each suspected defect area, its geometric center point is extracted to represent its spatial center of gravity, and it is determined whether the point is located within the valid detection sub-area. If it is located within the valid detection sub-area, it means that the main body of the entire area is located inside the paved surface, and its anomaly is more likely to originate from a real defect. In this case, it is confirmed to pass the spatial location verification; otherwise, it is a false defect.

[0065] In one example, the size of the structural element is set through the following process: First, a minimum safety distance (e.g., 30cm) is set based on paving process experience. This safety distance defines the physical length at which the effective detection sub-region should be far from the original boundary of the paving operation area. Then, the correspondence between image pixels and the physical dimensions of the road surface is obtained through camera calibration, thereby converting the safety distance into pixel length, i.e., the radius r of the circular structural element. The effective detection sub-region is obtained by performing an erosion operation on the paving operation area using this circular structural element with radius r.

[0066] Furthermore, although unreliable candidates located at the boundaries and leading edge transition zones of the paving operation area have been eliminated through spatial location verification, the remaining suspected defect areas may still contain anomalies caused by local texture fluctuations or lighting disturbances.

[0067] To accurately identify real defects, it is necessary to use morphological gradient verification to examine the degree of change in defect consistency. However, since the material near the paving front is not yet stable, the consistency coefficient is low and the gradient may be large. Even without defects, it may trigger a high gradient response. If morphological gradient analysis is performed directly on these areas, the process transition area will be misjudged as a defect. Therefore, the morphological gradient verification of this invention is for areas that have passed spatial verification, thereby ensuring the effectiveness of gradient verification.

[0068] See Figure 3 As shown, the morphological gradient verification process is as follows: For the suspected defect region that has passed the spatial verification, obtain its minimum bounding rectangle.

[0069] Since the actual defects in asphalt paving are usually manifested as uneven material distribution, resulting in irregular geometric shapes and difficulty in filling the circumscribed rectangle, the area filling rate is defined as the ratio of the area of ​​the suspected defect area to the area of ​​its smallest circumscribed rectangle. The lower the filling rate, the more irregular the geometric shape.

[0070] Considering that the characteristic consistency coefficient reflects the degree of matching between the current local area and the upstream paved surface in terms of apparent characteristics, while the consistency coefficient of the real defect area shows a steep gradient along the paving direction due to material abrupt changes, and that interference such as water stains and shadow noise usually only cause local isolated anomalies with gentle consistency changes, the absolute mean of the gradient of the consistency coefficient along the paving direction is calculated within the grid cells covered by the suspected defect area that has passed spatial verification. The specific calculation process is as follows: For each grid cell covered by the suspected defect area, the difference in the consistency coefficient between the cell and its downstream adjacent cell is calculated along the paving direction. The arithmetic mean of the absolute values ​​of the differences in the consistency coefficients of all cells in the region is taken as the absolute mean of the gradient of the region. The higher the absolute mean of the gradient, the more severe the structural discontinuity.

[0071] If a suspected defective region simultaneously meets the following conditions: the region's fill rate is lower than the fill rate judgment value and the absolute mean of the gradient is higher than the gradient judgment value, then it is determined to be a real defective region; otherwise, it is not determined to be a real defective region.

[0072] As an example of the above operation, the fill rate determination value and gradient determination value can be taken as the median of the fill rate and the median absolute mean of the gradient of all suspected defective regions that have passed spatial verification in the current frame.

[0073] It should be noted that the median was chosen as the decision limit because, during normal paving, there are relatively few actual defect areas caused by process or material issues. Therefore, most of the suspected defect areas identified through spatial verification are false anomalies. The fill rate and gradient mean of these false anomalies usually concentrate within a relatively stable range, while the fill rate and gradient mean of actual defects often exhibit extreme values. The median is not sensitive to extreme values ​​and can better represent the central trend of false anomalies; therefore, it was chosen as the decision limit.

[0074] S6. Extract the contour and perform geometric measurements on the actual defect area, and output the defect features.

[0075] After verifying the actual defect area, in order to achieve a quantitative assessment of the defect, the physical size and morphological characteristics of the defect can be obtained by extracting the contour and measuring the geometry of the actual defect area.

[0076] The specific implementation is as follows: First, for each real defect area, extract its pixel contour in the image.

[0077] To obtain the true physical size of the defect, the pixel outline of the defect in the image needs to be transformed into the physical outline in the road surface coordinate system. The road surface coordinate system is a right-handed coordinate system with the X-axis along the paver's travel direction, the Y-axis along the road surface laterally, and the Z-axis perpendicular to the road surface upwards. The origin of the coordinate system can be set at the projection point of the bottom center of the camera image onto the road surface.

[0078] The specific conversion process is carried out in the following steps: (1) For each pixel on the pixel contour, the intrinsic parameter matrix and lens distortion coefficient obtained by camera calibration are used to correct and back-project to the normalized camera coordinate system to obtain a three-dimensional direction vector.

[0079] (2) Based on the rotation matrix and translation vector of the camera relative to the road coordinate system, the direction vector obtained in the previous step is transformed to the road coordinate system through coordinate transformation. At this time, each pixel corresponds to a ray that starts from the optical center of the camera and points to the scene.

[0080] (3) In the road surface coordinate system, the paving surface can be represented as a plane with Z=0. The intersection point of the ray obtained in step (2) with the plane is the (X, Y) coordinate of the intersection point, which is the actual physical location of the road surface corresponding to the pixel.

[0081] (4) Traverse all points of the pixel contour to obtain a series of ordered physical coordinate points. Connect these points in order to form a closed planar polygon on the XY plane of the road surface coordinate system. This polygon is the physical contour of the defect.

[0082] Furthermore, the physical contour is orthogonally projected onto the horizontal road surface plane, and its projected area is calculated using the polygon area integral formula, which is taken as the actual coverage area of ​​the defect.

[0083] Finally, the minimum bounding rectangle of the physical profile is obtained and the coordinate axes are aligned according to the paving direction. The length of the long side of the rectangle represents the extension scale of the defect along the paving direction, and the width of the short side represents its lateral distribution scale, thereby realizing the directional quantification of the spatial distribution characteristics of the defect.

[0084] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0085] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0087] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0088] Finally, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying defects in asphalt pavement paving based on video images, characterized in that, Includes the following steps: The video capture device continuously captures video images of the paving operation surface, extracts the newly paved material area based on block feature matching, and extracts the dynamic area of ​​the paving front based on multi-frame difference. By integrating the newly paved material area with the dynamic area at the front edge of the paving, the paving operation area in the current frame image is identified; The identified paving area is divided into grid cells, and a local feature vector containing color and texture information is extracted for each grid cell; Based on the direction of travel of the paver, a unidirectional spatial propagation direction is defined from the paved area to the front edge of the paving. For any grid cell located at the current propagation position, select all cells from the two nearest adjacent grid rows upstream in the reverse propagation direction to form a reference cell set; Calculate the Mahalanobis distance between the local eigenvector of the current grid cell and the arithmetic mean of the local eigenvectors of all cells in the reference cell set, and use it as the degree of difference between the two. The difference is reverse normalized to obtain the feature consistency coefficient of the corresponding grid cell. Traverse all grid cells within the current frame's paving area to generate a feature consistency coefficient distribution map that corresponds one-to-one with each spatial grid cell; Define a local neighborhood window centered on the current grid cell to be analyzed, covering all adjacent grid cells within a fixed radius around it; The median value of the feature consistency coefficient of all grid cells within the local neighborhood window is used as the background consistency level. Calculate the absolute deviation between the feature consistency coefficient of the current grid cell and the background consistency level. If the absolute deviation is greater than the allowable deviation, the grid cell is determined to be an initial anomalous cell. Cluster analysis based on spatial connectivity is performed on the initially marked anomalous units to aggregate spatially adjacent anomalous units into one or more independent connected regions, and each connected region is a suspected defect region; For suspected defect areas, spatial location verification and morphological gradient verification are performed sequentially to screen out the actual defect areas; The contours of the actual defect areas are extracted and geometrically measured to output the defect features.

2. The method for identifying asphalt pavement paving defects based on video images as described in claim 1, characterized in that: The extraction of the newly laid material area based on block feature matching is implemented as follows: The acquired continuous video frame images are segmented into blocks, dividing each frame image into several sub-regions; For each sub-region, extract its color histogram and texture spectrum; Calculate the feature distance between the color histogram and texture spectrum of each sub-region and the pre-established feature template of the newly paved asphalt mixture; Sub-regions with a feature distance less than the tolerance threshold are aggregated and used as new material application areas.

3. A method for identifying defects in an asphalt pavement placement based on video images as recited in claim 1, wherein: The process of extracting the dynamic region of the paving front based on multi-frame difference is as follows: Extract three temporally adjacent frames from a continuous video image, and label them as the previous frame, the current frame, and the next frame, respectively. Calculate the absolute inter-frame difference between the current frame and the frames before and after it to obtain the forward difference map and the backward difference map, and perform a pixel-by-pixel minimum operation on the two to generate a joint difference map; Median filtering and morphological closing operations are performed sequentially on the joint difference graph; The processed image is labeled with 8-connected components to obtain a set of unconnected candidate regions; The candidate regions were filtered as follows: i) Retain candidate regions located in the lower half of the image; ii) Remove elongated areas with an aspect ratio greater than the preset ratio; iii) Retain candidate regions that have spatial overlap in adjacent time frames; The selected candidate areas are merged to form the dynamic area at the front of paving.

4. The method for identifying asphalt pavement paving defects based on video images as described in claim 1, characterized in that: The identification of the paving operation area in the current frame image includes the following: The intersection of the newly laid material area and the dynamic area of ​​the paving front in the current frame image is calculated to obtain a fused region. Morphological closing operations are performed on the fused region to fill the internal holes, and then connected component analysis is performed to retain the connected component with the largest area as the main paving region of the current frame. Starting from the edge pixels of the main paving area, a search is performed within a limited distance along its outer normal direction, and the pixel with the largest image gradient magnitude that exceeds the edge intensity limit is determined as the candidate boundary point. Connect all candidate boundary points sequentially according to the polar angle of their connection to the geometric center of the paving main area to form the initial boundary polyline; The boundary polylines are connected at their endpoints and smoothed by filtering to form closed polygons, which serve as the geometric boundaries of the paving area.

5. The method for identifying asphalt pavement paving defects based on video images as described in claim 1, characterized in that: The process of dividing the identified paving area into grid cells and extracting a local feature vector containing color and texture information for each grid cell is as follows: The current frame paving area is divided into multiple non-overlapping grid cells using a square grid. Calculate the gray-level co-occurrence matrix of the gray-level image for each grid cell, and extract the contrast and energy features from the matrix to form a texture statistics sub-vector; Each grid cell is converted from the RGB color space to the LAB color space, and the mean and standard deviation of the pixel values ​​of its L, A, and B channels are calculated to form a color matrix vector. The texture statistics subvector and the color moment subvector are concatenated in the order of texture first and then color to form the local feature vector of the grid cell.

6. A method for identifying defects in an asphalt pavement placement based on video images as recited in claim 1, wherein: The spatial location verification process is described below: For the paving operation area already identified in the current frame, perform morphological erosion operation using structuring elements of preset size to generate an effective detection area located inside the operation area; For each suspected defect area, extract its geometric center point; If the geometric center point of a suspected defect area is located within the effective detection area, it will pass the spatial location verification; otherwise, it will be determined as a false defect.

7. A method for identifying defects in an asphalt pavement placement based on video images as recited in claim 1, wherein: The morphological gradient verification process is described below: For suspected defect areas that pass spatial location verification, obtain their minimum bounding rectangle; The ratio of the area of ​​the suspected defect region to the area of ​​its smallest bounding rectangle is defined as the region fill rate. Within the grid cells covered by the suspected defect area, calculate the absolute mean of the gradient of the consistency coefficient along the paving direction; If a suspected defective region simultaneously meets the following conditions: the region's fill rate is lower than the fill rate judgment value and the absolute mean of the gradient is higher than the gradient judgment value, then it is determined to be a real defective region.

8. A method for identifying defects in an asphalt pavement placement based on video images as recited in claim 1, wherein: The process of extracting the contour and performing geometric measurements on the actual defect area, and outputting the defect features, is as follows: For each real defect region, extract its pixel contour in the image; Convert the pixel contours of the real defect area into physical contours in the road surface coordinate system; Calculate the projected area of ​​the physical profile in the road surface coordinate system as the actual area covered by the defect; Find the minimum bounding rectangle of the physical contour of the actual defect, and use its long side length and short side width to characterize the extension scale of the defect along the paving direction and the lateral distribution scale, respectively.

Images