A deep learning-based asphalt mixture aggregate distribution monitoring method and system

Abstract

The present application relates to the technical field of road engineering, and especially relates to a kind of asphalt mixture aggregate distribution monitoring method and system based on deep learning, comprising the image after pre-processing is input into instance segmentation model of migration training, obtains the binary instance mask corresponding to each aggregate particle;The contour, size feature and barycentric coordinate of each aggregate particle are extracted, the particle size range is determined, and the weighted mass and contribution factor are calculated;With barycentric coordinate to determine spatial position, iterative search nearest neighbor particle constructs distance chain, and the distance and moment between adjacent particles are calculated;The image is divided into several equal-area partitions, and the moment sum of each partition and the whole image is calculated;According to the relative deviation of the average level of each partition moment sum and the average level of the whole image moment sum, the aggregate distribution uniformity index is calculated.The present application realizes real-time, quantitative monitoring to the uniformity of asphalt mixture paving.

Description

Technical Field

[0001] This invention relates to the field of road engineering technology, and in particular to a method and system for monitoring the aggregate distribution of asphalt mixtures based on deep learning. Background Technology

[0002] In recent years, with the continuous improvement of highway transportation infrastructure, asphalt mixtures have become the main material for paving or overlaying high-grade highways due to their good mechanical properties and driving comfort. However, the problems of early damage and insufficient durability of asphalt pavements are quite prominent. Long-term practice and research have shown that the paving effect of asphalt mixtures is a key factor affecting the performance of asphalt pavements. Early damage to asphalt pavements is mostly caused by material segregation and uneven paving, which in turn leads to rutting, potholes, swells and other defects.

[0003] In existing technologies, the quality inspection of asphalt pavement mainly relies on post-sampling inspection, that is, quality assessment is carried out by drilling cores or sampling tests after paving is completed. This method not only has the problems of detection lag and limited representativeness, but also makes it difficult to grasp the paving status in real time during the paving process, and cannot carry out comprehensive process control of paving quality.

[0004] In terms of paving uniformity evaluation, existing technologies have failed to achieve automatic segregation determination based on image intelligent recognition: on the one hand, the identification and segmentation of aggregate particles on the paved surface still lacks highly accurate and adaptable automated means, making it difficult to accurately obtain information on the outline, size, and spatial location of aggregate particles; on the other hand, existing evaluation methods lack quantitative descriptions of aggregate distribution differences, and cannot use quantitative indicators to characterize the uniformity of aggregate particle distribution in the transverse and longitudinal directions of the road surface. The evaluation results are easily affected by human factors, and objectivity and accuracy are difficult to guarantee. At present, a deep learning-based method and system for monitoring the aggregate distribution of asphalt mixtures is needed. Summary of the Invention

[0005] To address the technical problems of lagging, lack of objectivity and accuracy in the evaluation of paving uniformity in existing asphalt pavement paving quality testing, this invention provides a deep learning-based method and system for monitoring the aggregate distribution of asphalt mixtures.

[0006] In a first aspect, the present invention provides a deep learning-based method for monitoring aggregate distribution in asphalt mixtures, employing the following technical solution: A deep learning-based method for monitoring aggregate distribution in asphalt mixtures includes: S1. Collect images of the asphalt mixture surface during the paving process and preprocess them. Input the preprocessed images into the instance segmentation model trained by transfer learning to obtain the binary instance mask corresponding to each aggregate particle. S2. Extract the outline, size features and centroid coordinates of each aggregate particle based on the binary instance mask, determine the particle size range of each aggregate particle, and calculate the weighted mass and contribution factor of each aggregate particle. S3. Determine the spatial position of each aggregate particle using the center of gravity coordinates, select the starting particle of the distance chain, calculate the distance and torque between adjacent particles based on the starting particle, until all aggregate particles are traversed to form a distance chain. S4. Divide the surface image into several equal-area partitions, construct partition distance chains within each partition, and calculate the sum of moments for each partition and the sum of moments for the entire image. S5. Based on the relative deviation between the average level of the sum of moments in each zone and the average level of the sum of moments across the entire map, calculate the aggregate distribution uniformity index to evaluate the aggregate distribution uniformity of the asphalt mixture.

[0007] Furthermore, the acquisition and preprocessing of asphalt mixture surface images during the paving process includes cropping the asphalt mixture surface image based on the short side of the asphalt mixture surface image to obtain a cropped image, and converting the cropped image into a grayscale image using a weighted average method. Median filtering is used to suppress noise in grayscale images, and histogram equalization is performed on the noise-suppressed images to obtain enhanced images. The size of the enhanced images is then uniformly adjusted to a preset target size to obtain standardized images.

[0008] Furthermore, determining the particle size range of each aggregate particle includes establishing a polar coordinate system with the centroid coordinate as the pole, scanning the contour along the circumference with a preset angle step size, obtaining the first intersection point of the ray in this direction with the contour, and the first intersection point of the reverse extension line in this direction with the contour, and calculating the distance between the two intersection points as the chord length in this scanning direction. Traverse all scanning directions and take the minimum value among all the through-chord lengths as the minimum through-chord length of the aggregate particle; Based on the image size calibration relationship, the minimum chord length is converted into the actual size to determine the sieve aperture size that the aggregate particle can pass through, and the particle size class to which the aggregate particle belongs is determined according to the preset coarse aggregate particle size classification sequence.

[0009] Further, the calculation of the weighted mass and contribution factor of each aggregate particle includes determining the sphericity and bulk density of aggregates at each particle size range, expressing the volume of the aggregate particle as a weighted volume of a cube volume and a sphere volume weighted by sphericity based on the geometric shape of the aggregate particle, and calculating the weighted mass of the aggregate particle based on the weighted volume and bulk density. The expression for the weighted mass is as follows: , , in, Let be the weighted mass of the aggregate particles in the i-th grade. Let be the bulk density of the i-th aggregate. Let be the weighted volume of the i-th aggregate. Let be the sphericity of the i-th aggregate. This refers to the volume of a cube calculated using equivalent particle size. This is the volume of a sphere calculated using the equivalent particle size.

[0010] Furthermore, the calculation of the weighted mass and contribution factor of each aggregate particle also includes defining the contribution factor of each aggregate size range based on the weighted volume of the largest particle size range aggregate as a normalization benchmark, and the calculation formula of the contribution factor is as follows: , in, , representing the contribution factor of the i-th aggregate, is used to characterize the relative contribution of the i-th aggregate particles in the asphalt mixture skeleton. Let be the sieve aperture size of the (i+1)th aggregate. This is the preset conversion factor from sieve aperture size to equivalent particle size. The sphericity of the aggregate at the largest particle size range. This refers to the sieve aperture size corresponding to the maximum particle size range.

[0011] Further, step S3 specifically includes: S31. Determine the number of aggregate particles based on the number of binary instance masks, and select the first aggregate particle identified in the upper left corner of the image as the starting particle of the distance chain. S32. From the aggregate particles that have not yet been added to the distance chain, search for the particle closest to the current particle according to the particle size priority principle and add it to the distance chain as the next particle. The distance between the current particle and the next particle is calculated using Euclidean distance. S33. Calculate the weighted mass of the current particle and the distance as the torque corresponding to the current particle; S34. Take the next particle as the new current particle and repeat steps S32 to S33 until all aggregate particles have been traversed.

[0012] Furthermore, the particle size priority principle is specifically determined in the following order of priority: The first stage involves comparing the particle size ranges of each candidate particle and selecting the candidate particle with the highest particle size range as the next particle. The second level is to calculate the angle between the line connecting each candidate particle to the current particle and the current chain direction when multiple candidate particles belong to the same particle size range. The candidate particle with the smallest angle is selected as the next particle. The current chain direction is the direction from the previous particle in the chain to the current particle. The third stage involves determining the next particle from left to right and from top to bottom according to the image scanning sequence.

[0013] Further, step S4 includes using the bisection method to divide the asphalt mixture surface image into multiple equal-area rectangular partitions along the longitudinal and transverse midlines, constructing partition distance chains in each partition, and sequentially linking each link in each partition distance chain geometrically in the image to obtain partition distance chains corresponding to the aggregate skeleton distribution. The number of aggregate particles in each zone and the total number of aggregate particles in the entire map are counted separately. The sum of the torques in each zone and the sum of the torques in the entire map are calculated. The expression for the sum of the torques is as follows: , in, This is the region identifier, and its value can be any partition or the entire map. For the region The sum of the torques of the inner distance chain, For the region The number of internal aggregate particles, For the region Inner distance chain The weighted mass of each link corresponds to a particle. For the first The distance between the current particle and the next particle in each link. The contribution factor is the particle size range to which the r-th link corresponds.

[0014] Furthermore, the evaluation of the uniformity of aggregate distribution in asphalt mixtures includes calculating the average moment of each zone based on the sum of the moments of each zone and the number of chain links of the distance chain within the corresponding zone, and using each average moment as the average level of the sum of moments of the corresponding zones; The average moment across the entire map is calculated and used as a normalized benchmark to characterize the uniform distribution state. Based on the ratio of the mean of the average moment in each zone to the average moment across the entire map, an aggregate distribution uniformity index is determined to evaluate the aggregate distribution uniformity of asphalt mixtures. The expression for the aggregate distribution uniformity index is as follows: , in, The aggregate distribution uniformity index. , , and These are the sums of the torques of the distance chains within the corresponding partitions. , , and These represent the number of aggregate particles within the corresponding zones. This represents the total number of aggregate particles within the entire map area. Let m be the torque of the m-th link in the chain that is the distance to the entire graph. This means that, starting from the first link, the torques of all links in the distance chain across the entire graph are accumulated one by one.

[0015] Secondly, a deep learning-based asphalt mixture aggregate distribution monitoring system includes: The image acquisition and segmentation module is used to acquire and preprocess images of the asphalt mixture surface during the paving process. The preprocessed images are then input into the instance segmentation model trained by transfer learning to obtain the binary instance mask corresponding to each aggregate particle. The feature extraction and weighting module is used to extract the contour, size features and centroid coordinates of each aggregate particle based on the binary instance mask, determine the particle size range of each aggregate particle, and calculate the weighted mass of each aggregate particle. The distance chain construction module is used to determine the spatial position of each aggregate particle using the centroid coordinates, select the starting particle of the distance chain, calculate the distance and torque between adjacent particles based on the starting particle, until all aggregate particles are traversed to form a distance chain. The partition statistics module is used to divide the surface image into several equal-area partitions, construct partition distance chains within each partition, and calculate the torque sum of each partition and the torque sum of the entire image. The uniformity evaluation module is used to calculate the aggregate distribution uniformity index based on the relative deviation between the average level of the sum of moments in each zone and the average level of the sum of moments across the entire map, and to evaluate the aggregate distribution uniformity of asphalt mixtures.

[0016] In summary, the present invention has the following beneficial technical effects: 1. This invention obtains the chord length of the aggregate by circumferential scanning with the center of gravity as the pole and takes its minimum value to determine the particle size range of the aggregate particles. It accurately characterizes the size characteristics of the aggregate particles that can pass through the sieve holes and realizes the objective and automatic determination of the particle size range. 2. This invention represents the aggregate particle volume as a weighted volume of a combination of cubic and spherical volumes weighted by sphericity to calculate the weighted mass, and uses the weighted volume of the aggregate with the largest particle size as the normalization benchmark to define the contribution factor, so that the mass estimation of aggregate particles conforms to their true geometric shape.

[0017] 3. This invention uses the particle size priority principle to sequentially determine candidate particles with the same distance, and performs validity determination on boundary truncation particles, which ensures the uniqueness and continuity of the distance chain construction results and avoids the calculation error introduced by particles with missing contours. 4. This invention uses the average moment of the whole map as the normalization benchmark for the theoretical uniform distribution state, and constructs an aggregate distribution uniformity index based on the relative deviation between the average moment of each zone and the average moment of the whole map. This achieves a sensitive and quantitative evaluation of local differences in aggregate distribution. The index value has a clear correspondence with the degree of segregation. The evaluation results are objective and accurate, and can effectively guide the intelligent paving construction of asphalt pavement. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall process of the deep learning-based asphalt mixture aggregate distribution monitoring method in an embodiment of the present invention; Figure 2 This is a schematic diagram of the adjacency distance chain in an embodiment of the present invention; Figure 3 This is a schematic diagram of the zoning annotation of the asphalt mixture surface image in an embodiment of the present invention; Figure 4 This is a schematic diagram of the identification particles and the partition distance chain in each partition of the present invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to the accompanying drawings.

[0020] Example 1 Reference Figure 1 This embodiment of a deep learning-based method for monitoring aggregate distribution in asphalt mixtures includes: S1. Collect images of the asphalt mixture surface during the paving process and preprocess them. Input the preprocessed images into the instance segmentation model trained by transfer learning to obtain the binary instance mask corresponding to each aggregate particle. S2. Extract the outline, size features and centroid coordinates of each aggregate particle based on the binary instance mask, determine the particle size range of each aggregate particle, and calculate the weighted mass and contribution factor of each aggregate particle. S3. Determine the spatial position of each aggregate particle using the center of gravity coordinates, select the starting particle of the distance chain, calculate the distance and torque between adjacent particles based on the starting particle, until all aggregate particles are traversed to form a distance chain. S4. Divide the surface image into several equal-area partitions, construct partition distance chains within each partition, and calculate the sum of moments for each partition and the sum of moments for the entire image. S5. Based on the relative deviation between the average level of the sum of moments in each zone and the average level of the sum of moments across the entire map, calculate the aggregate distribution uniformity index to evaluate the aggregate distribution uniformity of the asphalt mixture.

[0021] Specifically, a deep learning-based method for monitoring aggregate distribution in asphalt mixtures includes the following: like Figure 1As shown in Figure S1, an industrial camera is used to acquire images of the asphalt mixture surface during the paving process. The industrial camera is fixedly installed behind the screed of the paver, with the lens pointing vertically towards the paved asphalt mixture surface. During the continuous paving operation, images of the asphalt mixture surface are acquired at a preset acquisition frequency to provide raw image data for the subsequent automatic identification of aggregate particles.

[0022] The acquired asphalt mixture surface image undergoes standardized preprocessing, specifically as follows: First, using the shorter side of the original image as a reference, a central square crop is performed on the asphalt mixture surface image. This involves cropping a square region with the image center as the cropping center and the shorter side as the side length, thus removing irrelevant background areas at the image edges and ensuring that the image region for subsequent processing is centered in the field of view and is square. Second, the cropped image is converted to a grayscale image using a weighted average method. This involves assigning different weighting coefficients to the red, green, and blue color channels according to their contribution to pixel brightness, converting the color image into a single-channel grayscale image. The grayscale calculation formula is as follows: , in, , and These represent the pixel values ​​of the red, green, and blue channels in the original color image, respectively. This represents the pixel value after grayscale processing. 0.299, 0.587, and 0.114 are the preset weighting coefficients for the red, green, and blue channels, respectively. The sum of the three is 1. The value of each weighting coefficient is determined based on the sensitivity of the human eye to different colors of light. The weighting coefficient of the green channel is the largest, followed by the weighting coefficient of the red channel, and the weighting coefficient of the blue channel is the smallest, so that the brightness distribution of the grayscale image is consistent with the visual perception of the human eye and the brightness difference information between aggregate particles and asphalt matrix is ​​preserved.

[0023] Next, median filtering is used to suppress noise in the grayscale image. This involves taking a filter window of a preset size centered on each pixel, sorting the grayscale values ​​of all pixels within the window by size, and then taking the median value as the filtered grayscale value for the central pixel. Median filtering can effectively suppress impulse interference such as salt-and-pepper noise in the paved surface image while preserving the edge information of the aggregate particles. Then, histogram equalization is performed on the noise-suppressed image. This redistributes the grayscale levels of the image through grayscale transformation, making the grayscale distribution more uniform and enhancing the grayscale difference between the aggregate particles and the asphalt matrix, resulting in an enhanced image. Finally, the size of the enhanced image is uniformly adjusted to a preset target size. In this embodiment, it is adjusted to 640×640 pixels to obtain a standardized image that meets the input size requirements of the instance segmentation model.

[0024] An instance segmentation model for aggregate particle identification is constructed. This embodiment uses the YOLO26n-seg instance segmentation model, and its construction and training process is as follows: Using the Labelme annotation tool, the edge contours of each aggregate particle in the standardized image are depicted in a polygonal annotation manner, so that the annotation boundary corresponds to the actual edge of the aggregate particle; each aggregate particle in the same image is treated as an independent instance, all aggregate particles use the same aggregate category label, and the unlabeled area is used as the background, generating a JSON format annotation file containing aggregate particle category information, instance information, and polygon contour coordinate information; the polygon contour coordinates corresponding to each aggregate particle in the JSON format annotation file are read, and the contour coordinates are normalized according to the width and height of the standardized image, converting the JSON format annotation file into the training labels required by the YOLO26n-seg instance segmentation model, and constructing a self-built asphalt mixture aggregate particle instance segmentation dataset; YOLO26n-seg pre-trained weights are loaded as the initial weights of the model. Transfer training is performed using a self-built asphalt mixture aggregate particle instance segmentation dataset. This allows the model to further learn the appearance features of asphalt mixture aggregate particles based on its existing feature extraction capabilities. After training, model weight parameters for aggregate particle identification are obtained.

[0025] Load the model weight parameters obtained after transfer training, input the standardized image into the YOLO26n-seg instance segmentation model to obtain the prediction confidence and instance mask corresponding to each aggregate particle; perform binarization processing on the instance mask corresponding to each aggregate particle to generate a binary instance mask corresponding to each aggregate particle. In the binary instance mask corresponding to any aggregate particle, the pixel value of the current aggregate particle region is set to 1, and the pixel value of the other regions is set to 0.

[0026] Geometric calculations are performed on the foreground pixel set in each binary instance mask to extract the contour boundary of the corresponding aggregate particle, and the dimensional features such as the projected area, major axis, and minor axis of the minimum bounding rectangle of the aggregate particle are calculated. The centroid coordinates of each aggregate particle are obtained by calculating the spatial centroid of the foreground pixels in each binary instance mask; that is, the average of the abscissas of all foreground pixels in the binary instance mask is used as the abscissa of the aggregate particle's centroid, and the average of the ordinates of all foreground pixels is used as the ordinate of the aggregate particle's centroid. These centroid coordinates serve as the positional parameters of the aggregate particle in image space.

[0027] Specifically, geometric calculations are performed on the foreground pixel set in each binary instance mask to extract the contour boundaries of the corresponding aggregate particles, including: Let the first The binary instance mask corresponding to each aggregate particle is: ,in, and These are the x and y coordinates of the pixel, respectively; when the pixel... Belongs to the When there are individual aggregate particles in a region ,otherwise . No. The set of foreground pixels for each aggregate particle is represented as: , The foreground pixel set is traversed using an eight-neighbor boundary search method. When at least one background pixel with a value of 0 exists within the eight-neighborhood of any foreground pixel, that foreground pixel is defined as a contour pixel, and the entire contour pixel constitutes the first contour pixel. The outline boundary of each aggregate particle: , in, For the first The outline boundary of each aggregate particle For pixels The set of eight neighboring pixels, For eight neighboring areas The x and y coordinates of any adjacent pixels within the particle are used to connect the contour pixels sequentially in a clockwise or counterclockwise direction to obtain the closed contour of the aggregate particle. The projected area of ​​each aggregate particle is calculated based on the number of foreground pixels in its binary instance mask: , in, For the first The pixel projection area of ​​each aggregate particle in the image. and Let be the width and height of the binary instance mask, respectively. Calculate the minimum area bounding rectangle for the given contour boundary, assuming the two side lengths of this minimum bounding rectangle are respectively... and Then the major axis of the smallest circumscribed rectangle and short axis They are respectively: , , This yields the dimensional features of the aggregate particles, including their projected area, major axis, and minor axis. The centroid coordinates of each aggregate particle are obtained by calculating the spatial centroid of the foreground pixels in each binary instance mask. The center of gravity coordinates of each aggregate particle Calculate according to the following formula: , , The average of the abscissas of all foreground pixels in the binary instance mask is used as the abscissa of the aggregate particle's centroid, and the average of the ordinates of all foreground pixels is used as the ordinate of the aggregate particle's centroid. The contour boundary is used for subsequent calculation of the minimum through-center chord length, and the centroid coordinates are used as the position parameters of the aggregate particle in the image space, and are also used for subsequent calculation of particle distances and distance chains.

[0028] S2. Establish a polar coordinate system with the centroid coordinates of the aggregate particle as the pole, and scan the outline of the aggregate particle along the circumferential direction with a preset angle step size; for any scanning direction, obtain the first intersection point of the ray in that direction with the outline, and at the same time obtain the first intersection point of the reverse extension line of that direction with the outline, calculate the distance between the two intersection points, and take it as the chord length in that scanning direction; after traversing all scanning directions, take the minimum value among all chord lengths as the minimum chord length of the aggregate particle.

[0029] Since the size of the sieve openings through which aggregate particles can pass depends on their minimum size direction, the minimum chord length can accurately characterize the size characteristics of the aggregate particles passing through the sieve openings. Subsequently, the minimum chord length is converted from pixel size to actual size according to the image size calibration relationship to determine the size of the sieve openings through which the aggregate particle can pass, and the particle size class to which the aggregate particle belongs is determined according to the preset coarse aggregate particle size classification sequence. In this embodiment, the coarse aggregate particle size classifications are 4.75mm, 9.5mm, 13.2mm, 16mm, 19mm, and 26.5mm, where the size range of the 4.75mm aggregate is 4.75mm to 9.5mm, the size range of the 9.5mm aggregate is 9.5mm to 13.2mm, and so on for the other classes; for example, when the actual size of an aggregate particle identified in the captured image is 7mm, it is determined to be a 4.75mm aggregate.

[0030] Next, the weighted mass of each aggregate particle is calculated. In this embodiment, the sphericity of each grade of aggregate used in the actual road project is determined using the AIMS II image measurement system, and the bulk density of the aggregate is also measured. Based on the geometric shape of the aggregate particles, the volume of the aggregate particles is expressed as a weighted volume of the cubic volume and the spherical volume weighted according to sphericity, to characterize the true shape of the aggregate particles between a cube and a sphere. Then, the weighted mass of the aggregate particle is calculated based on the weighted volume and the bulk density. The calculation formula is as follows: , , in, Let be the weighted mass of the aggregate particles in the i-th grade. Let be the bulk density of the i-th aggregate. Let be the weighted volume of the i-th aggregate. Let be the sphericity of the i-th aggregate. This refers to the volume of a cube calculated using equivalent particle size. In this embodiment, the equivalent particle size is the volume of a sphere calculated using the equivalent particle size. i The sieve aperture size of the +1 grade aggregate is calculated using a preset conversion factor. The formulas for calculating the volume of a cube and the volume of a sphere are as follows: , in, Let be the equivalent particle size of the i-th aggregate. For the first i +1 grade aggregate sieve aperture size, In this embodiment, the conversion factor from the preset sieve aperture size to the equivalent particle size is used. Next, the contribution factors of each aggregate size class are defined. Large-diameter aggregates play a major role in load transfer within the asphalt mixture skeleton. To quantitatively characterize the relative contribution of each aggregate size class to the skeleton, the weighted volume of the largest size aggregate is used as the normalization benchmark. Based on the weighted volume of each size class, the contribution factors of each aggregate size class are defined. Different aggregate sizes have varying degrees of influence on distribution uniformity and asphalt mixture skeleton stability. Therefore, the contribution factors of each aggregate size class are defined to characterize the relative contribution of aggregates to segregation and skeleton stability. The calculation formula is as follows: , in, , representing the contribution factor of the i-th aggregate, is used to characterize the relative contribution of the i-th aggregate particles in the asphalt mixture skeleton. Let be the sieve aperture size of the (i+1)th aggregate. The sphericity of the aggregate at the largest particle size range. This refers to the sieve aperture size corresponding to the maximum particle size range.

[0031] In this embodiment, the preset sieve aperture size to equivalent particle size conversion factor The value is 1.11, meaning the equivalent particle size of the aggregate is calculated by dividing its corresponding sieve aperture size by 1.11. This represents the sieve aperture size corresponding to the largest particle size range. Take 37.5mm, which is the maximum nominal particle size range for coarse aggregates in asphalt mixtures; Correspondingly, this refers to the sphericity of the 26.5mm aggregate, which is the measured sphericity value of the largest coarse aggregate (particle size range 26.5mm~37.5mm), and... The same AIMS II image measurement system was used for measurement. In the remaining grades, The thicknesses were 9.5mm, 13.2mm, 16mm, 19mm, 26.5mm and 37.5mm respectively, corresponding to aggregates with i values ​​of 4.75mm, 9.5mm, 13.2mm, 16mm, 19mm and 26.5mm.

[0032] like Figure 2 As shown, S3, based on the centroid coordinates of each aggregate particle obtained in step one and the particle size range and weighted mass of each aggregate particle obtained in step two, constructs an adjacency distance chain. Specifically: The number of aggregate particles is determined based on the number of binary instance masks. Let the number of aggregate particles identified be n. The centroid coordinates of each aggregate particle represent the position of each aggregate particle in the image space. The centroid coordinates are mapped to the corresponding spatial reference coordinate system of the image according to the image calibration relationship, thereby representing the distribution position of each aggregate particle in the horizontal and vertical directions of the road surface. The road surface image is divided into an equal area grid. The first aggregate particle identified in the upper left corner of the image is selected as the starting particle of the distance chain, and this starting particle is used as the current particle.

[0033] From the aggregate particles that have not yet been added to the distance chain, search for the particle that is closest to the current particle and add it to the distance chain as the next particle. The distance between the current particle and the next particle is calculated using Euclidean distance. , in, Here, represents the centroid coordinates of the current particle and the next particle, respectively; L is the distance between the current particle and the next particle. Calculate the weighted mass of the current particle multiplied by the distance, which is used as the torque corresponding to the current particle; then, using the next particle as the new current particle, repeat the above search, distance measurement, and torque calculation process until all aggregate particles have been traversed.

[0034] Specifically, let the weighted mass of the starting particle be... The distance between the second particle, which is closest to the starting particle, and the starting particle is... The corresponding torque is Using the second particle as the new current particle, the weighted mass of the second particle is... The distance between the second particle and the nearest third particle is The corresponding torque is ; Calculate the distance and torque between adjacent particles in sequence according to the above method. When the number of untraversed aggregate particles is 0, the traversal ends. Link each calculated distance chain in sequence on the image geometry to obtain a distance chain that traverses all aggregate particles.

[0035] When multiple candidate particles at the same distance exist around a given aggregate particle, a particle size priority principle is adopted for linking to ensure the uniqueness and continuity of the distance chain construction result. Specifically, the following priority order is used for sequential judgment: First level: compare the particle size range of each candidate particle. Since large-diameter aggregates bear the main load transfer role in the asphalt mixture skeleton, prioritizing the linking of large-diameter aggregates can better reflect the mechanical characteristics of the skeleton structure. Therefore, the candidate particle with the highest particle size range is selected as the next particle. Second level: when multiple candidate particles belong to the same particle size range, calculate the angle between the direction of the line connecting each candidate particle to the current particle and the direction of the current chain. Select the candidate particle with the smallest angle as the next particle to ensure chain continuity. The direction of the current chain is the direction from the previous particle in the distance chain to the current particle. Third level: if it is still impossible to distinguish after the second level of judgment, the next particle is determined according to the image scanning order from left to right and from top to bottom.

[0036] When some aggregate particles near the image boundary have their outlines truncated, in order to avoid spatial position calculation errors due to missing outlines, the validity of the aggregate particles at the boundary is determined: aggregate particles that are in contact with the image boundary and have missing outline information are determined as invalid particles, and invalid particles do not participate in the construction of the distance chain; the remaining fully identified aggregate particles are determined as valid particles, and valid particles participate in the construction of the distance chain and subsequent uniformity calculation.

[0037] like Figure 3 , Figure 4 As shown in step S4, based on the distance chain constructed in step three and the torque of each link calculated, the asphalt mixture surface image is divided into multiple zones. Specifically, the asphalt mixture surface image is divided into four equal-area rectangular zones along the longitudinal and transverse midlines using a bisection method. That is, the image is divided into four rectangular areas of equal area using the longitudinal and transverse midlines as the dividing lines, and labeled as zones A, B, C, and D respectively. By dividing the image into equal-area zones, each zone has the same statistical reference area, providing a unified spatial reference for subsequent comparison of aggregate distribution differences between local areas and the entire image.

[0038] Construct partition distance chains within each partition. The construction method for the distance chains within each partition is the same as in step three: within each partition, select the first aggregate particle identified in the upper left corner of the partition as the starting particle of the partition distance chain, search for the particle with the closest distance to the current particle from the aggregate particles that have not yet been added to the partition distance chain as the next particle, calculate the distance and torque between adjacent particles, and traverse all aggregate particles in the partition in turn; link each link in the partition distance chain in the image geometry in turn to obtain the partition distance chain corresponding to the distribution of the aggregate skeleton. At the same time, construct the distance chain in the entire image in the same way as in step three to obtain the full image distance chain.

[0039] Count the number of aggregate particles in each zone and the total number of aggregate particles in the entire map. Let the number of aggregate particles in zones A, B, C, and D be a, b, c, and d, respectively, and the total number of aggregate particles in the entire map be e. Calculate the sum of the torques in each zone and the sum of the torques in the entire map using the following formula: , in, This is the region identifier, and its value can be any partition or the entire map. For the region The sum of the torques of the inner distance chain, For the region The number of internal aggregate particles, For the region Inner distance chain The weighted mass of each link corresponds to a particle. For the first The distance between the current particle and the next particle in each link, where the region The number of internal aggregate particles is At that time, the corresponding distance chain contains One link, The contribution factor is the particle size range to which the r-th link corresponds.

[0040] Specifically, the area identifier Taking partitions A, B, C, and D, and the entire graph respectively, the sum of moments for partitions A, B, C, and D, and the sum of moments for the entire graph are respectively: , , in, The sum of the torques for zones A, B, C, and D are respectively. The sum of the torques across the entire graph. , Let A represent the weighted mass of the particle corresponding to the i-th link in the chain and its distance from the next particle, respectively. , Let B be the weighted mass of the particle corresponding to the j-th link in the chain and its distance from the next particle, respectively. , These represent the weighted mass of the particle corresponding to the k-th link in the chain, and its distance from the next particle, respectively, for partition C. , These are the distance chains in partition D, respectively. Each link corresponds to the weighted mass of the particle and its distance from the next particle. , These represent the weighted mass of the particle corresponding to the q-th link in the global distance chain and its distance to the next particle, respectively.

[0041] S5. Based on the sum of the moments in each zone and the number of chain links in the corresponding distance chain within that zone, calculate the average moment of each zone. This is achieved by dividing the sum of the moments in each zone by the number of chain links in the corresponding distance chain, thus obtaining the average moment per chain link in each zone. This average moment is then used as the average level of the sum of moments in the corresponding zone. Similarly, the sum of the moments across the entire map is divided by the number of chain links in the entire map to obtain the average moment per chain link across the entire map. This average moment across the entire map is used as a normalization benchmark to characterize the theoretically uniform distribution state. When the aggregate particles are ideally uniformly distributed on the road surface, the average moment of each zone tends to be consistent with the average moment of the entire map. When aggregate accumulation or sparseness occurs in a local area, the average moment of the corresponding zone will deviate from the average moment of the entire map. This allows for the quantification of local differences in aggregate distribution.

[0042] The aggregate distribution uniformity index is determined based on the ratio of the average moment of each zone to the average moment of the entire map. This index is used to evaluate the aggregate distribution uniformity of asphalt mixtures. The expression for the aggregate distribution uniformity index is as follows: , in, The aggregate distribution uniformity index. , , and These are the sums of the torques of the distance chains within the corresponding partitions. , , and These represent the number of aggregate particles within the corresponding zones. This represents the total number of aggregate particles within the entire map area. Let m be the torque of the m-th link in the chain that is the distance to the entire graph. This is represented by the torques of all links in the distance chain across the entire graph, starting from the first link. These represent the summation of the torques of each link within partitions A, B, C, and D, respectively.

[0043] The uniformity of aggregate distribution in asphalt mixtures is evaluated based on the aggregate distribution uniformity index. Characterizes the degree of deviation of the actual distribution moment of aggregates from the theoretical uniform distribution state. The smaller the value, the closer the aggregate distribution in each zone is to the average state of the whole map, and the more uniform the aggregate distribution. The larger the value, the more obvious the local aggregate accumulation or sparsity, and the more severe the segregation of the asphalt mixture; This corresponds to an ideal uniform distribution.

[0044] Example 2 The difference between this embodiment and Embodiment 1 is that this embodiment provides a simulation experiment of a deep learning-based method for monitoring the aggregate distribution of asphalt mixtures; The weighted mass and contribution factor were calculated using measured data of aggregates used in actual road engineering projects, and the uniformity of aggregate distribution on the paved surface was evaluated.

[0045] Specifically, the basic physical parameters were first obtained. The bulk relative density of each grade of coarse aggregate used in the actual road project was measured, and the results are shown in Table 1. The sphericity of each grade of aggregate was measured using the AIMS II image measurement system, and the results are shown in Table 2.

[0046] Table 1. Bulk Relative Density of Coarse Aggregates

[0047] Table 2. Sphericity of Aggregates

[0048] The weighted volume and weighted mass of each aggregate grade were calculated based on the measured data in Tables 1 and 2. Taking the 4.75mm aggregate grade as an example, Tables 1 and 2 show the bulk density of this aggregate grade. It is 2.703 g / cm³, and its sphericity is... It is 0.700, and its corresponding number is... i +1 Screen aperture size Pi +1 is 9.5mm; Substitute the above parameters into the cube volume formula. Formula for the volume of a sphere Weighted volume formula and weighted mass formula The cubic volume, spherical volume, weighted volume, and weighted mass of the aggregate in each size range are calculated sequentially. Using the same method, the sieve size, sphericity, and bulk density of the remaining aggregate sizes are substituted into the above formula to obtain the weighted volume and weighted mass of all aggregate size ranges. Then, using the weighted volume of the largest aggregate size range as the normalization benchmark, the sieve size and sphericity of each aggregate size range are substituted into the contribution factor formula to calculate the contribution factor of each aggregate size range. .

[0049] The acquired paving surface images were preprocessed, segmented, and their particle size ranges determined according to the steps described in Example 1. Adjacent distance chains were constructed, and multi-zone torque summation statistics were performed sequentially. The weighted mass of the particles corresponding to each chain segment, the chain segment distance, and the contribution factor of the particle size range were substituted into the torque summation formula to obtain the torque summation for the four zones A, B, C, and D, as well as the torque summation for the entire image. Then, the torque summation for each zone, the number of aggregate particles in each zone and the entire image, and the torque summation for the entire image were substituted into the aggregate distribution uniformity index formula to calculate the paving uniformity index. The calculated result is close to 0, indicating that the average moment of each zone on the paved surface deviates little from the average moment of the entire surface, the aggregate distribution is close to the theoretical uniform distribution, and the paving quality is good. This simulation experiment shows that the monitoring method of this invention can provide real-time and quantitative evaluation of the apparent segregation degree of asphalt mixture paving, with a high degree of automation and objective and accurate results.

[0050] Example 3 A deep learning-based asphalt mixture aggregate distribution monitoring system includes: The image acquisition and segmentation module is used to acquire and preprocess images of the asphalt mixture surface during the paving process. The preprocessed images are then input into the instance segmentation model trained by transfer learning to obtain the binary instance mask corresponding to each aggregate particle. The feature extraction and weighting module is used to extract the contour, size features and centroid coordinates of each aggregate particle based on the binary instance mask, determine the particle size range of each aggregate particle, and calculate the weighted mass of each aggregate particle. The distance chain construction module is used to determine the spatial position of each aggregate particle using the centroid coordinates, select the starting particle of the distance chain, calculate the distance and torque between adjacent particles based on the starting particle, until all aggregate particles are traversed to form a distance chain. The partition statistics module is used to divide the surface image into several equal-area partitions, construct partition distance chains within each partition, and calculate the torque sum of each partition and the torque sum of the entire image. The uniformity evaluation module is used to calculate the aggregate distribution uniformity index based on the relative deviation between the average level of the sum of moments in each zone and the average level of the sum of moments across the entire map, and to evaluate the aggregate distribution uniformity of asphalt mixtures.

[0051] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for monitoring aggregate distribution in asphalt mixtures based on deep learning, characterized in that, include: S1. Collect images of the asphalt mixture surface during the paving process and preprocess them. Input the preprocessed images into the instance segmentation model trained by transfer learning to obtain the binary instance mask corresponding to each aggregate particle. S2. Extract the outline, size features and centroid coordinates of each aggregate particle based on the binary instance mask, determine the particle size range of each aggregate particle, and calculate the weighted mass and contribution factor of each aggregate particle. S3. Determine the spatial position of each aggregate particle using the center of gravity coordinates, select the starting particle of the distance chain, calculate the distance and torque between adjacent particles based on the starting particle, until all aggregate particles are traversed to form a distance chain. S4. Divide the surface image into several equal-area partitions, construct partition distance chains within each partition, and calculate the sum of moments for each partition and the sum of moments for the entire image. S5. Based on the relative deviation between the average level of the sum of moments in each zone and the average level of the sum of moments across the entire map, calculate the aggregate distribution uniformity index to evaluate the aggregate distribution uniformity of the asphalt mixture.

2. The method for monitoring aggregate distribution in asphalt mixtures based on deep learning according to claim 1, characterized in that, The process of acquiring and preprocessing images of the asphalt mixture surface during paving includes cropping the asphalt mixture surface image based on the short side to obtain a cropped image, and converting the cropped image into a grayscale image using a weighted average method. Median filtering is used to suppress noise in grayscale images, and histogram equalization is performed on the noise-suppressed images to obtain enhanced images. The size of the enhanced images is then uniformly adjusted to a preset target size to obtain standardized images.

3. The method for monitoring aggregate distribution in asphalt mixtures based on deep learning according to claim 1, characterized in that, The process of determining the particle size range of each aggregate particle includes establishing a polar coordinate system with the centroid coordinate as the pole, scanning the contour along the circumference with a preset angle step size, obtaining the first intersection point of the ray in this direction with the contour, and the first intersection point of the reverse extension line in this direction with the contour, and calculating the distance between the two intersection points as the chord length in this scanning direction. Traverse all scanning directions and take the minimum value among all the through-chord lengths as the minimum through-chord length of the aggregate particle; Based on the image size calibration relationship, the minimum chord length is converted into the actual size to determine the sieve aperture size that the aggregate particle can pass through, and the particle size class to which the aggregate particle belongs is determined according to the preset coarse aggregate particle size classification sequence.

4. The method for monitoring aggregate distribution in asphalt mixtures based on deep learning according to claim 3, characterized in that, The calculation of the weighted mass and contribution factor of each aggregate particle includes determining the sphericity and bulk density of aggregates at each particle size range. Based on the geometric shape of the aggregate particles, the volume of the aggregate particles is expressed as a weighted volume of a cube volume and a sphere volume weighted by sphericity. The weighted mass of the aggregate particle is then calculated based on the weighted volume and bulk density. The expression for the weighted mass is as follows: ， ， in, Let be the weighted mass of the aggregate particles in the i-th grade. Let be the bulk density of the i-th aggregate. Let be the weighted volume of the i-th aggregate. Let be the sphericity of the i-th aggregate. This refers to the volume of a cube calculated using equivalent particle size. This is the volume of a sphere calculated using the equivalent particle size.

5. The method for monitoring aggregate distribution in asphalt mixtures based on deep learning according to claim 4, characterized in that, The calculation of the weighted mass and contribution factor of each aggregate particle also includes defining the contribution factor of each aggregate size range based on the weighted volume of the largest particle size range aggregate as a normalization benchmark, and the calculation formula of the contribution factor is as follows: ， in, , representing the contribution factor of the i-th aggregate, is used to characterize the relative contribution of the i-th aggregate particles in the asphalt mixture skeleton. Let be the sieve aperture size of the (i+1)th aggregate. This is the preset conversion factor from sieve aperture size to equivalent particle size. The sphericity of the aggregate at the largest particle size range. This refers to the sieve aperture size corresponding to the maximum particle size range.

6. The method for monitoring aggregate distribution in asphalt mixtures based on deep learning according to claim 1, characterized in that, Step S3 specifically includes: S31. Determine the number of aggregate particles based on the number of binary instance masks, and select the first aggregate particle identified in the upper left corner of the image as the starting particle of the distance chain. S32. From the aggregate particles that have not yet been added to the distance chain, search for the particle closest to the current particle according to the particle size priority principle and add it to the distance chain as the next particle. The distance between the current particle and the next particle is calculated using Euclidean distance. S33. Calculate the weighted mass of the current particle and the distance as the torque corresponding to the current particle; S34. Take the next particle as the new current particle and repeat steps S32 to S33 until all aggregate particles have been traversed.

7. The method for monitoring aggregate distribution in asphalt mixtures based on deep learning according to claim 6, characterized in that, The particle size priority principle is determined in the following order of priority: The first stage involves comparing the particle size ranges of each candidate particle and selecting the candidate particle with the highest particle size range as the next particle. The second level is to calculate the angle between the line connecting each candidate particle to the current particle and the current chain direction when multiple candidate particles belong to the same particle size range. The candidate particle with the smallest angle is selected as the next particle. The current chain direction is the direction from the previous particle in the chain to the current particle. The third stage involves determining the next particle from left to right and from top to bottom according to the image scanning sequence.

8. The method for monitoring aggregate distribution in asphalt mixtures based on deep learning according to claim 1, characterized in that, Step S4 includes using the bisection method to divide the asphalt mixture surface image into multiple equal-area rectangular partitions along the longitudinal and transverse midlines, constructing partition distance chains in each partition, and sequentially linking each link in the partition distance chain geometrically to obtain partition distance chains corresponding to the aggregate skeleton distribution. The number of aggregate particles in each zone and the total number of aggregate particles in the entire map are counted separately. The sum of the torques in each zone and the sum of the torques in the entire map are calculated. The expression for the sum of the torques is as follows: ， in, This is the region identifier, and its value can be any partition or the entire map. For the region The sum of the torques of the inner distance chain, For the region The number of internal aggregate particles, For the region Inner distance chain The weighted mass of each link corresponds to a particle. For the first The distance between the current particle and the next particle in each link. The contribution factor is the particle size range to which the r-th link corresponds.

9. The method for monitoring aggregate distribution in asphalt mixtures based on deep learning according to claim 1, characterized in that, The evaluation of the uniformity of aggregate distribution in asphalt mixtures includes calculating the average moment of each zone based on the sum of the moments of each zone and the number of chain links of the distance chain within the corresponding zone, and using each average moment as the average level of the sum of moments of the corresponding zones. The average moment across the entire map is calculated and used as a normalized benchmark to characterize the uniform distribution state. Based on the ratio of the mean of the average moment in each zone to the average moment across the entire map, an aggregate distribution uniformity index is determined to evaluate the aggregate distribution uniformity of asphalt mixtures. The expression for the aggregate distribution uniformity index is as follows: ， in, The aggregate distribution uniformity index. , , and These are the sums of the torques of the distance chains within the corresponding partitions. , , and These represent the number of aggregate particles within the corresponding zones. This represents the total number of aggregate particles within the entire map area. Let m be the torque of the m-th link in the chain that is the distance to the entire graph. This means that, starting from the first link, the torques of all links in the distance chain across the entire graph are accumulated one by one.

10. A deep learning-based asphalt mixture aggregate distribution monitoring system, executing the method of claim 1, characterized in that, include: The image acquisition and segmentation module is used to acquire and preprocess images of the asphalt mixture surface during the paving process. The preprocessed images are then input into the instance segmentation model trained by transfer learning to obtain the binary instance mask corresponding to each aggregate particle. The feature extraction and weighting module is used to extract the contour, size features and centroid coordinates of each aggregate particle based on the binary instance mask, determine the particle size range of each aggregate particle, and calculate the weighted mass of each aggregate particle. The distance chain construction module is used to determine the spatial position of each aggregate particle using the centroid coordinates, select the starting particle of the distance chain, calculate the distance and torque between adjacent particles based on the starting particle, until all aggregate particles are traversed to form a distance chain. The partition statistics module is used to divide the surface image into several equal-area partitions, construct partition distance chains within each partition, and calculate the torque sum of each partition and the torque sum of the entire image. The uniformity evaluation module is used to calculate the aggregate distribution uniformity index based on the relative deviation between the average level of the sum of moments in each zone and the average level of the sum of moments across the entire map, and to evaluate the aggregate distribution uniformity of asphalt mixtures.

Images