Cement concrete pavement maintenance decision-making system and method based on vehicle-mounted image

Abstract

The invention discloses a cement concrete pavement maintenance decision-making system and method based on a vehicle-mounted image, and particularly relates to the technical field of intelligent traffic and maintenance management, and the method comprises the steps that a vehicle-mounted image collection module collects a pavement image through a multispectral sensor array and carries out standardization processing; the pavement state quantification module is used for realizing pixel-level segmentation of four types of diseases including cracks, plate corner fracture, joint damage and surface peeling by utilizing a multi-task deep learning network, extracting geometric features, calculating a disease severity index and generating a road section-level quantification report; the maintenance strategy generation module matches and outputs a preliminary maintenance scheme based on the multi-dimensional knowledge base; the decision optimization module fuses traffic, weather and resource data, constructs a multi-target integer programming model for optimization solution, and outputs a comprehensive optimal maintenance scheme and matched construction scheduling and traffic organization suggestions. According to the invention, precise identification, intelligent decision and construction resource collaborative optimization of pavement diseases are realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and maintenance management technology, and more specifically, to a decision-making system and method for cement concrete pavement maintenance based on vehicle-mounted images. Background Technology

[0002] With the rapid development of highway transportation networks, the maintenance and management of cement concrete pavements has become an increasingly important aspect of ensuring road safety and service life. In recent years, pavement defect detection technology based on image recognition has gradually become a research hotspot, providing the possibility for automated assessment of pavement conditions.

[0003] Existing technical solutions typically use vehicle-mounted cameras to collect road surface images, employ a single deep learning model to identify and classify typical defects such as cracks, set fixed thresholds based on simple statistical information of the defects, match them with a pre-set, relatively simple maintenance rule library, and recommend general maintenance processes. The implementation process generally involves image acquisition, defect identification, rule matching, and outputting suggested solutions, which to some extent automates the process from detection to preliminary recommendations.

[0004] However, in practical use, it still has some shortcomings, such as limited accuracy in identifying single types of defects, difficulty in effectively distinguishing complex defects with different forms such as cracks, corner fractures, joint damage, and surface peeling, and susceptibility to interference from light and shadow. The decision-making process is simplistic and crude, usually relying on only a few static rules without comprehensively considering key factors such as combinations of multiple defects, spatial distribution of severity, real-time traffic impact, refined meteorological conditions, and dynamic resource constraints. This results in maintenance plans that are often cost-effective and have poor construction feasibility, making them difficult to directly apply and optimize in complex real-world environments. Therefore, there is an urgent need for an integrated system and method that can achieve accurate quantification of multiple defects and integrate multi-source dynamic information for intelligent optimization decision-making. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a cement concrete pavement maintenance decision-making system and method based on vehicle-mounted images, which solves the problems mentioned in the background art through the following solutions.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a cement concrete pavement maintenance decision system based on vehicle-mounted images, including a vehicle-mounted image acquisition module: a multispectral image sensor array is integrated at the front end of the maintenance vehicle to acquire original images of the road surface ahead of the vehicle in a constant interval triggering mode, and outputs a standardized road surface image sequence after processing; The road surface condition quantification module receives the standardized road surface image sequence, inputs it into a multi-task deep learning segmentation network, and outputs pixel-level semantic segmentation masks for four types of defects: cracks, corner fractures, joint damage, and surface peeling. It calculates three geometric features: area occupancy, average width, and dispersion of extension direction. Based on a preset feature-weight mapping table, it calculates the severity index of a single defect. Based on the geolocation encoding of the image sequence, it performs spatial clustering and statistical analysis of the severity index of a single defect by lane and mileage, and generates a comprehensive quantitative report of road surface condition at the road segment level. Maintenance strategy generation module: It has a built-in multi-dimensional maintenance decision rule knowledge base. It takes the multi-disease combination matrix, severity index interval and spatial distribution pattern of each disease in the comprehensive quantitative report of road surface condition at the road section level as input conditions. Through the pattern matching engine, it matches the condition rules in the quantitative report and the knowledge base, and outputs a preliminary set of feasible maintenance process schemes. Decision optimization module: It integrates real-time traffic flow monitoring data, refined meteorological data for the next 72 hours, and current available maintenance manpower, equipment, and material resources status data. Based on the preliminary set of maintenance plans as the solution space, it constructs a multi-objective integer programming model, and iteratively solves the model with material cost, traffic delay cost, resource switching cost, and meteorological risk probability as key constraints. It outputs the target maintenance decision plan and simultaneously generates construction sequence arrangements, resource allocation paths, and dynamic traffic organization suggestions.

[0007] The cement concrete pavement maintenance decision-making method based on vehicle-mounted images includes S1: pavement image acquisition and standardization processing. The original pavement images are acquired at preset intervals through a vehicle-mounted multispectral sensor array. Combined with color temperature correction, geometric distortion elimination and adaptive contrast enhancement technology, a standardized image sequence with high-precision positioning information is generated. S2: Use a multi-task deep learning model to perform pixel-level segmentation of image sequences, identify four types of defects: cracks, corner fractures, joint damage, and surface peeling, extract the geometric features of each defect, calculate the defect severity index based on the pre-trained feature-weight mapping table, and generate a road segment-level quantitative report based on spatial clustering analysis. S3: Knowledge-based maintenance strategy matching, which matches the disease combination, severity and distribution pattern in the quantitative report with the multi-dimensional maintenance decision rule knowledge base, and outputs a preliminary set of feasible maintenance process solutions, including process type, material specifications, engineering quantity and cost and time data. S4: Integrate real-time traffic flow, refined weather forecasts, and maintenance resource status data. Based on the preliminary plan, construct a multi-objective optimization model with the core objectives of maximizing the cost-benefit ratio and minimizing traffic impact. Solve the model under multiple constraints of cost, resources, and weather to output the comprehensive optimal maintenance decision plan. S5: Based on the optimization plan, develop detailed construction sequence, resource allocation routes and time-based traffic control plans, and output complete maintenance implementation guidance documents including construction Gantt charts, resource scheduling charts and traffic impact assessment reports.

[0008] The technical effects and advantages of this invention are as follows: This invention uses an on-board multispectral image sensor and an improved multi-task deep learning segmentation network to identify four typical defects—cracks, corner fractures, joint damage, and surface peeling—in parallel and with high precision. It automatically extracts multidimensional geometric features and calculates a scientific and objective defect severity index, effectively overcoming the problems of strong subjectivity and low efficiency in traditional manual inspections, as well as the single type and insufficient accuracy of existing image recognition methods. This provides a reliable data foundation for subsequent decision-making. This invention achieves comprehensive optimization under multiple objectives by constructing and solving a multi-objective integer programming model. The output decision scheme takes into account the optimality of technical feasibility, economy and implementation timing, and significantly improves the scientific level and practical operability of maintenance decisions. This invention outputs the optimal maintenance plan and simultaneously generates detailed construction sequence Gantt charts, optimal resource allocation paths, and time-segmented dynamic traffic organization suggestions. This achieves seamless connection and coordinated scheduling of all aspects of road surface maintenance, including defect identification, decision optimization, and construction execution. It greatly improves the planning of maintenance operations and the efficiency of resource utilization, minimizes the interference of construction on traffic operations, and forms a new model of intelligent and refined road surface maintenance management. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a schematic diagram of the image acquisition and processing of the present invention; Figure 3 This is a schematic diagram illustrating the disease identification and quantification method of the present invention; Figure 4 This is a schematic diagram illustrating the matching of maintenance strategies according to the present invention; Figure 5 This is a schematic diagram illustrating the decision optimization of the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] As attached Figure 1 Appendix Figure 2The cement concrete pavement maintenance decision system shown includes an on-board image acquisition module: a multispectral image sensor array is integrated at the front of the maintenance vehicle to acquire original images of the road surface ahead of the vehicle in a constant interval triggering mode, and outputs a standardized road surface image sequence after processing.

[0012] It should be specifically noted that the vehicle-mounted image acquisition module performs adaptive white balance correction based on the road surface reference color temperature, eliminates geometric distortion based on sensor calibration parameters, and enhances contrast based on regional grayscale statistics on the acquired raw images.

[0013] It should be further explained that a vehicle-mounted mobile measurement platform is used, which is installed on a maintenance and inspection vehicle. A multispectral image sensor array is integrated in the center of the front bumper of the vehicle. The sensor array consists of three industrial-grade CMOS sensors, corresponding to the visible light band (400-760nm), the near-infrared band (780-1100nm), and the short-wave infrared band (1100-2500nm), respectively. The sensor frame rate is set to 30fps, the resolution is 4096×3072 pixels, the lens focal length is 25mm fixed focus lens, the shooting distance is 1.5-2.0m, and the road surface image pixel resolution reaches 0.1mm / pixel, which meets the identification requirements of fine cracks (width ≥0.1mm).

[0014] The vehicle's real-time position is obtained through an onboard high-precision GPS unit (positioning accuracy ±1cm). Combined with an inertial navigation unit (sampling rate 100Hz), the positional offset caused by vehicle bumps and turns is compensated. The sampling interval is set to 0.5m. When the vehicle's travel distance is 0.5m from the previous sampling point, the sensor array is triggered to capture images synchronously, ensuring that there is a 10% overlap between adjacent images. This provides a basis for subsequent image stitching and spatial positioning. At the same time, the sensor array is equipped with an active fill light unit, which adaptively adjusts the fill light power (adjustment range 50-300W) according to the ambient light intensity to avoid the impact of strong light glare and weak light blur on image quality.

[0015] The road surface reference color temperature dataset was selected as the basis for calibration. Color temperature samples of healthy cement concrete pavement were collected in advance under different regions (humid and hot areas in the south and cold areas in the north), different time periods (early morning, noon, evening), and different weather conditions (sunny, cloudy, rainy). A color temperature sample library with more than 100,000 samples was constructed. The sample dimensions include three core parameters: ambient light color temperature, road surface reflected light color temperature, and atmospheric transmittance.

[0016] During preprocessing, the color temperature features of the currently acquired image are extracted using the image grayscale world algorithm, and similarity matching is performed with the sample library to determine the reference color temperature range corresponding to the current scene. The gamma correction algorithm is then used to dynamically adjust the gain of the image's RGB channels to achieve adaptive white balance correction.

[0017] Sensor calibration parameter sets and road surface smoothness auxiliary correction data were selected. The sensor calibration parameter set was pre-acquired using the Zhang calibration method and includes intrinsic parameters (focal length, principal point coordinates, distortion coefficients) and extrinsic parameters (relative attitude angle and distance between the sensor and the road surface). The road surface smoothness auxiliary correction data came from real-time data collected by a vehicle-mounted laser smoothness meter (measurement accuracy ±0.1mm). During processing, preliminary radial and tangential distortion corrections were performed on the image based on the sensor intrinsic parameters. Combined with the road surface undulation data collected by the laser smoothness meter, a three-dimensional road surface undulation model was constructed. Secondary geometric corrections were performed on the pre-corrected image. Local coordinate mapping and pixel resampling were then performed on the pre-corrected image to compensate for local image stretching / compression distortion caused by road surface unevenness, providing an accurate image basis for subsequent measurement of the geometric features of road defects.

[0018] Image region grayscale statistics and disease grayscale feature thresholds were selected as the processing basis. The region grayscale statistics were obtained by dividing the image into 16×16 sub-regions and calculating the mean, variance, and entropy of grayscale in each sub-region. The disease grayscale feature thresholds were obtained by combining the Otsu algorithm with historical disease image samples. The specific steps were as follows: effective samples covering four types of diseases with different severity and different lighting conditions were screened from the historical database. After grayscale conversion, noise and interference, including markings, were removed. The preprocessed samples were input into the algorithm to find the grayscale value that maximizes the variance between the disease foreground and the healthy background as the initial threshold. Multiple sets of initial thresholds were collected, and after removing outliers, the interval mean was taken to obtain the final threshold.

[0019] During processing, the adaptive histogram equalization (CLAHE) algorithm is used to dynamically adjust the equalization parameters according to the gray-level characteristics of different sub-regions. The contrast gain is increased for dark areas with a gray-level variance of less than 20 (such as under the shade of trees or inside cracks), while the gain is reduced for bright areas with a gray-level variance of greater than 80 (such as areas under strong midday sunlight). At the same time, the intensity is enhanced based on the threshold constraint of the gray-level characteristics of the disease to avoid noise amplification caused by excessive enhancement.

[0020] After preprocessing, a standardized road surface image sequence is output. Each frame of the image is accompanied by GPS positioning information (latitude, longitude, and altitude), acquisition timestamp, and sensor parameter data. The storage format is TIFF (lossless compression) for easy access by subsequent modules.

[0021] As attached Figure 3As shown, the pavement condition quantification module receives the standardized pavement image sequence, inputs it into a multi-task deep learning segmentation network, and outputs pixel-level semantic segmentation masks for four types of defects: cracks, corner fractures, joint damage, and surface peeling. It calculates three geometric features: area occupancy, average width, and dispersion of extension direction. Based on a preset feature-weight mapping table, it calculates the severity index of a single defect. Based on the geolocation encoding of the image sequence, it performs spatial clustering and statistical analysis of the severity index of a single defect by lane and mileage, and generates a comprehensive quantitative report of pavement condition at the road segment level.

[0022] It should be specifically noted that the multi-task deep learning segmentation network is an improvement on the U-Net++ framework, including the introduction of a cross-scale feature fusion unit to adjust the feature maps of different levels of the encoder to the same scale for fusion; setting independent segmentation branches for the four types of diseases at the decoder output, and adding an adaptive threshold adjustment layer after each branch to dynamically adjust the segmentation threshold according to the disease difficulty level of the input image; and using a combination of Dice loss and Focal loss as the loss function for network training.

[0023] The feature-weight mapping table is obtained by training on historical road disease geometric feature data and corresponding maintenance cost and effect data using the random forest algorithm. The feature weights in the table are dynamically adjusted according to the road segment level.

[0024] It should be further explained that a pavement defect segmentation dataset of over 80,000 standardized pavement images was constructed. Each image was pixel-level annotated, with annotation categories including four types of defects: cracks (longitudinal, transverse, and mesh-like), corner fractures, joint damage (filler detachment, joint opening), and surface peeling, as well as background (healthy pavement, road markings, and debris). The dataset was expanded to over 320,000 images using data augmentation techniques, including random rotation of -15° to -15°, scaling by 0.8 to 1.2 times, flipping, adding Gaussian noise with a variance of 0.01 to 0.03, and multispectral band fusion to simulate image features under different lighting conditions. Simultaneously, a "defect difficulty level label" was introduced, categorizing defects into three levels—simple, medium, and complex—based on defect size and clarity, for training purposes. The adaptive weight allocation during the process is based on the following specific grading standards: Simple: In terms of size, crack width ≤ 0.3mm, area ≤ 0.1㎡, and other surface-like diseases area ≤ 0.5㎡; in terms of clarity, disease edges are clear, with high contrast with the background and no obvious occlusion or noise interference; Medium: In terms of size, crack width 0.3-1.0mm, area 0.1-0.5㎡, and other surface-like diseases area 0.5-2.0㎡; in terms of clarity, disease edges are relatively clear, with slight occlusion in some areas, and medium contrast with the background; Complex: In terms of size, crack width > 1.0mm, area > 0.5㎡, and other surface-like diseases area > 2.0㎡; in terms of clarity, disease edges are blurred, with severe occlusion or strong noise, low contrast with the background, and easily confused with other diseases.

[0025] Based on the U-Net++ framework, the following structural units are introduced: Cross-scale feature fusion unit: This unit adjusts the feature maps (C1-C5) from different levels of the encoder to the same scale through transposed convolution; Multi-task branch design unit: Four independent segmentation branches are set at the decoder output, corresponding to four types of diseases. An adaptive threshold adjustment layer is added after each branch to dynamically adjust the segmentation threshold according to the difficulty level of the disease in the input image. Simple diseases use the baseline segmentation threshold (default 0.5) without additional adjustment; medium diseases have the threshold lowered by 0.05-0.1 to balance recognition completeness and accuracy; complex diseases have the threshold further lowered by 0.1-0.15; Loss function optimization unit: This unit uses a combined loss function of "Dice loss + Focal loss," where Dice loss addresses the low pixel ratio of diseases, and Focal loss improves the segmentation accuracy of complex disease regions.

[0026] The network was trained using the AdamW optimizer with an initial learning rate of 1e-4. A cosine annealing learning rate scheduling strategy was adopted, with 100 training epochs and a batch size of 16.

[0027] For the pixel-level semantic segmentation masks of the four types of defects output, three core geometric features are extracted: area occupancy, average width, and dispersion of extension direction. The specific calculation methods are as follows: Area occupancy: The connected region labeling algorithm (8-neighborhood) is used to extract the connected regions of the defects, calculate the number of pixels in the region, convert it into the actual area by combining the image pixel resolution, and then divide it by the total area of ​​the corresponding road surface area to obtain the area occupancy; Average width: For linear defects including cracks and joint damage, the "skeleton extraction + equidistant sampling" method is used. Every 5 pixels on the defect skeleton are sampled, and the defect width (pixel distance perpendicular to the skeleton direction) at each sampling point is calculated. The average value is taken and converted into the actual width; For planar defects including corner fractures and surface peeling, the "equivalent width" (area / equivalent length) is used for calculation; Dispersion of extension direction: Principal component analysis (PCA) is used to analyze the pixel coordinates of the defect skeleton, and the variance ratio of the main direction to the auxiliary direction is calculated. The smaller the variance ratio, the more dispersed the extension direction (such as network cracks).

[0028] A "feature-weight mapping table" was constructed, trained using historical maintenance data (disease development trends, maintenance costs, and effects of different road sections over 5 years) and a random forest algorithm. Specific data included: historical disease geometric feature data, corresponding road section maintenance process types, maintenance costs, post-maintenance disease repair rates, and service life extension data. In the mapping table, the feature weights for different disease types differed, and the weights were dynamically adjusted according to the road section level (expressway, national highway, county road). Based on the mapping table, a weighted summation formula was used to calculate the single disease severity index (SSI), with a value range of 0-10, where 0-3 represents mild disease, 3-7 represents moderate disease, and 7-10 represents severe disease.

[0029] A dual-dimensional spatial coding system of "lane-mileage" is introduced to convert GPS positioning information of standardized image sequences into specific lane numbers (e.g., two-way four-lane roads are divided into lanes 1-4) and mileage markers (accuracy ±0.5m). The density clustering algorithm (DBSCAN) is used to spatially cluster the single disease severity index of the same lane and adjacent mileage. The clustering parameters (neighborhood radius, minimum number of samples) are adaptively set according to the road segment lane width (3.75m standard lane) and the acquisition interval (0.5m). Disease areas with a distance of less than 1m and the same severity index level are clustered into a disease unit to avoid duplicate statistics caused by image overlap.

[0030] Based on clustering results, a comprehensive quantitative report on pavement condition at the road segment level is generated. The report includes basic road segment information: segment name, mileage range, number of lanes, and pavement construction year; a multi-disease combination matrix: the distribution and proportion of mild / moderate / severe defects for four types of defects; statistics on the severity index intervals for each defect (e.g., mild cracks account for 65%, moderate joint damage accounts for 28%); spatial distribution heatmap data: with mileage as the horizontal axis and lanes as the vertical axis, the severity index distribution of different defects is marked; and defect development trend prediction data: based on historical data from the same period, an LSTM neural network is used to predict the change in defect severity index over the next 6 months. The report is stored in JSON format and also generates visual charts (bar charts, heatmaps), providing intuitive and accurate input for subsequent maintenance strategy generation.

[0031] As attached Figure 4 As shown, the maintenance strategy generation module has a built-in multi-dimensional maintenance decision rule knowledge base. It takes the multi-disease combination matrix, severity index intervals and spatial distribution patterns of each disease in the comprehensive quantitative report of road surface conditions at the road segment level as input conditions. Through the pattern matching engine, it matches the condition rules in the quantitative report and the knowledge base, and outputs a preliminary set of feasible maintenance process schemes.

[0032] It should be specifically noted that the multi-dimensional maintenance decision rule knowledge base adopts a three-layer structure of "rule base + case base + parameter base": the rule base is represented by production rules, the premise of which is the combination of the multi-disease combination matrix, severity index interval and spatial distribution pattern, and the conclusion is the corresponding maintenance process type; the case base stores complete cases including historical disease data, maintenance plans and effect feedback; the parameter base stores the material technical specifications, estimated engineering quantity calculation standards and basic cost and construction period quota data corresponding to various maintenance processes.

[0033] It should be further explained that three types of core data were selected to build the knowledge base, specifically including industry standard data: "Technical Specification for Maintenance of Cement Concrete Pavement of Highway" JTG / T H20-2011 and local maintenance specifications from various regions; historical maintenance case data: maintenance cases from 20 provinces and more than 500 road sections across the country were collected, covering maintenance process selection, material use, cost and construction period data under different combinations of defects and different degrees of severity; and technical parameter data for new materials / new processes: technical parameters, applicable conditions and cost data of newly emerging maintenance materials such as high-elasticity polyurethane joint filler and rapid repair mortar, as well as new processes such as microwave heating maintenance and mechanical automated crack sealing processes were introduced.

[0034] The system employs a three-tiered structure: a rule base, a case base, and a parameter base. The rule base uses a multi-disease combination matrix, a severity index range, and a spatial distribution pattern as prerequisites and "maintenance process type" as the conclusion. It uses production rules, meaning that when the set prerequisites are met, the corresponding conclusion is derived or a specified action is executed. The case base stores complete information on historical maintenance cases, including input conditions (disease data), maintenance plans, implementation effects, and cost and timeframes, used for case reasoning when rule matching fails. The parameter base stores the material technical specifications (such as tensile strength and softening point of crack sealant), estimated work quantity calculation standards (such as crack sealing work quantity calculated as length × width × depth), basic costs (material unit price, labor unit price, equipment rental unit price), and work period quota data (such as a 2-day work period for sealing 1km of minor cracks).

[0035] Establish a knowledge base update interface to regularly (quarterly) access the latest industry standards, new maintenance cases, and data on new materials / processes. After expert review (inviting 5 experts in the field of road maintenance), update the knowledge base. At the same time, combine the feedback data on the effect of the maintenance plan implementation, and use reinforcement learning algorithms to optimize the rule weights and improve the accuracy of rule matching.

[0036] The multi-disease combination matrix, severity index intervals, and spatial distribution patterns of each disease in the comprehensive quantitative report of road surface conditions at the road segment level are structurally extracted and converted into matching vectors. The cosine similarity algorithm is used to calculate the similarity with the premise vectors in the rule base. The multi-disease combination, severity index intervals, and spatial distribution patterns in the quantitative report are converted into numerical features of a unified dimension (e.g., mild 1, moderate 2, severe 3, concentrated distribution 1, dispersed distribution 2). The encoded numerical features are arranged in a fixed order to form matching vectors. Each premise in the rule base is also constructed into a condition vector according to the same rules to ensure that the dimensions of the two are consistent. The cosine similarity formula is used to calculate the cosine value of the angle between the matching vector and each condition vector. The closer the value is to 1, the higher the similarity; the closer it is to 0, the lower the similarity.

[0037] When the similarity is ≥0.85, the match is considered successful, and the corresponding maintenance process type is extracted; if there are multiple successfully matched rules, they are sorted according to the rule confidence (determined by the historical matching success rate), and the processes corresponding to the top 3 rules with the highest confidence are selected.

[0038] When rule matching fails (similarity < 0.85), a case reasoning mechanism is initiated. The K-nearest neighbor algorithm (K=5) is used to select the 5 historical cases most similar to the current quantitative report input conditions from the case library. When calculating case similarity, three core dimensions are considered: disease combination type, severity index distribution, and road segment level. The maintenance schemes corresponding to the 5 selected cases are fused and analyzed to extract common process types and material specifications. The engineering quantity and cost data are adjusted in combination with the specific conditions of the current road segment (such as climate conditions and traffic volume) to form a feasible maintenance process scheme.

[0039] The final preliminary solution set includes 3-5 feasible maintenance schemes, each specifying the following data: process type and implementation sequence (e.g., "Step 1: Crack sealing, Step 2: Crack filler replacement"); material specifications (e.g., polyurethane crack sealant with tensile strength ≥2.5MPa and softening point ≥70℃); estimated workload (e.g., crack sealing length 1200m, crack filler replacement length 800m); basic cost data (material cost, labor cost, equipment cost, and total basic cost); and schedule data (schedule for each process and total schedule).

[0040] As attached Figure 5 As shown, the decision optimization module integrates real-time traffic flow monitoring data, refined meteorological data for the next 72 hours, and current available maintenance personnel, machinery, and material resources. Based on the preliminary set of maintenance plans as the solution space, it constructs a multi-objective integer programming model. It iteratively solves the model with material costs, traffic delay costs, resource switching costs, and meteorological risk probabilities as key constraints, outputs the target maintenance decision plan, and simultaneously generates construction sequence arrangements, resource allocation paths, and dynamic traffic organization suggestions.

[0041] It should be specifically noted that the constraints of the multi-objective integer programming model include: cost constraints: the total implementation cost of the current maintenance plan does not exceed the preset budget threshold; resource constraints: the number of personnel, equipment and materials required for construction does not exceed the currently available resources, and the resource allocation time meets the construction requirements; meteorological constraints: the meteorological risk level during construction must not exceed the preset safety threshold; and process constraints: the implementation sequence of different maintenance processes must meet the preset process flow requirements.

[0042] The decision optimization module also uses an improved non-dominated sorting genetic algorithm to solve the multi-objective optimization model. The initial population is composed of the preliminary scheme set and randomly generated feasible schemes. The two core objective functions are dynamically balanced using the weight coefficient method, and the weight values ​​are set according to the road segment level.

[0043] It should be further explained that the three types of core dynamic data accessed include: real-time traffic flow monitoring data: from traffic monitoring cameras and loop detectors along the road section, the data includes current hourly traffic flow, vehicle type distribution (proportion of small cars and large cars), and average vehicle speed; refined meteorological data for the next 72 hours: from the National Meteorological Science Data Center and local meteorological stations, the data includes temperature, precipitation probability, precipitation, wind speed, and wind direction, with a time resolution of 1 hour and a spatial resolution of 1km×1km; and current available maintenance personnel, machinery, and material resource status data: from the maintenance unit's resource management system, the data includes the number and skill level of maintenance personnel, the location, status (available / under maintenance) and operating efficiency of maintenance equipment (such as crack sealing machines and crack filling machines), the inventory quantity of maintenance materials, storage location, and transportation costs.

[0044] The multi-source data is fused and processed: traffic flow data is smoothed using a moving average algorithm (window size of 5 minutes) to remove outliers, and the traffic flow trend during the construction period is predicted based on the same time period over the past 30 days; meteorological data is fused using a weighted average algorithm, with national meteorological data and local meteorological station data having weights of 0.6 and 0.4 respectively, to generate accurate meteorological data for road sections. At the same time, based on parameters such as precipitation probability and wind speed, the meteorological risk level (level 1-5, with level 5 being the highest risk) is calculated; resource data is represented by a real-time updated resource status matrix, where rows represent resource types (personnel, equipment, materials), columns represent resource numbers, and elements represent the real-time status and availability of resources. At the same time, the allocation time and cost of resources between different road sections are calculated based on road mileage and traffic conditions.

[0045] Based on the preliminary set of maintenance schemes as the solution space, a multi-objective integer programming model with "dual core objectives and multiple constraints" is constructed. The specific model design is as follows: Objective function: The core objectives include maximizing the life-cycle benefit-cost ratio and minimizing the traffic impact index during construction. Life-cycle benefits include the benefits of extended pavement lifespan after maintenance (based on historical data, the economic benefit of each year of extended lifespan is calculated at 5% of the average annual toll revenue of the road segment) and cost savings in subsequent maintenance due to reduced pavement distress. Life-cycle costs include the implementation costs of the current maintenance plan (materials, labor, equipment), subsequent maintenance costs over the next 5 years, and traffic delay costs (time costs incurred due to vehicle deceleration and detours caused by construction). Benefit-cost ratio = life-cycle benefit / life-cycle cost. The traffic impact index is represented by a weighted sum of the average speed reduction rate, traffic congestion duration, and detour vehicle ratio during the construction period. The weights are determined using the Analytic Hierarchy Process (AHP), with an average speed reduction rate weight of 0.4, congestion duration weight of 0.3, and detour vehicle ratio weight of 0.3.

[0046] Constraints: Cost constraint: The total implementation cost of the current maintenance plan ≤ 30% of the maintenance unit's annual maintenance budget, to avoid excessive investment in a single maintenance operation; Resource constraint: The number of personnel, equipment, and materials required for construction ≤ the current available resources, and the resource allocation time ≤ the construction preparation period; Meteorological constraint: The meteorological risk level during construction ≤ Level 3 (avoid construction during severe weather such as heavy rain and strong winds). If there is a period with a meteorological risk level ≥ Level 4 during the construction period of a certain plan, the construction sequence will be adjusted to avoid severe weather; Construction period constraint: The total construction period ≤ the longest construction period allowed by road traffic control (e.g., construction period for expressway sections ≤ 7 days, national highway sections ≤ 15 days); Technological constraint: The implementation sequence of maintenance processes must meet technical requirements (e.g., crack treatment first, then surface repair).

[0047] Model Solution: An improved non-dominated sorting genetic algorithm (NSGA-Ⅲ) is used for iterative solution. Specific optimization strategies include: using 3-5 schemes from the initial scheme set as the initial population, and randomly generating 20 feasible schemes to expand the population size; constructing a fitness function based on two core objective functions, and using a weighted coefficient method to balance the two objectives, with the weights dynamically adjusted according to the road segment level, the traffic impact index weight for highway segments being 0.6 and the benefit-cost ratio weight being 0.4; the opposite is true for county road segments; setting the number of iterations to 100, the crossover probability to 0.8, and the mutation probability to 0.1; after each iteration, feasible solutions are selected through constraints, and finally, the optimal scheme (the scheme with the highest comprehensive score) in the Pareto optimal solution set is output.

[0048] After the solution is completed, the target maintenance decision plan is output. The plan includes the optimized maintenance process type, material technical specifications, accurate engineering quantity (error ≤ 5%), final cost (cost reduced by 8-15% compared with the initial plan) and construction period (construction period shortened by 10-20% compared with the initial plan).

[0049] At the same time, three types of supporting documents are generated: using Gantt charts, the start and end times, required resources and work sections of each maintenance process are clearly defined, key nodes (such as material arrival time and process conversion time) are marked, and a 20% buffer period is reserved to deal with emergencies (such as temporary traffic control and equipment failure).

[0050] Based on real-time traffic data and resource storage locations, the Dijkstra algorithm is used to optimize resource allocation paths, generate optimal transportation routes and time schedules for personnel, equipment, and materials, and reduce resource allocation costs and time.

[0051] Based on real-time traffic flow forecast data, develop time-based and lane-based traffic control plans, including: the location and size of construction area fencing; the number and location of traffic guidance signs; a construction suspension mechanism during peak hours (e.g., 7-9 am and 5-7 pm); and an emergency lane reservation plan to ensure the passage of rescue vehicles. Simultaneously, generate a traffic impact assessment report, including basic construction information: maintenance construction sections, construction schedule, and the scope of each stage of construction; traffic flow forecast: predicting changes in traffic flow and speed at different times during construction, as well as congestion risk points; traffic impact analysis: analyzing the specific impact of construction on vehicle traffic efficiency and pedestrian safety; and response measures: clarifying the traffic control plan, detour suggestions, emergency lane setup, and traffic guidance sign deployment plan during construction, predicting traffic flow changes and congestion points during construction, and providing decision-making basis for traffic management departments.

[0052] The cement concrete pavement maintenance decision-making method based on vehicle-mounted images includes S1: pavement image acquisition and standardization processing. The original pavement images are acquired at preset intervals through a vehicle-mounted multispectral sensor array. Combined with color temperature correction, geometric distortion elimination and adaptive contrast enhancement technology, a standardized image sequence with high-precision positioning information is generated.

[0053] S2: Utilize a multi-task deep learning model to perform pixel-level segmentation of image sequences, identify four types of defects: cracks, corner fractures, joint damage, and surface peeling, extract the geometric features of each defect, calculate the defect severity index based on a pre-trained feature-weight mapping table, and generate a road segment-level quantitative report based on spatial clustering analysis.

[0054] S3: Knowledge-based maintenance strategy matching, which matches the disease combinations, severity and distribution patterns in the quantitative report with the multi-dimensional maintenance decision rule knowledge base, and outputs a preliminary set of feasible maintenance process solutions, including process type, material specifications, engineering quantity and cost and time data.

[0055] S4: Integrating real-time traffic flow, refined weather forecasts, and maintenance resource status data, based on the preliminary plan, a multi-objective optimization model is constructed with the goal of maximizing the cost-benefit ratio and minimizing traffic impact. The model is then solved under multiple constraints of cost, resources, and weather to output the comprehensive optimal maintenance decision plan.

[0056] S5: Based on the optimization plan, develop detailed construction sequence, resource allocation routes and time-based traffic control plans, and output complete maintenance implementation guidance documents including construction Gantt charts, resource scheduling charts and traffic impact assessment reports.

[0057] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments of this disclosure. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cement concrete pavement maintenance decision-making system based on vehicle-mounted images, characterized in that, include: Vehicle-mounted image acquisition module: A multispectral image sensor array is integrated at the front of the maintenance vehicle to acquire raw images of the road surface ahead of the vehicle in a constant interval triggering mode, and outputs a standardized road surface image sequence after processing; The road surface condition quantification module receives the standardized road surface image sequence, inputs it into a multi-task deep learning segmentation network, and outputs pixel-level semantic segmentation masks for four types of defects: cracks, corner fractures, joint damage, and surface peeling. It calculates three geometric features: area occupancy, average width, and dispersion of extension direction. Based on a preset feature-weight mapping table, it calculates the severity index of a single defect. Based on the geolocation encoding of the image sequence, it performs spatial clustering and statistical analysis of the severity index of a single defect by lane and mileage, and generates a comprehensive quantitative report of road surface condition at the road segment level. Maintenance strategy generation module: It has a built-in multi-dimensional maintenance decision rule knowledge base. It takes the multi-disease combination matrix, severity index interval and spatial distribution pattern of each disease in the comprehensive quantitative report of road surface condition at the road section level as input conditions. Through the pattern matching engine, it matches the condition rules in the quantitative report and the knowledge base, and outputs a preliminary set of feasible maintenance process schemes. Decision optimization module: It integrates real-time traffic flow monitoring data, refined meteorological data for the next 72 hours, and current available maintenance manpower, equipment, and material resources status data. Based on the preliminary set of maintenance plans as the solution space, it constructs a multi-objective integer programming model, and iteratively solves the model with material cost, traffic delay cost, resource switching cost, and meteorological risk probability as key constraints. It outputs the target maintenance decision plan and simultaneously generates construction sequence arrangements, resource allocation paths, and dynamic traffic organization suggestions.

2. The cement concrete pavement maintenance decision-making system based on vehicle-mounted images according to claim 1, characterized in that: The vehicle-mounted image acquisition module performs adaptive white balance correction based on the road surface reference color temperature, eliminates geometric distortion based on sensor calibration parameters, and enhances contrast based on regional grayscale statistics on the acquired raw images.

3. The cement concrete pavement maintenance decision-making system based on vehicle-mounted images according to claim 1, characterized in that: The multi-task deep learning segmentation network is based on the U-Net++ framework and improved by introducing a cross-scale feature fusion unit to adjust the feature maps of different levels of the encoder to the same scale for fusion. Independent segmentation branches are set for the four types of diseases at the decoder output, and an adaptive threshold adjustment layer is added after each branch to dynamically adjust the segmentation threshold according to the disease difficulty level of the input image; a combination of Dice loss and Focal loss is used as the loss function for network training.

4. The cement concrete pavement maintenance decision-making system based on vehicle-mounted images according to claim 1, characterized in that: The feature-weight mapping table is obtained by training on historical road disease geometric feature data and corresponding maintenance cost and effect data using the random forest algorithm. The feature weights in the table are dynamically adjusted according to the road segment level.

5. The cement concrete pavement maintenance decision-making system based on vehicle-mounted images according to claim 1, characterized in that: The multi-dimensional maintenance decision rule knowledge base adopts a three-layer structure of "rule base + case base + parameter base": the rule base is represented by production rules, the premise of which is the combination of the multi-disease combination matrix, severity index interval and spatial distribution pattern, and the conclusion is the corresponding maintenance process type; the case base stores complete cases including historical disease data, maintenance plans and effect feedback; the parameter base stores the material technical specifications, estimated engineering quantity calculation standards and basic cost and construction period quota data corresponding to various maintenance processes.

6. The cement concrete pavement maintenance decision-making system based on vehicle-mounted images according to claim 1, characterized in that: The multi-objective integer programming model has the following constraints: cost constraint: the total implementation cost of the current maintenance plan does not exceed the preset budget threshold; resource constraint: the number of personnel, equipment and materials required for construction does not exceed the currently available resources, and the resource allocation time meets the construction requirements; meteorological constraint: the meteorological risk level during construction must not exceed the preset safety threshold; and process constraint: the implementation sequence of different maintenance processes must meet the preset process flow requirements.

7. The cement concrete pavement maintenance decision-making system based on vehicle-mounted images according to claim 1, characterized in that: The decision optimization module also uses an improved non-dominated sorting genetic algorithm to solve the multi-objective optimization model. The initial population is composed of the preliminary scheme set and randomly generated feasible schemes. The two core objective functions are dynamically balanced using the weight coefficient method, and the weight values ​​are set according to the road segment level.

8. A method for cement concrete pavement maintenance decision-making based on vehicle-mounted images, used to implement the cement concrete pavement maintenance decision-making system based on vehicle-mounted images as described in any one of claims 1-7, characterized in that, include: S1: Road surface image acquisition and standardization processing. Through the vehicle-mounted multispectral sensor array, original road surface images are acquired at preset intervals. Combined with color temperature correction, geometric distortion elimination and adaptive contrast enhancement technology, a standardized image sequence with high-precision positioning information is generated. S2: Use a multi-task deep learning model to perform pixel-level segmentation of image sequences, identify four types of defects: cracks, corner fractures, joint damage, and surface peeling, extract the geometric features of each defect, calculate the defect severity index based on the pre-trained feature-weight mapping table, and generate a road segment-level quantitative report based on spatial clustering analysis. S3: Knowledge-based maintenance strategy matching, which matches the disease combination, severity and distribution pattern in the quantitative report with the multi-dimensional maintenance decision rule knowledge base, and outputs a preliminary set of feasible maintenance process solutions, including process type, material specifications, engineering quantity and cost and time data. S4: Integrate real-time traffic flow, refined weather forecasts, and maintenance resource status data. Based on the preliminary plan, construct a multi-objective optimization model with the core objectives of maximizing the cost-benefit ratio and minimizing traffic impact. Solve the model under multiple constraints of cost, resources, and weather to output the comprehensive optimal maintenance decision plan. S5: Based on the optimization plan, develop detailed construction sequence, resource allocation routes and time-based traffic control plans, and output complete maintenance implementation guidance documents including construction Gantt charts, resource scheduling charts and traffic impact assessment reports.

Images