Intelligent decision and control method and system for road marking maintenance

Abstract

The application provides a kind of intelligent decision and control method and system of road marking maintenance, belong to intelligent traffic infrastructure maintenance technical field.The method includes: obtaining marking image by visual sensor, and using pre-trained retroreflective coefficient estimation model to map image gray feature into norm-defined retroreflective coefficient estimation value;Receive and analyze natural language job instruction, based on instruction semantics, retroreflective coefficient estimation value and vehicle state, generate structured strategy parameters containing spraying trigger threshold and model predictive control weight through large model decision unit LLM inference;Finally, by model predictive controller according to the strategy parameter rolling optimization, output driving and spraying instruction, realize the closed-loop control of tracking and filling.This application realizes the online quantitative perception of marking retroreflective performance and adaptive decision based on natural language, improves the accuracy, intelligent level and scene adaptability of maintenance work.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation infrastructure maintenance technology, and in particular to an intelligent decision-making and control method and system for road marking maintenance. Background Technology

[0002] Road markings are a key infrastructure for ensuring traffic safety and smooth flow. Their geometric continuity and retroreflective properties (i.e., retroreflective coefficient (R²) are crucial. L Road markings are directly related to driving safety at night and in adverse weather conditions. Currently, the maintenance of road markings mainly relies on manual inspections and semi-automatic marking equipment.

[0003] In the prior art, US Patent No. US10113277B2 discloses a system that identifies damaged road markings and automatically repairs them using a front-facing camera and laser scanner. This system can achieve automated operation based on geometric recognition. However, such solutions have at least the following shortcomings: First, its detection dimension is limited to the geometric integrity and position of the road markings, and it cannot continuously and quantitatively evaluate the retroreflection coefficient, a core indicator that determines nighttime safety, online. This results in a lack of accurate data support for maintenance decisions, and road markings that "look complete but are no longer reflective" may not be maintained in a timely manner. Second, traditional PID control schemes cannot resolve fuzzy semantic instructions with multi-objective trade-offs (such as "strengthen spraying while ensuring smoothness"), resulting in poor operational flexibility.

[0004] Furthermore, although there have been studies on road marking status assessment based on image processing, they mostly remain at the level of qualitative or simple threshold judgments such as brightness and color. They are difficult to directly and reliably map to the retroreflection coefficient, a physical quantity clearly defined in industry standards, and thus cannot serve as a direct basis for accurate maintenance.

[0005] Therefore, there is an urgent need in this field for an integrated intelligent maintenance solution that can continuously and quantitatively perceive the performance of road markings and autonomously generate and execute optimal control strategies based on high-level, naturalized operational intentions, in order to overcome the problems of single detection dimensions, insufficient intelligent decision-making, and limited control accuracy and adaptability in existing technologies.

[0006] The embodiments of the present invention are improvements made to solve the above problems. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for online detection and intelligent maintenance decision-making of road marking retroreflective performance based on multi-source perception fusion. By implementing this invention, ordinary grayscale features can be transformed into retroreflective coefficients with clear physical meaning through a data-driven mapping model, realizing the quantification of low-cost perception. The introduction of a large model decision unit (LLM) solves the problem of adaptive transformation from "unstructured requirements" to "underlying mathematical parameters".

[0008] To achieve the aforementioned objective, in a first aspect, embodiments of the present invention provide an intelligent decision-making and control method for road marking maintenance, the technical solution of which is: The method includes a visual perception step, a reflectivity quantification step, an intelligent decision-making step, and a model predictive control step.

[0009] In the visual perception step, image data containing road marking areas is acquired. In the reflectivity quantification step, based on the grayscale information in the image data, the estimated retroreflection coefficient of the markings is determined using a pre-trained retroreflection coefficient estimation model. The retroreflection coefficient estimation model establishes a mapping relationship between grayscale features and the standardized retroreflection coefficient values. In the intelligent decision-making step, natural language operation instructions are acquired, and based on the semantics of the instructions, the estimated retroreflection coefficient values, and vehicle state data, inference is performed using a Large Model Decision Unit (LLM) to generate structured control strategy parameters. These strategy parameters include at least the spraying operation trigger threshold and the optimization target parameters of the model predictive controller. In the model predictive control step, based on the structured control strategy parameters, the geometric information of the markings, and the vehicle state data, rolling optimization is performed using the model predictive controller to generate drive instructions for vehicle tracking and spraying instructions for marking completion.

[0010] Specifically, the reflectivity quantification step may further include: extracting at least one grayscale statistical feature from the region of interest of the road marking from the image data; inputting the grayscale statistical feature into a pre-trained retroreflection coefficient estimation model; and having the model output a value conforming to the definition and dimensions of the retroreflection coefficient in the road engineering lighting specifications, as the estimated retroreflection coefficient. The pre-trained retroreflection coefficient estimation model can be obtained by: collecting multiple sets of training data, each set including sample grayscale images of the same road marking area and the true values ​​simultaneously measured by a standard retroreflection coefficient measuring instrument; extracting the same grayscale statistical features from each sample grayscale image; and training a pre-set regression model using the grayscale statistical features as input and the corresponding true values ​​as output to obtain the retroreflection coefficient estimation model.

[0011] This approach maps the grayscale image features captured by ordinary industrial cameras into retroreflection coefficients with clear physical meaning through a data-driven model. This enables low-cost sensors to continuously and quantitatively perceive high-value professional indicators, providing a direct basis for precise maintenance.

[0012] Optionally, in the intelligent decision-making process, the Large Model Decision Unit (LLM) can be configured to: parse natural language job instructions to understand the optimization objectives and operational constraints of the job; combine retroreflection coefficient estimates with vehicle state data to evaluate the current lane marking performance and operating conditions; and dynamically infer and output the spraying job trigger threshold, expected vehicle speed, and weighting coefficients used to adjust the balance between trajectory tracking accuracy and motion smoothness of the model predictive controller. This configuration allows high-level, fuzzy natural language intents (such as "prioritize repainting sections with poor reflectivity" or "operate with caution on wet and slippery surfaces") to be automatically parsed and transformed into specific mathematical parameters executable by the underlying controller, greatly improving the system's usability and scenario adaptability, and avoiding complex manual parameter configuration.

[0013] Furthermore, the method includes a closed-loop optimization step: after executing the spraying command, re-inspection data of the re-marked lines is acquired, including thickness data and / or estimated retroreflection coefficient values; based on the difference between the re-inspection data and the expected standard, parameter adjustment amounts are generated; and the parameter adjustment amounts are used to correct the structured control strategy parameters in subsequent executions online. This method forms a closed loop of "decision-execution-evaluation-re-decision," enabling the system to learn and continuously optimize itself based on actual operational results, effectively ensuring and continuously improving the consistency of maintenance quality.

[0014] Preferably, the structured control strategy parameters also include spray flow rate parameters. When generating spraying instructions in the model predictive control step, the control quantity of the spraying actuator is dynamically calculated and adjusted based on the current vehicle speed and spray flow rate parameters to keep the paint deposition amount per unit area stable. This design solves the technical problem of uneven marking coating thickness that is easily caused when the vehicle changes speed due to road conditions or tracking needs, thus improving the stability of the work quality.

[0015] Secondly, embodiments of this application also provide an intelligent decision-making and control system for road marking maintenance, which includes a visual perception module, a reflectivity quantification module, an intelligent decision-making module, a model prediction control module, and an instruction output module.

[0016] The visual perception module acquires image data containing road marking areas; the reflectivity quantification module determines the estimated retroreflection coefficient of the markings based on grayscale information in the image data using a pre-trained retroreflection coefficient estimation model; the retroreflection coefficient estimation model establishes a mapping relationship between grayscale features and the standardized retroreflection coefficient values; the intelligent decision-making module integrates a large model decision unit (LLM) to generate structured control strategy parameters based on the semantics of natural language operation instructions, the estimated retroreflection coefficient values, and vehicle status data; the strategy parameters include at least the spraying operation trigger threshold and the optimization target parameters of the model predictive controller; the model predictive control module performs rolling optimization based on the structured control strategy parameters, the geometric information of the markings, and vehicle status data, generating drive and spraying instructions; the instruction output module outputs the drive and spraying instructions.

[0017] The reflectivity quantization module may include a feature extraction unit and a model calculation unit. The feature extraction unit is used to extract at least one gray-level statistical feature of the region of interest of the tracing from the image data. The model calculation unit stores and runs a pre-trained retroreflection coefficient estimation model, which is used to receive the gray-level statistical features and output the estimated retroreflection coefficient value.

[0018] Among them, the Large Model Decision Unit (LLM) in the Intelligent Decision Module can be configured to: parse natural language job instructions to understand the optimization objectives and operational constraints of the job; combine the retroreflection coefficient estimate with vehicle state data to evaluate the current lane marking performance and working conditions; dynamically reason and output the spraying job trigger threshold, the expected vehicle speed, and the weight coefficients used to adjust the balance between trajectory tracking accuracy and motion smoothness of the model predictive controller.

[0019] Furthermore, the system also includes a closed-loop optimization module, which is used to obtain re-inspection data of the re-marked lines after the spraying is performed; analyze the difference between the re-inspection data and the expected standard, and generate parameter adjustment amounts; and feed the parameter adjustment amounts back to the intelligent decision-making module or the reflectivity quantification module for dynamic optimization of the strategy or model.

[0020] Furthermore, the above summary does not enumerate all the features required for embodiments of the present invention, and other combinations of these feature groups may also constitute embodiments of the present invention. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the background art, the drawings used in the embodiments of the present invention or the background art will be described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a system functional block diagram of a road intelligent marking robot provided in an embodiment of the present invention.

[0023] Figure 2 A road marking reflectivity detection module and grayscale-retroreflection coefficient (GRP) provided in an embodiment of the present invention R L ) Calibration process diagram.

[0024] Figure 3 This is a block diagram illustrating the principle of collaborative operation between the Large Model Decision Unit (LLM) and the Model Prediction Control Unit in an intelligent decision control module provided in an embodiment of the present invention.

[0025] Figure 4 This is a flowchart illustrating an intelligent decision-making and control method for road marking maintenance provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the technical means, creative features, objectives and effects of the embodiments of the present invention easier to understand, the embodiments of the present invention are further described below in conjunction with the figures and specific embodiments. It should be understood that the specific embodiments described herein are merely for explaining the embodiments of the present invention and are not intended to limit the embodiments of the present invention.

[0027] To better understand the embodiments of the present invention, please refer to Figures 1 to 4 As shown.

[0028] Example 1: This invention illustrates the overall hardware architecture and core workflow of an intelligent decision-making and control system for road marking maintenance. See also... Figure 1 As shown, the system of this embodiment adopts a layered architecture design, mainly including a perception layer, a decision layer, and an execution layer. The perception layer consists of a lane marking tracking module 2, a lane marking reflection detection module 3, and various sensors (such as wheel speed encoders), which are responsible for collecting raw road environment data. The decision layer is centered on an intelligent decision control module 4, which integrates a large model decision unit LLM42 and a model prediction control unit 44. The execution layer includes a lane marking completion module 5 and a chassis drive mechanism, which are responsible for converting decision commands into physical actions. The remote monitoring and human-machine interaction module 7 provides operators with a channel for remote supervision and strategy input.

[0029] See Figure 4 The method begins with step S401, when the system is powered on and initialized, and each sensor and module completes its self-test. In step S402, the perception layer starts working. The road marking tracking module 2 acquires road images through an OpenMV camera, extracts the geometric center of the road markings, and calculates the tracking deviation. At the same time, the road marking reflection detection module 3 acquires grayscale images of the road markings through a monochrome industrial camera. In step S403, the quantization perception step, the edge computing unit (such as Jetson Nano) in the intelligent decision control module 4, based on the grayscale image, calls a pre-trained retroreflection coefficient estimation model to calculate the estimated retroreflection coefficient value of the current marking position in real time. Specifically, the construction of the retroreflection coefficient estimation model involves offline training and online mapping. In the offline stage, grayscale images of markings with different road conditions and different wear levels are collected, and their actual retroreflection coefficient values ​​are simultaneously measured using a standard retroreflectometer. R L The model establishes a mapping relationship through regression analysis. In this embodiment, a linear regression formula is used. Calculate the estimated value of the retroreflection coefficient, where μ The grayscale average of the marked area. σ The grayscale standard deviation, in one example, is obtained through training and fitting. , , For scenarios with significant nonlinear characteristics of uneven illumination, the model can also use support vector regression (SVR) or a shallow neural network with 10 neurons in the hidden layer for feature fitting. In step S404, the intelligent decision-making process is initiated. The large model decision unit (LLM42) receives natural language instructions (such as "prioritize repainting non-reflective markings within the tunnel") from the remote monitoring and human-machine interaction module 7. Combining this with the retroreflection coefficient estimate obtained in step S403, the current vehicle speed, and other contextual information, it performs reasoning to dynamically generate a set of structured control strategy parameters. These parameters include at least the spraying trigger threshold, the target vehicle speed, and the optimized weights of the model prediction control unit 44. In step S405, the model prediction control unit 44 performs rolling optimization calculations based on the aforementioned strategy parameters, real-time tracking deviation, and vehicle state to solve for the optimal chassis drive command and paint control command within a future time domain. Specifically, the rolling optimization calculation minimizes the cost function. J accomplish:

[0030] in, N p To predict the time domain, N c To control the time domain, To predict the output, For reference trajectory, To control the increment; Q and R These are the weight matrices issued by the large model decision unit LLM42.

[0031] Step S406: Execute the layer response command to drive the chassis smoothly along the markings, and at the retroreflection coefficient (R... L For road sections below the trigger threshold, the precise control module 5 for lane marking replacement will perform spraying operations. Step S407: After the re-marking is completed, the system re-inspects the work effect through a post-detection device (such as a thickness sensor) and feeds the data back to the decision-making level. In step S408, the system determines whether adaptive correction of the control strategy parameters or the retroreflection coefficient estimation model is needed based on the deviation between the re-inspection results and the expected target, thereby forming a complete intelligent closed loop of "perception-decision-execution-evaluation-optimization".

[0032] In an embodiment of the present invention, by Figure 1 The modular hardware architecture shown is Figure 4 The closed-loop software process shown is combined to build a complete system that can autonomously complete the quantitative assessment of the marking status, intelligent operation decision-making and high-precision execution. This setup realizes the transformation from traditional manual experience-driven to data and model-driven, improving the accuracy, consistency and efficiency of maintenance operations.

[0033] Example 2: See Figure 2 As shown, the grayscale image is converted to the inverse reflection coefficient (...). R L The quantitative mapping of ) is divided into two stages: clear offline calibration and online application.

[0034] In some implementations, the offline calibration phase is performed according to the following steps: First, the preparation stage: place the robot platform on a test section containing various reflective markings (such as newly painted, worn, and dirty), and ensure that the black and white industrial camera in the marking reflectivity detection module 3 and the measurement field of view of the marking retroreflection coefficient detector are aligned with the same area. Then, data acquisition is performed: the system control platform moves slowly or measures at a fixed point, simultaneously triggering the black and white industrial camera to take pictures and the retroreflection coefficient detector to take readings, recording the pairs of "marker grayscale images" and "measured retroreflection coefficient values", repeating this process hundreds of times to cover sufficient data variation.

[0035] Next, data annotation and feature engineering are performed: For each acquired grayscale image, the precise area covered by the retroreflection coefficient detector's spot is selected as the Region of Interest (ROI) using image processing algorithms (such as thresholding). Specifically, the extraction of the ROI is automatically completed through the following steps: First, adaptive thresholding or Canny edge detection is performed on the grayscale image acquired by the black and white industrial camera to initially identify the outline of the marking area in the image; then, combining the known installation location of the marking retroreflection coefficient detector, the size of the measured spot, and the camera's intrinsic and extrinsic parameters (obtained through joint calibration), the physical measurement points of the detector are mapped to the image coordinate system, thereby accurately locating the ROI corresponding to the measured value within the initially identified marking area, and calculating the grayscale mean of all pixels within the ROI. μ ) and gray standard deviation ( σ ) as feature vector .

[0036] Calculate the image features within the ROI, such as the gray mean (reflecting overall brightness) and gray standard deviation (reflecting texture uniformity), and pair these calculated feature values ​​with the corresponding measured retroreflection coefficient values ​​to form a structured dataset.

[0037] Finally, model training: An appropriate machine learning regression algorithm (e.g., linear regression, support vector regression (SVR), or a shallow neural network) is selected. The aforementioned grayscale features (such as grayscale mean and standard deviation) are used as input (X), and the measured inverse reflection coefficient values ​​are used as output labels (y). The model is then trained by minimizing the prediction error (e.g., mean squared error). This is the "pre-trained inverse reflection coefficient estimation model." This model encapsulates the complex mapping relationship from the image feature space to the physical performance space.

[0038] As a specific example, when using a linear regression model, the inverse reflection coefficient estimation model obtained after training can be expressed as: ,in , These are the weighting coefficients. b For example, a set of sample parameters obtained after training on a certain dataset: w 1 = 2.5 w 2 = -0.8 b =30, then when a certain ROI's μ =120, σ When =15, the estimated retroreflection coefficient In another implementation, support vector regression (SVR) or a shallow neural network containing a single hidden layer (such as 10 neurons) can be used as the regression model to capture the possible nonlinear relationship between grayscale features and retroreflection coefficients.

[0039] The online application phase corresponds to real-time inference during operation. In step S403, the black-and-white industrial camera 31 continuously acquires grayscale images of the road surface. The system automatically identifies the marking areas in the images and slides a window along the forward direction. For each window area, it calculates the same grayscale features (such as grayscale mean and standard deviation) as in the offline phase. Then, these feature values ​​are input into the deployed, previously trained retroreflection coefficient estimation model. The model immediately outputs a value that directly corresponds to the retroreflection coefficient defined in road engineering specifications (such as JT / T 280). R L Its dimension is mcd·m - ²·lx - ¹ It enables continuous, online conversion from “pixel grayscale values” to “standard physical quantities”.

[0040] In an embodiment of the present invention, by Figure 2 The standardized process shown provides a concrete and feasible solution for converting low-cost industrial cameras into semi-professional-grade online retroreflectance coefficient measuring instruments. Example 3:

[0041] See Figure 3 As shown, the Large Model Decision Unit (LLM42) and the Model Prediction Control Unit (MPC) 44 do not operate in isolation, but are tightly coupled through a structured data flow; in some feasible implementations, the Large Model Decision Unit (LLM42) is deployed on edge computing devices (such as... Figure 1 On the Jetson Nano 41, the input comes from multiple channels: first, natural language operation instructions text from the remote monitoring and human-machine interaction module 7 (for example, the operator's voice input "School area ahead, please perform low-speed and fine operation"); second, real-time retroreflection coefficient estimation data stream from the road marking reflection detection module 3; and third, vehicle status information such as tracking deviation and current vehicle speed from the road marking tracking module 2.

[0042] The Large Model Decision Unit (LLM42) is a Large Model Decision Unit deployed on a Jetson Nano 41. To achieve the structured conversion of natural language instructions into control parameters, its implementation includes the following two optional implementation methods: Implementation Method A (based on fine-tuning of the language model): In this embodiment, the large model decision unit (LLM) uses a pre-trained model based on the Transformer architecture with a parameter count between 1B and 7B (e.g., an INT4 quantized version of TinyLLaMA-1.1B or ChatGLM-6B, or a pruned version of Qwen-7B). To achieve domain-specific adaptation, supervised fine-tuning (SFT) is used for training. 1. Training Data Construction: Thousands of sample pairs were collected, constructed by domain experts. Each sample contains: Input: A textual prompt consisting of a natural language instruction, current vehicle status (e.g., speed), environmental information (e.g., road type), and a real-time retroreflection coefficient estimate. For example: "Instruction: Prioritize adding the left-side non-reflective edge line within the tunnel. Context: Current speed = 6 km / h, road type = tunnel, left edge line retroreflection coefficient = 105 mcd·m - ²·lx - ¹. Output: The corresponding structured strategy parameter text or JSON format. For example: {"Spraying threshold":180,"Target speed":5,"MPC weight":{"Lateral error":0.9,"Heading error":0.08,"Control smoothness":0.02},"Priority lane":"Left side"}.

[0043] 2. Model fine-tuning: Using the above sample pairs as training data, supervised fine-tuning is performed on the selected pre-trained language model to optimize its ability to generate structured output based on multimodal context (instructions + data).

[0044] 3. Online reasoning: During the task, the system constructs prompt words from real-time information in the same format and inputs them into the deployed fine-tuning model. The model directly outputs structured strategy parameters for subsequent modules to parse.

[0045] Implementation method B (based on rule engine and semantic template): As another, lighter and more interpretable implementation, the large model decision unit LLM42 can be composed of a rule engine combined with a semantic parsing template.

[0046] 1. Rule Base: Predefines a series of keywords (such as "priority," "refined," "tunnel," "school area," "curve") and the parameter adjustment rules they trigger. For example, the rule base contains: The IF instruction includes "tunnel" THEN spray threshold = spray threshold + 20; target vehicle speed = min(target vehicle speed, 5); the IF instruction includes "fine" THEN MPC weight.control smoothness = MPC weight.control smoothness * 0.5.

[0047] 2. Semantic parsing: The input natural language instructions are segmented, keywords are extracted, and intent is recognized (using a simple bag-of-words model or pre-trained word vectors), and matched with keywords in the rule base.

[0048] 3. Parameter synthesis: Combining the extracted keywords and current real-time data (inverse reflection coefficient, vehicle speed), the corresponding rule chain is triggered to dynamically adjust and synthesize a set of default policy parameter base values ​​to generate the final structured policy parameters.

[0049] The model prediction control unit 44 receives structured strategy parameters from the large model decision unit LLM42. To adapt to different scenarios, the objective function weights of the MPC can be dynamically configured. For example, in the strategy parameters output in implementation method A or B, when the MPC weight lateral error is set to 0.9 and the MPC weight control smoothness is set to 0.1, the controller focuses on trajectory tracking accuracy; when both are set to 0.5 and 0.5, a balance is achieved between tracking accuracy and motion smoothness, making it more suitable for curves or bumpy roads. Example 4:

[0050] See Figure 1 The intelligent decision-making and control system for road marking maintenance can be specifically implemented as an intelligent marking robot as shown in the figure. The function of the visual perception module is jointly realized by the OpenMV camera in the marking tracking module 2 and the black and white industrial camera in the marking reflection detection module 3; the reflection performance quantification module can run in software form on the Jetson Nano 41 of the intelligent decision-making and control module 4, which calls the pre-trained model stored therein and processes the image stream from the black and white industrial camera.

[0051] The intelligent decision-making module specifically corresponds to Figure 1 The large model decision unit (LLM42) is located within the intelligent decision control module 4. It can be an LLM service process running on a Jetson Nano, receiving instruction text from the remote monitoring and human-machine interaction module 7 and real-time data from other modules via an internal API.

[0052] The model predictive control module corresponds to Figure 1 The model prediction control unit 44, which also runs on Jetson Nano, receives strategy parameters from the large model decision unit LLM42 and vehicle status data from the tracking microcontroller 22 via a high-speed internal bus (such as SPI or shared memory). The function of the instruction output module can be jointly undertaken by the tracking microcontroller 22 and its connected motor driver, as well as the control valves in the line marking module 5, which are responsible for converting digital instructions into motor PWM signals and solenoid valve switching signals.

[0053] Furthermore, the closed-loop optimization function can be implemented by adding a data management and analysis submodule to the system. This submodule collects data from the pressure sensor in the marking module 5, data from the post-thickness detection device, and the estimated retroreflection coefficient after marking. After analysis, it generates model correction suggestions and updates the retroreflection coefficient estimation model or adjusts the internal strategy preferences of the large model decision unit LLM42 through Jetson Nano 41. Example 5:

[0054] In a specific application scenario, the core solution of this invention can be used for the maintenance of road markings in urban tunnels. The lighting inside the tunnel is dim and changes drastically (entrance / exit), the road markings are easily contaminated and their reflectivity decays quickly, requiring extremely high precision and safety in maintenance.

[0055] The operator issues a voice command via a remote terminal: "Enter the tunnel for work, focusing on the left lane edge line, ensuring uniform reflection, safety first." The system responds as follows: The black-and-white industrial camera in the road marking reflection detection module 3 can still capture clear grayscale images under low light conditions; after parsing the command, the large model decision unit LLM42 in the intelligent decision control module 4, combined with the "tunnel" environment label, generates a set of highly targeted structured control strategy parameters: setting the spraying trigger threshold higher than in open areas (e.g., 180 mcd·m). - ²·lx - ¹, because the reflectivity of lane markings is more prominent in tunnels), the target vehicle speed is limited to a low 5km / h, and a set of parameters with high tracking accuracy weights are set for the model prediction control unit 44 (e.g., lateral error weight = 0.9, control smoothness weight = 0.1) to ensure that the vehicle drives smoothly close to the left lane line in the narrow tunnel; at the same time, the intention of "uniform reflectivity" in the instruction is translated into parameters that require higher stability for the pressure closed-loop control of the plunger pump in the lane marking module 5.

[0056] For example, in a specific application scenario, when an operator issues the instruction "Entering a school zone ahead, please work slowly and carefully and ensure the markings are clear," if the system uses implementation method A, this instruction and the real-time context are used to construct a prompt word input fine-tuning model. The model may output: {"Spraying threshold":150,"Target vehicle speed":4,"MPC weight":{"Lateral error":0.85,"Control smoothness":0.15},"Spraying pressure gain":1.1}.

[0057] If implementation method B is adopted, the rule engine recognizes keywords such as "school area", "low speed" and "fine" and triggers the corresponding rules: the default vehicle speed is reduced from 8km / h to 4km / h, the spraying trigger threshold is adjusted from 130 to 150, and the MPC weight is fine-tuned to enhance smoothness. The system then controls based on the updated parameters.

[0058] In this embodiment, through the flexible configuration of natural language commands, the system can automatically adapt to working environments with vastly different lighting and spatial constraints, such as those from open-air environments to tunnels, and adjust the core control objectives accordingly (switching from "efficiency" priority to "safety and uniformity" priority), which is something that traditional equipment using fixed rules cannot achieve. Example 6:

[0059] To further verify the "natural language-based LLM dynamic decision-making" and "grayscale-reflection coefficient (R)" in the technical solution of this invention from a reverse perspective L To emphasize the necessity of the two key technical features of "quantitative model", the following comparative model is set up.

[0060] Comparative Example 1 (lacking LLM decision-making, using fixed threshold rules): Reproducing an automatic line marking machine based on fixed rules mentioned in the background technology, whose spraying trigger threshold is a fixed value of 150 mcd·m - ²·lx - ¹, The target vehicle speed is fixed at 8km / h, and the weights of the MPC controller are fixed.

[0061] Comparative Example 2 (Missing quantization model, using a simple brightness threshold): Reproducing another common approach, using a regular color camera, determining whether additional marking is needed solely based on the average RGB brightness of the marked area, and setting a brightness threshold.

[0062] Embodiment of the present invention: adopts a complete technical solution.

[0063] A comparative test was conducted on a 5-kilometer comprehensive test section that included tunnels, curves, and road markings with varying degrees of wear. The key performance indicators are compared in the table below: Performance indicators Comparative Example 1 (Fixed Rule) Comparative Example 2 (Simple Brightness Judgment) Embodiments of the present invention Accuracy of supplementary demarcation decisions 75% 60% 98% (Refers to the percentage of road sections that actually need to be re-marked that were correctly triggered for operation) (The tunnel was over-sprayed due to a fixed threshold; the wear areas on the curves were missed because the driver did not slow down.) (Brightness assessment is severely distorted on cloudy days and inside tunnels, leading to numerous misjudgments) (Quantization model is robust to illumination interference; LLM dynamically adjusts the threshold) Average operating speed 8km / h (fixed) 7.5km / h 6.8km / h (Failure to slow down in curves and tunnels poses a safety hazard) (Frequent start-stop cycles due to misjudgment) (LLM proactively reduces speed in complex road sections for enhanced safety) Coating thickness uniformity (standard deviation) 0.25 mm 0.30 mm 0.12 mm (Reflects the quality of the road markings) (With a fixed vehicle speed, the amount of paint applied was not adjusted accordingly) (Misjudgment led to disordered spraying timing) (MPC coordinates vehicle speed for dynamic traffic flow compensation) Number of operator interventions 15 times 22 times 3 times (Number of times manual intervention or parameter adjustment is required) (Global threshold needs to be frequently modified) (Brightness threshold and start / stop settings need to be continuously adjusted) (Natural language commands are set once, and the system adapts accordingly) The comparative experimental data above show that the absence of any of the core features of this invention (LLM dynamic decision-making or quantitative perception model) will lead to a significant decrease in operational accuracy, safety, quality consistency, and automation. Example 7:

[0064] In one possible application scenario, a vehicle is following a path when a small patch of oil stain suddenly appears in the road marking area ahead. This situation is not explicitly defined in the training data, and traditional rule-based systems may misjudge it as severe wear and incorrectly repaint it, or directly lose the trajectory. In this embodiment of the invention, the grayscale image of the road marking reflectivity detection module 3 shows abnormal grayscale characteristics in the area (such as a sudden change in mean or a sharp increase in standard deviation). When the large model decision unit LLM42 is inferring, although it does not directly identify the "oil stain", combined with the context of "drastic change in grayscale characteristics" and "currently a straight and dry road section", its built-in common sense reasoning ability may make it judge that this is atypical wear, but temporary contamination. Therefore, it may dynamically generate a temporary, conservative strategy: not to trigger repainting, but to instruct the model prediction control unit 44 to temporarily increase the weight to maintain the current heading ("patient" mode), and at the same time send a prompt to the operator through the remote monitoring and human-machine interaction module 7: "An abnormality has been detected in the road marking area ahead, it is recommended to observe". After the vehicle smoothly passes through the contaminated area, the system quickly returns to normal.

[0065] In this embodiment, the present invention demonstrates its preliminary ability to handle out-of-distribution situations, enabling its decisions to not rigidly rely on a preset rule base, thereby improving the system's processing capabilities in real-world complex environments. Example 8:

[0066] In other specific application scenarios, the aforementioned "inverse reflection coefficient estimation model" can be implemented in various ways, all of which can achieve the mapping from image feature space to physical performance space.

[0067] In some implementations, this mapping relationship can be constructed as a simple linear regression model, assuming the input feature vector is... ,in grayscale mean , grayscale standard deviation , If the feature dimension is used, then the estimated value of the retroreflection coefficient is... It can be represented as:

[0068] in, ∈R n This is the weight coefficient vector. b ∈R represents the bias term, with weight w and bias... b From the calibration dataset using the least squares method The above is what we learned, that is, to solve:

[0069] This mapping relationship is clear and computationally inexpensive, making it suitable for scenarios with extremely high real-time requirements.

[0070] In other implementations, to capture more complex nonlinear relationships, a support vector regression (SVR) model or a shallow feedforward neural network (e.g., a multilayer perceptron (MLP) with one hidden layer) can be used. For the SVR model, its mapping function is typically expressed as:

[0071] in, For Lagrange multipliers, K (·,·) represents the kernel function (such as the radial basis function RBF). x j These are support vectors.

[0072] For a neural network with one hidden layer, let the hidden layer have... h There are neurons, and the activation function is... σ (·) (such as ReLU), then the mapping relationship is expressed as:

[0073] Where W∈R h*n Let b1 ∈ R be the weight matrix from the input layer to the hidden layer. h Let v be the hidden layer bias vector, and v∈R h The weight vector from the hidden layer to the output layer. b 2∈R represents the output layer bias, and the model parameter set. θ ={W,b1,v, b 2} The loss function (e.g., mean squared error) is optimized on the calibrated data using a gradient descent algorithm (e.g., backpropagation) to obtain:

[0074] Whether linear or nonlinear, the essence of the model is to establish a transition from an easily accessible "image feature space" (dimensions such as gray-level mean, standard deviation, etc.) to a crucial but difficult-to-measure "physical performance space" (single-dimensional retroreflection coefficient). R L Mapping bridge . Example 9:

[0075] In one feasible implementation, when the operator issues the instruction "perform preventative resurfacing on wet, slippery surfaces after rain, with caution," the large model decision unit LLM42 may output structured parameters in JSON format as follows: { "perception": { "reflectivity_threshold": 160 }, "motion_control": { "target_speed": 4.0, "mpc_weights": { "lateral_error": 0.85, "heading_error": 0.10, "control_effort": 0.05 }, "constraints": { "max_speed": 5.0, "max_accel": 0.3 } }, "spray_control": { "flow_rate_base": "medium", "pressure_adaptive_gain": 1.2 }, "context_tag": ["wet_road", "safety_first"] } These parameters are no longer natural language, but key-value pairs that can be directly parsed and used by downstream control modules (sensing threshold judgment, MPC, spraying mechanism); for example, the MPC module will read the values ​​under mpc_weights to configure its optimizer, and the spraying controller will set the initial flow rate and enable pressure adaptive gain according to flow_rate_base and pressure_adaptive_gain, thereby achieving seamless connection between intelligent decision-making and underlying execution.

[0076] It should be noted that the optimization objective parameters of the model predictive controller, specifically, are the weight matrix and reference trajectory defined in a discrete-time optimization problem. Typically, its optimization objective function (cost function) J can be formalized as:

[0077] in: y( k )∈ Is the system in the prediction time domain? k The predicted output of each step (such as the vehicle's lateral position and heading angle).

[0078] r( k )∈ It is the corresponding reference trajectory vector, which originates from the geometric center of the caliper.

[0079] Δu( k )=u( k )−u( k −1)∈ It is the first in the control time domain k Step control increments (such as left and right wheel acceleration).

[0080] N p It predicts the length of the time domain. N c It controls the length of the time domain, and usually... N c ≤ N p .

[0081] Let M denote the weighted quadratic norm of vector v, where M is a positive definite or semi-positive definite weight matrix.

[0082] and The "optimization objective parameters" (i.e., weight matrix) are generated by the large model decision unit (LLM). For example, when "precise tracking" is required, the large language model tends to output a larger target parameter. The matrix (or increasing the weighting coefficients of the constraint error terms); when "run smoothing" is required, a larger output is preferred. R matrix.

[0083] Rolling optimization refers to optimizing the sampling process in each sampling period. T s (like T s =10ms), the model prediction controller executes the following steps: 1) Measurement: Obtain the current system state x( t (e.g., vehicle position, speed); 2) Prediction: Based on the current state x( t ), system dynamic model and possible future control input sequences Predicting systems in the future Np Output sequence within a step ; 3) Optimization: Solve the following finite-time optimal control problem to obtain the optimal control sequence. :

[0084] st , k = t ,…, t + Np- 1 (System Dynamics Model) , k = t ,…, t + Nc- 1 (Control Constraints) , k = t ,…, t + Nc- 1 (Control Incremental Constraints) , k = t+1 ,…, t + Np (Output constraints) Among them, constraints such as umax and Δumax can be partially given dynamically by the large language model according to the job instructions.

[0085] 4) Execution: The first control variable in the optimized control sequence is executed. Apply it to the actual system (vehicle).

[0086] Reaching the next sampling period t When +1, repeat the entire process: based on the new state measurement value x( t +1), perform prediction-optimization again, and set the first control variable of the new solution. This method is applied to the system; and by repeating this process, it moves forward like a "rolling" motion. This method can continuously use the latest feedback information to compensate for model errors and external disturbances, achieving adaptive and precise control.

[0087] Example 10: To further illustrate the decision-making logic of this invention in actual operation, this embodiment provides a method for adjusting the control strategy according to different working conditions. This embodiment gives parameter mapping examples in two typical scenarios: Scenario 1: Nighttime preventive maintenance of high-grade highways. In this scenario, the operation requirements are "high-precision tracking" and "smooth operation traces" to ensure the visual appeal and guidance continuity of the road markings.

[0088] Perceptual input: The retroreflection coefficient estimation model calculates the retroreflection coefficient of the current marking as follows: (The markings are not completely ineffective, but their brightness has decreased.)

[0089] Manual / system instruction: "Maintain high precision, apply smoothly."

[0090] LLM decision logic: The large model decision unit LLM42 parses the semantics of the instructions and identifies the core requirements as "high precision (small error)" and "smoothness (slow action)".

[0091] Parameter generation: In the intelligent decision-making process, the large model decision unit LLM42 transforms the decision results into specific structured control parameters based on the identified "high-precision" semantic weights. 1) Set the spraying trigger threshold T threshold =220; 2) Weight matrix parameter mapping: LLM42 increases the cost function J State deviation weighting coefficient q 11 (For example, a value of 15.0), while increasing the control increment penalty weight coefficient. r 11 (For example, the value is 1.2); The system increases q 11 This ensures the vehicle stays close to the center of the lane markings (reducing tracking error) by increasing... r 11 Increase penalties for violent movements of the actuator, suppress rapid vibrations of the steering mechanism, and achieve high-quality, high-continuity lane marking.

[0092] Scenario 2: Maintenance of complex curvature road sections in urban low-speed areas. In this scenario, the working environment is complex and the curvature of the road section changes rapidly. The core requirement of the system changes from "smoothness" to "high dynamic tracking response".

[0093] Perceptual input: The visual sensor detects that the road ahead is entering a continuous curved section, and the retroreflection coefficient estimation model calculates... .

[0094] Manual / system instruction: "Enhance tracking response and ensure coverage."

[0095] LLM Decision Logic: The large model decision unit LLM42 parses the instruction semantics and identifies that "real-time tracking" has a higher priority than "smooth action" under the current working condition.

[0096] Parameter generation: LLM42 achieves online switching of control laws by adjusting the weight matrix.

[0097] 1. Reduce the weighting coefficient of the control increment penalty. r 11 For example, r 11 Reduced from 1.2 in Scenario 1 to 0.3; 2. Maintain or appropriately increase the weighting coefficient of state deviation. q 11 For example, keep it at 15.0.

[0098] By reducing r 11The system reduces the penalty for the movement range of the actuators, allowing the chassis steering mechanism and the painting robot arm to make faster and larger movement responses. When the trajectory is redrawn on sharp turns, it can closely cover the original markings and avoid the phenomenon of missing the target due to control lag.

[0099] It should be understood that the terms "one embodiment," "an embodiment," "a feasible implementation," or "some implementations" used throughout the specification mean that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present invention. Therefore, "one embodiment," "an embodiment," "a feasible implementation," or "some implementations" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to the embodiments of the present invention.

[0100] The above description is merely a specific embodiment of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included within the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.

Claims

1. An intelligent decision-making and control method for road marking maintenance, characterized in that, include: Visual perception step: Acquire image data containing the area of ​​road markings using a visual sensor; Reflectivity quantification step: Based on the grayscale information in the image data, determine the estimated value of the retroreflection coefficient of the marking using a pre-trained retroreflection coefficient estimation model. R L The retroreflection coefficient estimation model establishes a mapping relationship between gray-scale features and the standard-defined retroreflection coefficient values. Intelligent decision-making steps: Obtain natural language operation instructions, parse the semantics of the instructions using a large model decision unit (LLM), and combine the estimated retroreflection coefficient and vehicle state data to perform inference, generating a threshold for spraying operation. T threshold Structured control strategy parameters, including desired vehicle speed and model predictive controller weight matrix; Model predictive control steps: Based on the structured control strategy parameters, the geometric information of the markings, and the vehicle state data, the model predictive controller performs rolling optimization to generate driving commands for vehicle tracking and spraying commands for marking completion in real time.

2. The method according to claim 1, characterized in that, The specific steps for quantifying reflective properties include: Extract the grayscale mean of the region of interest (ROI) for the datum lines from the image data. μ and grayscale standard deviation σ ; The gray average value ( μ ) and gray standard deviation ( σ The input is fed into the pre-trained retroreflection coefficient estimation model; The model outputs a numerical value conforming to the dimensions of road engineering lighting specifications, which is used as the estimated value of the retroreflection coefficient.

3. The method according to claim 1, characterized in that, The Large Model Decision Unit (LLM) is configured to execute the following inference logic: Identify the semantics of the job scenario in natural language instructions and map them into weight preferences; If the semantic recognition is "high precision", then increase the state bias weighting coefficient of the model predictor controller. Q If the semantic recognition is "smooth", then increase the control increment weight coefficient. R ; The output contains a JSON-formatted policy message that includes the spraying trigger threshold and control weights.

4. The method according to claim 1, characterized in that, In the intelligent decision-making step, the Large Model Decision Unit (LLM) is configured as follows: Parse the natural language job instructions to understand the job's optimization objectives and operational constraints; By combining the estimated retroreflection coefficient with the vehicle status data, the current road marking performance and operating conditions are evaluated. The system dynamically infers and outputs the spraying operation trigger threshold, the desired vehicle speed, and the weighting coefficients used to adjust the balance between trajectory tracking accuracy and motion smoothness of the model predictive controller.

5. The method according to claim 1, characterized in that, The method also includes a closed-loop optimization step: After executing the spraying command, obtain the re-inspection data of the re-marked lines, the re-inspection data including thickness data and / or the estimated value of the re-inspection retroreflection coefficient; Based on the difference between the re-inspection data and the expected standard, parameter adjustment amounts are generated; The parameters of the structured control strategy are adjusted online using the aforementioned parameter adjustment amount.

6. The method according to claim 1 or 5, characterized in that, The structured control strategy parameters also include spraying flow rate parameters; when generating the spraying command in the model predictive control step, the control quantity of the spraying actuator is dynamically calculated and adjusted based on the current vehicle speed and the spraying flow rate parameters, so as to keep the amount of paint deposited per unit area stable.

7. An intelligent decision-making and control system for road marking maintenance, characterized in that, include: The visual perception module is used to acquire image data containing the area of ​​road markings; The reflectivity quantization module is used to determine the estimated value of the retroreflection coefficient of the marking based on the grayscale information in the image data and through a pre-trained retroreflection coefficient estimation model, wherein the retroreflection coefficient estimation model establishes a mapping relationship between grayscale features and the standard-defined retroreflection coefficient value; The intelligent decision-making module integrates a large model decision unit (LLM) to generate structured control strategy parameters based on the semantics of natural language operation instructions, the estimated retroreflection coefficient, and vehicle state data. The strategy parameters include at least the spraying operation trigger threshold and the optimization target parameters of the model predictive controller. The model predictive control module is used to generate driving commands and painting commands by performing rolling optimization through the model predictive controller based on the structured control strategy parameters, the geometric information of the markings, and the vehicle state data. The instruction output module is used to output the driving instruction and the spraying instruction.

8. The system according to claim 7, characterized in that, The reflectivity quantification module includes: The feature extraction unit is used to extract at least one gray-level statistical feature of the region of interest of the tracing from the image data; The model calculation unit stores and runs the pre-trained retroreflection coefficient estimation model, and is used to receive the gray-scale statistical features and output the estimated retroreflection coefficient values.

9. The system according to claim 7, characterized in that, The Large Model Decision Unit (LLM) in the intelligent decision-making module is configured as follows: Parse the natural language job instructions to understand the job's optimization objectives and operational constraints; By combining the estimated retroreflection coefficient with the vehicle status data, the current road marking performance and operating conditions are evaluated. The system dynamically infers and outputs the spraying operation trigger threshold, the desired vehicle speed, and the weighting coefficients used to adjust the balance between trajectory tracking accuracy and motion smoothness of the model predictive controller.

10. The system according to claim 7, characterized in that, The system also includes a closed-loop optimization module, used for: After the spraying is carried out, obtain the re-inspection data of the repainted lines; Analyze the differences between the re-inspection data and the expected standards, and generate parameter adjustment amounts; The parameter adjustment amount is fed back to the intelligent decision-making module or the reflectivity quantification module for dynamic optimization of the strategy or model.

Images