A road marking repair and drawing method based on image recognition

Abstract

The application discloses a road marking repair and drawing method based on image recognition, and relates to the technical field of image data processing.The method comprises the following steps: collecting high-resolution road marking images and synchronously collecting spatial pose data through a multi-sensor fusion system; performing brightness equalization, noise filtering and semantic segmentation on the images to accurately identify the marking area; detecting the damaged area based on morphological processing and geometric feature analysis; generating a repair path fitted by a Bezier curve and calculating drawing parameters; mapping the image coordinates to a world coordinate system to generate device control instructions; driving a mechanical arm to perform high-precision repair work and supporting real-time quality detection and multi-device collaborative maintenance.The application aims to solve the problems of low recognition accuracy, poor repair efficiency and insufficient automation in existing road marking maintenance, realizes automatic identification, accurate positioning and efficient repair of road marking damage, and significantly improves the accuracy, consistency and intelligent level of maintenance work.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a method for repairing and applying road markings based on image recognition. Background Technology

[0002] With the continuous evolution of intelligent transportation systems and automated road maintenance technologies, road markings, as a key infrastructure guiding vehicle traffic and ensuring traffic safety, directly impact traffic efficiency and accident prevention capabilities in terms of their integrity and visibility. Over long-term use, road markings are susceptible to fading, peeling, or breakage due to factors such as vehicle pressure, UV aging, rain erosion, and snow and ice, necessitating regular inspection and repair. Traditional road marking maintenance relies primarily on manual inspections and experience-based judgment, which suffers from low efficiency, strong subjectivity, and limited coverage, making it difficult to meet the high-frequency maintenance needs of large-scale road networks. In recent years, intelligent detection technology based on image recognition has been gradually introduced into the field of road marking condition assessment. This involves using vehicle-mounted or drone-borne visual sensors to collect road surface images, combined with computer vision algorithms to identify damaged marking areas, providing data support for automated repair.

[0003] Among them, the image recognition-based road marking repair and application method focuses on accurately locating the geometric shape and damaged location of road markings from acquired images, and generating repair paths and application parameters accordingly. The core of this method lies in achieving semantic segmentation of road markings, quantitative assessment of damaged areas, and high-precision mapping of repair trajectories to drive automated application equipment to complete precise operations. An ideal technical solution should be able to stably extract the topological structure of road markings even under non-ideal conditions such as complex lighting, shading, road surface stains, and partial missing markings, and seamlessly convert the recognition results into executable mechanical control commands.

[0004] However, existing technologies have significant shortcomings in the closed-loop integration of image recognition and road marking restoration. Current road marking recognition models mostly employ general semantic segmentation networks, failing to optimize for the slender structure, low contrast, and localized fracture characteristics of road markings. This results in insufficient edge localization accuracy, particularly in areas of severe wear, leading to false positives or false negatives. Furthermore, the lack of a dynamic coupling mechanism between the recognition results and the motion control of the marking equipment means that repair path generation relies on static geometric fitting, failing to adaptively adjust process parameters such as paint flow rate, nozzle height, and travel speed based on the actual degree of damage to the markings. In addition, existing systems lack a robust spatial mapping relationship between the image coordinate system and the equipment's operating coordinate system, making them susceptible to positional shifts due to vehicle movement or sensor pose deviations, resulting in misalignment between the restored markings and the original trajectory. Summary of the Invention

[0005] The purpose of this invention is to provide a road marking repair and application method based on image recognition, so as to solve the problems of low recognition accuracy, poor repair efficiency and insufficient automation in the existing technology for road marking maintenance.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A road marking repair and application method based on image recognition includes the following specific steps:

[0008] Step S1: Collect road marking image data. A high-resolution image of the road marking is obtained through a multi-sensor fusion acquisition system installed on the top of the detection vehicle. The system integrates a 20-megapixel color industrial camera and a near-infrared supplementary light device. The acquisition frequency is 30 frames per second, and the image resolution is 3840 pixels by 2160 pixels. At the same time, the spatial position and attitude data at the time of acquisition are recorded through the global positioning system and inertial measurement unit.

[0009] Step S2: Preprocess the image data, perform brightness equalization and color correction on the acquired raw image, use the limited contrast adaptive histogram equalization algorithm to enhance the local contrast of the image, and use the Gaussian filtering algorithm to eliminate noise interference. The filter kernel size is 5 pixels by 5 pixels and the standard deviation is 1.5.

[0010] Step S3: Identify the marking region. Input the preprocessed image into a deep convolutional neural network for semantic segmentation. The network adopts an encoder-decoder structure. The encoder part uses ResNet50 pre-trained on the ImageNet dataset as the backbone network. The decoder part fuses shallow and deep features through skip connections and outputs a probability map of each pixel belonging to the marking region. The probability threshold is set to 0.85.

[0011] Step S4: Detect the missing area. Based on the recognition result of the marking area, remove small noise through morphological opening operation. The structural element adopts a rectangular kernel of 3 pixels by 15 pixels. Then, calculate the connected components of the marking area and analyze the geometric features of each connected component, including area, perimeter and aspect ratio of the minimum bounding rectangle. When the area of ​​the marking area is less than 60% of the standard area of ​​the complete marking or there is a break gap greater than 5 cm, it is marked as a missing area.

[0012] Step S5: Generate a repair path. For the detected defective area, extract its contour point set, simplify the contour using the Douglas-Puk algorithm, with a distance threshold of 2 pixels, and then generate a repair path fitted by a Bézier curve based on the simplified contour points. At the same time, calculate the repair application parameters according to the area and type of the defective area, including paint flow rate, nozzle moving speed and ground clearance.

[0013] Step S6: Coordinate transformation and control command generation. The repair path in the image coordinate system is mapped to the world coordinate system through a perspective transformation matrix. This transformation matrix is ​​obtained by calibration through a calibration plate with a calibration error of less than 0.5 pixels. Then, the path is pose-compensated by combining data from the Global Positioning System and the Inertial Measurement Unit, and finally, motion control commands for the application device are generated.

[0014] Step S7: Perform the repair marking. The marking equipment drives the three-degree-of-freedom robotic arm to move along the repair path with the marking paint nozzle according to the received control command. At the same time, the paint flow rate and nozzle height are adjusted according to the preset marking parameters to complete the marking repair operation.

[0015] Preferably, the multi-sensor fusion acquisition system in step S1 further includes a lidar sensor for acquiring three-dimensional point cloud data of the road surface. The lidar has 32 lines, a horizontal angular resolution of 0.1 degrees, a vertical angular resolution of 1 degree, and a maximum detection distance of 100 meters. The point cloud data and image data are synchronized through timestamps and fused to assist in the extraction of three-dimensional geometric information of the road markings.

[0016] Preferably, the training process of the deep convolutional neural network in step S3 adopts a transfer learning strategy. First, it is pre-trained on the COCO dataset, and then fine-tuned on a self-built road marking dataset. This self-built dataset contains 100,000 labeled images, covering different weather conditions, light intensities and road surface types. Data augmentation methods include random rotation, brightness adjustment and adding Gaussian noise. Finally, the model achieves an average intersection-union ratio of 0.92 on the test set.

[0017] Preferably, the defect area detection in step S4 also includes the evaluation of the clarity of the marking edge. By calculating the gradient amplitude of the edge points of the marking area, when the average gradient amplitude is lower than the set threshold of 50, it is determined that there is a fading defect in the area and it needs to be included in the repair scope.

[0018] Preferably, the path repair generation in step S5 also takes into account the type of marking. For dashed markings, interpolation repair is performed according to the original marking interval pattern to ensure that the interval of the repaired dashed lines is consistent with the original markings, and the interval error is controlled within ±2 cm.

[0019] Preferably, in step S6, the coordinate transformation process uses the Zhang Zhengyou calibration method to obtain the camera intrinsic and extrinsic parameter matrices. The intrinsic parameter matrix includes the focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameter matrix describes the rotation and translation relationship between the camera coordinate system and the world coordinate system. The transformation parameters are optimized using the least squares method to ensure mapping accuracy.

[0020] Preferably, in step S7, the marking device adopts high-pressure airless spraying technology, the paint flow control accuracy is ±5 ml / min, the nozzle movement speed range is 0.1 m / s to 1 m / s, the ground clearance adjustment range is 5 cm to 20 cm, and the robotic arm repeatability is ±1 mm.

[0021] Preferably, it also includes a real-time quality detection step. During the repair and marking process, another set of cameras is used to collect images of the marked lines in real time. The same semantic segmentation model as in step S3 is used to identify the newly marked line area and calculate its similarity with the preset standard template. When the similarity is less than 95%, a re-marking mechanism is triggered.

[0022] Preferably, it also includes a data management and analysis module to establish a historical database of road marking repairs, record the location, time, damage type, repair parameters and quality assessment results of each repair operation, and use a time series analysis model based on the database to predict the lifespan and maintenance cycle of road markings, providing data support for road maintenance decisions.

[0023] Preferably, the method supports multi-device collaborative operation, connecting multiple detection vehicles and marking equipment to a central control platform via a wireless communication network. The platform optimizes resource scheduling based on a task allocation algorithm to achieve efficient collaborative maintenance of a large-scale road network, with a single device capable of repairing an average of 2 kilometers per day.

[0024] Compared with the prior art, the beneficial technical effects of the present invention are as follows:

[0025] This invention achieves accurate identification of damaged road marking areas through high-precision image acquisition and deep semantic segmentation technology. Combined with adaptive repair path generation and coordinate mapping mechanisms, it ensures high accuracy of the marking location. Real-time quality monitoring and parameter control strategies improve the reliability and consistency of repair operations. Finally, it constructs a complete automated closed-loop system from detection to repair, which significantly improves the efficiency and quality of road marking maintenance and reduces labor costs and traffic impact. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall technical solution architecture of the road marking repair and application method based on image recognition proposed in this invention;

[0027] Figure 2 This is a schematic diagram of the core principle framework of the line marking region recognition and defect detection based on deep convolutional neural networks in this invention. Detailed Implementation

[0028] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and not to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the invention.

[0029] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0030] In the embodiments of the present invention, the same reference numerals denote the same components, and for the sake of brevity, detailed descriptions of the same components are omitted in different embodiments. It should be understood that the thickness, length, width, and other dimensions of various components in the embodiments of the present invention shown in the accompanying drawings, as well as the overall thickness, length, width, and other dimensions of the integrated device, are merely illustrative and should not constitute any limitation on the present invention; the term "multiple" in the present invention refers to two or more (including two).

[0031] Example 1: To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0032] Currently, road markings suffer structural damage such as fading, peeling, or breakage due to multiple environmental factors during long-term service, including vehicle traffic, UV aging, rain erosion, and snow and ice. Traditional manual inspection and maintenance methods suffer from fundamental defects such as low efficiency, strong subjectivity, and limited coverage, making it difficult to meet the urgent needs of modern intelligent transportation systems for high-frequency, high-precision, and automated road maintenance. To address these technical problems, this invention proposes to achieve accurate identification of damaged areas in road markings through high-precision image acquisition and deep semantic segmentation technology. Combined with adaptive repair path generation and coordinate mapping mechanisms, it ensures high accuracy in marking location. Real-time quality monitoring and parameter control strategies improve the reliability and consistency of repair operations. Furthermore, it constructs a complete automated closed-loop system from detection to repair, significantly improving the efficiency and quality of road marking maintenance, reducing labor costs and traffic impact, and applying this method to a road marking repair and application method based on image recognition.

[0033] Reference Appendix Figure 1 The overall technical architecture of this invention includes a multi-sensor fusion acquisition system, an image preprocessing module, a deep semantic segmentation network, a defect region detection unit, a repair path generator, a coordinate transformation and control command generation module, a mapping execution mechanism, and an optional real-time quality detection and data management and analysis subsystem. These modules work collaboratively to form an end-to-end automated repair process.

[0034] Step S1: Collect road marking image data by acquiring high-resolution images of road markings through a multi-sensor fusion acquisition system installed on the top of the detection vehicle.

[0035] This system integrates a 20-megapixel color industrial camera and a near-infrared illumination device, capturing images at a frequency of 30 frames per second with a resolution of 3840 pixels by 2160 pixels. It simultaneously records the spatial position and attitude data at the moment of acquisition using a global positioning system and an inertial measurement unit. Specifically, the color industrial camera employs a global shutter CMOS sensor with a pixel size of 2.4 micrometers and a dynamic range of no less than 70 dB to ensure no motion blur in images during high-speed operation. The near-infrared illumination device operates at a wavelength of 850 nanometers with a peak power of 50 watts. It synchronizes with the camera exposure through pulse modulation, effectively suppressing ambient light interference, and significantly improving the contrast between road markings and the road surface background, especially in low-light scenarios such as dusk, dawn, or tunnels. The global positioning system employs a high-precision receiver supporting BeiDou, GPS, and GLONASS tri-mode positioning, achieving a positioning accuracy better than 0.1 meters. The inertial measurement unit includes a three-axis gyroscope and a three-axis accelerometer, with an angular random walk of less than 0.1 degrees per square root of hour and zero-bias stability better than 1 degree per hour. Data is fused using Kalman filtering to output attitude and position information at a frequency of 100 Hz. The multi-sensor fusion acquisition system also includes a lidar sensor for acquiring three-dimensional point cloud data of the road surface. The lidar has 32 lines, a horizontal angular resolution of 0.1 degrees, a vertical angular resolution of 1 degree, and a maximum detection range of 100 meters. Point cloud data and image data are synchronized with microsecond-level timestamps via hardware trigger signals. The fused data is used to assist in extracting the three-dimensional geometric information of road markings, such as using point cloud height information to distinguish road markings from white stains or tire tracks on the road surface, improving recognition robustness.

[0036] Step S2: Preprocess the image data by performing brightness equalization and color correction on the acquired raw image.

[0037] Specifically, the original RGB image is first converted to the LAB color space, where the L channel represents luminance and the A and B channels represent chroma. A contrast-limited adaptive histogram equalization algorithm is applied to the L channel. This algorithm divides the image into 8x8 local regions, calculates a histogram for each region, and performs equalization, while setting a contrast clipping threshold of 2.0% to prevent excessive noise amplification. The equalized L channel is then recombined with the original A and B channels to form a LAB image, which is then converted back to the RGB space. Subsequently, a Gaussian filter algorithm is applied independently to each channel of the RGB image to eliminate high-frequency noise interference. The filter kernel size is 5 pixels by 5 pixels, and the standard deviation is 1.5. This parameter combination strikes a balance between preserving the sharpness of the marker edges and suppressing salt-and-pepper noise, and experiments have shown that it can improve the signal-to-noise ratio by approximately 15 dB. The preprocessed image serves as the input to the subsequent deep learning model, with its pixel values ​​normalized to the 0-1 range.

[0038] Step S3: Identify the marked area and input the preprocessed image into a deep convolutional neural network for semantic segmentation.

[0039] This network employs an encoder-decoder architecture. The encoder uses a ResNet50 pre-trained on the ImageNet dataset as its backbone, while the decoder fuses shallow and deep features through skip connections, outputting a probability map of each pixel belonging to a datum region, with a probability threshold set to 0.85. Specifically, the encoder progressively downsamples the input image through five stages of residual blocks, extracting multi-scale feature maps at downsampling factors of 2, 4, 8, 16, and 32. The decoder progressively upsamples deep features through transposed convolutional layers, incorporating the corresponding scale feature map from the encoder through skip connections at each upsampling stage to recover spatial detail. The final output layer is a 1x1 convolution followed by a sigmoid activation function, generating a single-channel probability map. The training process of the deep convolutional neural network adopts a transfer learning strategy. It is first pre-trained on the COCO dataset, and then fine-tuned on a self-built road marking dataset containing 100,000 labeled images, covering different weather conditions such as sunny, cloudy, rainy, and snowy days, different light intensities such as strong midday light and weak evening light, and different road surface types such as asphalt, cement, and brick. Data augmentation methods include random rotation (angle range from -15 degrees to +15 degrees), brightness adjustment (gain factor from 0.8 to 1.2), and adding Gaussian noise (standard deviation of 0.01). The final model achieves an average intersection-union ratio (IU / U) of 0.92 on the independent test set, significantly outperforming benchmark models such as U-Net and DeepLabv3+.

[0040] Step S4: Detect the defective area. Based on the recognition results of the marking area, remove small noise through morphological opening operation. The structural element adopts a rectangular kernel of 3 pixels by 15 pixels. Then, calculate the connected components of the marking area and analyze the geometric features of each connected component, including area, perimeter and aspect ratio of the minimum bounding rectangle. When the area of ​​the marking area is less than 60% of the standard area of ​​the complete marking or there is a gap greater than 5 cm, it is marked as a defective area.

[0041] Specifically, morphological opening operations first perform erosion and then dilation, effectively removing isolated noise points with an area smaller than 45 pixels while preserving the main structure of the markings. Connectivity analysis uses the eight-neighbor connectivity criterion, calculating the total number of pixels (area), the number of boundary pixels (perimeter), and the aspect ratio of the minimum bounding rectangle for each connected region. The standard area of ​​a complete marking is pre-defined based on the marking type. For example, the standard width of a solid lane divider is 15 cm, which corresponds to an image area of ​​approximately 12,000 pixels under an image resolution of 3840 x 2160 and a vehicle height of 2 meters. If the area of ​​a connected region is less than 7200 pixels (i.e., 12000 x 60%), it is considered a localized wear defect. For break detection, the system scans along the main direction of the marking and calculates the Euclidean distance between the center points of adjacent connected regions. If this distance, projected in the world coordinate system, is greater than 5 cm, it is marked as a break defect. In addition, defect detection also includes an assessment of the clarity of the road marking edges. By calculating the gradient magnitude of the edge points of the road marking area, if the average gradient magnitude is lower than a set threshold of 50, the area is determined to have a fading defect and needs to be included in the repair scope. The gradient magnitude is calculated separately in the X and Y directions using the Sobel operator, and then the square root of the sum of the squares is taken. This indicator directly reflects the degree of contrast attenuation between the road marking and the road background.

[0042] Step S5: Generate a repair path. For the detected defective area, extract its contour point set, simplify the contour using the Douglas-Puk algorithm, with a distance threshold of 2 pixels, and then generate a repair path fitted by a Bézier curve based on the simplified contour points. At the same time, calculate the repair application parameters according to the area and type of the defective area, including paint flow rate, nozzle movement speed and ground clearance.

[0043] Specifically, contour extraction employs Canny edge detection combined with region growing to ensure closed contours without burrs. The Douglas-Puk algorithm recursively removes redundant points contributing less than 2 pixels to the overall shape, compressing the point set to less than 20% of its original size while retaining key inflection points, significantly reducing the computational complexity of subsequent curve fitting. Cubic Bézier curves are used for Bézier curve fitting, with control points solved using the least squares method to minimize the mean square error between the curve and the simplified contour points. The calculation of repair application parameters follows these rules: paint flow rate is directly proportional to the defect area, with a base flow rate of 100 ml / min; for every additional 100 square centimeters of defect area, the flow rate increases by 10 ml / min. Nozzle movement speed is inversely proportional to the severity of the defect; for slightly worn areas, the speed is set to 0.8 m / s, while for fractured or large-area peeling areas, the speed is reduced to 0.3 m / s. Ground clearance is dynamically adjusted based on road surface smoothness, using LiDAR point cloud computing to calculate local road surface curvature; the greater the curvature, the higher the clearance, ranging from 5 cm to 20 cm. In addition, the repair path generation also takes into account the type of marking. For dashed markings, the system first detects the interval pattern of the original dashed lines through Hough transform, calculates the average line segment length and interval distance, and then performs interpolation repair in the defect area according to the pattern to ensure that the interval of the repaired dashed lines is consistent with the original markings, and the interval error is controlled within ±2 cm.

[0044] Step S6, coordinate transformation and control command generation, maps the repair path in the image coordinate system to the world coordinate system through a perspective transformation matrix. This transformation matrix is ​​obtained by calibration through a calibration plate with a calibration error of less than 0.5 pixels. Then, the path is pose-compensated by combining data from the Global Positioning System and the Inertial Measurement Unit, and finally the motion control command of the application device is generated.

[0045] Specifically, the coordinate transformation process uses the Zhang Zhengyou calibration method to obtain the camera's intrinsic and extrinsic parameter matrices. The intrinsic parameter matrix includes the focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameter matrix describes the rotation and translation relationship between the camera coordinate system and the world coordinate system. The transformation parameters are optimized using the least squares method to ensure mapping accuracy. The calibration process is performed on a flat field using a 9x6 checkerboard calibration board, acquiring 20 sets of images from different viewpoints, with reprojection errors controlled within 0.3 pixels. The perspective transformation matrix H maps the image coordinates (u, v) to world coordinates (X, Y), satisfying:

[0046] ;

[0047] Where H is a 3x3 homogeneous matrix. During vehicle movement, the camera pose changes in real time due to bumps. The system uses real-time pose data provided by the Global Positioning System (GPS) and Inertial Measurement Unit (INS) to dynamically correct the initially calibrated extrinsic parameter matrix. The corrected transformation matrix is ​​used to transform each control point of the repair path from image coordinates to world coordinates. Subsequently, the control command generation module converts the world coordinate path into the joint space trajectory of the robotic arm of the application device, using fifth-order polynomial interpolation to ensure continuous velocity and acceleration, and outputs a PWM signal to the servo driver.

[0048] Step S7: Perform the repair marking. The marking equipment drives the three-degree-of-freedom robotic arm to move along the repair path with the marking paint nozzle according to the received control command. At the same time, the paint flow rate and nozzle height are adjusted according to the preset marking parameters to complete the marking repair operation.

[0049] Specifically, the marking equipment employs high-pressure airless spraying technology, with paint flow control accuracy of ±5 ml / min, nozzle movement speed ranging from 0.1 m / s to 1 m / s, ground clearance adjustment ranging from 5 cm to 20 cm, and robotic arm repeatability accuracy of ±1 mm. The three-degree-of-freedom robotic arm includes X and Y translation axes and a Z lifting axis. The X and Y axes are driven by linear motors with a maximum stroke of 1.5 meters, while the Z axis is driven by a servo electric cylinder with a response time of less than 100 milliseconds. The paint supply system uses closed-loop pressure control, with pressure sensors monitoring pipeline pressure in real time and a PID controller adjusting the diaphragm pump speed to ensure stable flow. During the repair process, a real-time quality inspection step is also included. A camera installed behind the marking equipment captures images of the marked lines in real time. The same semantic segmentation model as in step S3 is used to identify the newly marked area, calculating its similarity to a preset standard template. When the similarity is below 95%, a re-marking mechanism is triggered. The similarity calculation uses the Structural Similarity Index (SSIM), comprehensively considering brightness, contrast, and structural information. In addition, the system includes a data management and analysis module, which establishes a historical database of road marking repairs, recording the location, time, damage type, repair parameters, and quality assessment results of each repair operation. Based on this database, a time series analysis model is used to predict the lifespan and maintenance cycle of road markings, providing data support for road maintenance decisions. This method supports multi-device collaborative operation, connecting multiple inspection vehicles and marking equipment to a central control platform via a 4G / 5G wireless communication network. The platform optimizes resource scheduling based on a task allocation algorithm, achieving efficient collaborative maintenance of large-scale road networks, with a single device capable of repairing an average of 2 kilometers per day.

[0050] To illustrate the technical effects of this invention more specifically, the following application example is constructed: During nighttime maintenance work on a city expressway, a detection vehicle travels at a speed of 40 km / h, and a multi-sensor fusion acquisition system acquires real-time road surface images and point cloud data. After preprocessing and deep semantic segmentation, the system identifies a solid line peeling area measuring 80 cm long and 12 cm wide, with an area approximately 45% of the original standard area, and an average edge gradient amplitude of 35, indicating severe damage. The repair path generator extracts its contour and simplifies it to 15 key points, fitting a smooth Bézier curve. Based on the damaged area, the system sets the paint flow rate to 140 ml / min, the nozzle movement speed to 0.4 m / s, and the ground clearance to 8 cm. The coordinate transformation module, combined with real-time pose data, accurately maps the image path to world coordinates, and control commands drive the robotic arm to complete the marking. Real-time detection by the rear camera shows that the SSIM value of the newly marked line and the standard template is 0.97, meeting the quality requirements. The entire process, from detection to repair completion, takes less than 30 seconds and requires no manual intervention.

[0051] Example 2: Building upon Example 1, this example further introduces a road marking geometry reconstruction and repair verification mechanism based on 3D point clouds to address scenarios where 2D image recognition fails due to extreme lighting or severe dirt coverage. (See attached document.) Figure 2 The output of a deep convolutional neural network not only includes a two-dimensional probability map, but also integrates LiDAR point cloud data to generate a three-dimensional geometric model of the markings.

[0052] Specifically, in step S1, the 32-line lidar outputs point cloud data at a frequency of 10 frames per second, with each point containing three-dimensional coordinates (X, Y, Z) and reflection intensity I. After the point cloud and image are strictly synchronized via timestamps, the point cloud is projected onto the image plane using the extrinsic parameter matrix calibrated in step S6, establishing a one-to-one correspondence between pixels and point clouds. In step S3, the output probability map of the semantic segmentation network is used to filter the subset of point clouds belonging to the marking region. In step S4, defect detection is based not only on two-dimensional connected component analysis but also incorporates three-dimensional geometric features: the local plane fitting residual of the marking point cloud is calculated; if the residual is greater than 2 mm, it is determined that there is a depression or bulge-type structural damage in the region; simultaneously, through point cloud density analysis, if the number of points per unit area is lower than a threshold (e.g., 5 points per square centimeter), it is determined to be a peeling defect. In step S5, the repair path generation is performed directly in three-dimensional space. The three-dimensional contour points of the damaged area are extracted, simplified using the spatial Douglas-Puk algorithm, and fitted with a three-dimensional Bézier curve. This curve also includes height information to guide the dynamic adjustment of the nozzle in the Z-axis direction. In the real-time quality inspection in step S7, the system not only analyzes the rear camera image but also scans the newly applied area using a LiDAR scanner in front of the application equipment, reconstructing its three-dimensional shape. The Hausdorff distance is calculated between this area and the three-dimensional template of the standard marking. If the distance exceeds 3 mm, re-applying is triggered. This three-dimensional enhancement mechanism significantly improves the system's operational reliability in complex scenarios such as flooded roads after heavy rain or snow-covered roads in winter.

[0053] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape, and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for repairing and applying road markings based on image recognition, characterized in that: The specific steps include the following: Step S1: Acquire road marking image data. A high-resolution image of the road marking is obtained through a multi-sensor fusion acquisition system installed on the top of the detection vehicle. This system integrates an industrial camera and a near-infrared supplementary lighting device, and simultaneously records the spatial position and attitude data at the time of acquisition through a global positioning system and an inertial measurement unit. The multi-sensor fusion acquisition system also includes a lidar sensor to acquire three-dimensional point cloud data of the road surface. The point cloud data and image data are synchronized through timestamps and fused to assist in the extraction of three-dimensional geometric information of the road markings. Step S2: Preprocess the image data, perform brightness equalization and color correction on the acquired raw image, use the limited contrast adaptive histogram equalization algorithm to enhance the local contrast of the image, and use the Gaussian filtering algorithm to eliminate noise interference; Step S3: Identify the road marking area. Input the preprocessed image into a deep convolutional neural network for semantic segmentation. The network adopts an encoder-decoder structure and outputs a probability map of each pixel belonging to the road marking area. The training process of the deep convolutional neural network adopts a transfer learning strategy. First, it is pre-trained on the COCO dataset, and then fine-tuned on a self-built road marking dataset. This self-built dataset covers different weather conditions, light intensity and road surface type. Step S4: Detect defective areas. Based on the marking area recognition results, small noise is removed through morphological opening operations. Then, the connected components of the marking area are calculated, and the geometric features of each connected component are analyzed. When the area of ​​the marking area is less than 60% of the standard area of ​​the complete marking or there is a gap greater than 5 cm, it is marked as a defective area. At the same time, the edge sharpness is evaluated by calculating the gradient magnitude of the edge points of the marking area. When the average gradient magnitude is lower than the set threshold of 50, it is determined that there is a fading defect in the area. The defective area detection is further based on the three-dimensional point cloud data obtained by LiDAR. Structural damage is determined by calculating the local plane fitting residual of the marking point cloud. If the residual is greater than 2 mm, it is determined to be a depression or bulge damage. At the same time, through point cloud density analysis, if the number of points per unit area is less than 5 points per square centimeter, it is determined to be a peeling defect. Step S5: Generate a repair path. For the detected defective area, extract its contour point set, simplify the contour using the Douglas-Pokal algorithm, and then generate a repair path fitted with a Bézier curve based on the simplified contour points. At the same time, adaptively calculate the repair application parameters according to the area and type of the defective area, including paint flow rate, nozzle moving speed, and ground clearance. The paint flow rate is directly proportional to the defective area, and the nozzle moving speed is inversely proportional to the severity of the defect. The repair path is generated in three-dimensional space. The three-dimensional contour points are simplified using the spatial Douglas-Pokal algorithm and a three-dimensional Bézier curve is fitted to guide the dynamic adjustment of the nozzle in the Z-axis direction. Step S6: Coordinate transformation and control command generation. The repair path in the image coordinate system is mapped to the world coordinate system through a perspective transformation matrix. The camera intrinsic and extrinsic parameter matrices are obtained through the Zhang Zhengyou calibration method. Then, the camera pose changes caused by vehicle bumps are compensated in real time by combining data from the Global Positioning System and the Inertial Measurement Unit. Finally, motion control commands for the application device are generated. Step S7: Perform the repair marking. The marking equipment drives the three-degree-of-freedom robotic arm to move along the repair path with the marking paint nozzle according to the received control command. At the same time, the paint flow rate and nozzle height are adjusted according to the preset marking parameters to complete the marking repair work. During the repair process, the marking image after marking is collected in real time by the rear camera. The same semantic segmentation model as in step S3 is used to identify the newly marked marking area and calculate its similarity with the preset standard template. When the similarity is less than 95%, the re-marking mechanism is automatically triggered.

2. The road marking repair and application method based on image recognition according to claim 1, characterized in that: The generated repair path also takes into account the type of marking. For dashed markings, interpolation repair is performed according to the original marking interval pattern to ensure that the interval of the repaired dashed lines is consistent with the original markings.

3. The road marking repair and application method based on image recognition according to claim 1, characterized in that: The intrinsic parameter matrix includes focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameter matrix describes the rotation and translation relationship between the camera coordinate system and the world coordinate system. The transformation parameters are optimized using the least squares method.

4. The road marking repair and application method based on image recognition according to claim 1, characterized in that: The marking equipment adopts high-pressure airless spraying technology, with paint flow control accuracy of ±5 ml / min, nozzle movement speed range of 0.1 m / s to 1 m / s, ground clearance adjustment range of 5 cm to 20 cm, and robotic arm repeatability accuracy of ±1 mm.

5. The road marking repair and application method based on image recognition according to claim 1, characterized in that: It also includes a data management and analysis module, which establishes a historical database of road marking repairs, records the location, time, damage type, repair parameters and quality assessment results of each repair operation, and uses a time series analysis model based on this database to predict the lifespan and maintenance cycle of road markings.

Images