Road marking identifying and spraying method and system based on image processing

Abstract

The invention discloses a road marking identifying and spraying method and system based on image processing, and the method comprises the steps: collecting a lane line image through a camera, and converting the lane line image into an aerial view; establishing rigid calibration of the pose relation between the nozzle and the camera to realize coordinate registration; converting the aerial view from an RGB space to an HSV space, introducing a self-adaptive threshold adjustment mechanism on this basis, and obtaining a lane marking area image from the lane line image according to self-adaptive segmentation; a lane image is converted from a pixel coordinate space to a parameter space, and automatic detection of lane edge lines and marked lines is realized through Hough transform; in combination with the detected lane marking area, the detection area is classified through a gray threshold value or a brightness feature, and the offset of a marking center line is calculated; the transverse position of the spraying mechanism is adjusted in real time according to the offset of the center line of the marked line, dynamic opening and closing control over the spraying head is achieved in combination with the vehicle running speed and the recognition signal state, and it can be guaranteed that the spraying track and the recognition center line are kept consistent.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation and machine vision technology, specifically to a road marking recognition and spraying method and system based on image processing. Background Technology

[0002] With the development of automation technology, vehicle-mounted and ride-on road marking vehicles have emerged. Vehicle-mounted road marking vehicles place the marking equipment on a vehicle, using the vehicle to propel the equipment forward, providing a continuous and stable driving force. However, manual control of the spraying position is still required, resulting in a lower level of automation. Ride-on road marking machines integrate the marking equipment with the driving power system, incorporating multiple sensors and controllers, and are equipped with a microcomputer control system. This enables rapid and precise spraying, significantly improving the level of automation.

[0003] Road detection technology primarily utilizes onboard cameras and LiDAR to detect lane markings and road conditions in real time. Due to the high cost of LiDAR, current autonomous driving technology in my country mainly employs onboard cameras to collect road information for identification and processing. Road detection is extremely complex and challenging, and developing a road detection algorithm adaptable to various complex environments is currently the biggest challenge.

[0004] Although the above two methods greatly improve the efficiency of road marking, they still require manual control of the road marking machine's direction of travel in real time to ensure that the positioning rod in front of the road marking vehicle is aligned with the pre-marked baseline. The spraying quality is easily affected by the operator's subjective bias. Summary of the Invention

[0005] The technical problem to be solved by this invention is to address the shortcomings of existing vehicle-mounted and ride-on road marking vehicles in terms of automation and accuracy, and to solve the problem of disconnect between the "identification-control-execution" links in the road marking spraying process.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A road marking recognition and spraying method based on image processing includes: The camera captures lane line images and converts them into a bird's-eye view. Establish rigid calibration of the pose relationship between the nozzle and the camera to achieve coordinate registration; The bird's-eye view is converted from RGB space to HSV space, and an adaptive threshold adjustment mechanism is introduced on this basis. The lane line image converted to HSV space is then segmented to obtain the lane marking area image. The segmented lane image is transformed from pixel coordinate space to parameter space, and the lane edge lines and markings are automatically detected through Hough transform. Based on the detected lane marking areas, the detected areas are classified by grayscale threshold or brightness features, and the offset of the lane marking centerline is calculated. Based on the offset of the center line of the road marking, the lateral position of the spraying mechanism is adjusted in real time. Combined with the vehicle's running speed and the status of the recognition signal, the dynamic opening and closing control of the nozzle is realized to ensure that the spraying trajectory is consistent with the recognition center line.

[0007] In one embodiment of the present invention, establishing a rigid calibration of the pose relationship between the nozzle and the camera to achieve coordinate registration includes: Arrange a QR code on the ground plane, draw a circle with the QR code as the center, and place four known black dots in the four directions of the circle: top, bottom, left, and right. ; Align the center of the nozzle tip with the four black dots on the circumference in sequence, and record the nozzle coordinates each time. Simultaneously, the camera recognizes the QR code and provides the true coordinates of the black dot on the circumference in the camera coordinate system. After four measurements, a set of corresponding point pairs was obtained. ;in, For relative to the first When there are black dots, the coordinates of the origin of the nozzle coordinate system in the base coordinate system; Three unknowns in the rigid body transformation of the nozzle relative to the camera: translation amount and rotation angle Define a two-dimensional rotation matrix, obtain the relationship between the nozzle coordinates and the camera recognition coordinates, and organize them into a system of linear equations; The system of linear equations is solved using the least squares method to obtain... This is used as the pose compensation quantity of the nozzle-camera coordinate system.

[0008] In one embodiment of the present invention, the two-dimensional rotation matrix is: ; The nozzle coordinates and the camera recognition coordinates satisfy the following relationship: ; Rearranged into a system of linear equations: ; In the formula, It is a two-dimensional rotation matrix. The rotation angle is... The coordinates of the black dot. Let these be the nozzle coordinates. The translation amount, To find the parameter vector, ; represents the matrix transpose. For the corresponding point vector, Constructed from observational data.

[0009] In one embodiment of the present invention, acquiring an image of a lane marking area includes: The conversion of the bird's-eye view from RGB space to HSV space is as follows: ; In the formula, This is the representation of the converted image in the HSV color space. Here is a conversion function for converting from RGB color space to HSV color space. This is the original image in the RGB color space. Set upper and lower thresholds for each channel in the converted image. And binarize it according to the following formula: ; In the formula, for The pixel value of the point; The threshold parameter is adaptively adjusted based on the overall brightness distribution of the image. ; ; In the formula, , To adaptively adjust the upper and lower thresholds of each channel in the image, The average brightness of the current image. As the reference brightness, This is the adjustment coefficient; The image is segmented based on an adaptively adjusted threshold to obtain the lane marking area image.

[0010] In one embodiment of the present invention, automatic detection of lane edge lines and markings is achieved, including: The segmented lane image is transformed from pixel coordinate space to parameter space, and the transformation relationship is as follows: ; In the formula, The shortest distance from the point to the origin. For pixels, For perpendicular line and The included angle of the axis; Furthermore, through transformation relationships, multiple points on the same straight line in image space form a cluster of intersection points in parameter space; an accumulator array is then established in the discrete parameter space. For each edge point Calculate all possible transformation relationships Combine and accumulate votes: ; The local maxima in the accumulator correspond to the latent straight lines in the image, which can be determined by setting a threshold. Extract all that satisfy Candidate lines; The equation of the straight line in the image space is calculated from the peak values ​​in the parameter space. ; For each detected straight line, its endpoint coordinates are plotted and superimposed onto the original image to form the line detection result.

[0011] In one embodiment of the present invention, calculating the offset of the center line of the marking includes: Let the input image be For, its grayscale threshold If , then the binarized image is represented as: ; in, For image Binary representation; Perform connected component analysis on the thresholded image to filter out candidate regions that meet the area condition: ; In the formula, For the first Candidate regions, The set of all detected connected components. The minimum effective area threshold, It is a connected region; Retain connected regions with an area greater than a threshold , as candidate detection areas; For each candidate detection region, calculate its horizontal projection intensity distribution and find the geometric center or maximum response point of the distribution curve; ; ; In the formula, For horizontal projection intensity distribution, The geometric center or maximum response point of the distribution curve. The coordinate axes; Obtain the horizontal position of the center line of the grading line in the image. Horizontal distance This indicates the offset of the center line relative to the vertical axis of symmetry of the image.

[0012] In one embodiment of the present invention, adjusting the lateral position of the spraying mechanism in real time according to the offset of the center line of the marking includes: Based on the offset of the marking centerline, the actuator is controlled to move laterally in the opposite direction to align the nozzle center with the marking center. The adjustment formula is as follows: ; In the formula, Indicates the initial position of the nozzle; This represents the proportional control coefficient, determined through actual testing. The testing method involves measuring the ratio of the actual ground distance to the camera's moving distance. This represents the offset of the currently identified lane marking centerline. To adjust the lateral position of the spraying mechanism after adjustment.

[0013] In one embodiment of the present invention, dynamic opening and closing control of the nozzle is achieved by combining vehicle operating speed and identification signal status, including: When the camera detects data and the vehicle speed exceeds the minimum required speed and the waiting time reaches [the specified time], The nozzle turns on when the camera fails to recognize data and the waiting time reaches [a certain threshold]. The nozzle will then shut off.

[0014] In one embodiment of the present invention, ensuring that the spraying trajectory remains consistent with the identification center line includes: when the vehicle is moving at a constant speed, the spray head opening and closing time interval... By speed Nozzle coverage width Decide: ; By using a formula to calculate the spray nozzle opening and closing time interval, the spraying time can be adaptively adjusted at different vehicle speeds, ensuring that the spraying width is consistent with the marking width.

[0015] The present invention also provides a system for road marking recognition and spraying method based on image processing described above, comprising: The image acquisition and preprocessing module is used to acquire lane line images by the camera and convert the lane line images into bird's-eye views. The camera and nozzle rigid body calibration module is used to establish the rigid calibration of the pose relationship between the nozzle and the camera, and to achieve coordinate registration; The image segmentation module is used to convert the bird's-eye view from RGB space to HSV space, and on this basis, an adaptive threshold adjustment mechanism is introduced to obtain the lane marking area image from the lane line image converted to HSV space according to the adaptive segmentation. The lane marking detection module is used to transform the segmented lane image from pixel coordinate space to parameter space, and realize the automatic detection of lane edge lines and markings through Hough transform; The centerline extraction module is used to classify the detected lane marking area by grayscale threshold or brightness feature, and calculate the centerline offset of the lane marking. The spraying control and deviation compensation module is used to adjust the lateral position of the spraying mechanism in real time according to the offset of the center line of the marking, and combined with the vehicle running speed and the recognition signal status, to realize the dynamic opening and closing control of the nozzle, so as to ensure that the spraying trajectory is consistent with the recognition center line.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention utilizes a series of innovative technologies, including high-precision camera-nozzle rigid body calibration, adaptive HSV marking segmentation, bird's-eye view centerline tracking, and vehicle speed adaptive start-stop control, to achieve key performance indicators such as spraying path accuracy ≤ ±1.0 cm, marking recognition rate ≥ 98%, obstacle avoidance success rate ≥ 90%, operating speed ≥ 2 km / h, and annual maintenance cost per unit ≤ 5000 yuan. Even under extreme conditions such as complex lighting, varying degrees of road damage, and intersections, it can still achieve fully automatic, high-precision, and zero-missing-spray marking repainting. Its comprehensive performance indicators significantly surpass existing technologies, substantially reducing the labor costs and safety risks of road marking maintenance, and possessing immense economic value and promotional significance.

[0017] This invention has a high level of automation and intelligence, enabling precise positioning and correction, improving the accuracy and robustness of line marking recognition, enhancing construction efficiency and work quality, accurately establishing camera-sprayer coordinate mapping, improving spraying accuracy and consistency, and achieving real-time dynamic control of the spraying process.

[0018] Traditional road marking equipment still heavily relies on human vision and operation. Drivers need to maintain their driving trajectory and nozzle control for extended periods, making them highly susceptible to fatigue or subjective judgment errors, leading to curved, offset, or uneven road markings. This makes it difficult to meet the precision, consistency, and standardization requirements of intelligent transportation systems. To address this, this invention introduces image processing and visual calibration technologies to achieve fully automated and closed-loop control of the spraying process. Its core technical challenges are: how to stably identify road markings using an onboard camera under complex lighting and diverse road conditions; how to establish a precise mapping between the camera coordinate system and the nozzle coordinate system, ensuring accurate guidance of spraying based on visual recognition results; and how to adjust the nozzle position and opening / closing timing in real time based on the marking center deviation during spraying, achieving automatic path alignment and spraying consistency. By constructing a fusion system of image recognition, coordinate transformation, and spraying control, this invention fundamentally solves the problems of traditional road marking equipment's reliance on manual labor, low precision, and poor robustness, providing a high-precision, intelligent road marking spraying solution for intelligent transportation construction. The current solution is suitable for both newly constructed roads and repainting of existing roads. Attached Figure Description

[0019] Figure 1 This is a flowchart of a road marking recognition and spraying method based on image processing according to an embodiment of the present invention.

[0020] Figure 2 This is a flowchart of the road marking recognition and spraying method according to an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the spraying device according to an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the spraying process according to an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram illustrating the pose relationship between the camera and the nozzle in an embodiment of the present invention.

[0024] Figure 6 This is a schematic diagram of lane recognition and calculation results according to an embodiment of the present invention. Detailed Implementation

[0025] To facilitate understanding of the technical solution of the present invention by those skilled in the art, the technical solution of the present invention will now be further described in conjunction with the accompanying drawings.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0027] Please see Figure 1 , 2 As shown, the present invention provides a road marking recognition and spraying method based on image processing. Before describing the road marking recognition and spraying method of the present invention, the spraying operation will be described first.

[0028] like Figure 3 and Figure 4 As shown, the vehicle is equipped with a recognition nozzle device, including a camera 100 and a spraying mechanism. The spraying mechanism includes a nozzle 200 and an actuator 300 for adjusting the position of the nozzle 200. The spraying operation includes stages of recognition without spraying, recognition with spraying, and no recognition with spraying; therefore, it is necessary to control the spraying start and stop times.

[0029] Specifically, road marking recognition spraying methods include: S10: The camera captures lane line images and converts them into a bird's-eye view.

[0030] In one embodiment of the present invention, camera intrinsic parameters are obtained through Zhang Zhengyou planar calibration. Distortion parameters External parameters of each view ,in, It is the radial distortion coefficient; It is the tangential distortion coefficient. This is the matrix transpose. Indicates the first Rotation matrix for each view; Indicates the first The translation vectors of each view are used to construct the bird's-eye view perspective transformation matrix. This is used to transform lane line images into bird's-eye views in real time. Specifically, it includes the following steps: S1.1: Photographing images containing a planar checkerboard pattern from different poses and distances. Zhang Biaodingtu Assume the calibration plate is defined in the world coordinate system. of On a plane, the two-dimensional coordinates of a corner point are: The actual side length of the grid is Rice. Among them, This represents the i-th calibration image.

[0031] For each Estimating the plane homography matrix using DLT (Direct Linear Transformation) (Depend on (From world plane to image plane) ,(1); in, The two-dimensional coordinates of a point in the image are represented by the x-coordinate and the y-coordinate. This represents the coordinates of a three-dimensional point in the world coordinate system. The former is the x-coordinate, and the latter is the y-coordinate. It is usually assumed that... ; It is a scale factor used to balance the coordinate values ​​on both sides; It is a planar homography matrix, representing the perspective transformation matrix from the world coordinate system plane to the image plane; It is a planar homography matrix The elements represent the projection transformation coefficients between planes. DLT is commonly used for projection transformations from 2D image coordinates to 3D world coordinates, or in computer vision for camera calibration and calculating perspective transformation matrices. It is a method for calculating transformation matrices by directly using the linear relationship between image points and world coordinate points. make For each Constructing vector functions: (2); in, Representing the homography matrix The Middle Line number Column, No. Column elements; World coordinates and Corresponding column index; This represents a 5-dimensional vector composed of these elements in a specific order, used for linear DLT solution of homography.

[0032] And establish a system of linear equations: (3); in, This represents the vectorized form of the camera intrinsic parameter matrix. It is then calculated using Singular Value Decomposition (SVD) after overlaying all views. ,recover: (4); in, These represent the equivalent focal length on the camera's u-axis (horizontal axis), the equivalent focal length on the camera's v-axis (vertical axis), the pixel coordinate axis skew coefficient, the horizontal pixel coordinates of the optical center (principal point) on the image, and the vertical pixel coordinates of the optical center (principal point) on the image, respectively. The analytical expression is obtained.

[0033] S1.2: Decomposition External reference The pinhole junction diameter / tangential distortion model was used to estimate... The full nonlinear optimization (BA) joint refining is performed by minimizing the reprojection error. Among them, the pinhole combined with diameter / tangential distortion model is a general camera imaging model.

[0034] S1.3: Select Reference View (road level) ), its homography Among them, the bolded font , , These are the extrinsic parameters of the view.

[0035] Image ground plane base transformation To obtain the metric bird's-eye view definition axis alignment rotation With scale Finally, we get: (5); Use first at runtime Distortion removal, then with Performing perspective transformation yields a bird's-eye view, thus converting the lane line image into a bird's-eye view.

[0036] S20 establishes a rigid calibration of the nozzle and camera pose relationship to achieve coordinate registration.

[0037] Please see Figure 1 , 2 As shown in Figure 5, in one embodiment of the present invention, the nozzle coordinate system is determined. Relative to camera coordinate system Two-dimensional rigid body transformation relationship This ensures that the center of the QR code recognized by the image corresponds one-to-one with the actual spray point of the nozzle. Specifically, it includes the following sub-steps: S2.1: Arrange a QR code on the ground plane, draw a circle with the QR code as the center, and place four known black dots in the four directions of the circle: top, bottom, left, and right. Camera 100 is fixedly mounted, and its relative pose to the QR code can be obtained through a recognition algorithm. The nozzle 200 and camera 100 are rigidly connected; translational and rotational deviations are measured in subsequent steps. For the first The coordinates of the black dot in the camera. And, the coordinates of the QR code with a radius of... Draw a circle.

[0038] S2.2: Align the center of the nozzle tip with the four black dots on the circumference in sequence, and record the nozzle coordinates each time. Simultaneously, the camera recognizes the QR code and provides the actual coordinates of the black dot on the circumference. After four measurements, a set of corresponding point pairs was obtained. .in, For relative to the first The coordinates of the nozzle when there is a black dot.

[0039] S2.3: Three unknowns in the rigid body transformation of the nozzle relative to the camera: translation amount and rotation angle Define a two-dimensional rotation matrix, obtain the relationship between the nozzle coordinates and the camera recognition coordinates, and organize them into a system of linear equations.

[0040] The two-dimensional rotation matrix is: (6); The nozzle coordinates and the camera recognition coordinates satisfy: (7); Rearranged into a system of linear equations: (8); in, Let be the parameter vector to be determined. Constructed from observational data, This is the vector of the corresponding point.

[0041] S2.4: Solve the linear equation system using the least squares method to obtain... This is used as the pose compensation quantity of the nozzle-camera coordinate system.

[0042] (9); get Subsequently, this was used as the pose compensation value for the nozzle-camera coordinate system. Since the height direction cannot be directly obtained through this two-dimensional calibration, it was set as a manually adjustable parameter. .

[0043] Since the camera 100 is fixed, the nozzle 200 has a fixed length in the y-direction relative to the camera 100, and is dynamically adjusted in the x-direction. The nozzle position is dynamically controlled by adjusting the lateral offset.

[0044] S30 converts the bird's-eye view from RGB space to HSV space, and introduces an adaptive threshold adjustment mechanism to obtain the lane marking area image from the lane line image converted to HSV space based on adaptive segmentation.

[0045] In one embodiment of the present invention, the image is converted from the RGB space to the more color-separable HSV space, and an adaptive threshold adjustment mechanism is introduced to set upper and lower thresholds. The segmentation threshold is automatically adjusted based on changes in overall image brightness, thus maintaining the stability and continuity of line recognition under various lighting conditions. Specifically, this includes the following sub-steps: S3.1: Convert the bird's-eye view from RGB space to HSV space. The conversion relationship is as follows: (10); In the formula, This is the representation of the converted image in the HSV color space. This is a conversion function (non-linear transformation) for converting from the RGB color space to the HSV color space. This is the original image represented in the RGB color space. Here, H, S, and V represent the hue, saturation, and lightness components, respectively. This step helps enhance the differentiation of road markings across the chroma channels.

[0046] S3.2: Set upper and lower thresholds for each channel in the converted image. And binarize it according to the following formula: ,(11) In the formula, for The pixel value of a point is set to 1 if it is greater than a threshold, and 0 if it is less than a threshold. The resulting binary image retains only the highlighted areas. This method achieves rapid color filtering by adjusting a small number of threshold parameters, and is computationally simple and real-time.

[0047] S3.3: Considering the impact of changes in illumination, the threshold parameter is adaptively adjusted according to the overall brightness distribution of the image: (12); ; In the formula, , To adaptively adjust the upper and lower thresholds of each channel in the image, The average brightness of the current image. As the reference brightness, This is the adjustment coefficient.

[0048] S3.4: Segment the image based on the adaptively adjusted threshold to obtain the lane marking region image. After this adaptive adjustment, the robustness of threshold segmentation to changes in illumination is significantly enhanced, maintaining the clarity of the lane marking region even in scenes with uneven brightness.

[0049] S40 transforms the segmented lane image from pixel coordinate space to parameter space, and uses Hough transform to achieve automatic detection of lane edge lines and markings.

[0050] In one embodiment of the present invention, the binarized lane image is converted from pixel coordinate space. The process involves converting to parameter space and then using the Hough Transform to automatically detect lane edge lines and markings. This includes the following sub-steps: S4.1: Transform the segmented lane image from pixel coordinate space to parameter space. The conversion relationship is as follows: (13); In the formula, The shortest distance from the point to the origin. For pixels, For perpendicular line and The included angle of the axis.

[0051] Furthermore, through transformation relationships, multiple points on the same straight line in image space form a cluster of intersection points in parameter space; an accumulator array is then established in the discrete parameter space. For each edge point Calculate all possible transformation relationships Combine and accumulate votes: (14); The local maxima in the accumulator correspond to the latent straight lines in the image, which can be determined by setting a threshold. Extract all that satisfy Candidate lines; S4.2: Calculate the equation of the straight line in the image space from the peak values ​​in the parameter space; (15); For each detected straight line, its endpoint coordinates are plotted and superimposed onto the original image to form the line detection result. If the number of detected straight lines exceeds a preset limit... Then, the votes are selected from highest to lowest based on the accumulator count. Each result is considered a valid test result.

[0052] S50, combined with the detected lane marking area, classifies the detected area by grayscale threshold or brightness feature, and calculates the offset of the lane marking centerline.

[0053] In one embodiment of the present invention, the lane marking area identified in the image is combined with the detection area to be classified by grayscale threshold or brightness feature, and the position parameter of the lane marking centerline is extracted. This provides precise geometric input for subsequent nozzle deviation compensation and control decisions. Specifically, it includes the following sub-steps: S5.1: After completing brightness and color preprocessing, the system converts the image to grayscale and uses a dynamic thresholding method to separate the marking regions. Let the input image... For, its grayscale threshold If , then the binarized image is represented as: ,(16) in, For image Binary representation; Perform connected component analysis on the thresholded image to filter out candidate regions that meet the area condition: (17); In the formula, For the first Candidate regions, The set of all detected connected components. The minimum effective area threshold, For connected regions, retain connected regions with an area greater than a threshold. , as a candidate detection area.

[0054] S5.2: For each candidate detection region, calculate its horizontal projection intensity distribution and find the geometric center or maximum response point of the distribution curve; (18); ,(19) In the formula, For horizontal projection intensity distribution, The geometric center or maximum response point of the distribution curve. ...

[0055] Obtain the horizontal position of the center line of the grading line in the image. Horizontal distance This represents the offset of the center line relative to the vertical axis of symmetry of the image. As shown in Figure 6, the white rectangular area represents the detected marking area, and the dashed line represents the center line, with its horizontal distance... This indicates the offset of the center line relative to the vertical axis of symmetry of the image.

[0056] S5.3: The calculated center coordinates will be transformed using the bird's-eye view transformation matrix. Transform to the actual ground coordinate system: (20); The results are input into the nozzle control module for real-time calculation of the spray path and correction of deviations.

[0057] S60 adjusts the lateral position of the spraying mechanism in real time based on the offset of the road marking centerline, and, in conjunction with the vehicle's speed and the status of the recognition signal, dynamically controls the opening and closing of the spray nozzle to ensure that the spraying trajectory remains consistent with the recognition centerline, including: S6.1: Based on the offset of the center line of the marking, control the actuator to move laterally in the opposite direction to align the center of the nozzle with the center of the marking. The adjustment formula is as follows: ,(twenty one); In the formula, Indicates the initial position of the nozzle; This represents the proportional control coefficient, determined through actual testing. The testing method involves measuring the ratio of the actual ground distance to the camera's moving distance. This represents the offset of the currently identified lane marking centerline. To adjust the lateral position of the spraying mechanism after adjustment, the system corrects the nozzle position in real time based on deviation feedback to ensure the spraying trajectory is centered.

[0058] S6.2: When the camera detects data and the vehicle speed exceeds the minimum required speed and the waiting time reaches [a certain threshold], [the following conditions apply]. The nozzle turns on. When the camera fails to recognize data and the waiting time reaches [a certain threshold]... The nozzle will then shut off.

[0059] When the recognition module outputs a "data detected" signal, the system enters the painting preparation state. This is combined with the vehicle's operating speed. With system response delay The nozzle activation logic is defined as follows: ,(twenty two); in, This is a function of combined speed and response time. And, This is the minimum required speed.

[0060] When the recognition signal disappears (i.e., no marking data is detected), the system enters the spraying shutdown logic, also taking into account delay time compensation. : ,(twenty three); S6.3: Sprayer opening and closing time interval when the vehicle is moving at a constant speed. By speed Nozzle coverage width Decide: ; By using a formula to calculate the spray nozzle opening and closing time interval, the spraying time can be adaptively adjusted at different vehicle speeds, ensuring that the spraying width is consistent with the marking width.

[0061] In summary, this invention, through a series of innovative technologies such as high-precision camera-nozzle rigid body calibration, adaptive HSV marking segmentation, bird's-eye view centerline tracking, and vehicle speed adaptive start-stop control, achieves key performance indicators such as spraying path accuracy ≤ ±1.0cm, marking recognition rate ≥ 98%, obstacle avoidance success rate ≥ 90%, operating speed ≥ 2km / h, and annual maintenance cost per unit ≤ 5000 yuan. Even under extreme conditions such as complex lighting, varying degrees of road damage, and intersections, it can still achieve fully automatic, high-precision, and zero-missing-spray marking repainting. Its comprehensive performance indicators significantly surpass existing technologies, substantially reducing the labor costs and safety risks of road marking maintenance, and possessing immense economic value and promotional significance.

[0062] In another embodiment of the present invention, the present invention also provides a system for a road marking recognition and spraying method based on image processing, comprising: The image acquisition and preprocessing module is used to acquire lane line images by the camera and convert the lane line images into bird's-eye view images.

[0063] The camera and nozzle rigid body calibration module is used to establish the rigid calibration of the pose relationship between the nozzle and the camera, and to achieve coordinate registration.

[0064] The image segmentation module is used to convert the bird's-eye view from RGB space to HSV space, and on this basis, an adaptive threshold adjustment mechanism is introduced to obtain the lane marking area image from the lane line image converted to HSV space according to the adaptive segmentation.

[0065] The lane marking detection module is used to transform the segmented lane image from pixel coordinate space to parameter space, and realize the automatic detection of lane edge lines and lane markings through Hough transform.

[0066] The centerline extraction module is used to classify the detected lane marking areas by grayscale threshold or brightness features, and calculate the centerline offset of the lane markings.

[0067] The spraying control and deviation compensation module is used to adjust the lateral position of the spraying mechanism in real time according to the offset of the center line of the marking, and combined with the vehicle running speed and the recognition signal status, to realize the dynamic opening and closing control of the nozzle, so as to ensure that the spraying trajectory is consistent with the recognition center line.

[0068] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0069] The above embodiments are merely examples of implementation methods of the invention. The scope of protection of the present invention is not limited to the above embodiments. For those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A road marking recognition and spraying method based on image processing, characterized in that, include: The camera captures lane line images and converts them into a bird's-eye view. Establish rigid calibration of the pose relationship between the nozzle and the camera to achieve coordinate registration; The bird's-eye view is converted from RGB space to HSV space, and an adaptive threshold adjustment mechanism is introduced on this basis. The lane line image converted to HSV space is then segmented to obtain the lane marking area image. The segmented lane image is transformed from pixel coordinate space to parameter space, and the lane edge lines and markings are automatically detected through Hough transform. Based on the detected lane marking areas, the detected areas are classified by grayscale threshold or brightness features, and the offset of the lane marking centerline is calculated. Based on the offset of the center line of the road marking, the lateral position of the spraying mechanism is adjusted in real time. Combined with the vehicle's running speed and the status of the recognition signal, the dynamic opening and closing control of the nozzle is realized to ensure that the spraying trajectory is consistent with the recognition center line.

2. The road marking recognition and spraying method based on image processing according to claim 1, characterized in that, Establish rigid calibration of the nozzle and camera pose relationship to achieve coordinate registration, including: Arrange a QR code on the ground plane, draw a circle with the QR code as the center, and place four known black dots in the four directions of the circle: top, bottom, left, and right. ;in, For the first The coordinates of the black dot in the camera; Align the center of the nozzle tip with the four black dots on the circumference in sequence, and record the nozzle coordinates each time. Simultaneously, the camera recognizes the QR code and provides the true coordinates of the black dot on the circumference in the camera coordinate system. After four measurements, a set of corresponding point pairs was obtained. ;in, For relative to the first When there are black dots, the coordinates of the origin of the nozzle coordinate system in the base coordinate system; Three unknowns in the rigid body transformation of the nozzle relative to the camera: translation amount and rotation angle Define a two-dimensional rotation matrix, obtain the relationship between the nozzle coordinates and the camera recognition coordinates, and organize them into a system of linear equations; The system of linear equations is solved using the least squares method to obtain... This is used as the pose compensation quantity of the nozzle-camera coordinate system.

3. The road marking recognition and spraying method based on image processing according to claim 2, characterized in that, The two-dimensional rotation matrix is: ； The nozzle coordinates and the camera recognition coordinates satisfy the following relationship: ； Rearranged into a system of linear equations: ； In the formula, It is a two-dimensional rotation matrix. The rotation angle is... The coordinates of the black dot. Let these be the nozzle coordinates. The translation amount, To find the parameter vector, ; represents the matrix transpose. For the corresponding point vector, Constructed from observational data.

4. The road marking recognition and spraying method based on image processing according to claim 1, characterized in that, Acquire images of the lane marking area, including: The conversion of the bird's-eye view from RGB space to HSV space is as follows: ； In the formula, This is the representation of the converted image in the HSV color space. Here is a conversion function for converting from RGB color space to HSV color space. This is the original image in the RGB color space. Set upper and lower thresholds for each channel in the converted image. And binarize it according to the following formula: ； In the formula, for The pixel value of the point; The threshold parameter is adaptively adjusted based on the overall brightness distribution of the image. ； ； In the formula, , To adaptively adjust the upper and lower thresholds of each channel in the image, The average brightness of the current image. As the reference brightness, This is the adjustment coefficient; The image is segmented based on an adaptively adjusted threshold to obtain the lane marking area image.

5. The road marking recognition and spraying method based on image processing according to claim 1, characterized in that, Automatic detection of lane edge lines and markings, including: The segmented lane image is transformed from pixel coordinate space to parameter space, and the transformation relationship is as follows: ； In the formula, The shortest distance from point A to the origin. For pixels, For perpendicular line and The included angle of the axis; Furthermore, through transformation relationships, multiple points on the same straight line in image space form a cluster of intersection points in parameter space; an accumulator array is then established in the discrete parameter space. For each edge point Calculate all possible transformation relationships Combine and accumulate votes: ； The local maxima in the accumulator correspond to the latent straight lines in the image, which can be determined by setting a threshold. Extract all that satisfy Candidate lines; The equation of the straight line in the image space is calculated from the peak values ​​in the parameter space. ； For each detected straight line, its endpoint coordinates are plotted and superimposed onto the original image to form the line detection result.

6. The road marking recognition and spraying method based on image processing according to claim 1, characterized in that, Calculate the offset of the center line of the marking, including: Let the input image be For, its grayscale threshold If , then the binarized image is represented as: ； in, For image Binary representation; Perform connected component analysis on the thresholded image to filter out candidate regions that meet the area condition: ； In the formula, For the first Candidate regions, The set of all detected connected components. The minimum effective area threshold, It is a connected region; Retain connected regions with an area greater than a threshold , as candidate detection areas; For each candidate detection region, calculate its horizontal projection intensity distribution and find the geometric center or maximum response point of the distribution curve; ； ； In the formula, For horizontal projection intensity distribution, The geometric center or maximum response point of the distribution curve. The coordinate axes; Obtain the horizontal position of the center line of the grading line in the image. Horizontal distance This indicates the offset of the center line relative to the vertical axis of symmetry of the image.

7. The road marking recognition and spraying method based on image processing according to claim 1, characterized in that, Based on the offset of the center line of the road marking, the lateral position of the spraying mechanism is adjusted in real time, including: Based on the offset of the marking centerline, the actuator is controlled to move laterally in the opposite direction to align the nozzle center with the marking center. The adjustment formula is as follows: ； In the formula, Indicates the initial position of the nozzle; This represents the proportional control coefficient, determined through actual testing. The testing method involves measuring the ratio of the actual ground distance to the camera's moving distance. This represents the offset of the currently identified lane marking centerline. To adjust the lateral position of the spraying mechanism after adjustment.

8. The road marking recognition and spraying method based on image processing according to claim 1, characterized in that, By combining vehicle speed and signal status, dynamic opening and closing control of the nozzles is achieved, including: When the camera detects the data and the vehicle speed exceeds the minimum required speed and the waiting time reaches [the specified time]... The nozzle turns on when the camera fails to recognize data and the waiting time reaches [a certain threshold]. The nozzle will then shut off.

9. The road marking recognition and spraying method based on image processing according to claim 8, characterized in that, To ensure the spraying trajectory remains consistent with the recognition centerline, this includes: the spray nozzle opening and closing time interval when the vehicle is moving at a constant speed. By speed Nozzle coverage width Decide: ； By using a formula to calculate the spray nozzle opening and closing time interval, the spraying time can be adaptively adjusted at different vehicle speeds, ensuring that the spraying width is consistent with the marking width.

10. A system for a road marking recognition and spraying method based on image processing according to any one of claims 1-9, characterized in that, include: The image acquisition and preprocessing module is used to acquire lane line images by the camera and convert the lane line images into bird's-eye views. The camera and nozzle rigid body calibration module is used to establish the rigid calibration of the pose relationship between the nozzle and the camera, and to achieve coordinate registration; The image segmentation module is used to convert the bird's-eye view from RGB space to HSV space, and on this basis, an adaptive threshold adjustment mechanism is introduced to obtain the lane marking area image from the lane line image converted to HSV space according to the adaptive segmentation. The lane marking detection module is used to transform the segmented lane image from pixel coordinate space to parameter space, and realize the automatic detection of lane edge lines and markings through Hough transform; The centerline extraction module is used to classify the detected lane marking area by grayscale threshold or brightness feature, and calculate the centerline offset of the lane marking. The spraying control and deviation compensation module is used to adjust the lateral position of the spraying mechanism in real time according to the offset of the center line of the marking, and combined with the vehicle running speed and the recognition signal status, to realize the dynamic opening and closing control of the nozzle, so as to ensure that the spraying trajectory is consistent with the recognition center line.

Images