Visual detection and path planning integrated road marking robot control system

Abstract

The application provides a visual detection and path planning integrated road marking robot control system, which comprises a visual input data generation module, a marking path data generation module, a construction quality attribute judgment module and a construction execution control module. The system acquires road images in real time, extracts a concerned area and evaluates visual effectiveness; divides the concerned area into continuous and adjacent longitudinal sampling bands along the advancing direction of the robot, and generates a smooth and continuous marking path; calculates an overall smoothness index and a local mutation risk index based on the discrete second-order difference of the path point sequence, and quantifies the construction quality; and dynamically adjusts the walking and spraying rhythm according to the quality, so as to realize quality-driven adaptive construction. The application does not need to preset a fixed sampling number, is efficient in calculation, is suitable for embedded platforms, and significantly improves the marking precision, robustness and automation level.

Description

Technical Field

[0001] This invention belongs to the field of data processing, and in particular relates to a road marking robot control system that integrates visual inspection and path planning. Background Technology

[0002] As a specialized operating device for road construction scenarios, the core task of a road marking robot is to acquire road structure or existing marking information by relying on visual perception in complex, dynamic and variable road environments, and then plan its walking and spraying path based on this information, ultimately completing high-quality and standardized road marking construction.

[0003] Existing technologies generally employ a sequential approach of "visual inspection—path planning—motion control." This involves using a vision system to acquire and process images of the road surface, extracting information about road edges, existing markings, or guiding structures. Based on these visual results, a path is generated, and a robot is controlled to perform the painting operation. However, in actual road construction environments, this approach reveals significant shortcomings. First, road scenarios inherently present problems such as drastic changes in lighting, road surface reflections, water stains, oil stains, shadows, and uneven wear of old lines. This leads to significant instability in visual inspection results both spatially and temporally. Existing technologies typically assume sufficient reliability in visual output, lacking a systematic constraint mechanism on the credibility of visual information. Second, road marking operations are inherently irreversible. Once painting is completed, incorrect markings are difficult to correct. If the path planning stage relies directly on biased or incomplete visual results, perceptual errors can easily be amplified into actual construction defects, such as broken lines, offsets, overlaps, or boundary violations. Furthermore, existing path planning methods primarily focus on geometric feasibility and kinematic constraints, while rarely conducting forward-looking assessments of the planned path from the perspective of construction outcome quality. They lack the ability to impose risk constraints on future marking effects during the planning phase. In these circumstances, localized failures of the vision system, blind reliance on perception results in path planning, and passive following of the planned path by control execution collectively constitute the main technical bottlenecks for stable construction of road marking robots in complex environments. Existing technologies struggle to simultaneously ensure construction continuity, stability, and controllable marking quality under conditions of visual uncertainty. Summary of the Invention

[0004] The purpose of this invention is to design an integrated road marking robot control system that combines visual inspection and path planning. This system can reliably generate executable paths that meet construction quality requirements even in road environments with fluctuating visual conditions, significantly improving the reliability and practicality of the road marking robot in real construction scenarios.

[0005] To achieve the above objectives, the present invention provides an integrated road marking robot control system combining visual inspection and path planning, the system comprising: The visual input data generation module is used to acquire road image sequences, extract the area of ​​interest in the road ahead corresponding to the range of action of the spraying mechanism, perform grayscale mapping and normalization on continuous image frames in the area of ​​interest to obtain grayscale image sequences, calculate the visual effectiveness results based on the grayscale differences of adjacent frames within the time window, and output road visual input data. The road marking path data generation module is used to divide the region of interest into continuous and adjacent longitudinal sampling bands along the robot's forward direction based on the road visual input data. The longitudinal sampling bands are arranged sequentially along the horizontal direction to cover the entire length of the region of interest. Based on the average grayscale distribution of the time window, candidate center points of each longitudinal sampling band are determined, and the candidate center points are continuously weighted and updated in combination with the visual validity results to generate an initial path point sequence. Then, the initial path point sequence is locally morphologically adjusted to output road marking path data. The construction quality attribute determination module is used for the road marking path data. It calculates the average amplitude of the discrete second-order difference of the path point sequence in a sliding window manner as the overall smoothness index, and calculates the maximum value of the discrete second-order difference as the local mutation risk index. It outputs the construction quality attribute containing the overall smoothness index and the local mutation risk index. The construction execution control module is used to combine the overall smoothness index and the local sudden change risk index into a linear risk quantity based on the road marking path data and the construction quality attributes. It generates an execution adjustment coefficient through monotonically decreasing mapping, and synchronously scales the basic walking rhythm and the basic spraying rhythm based on the execution adjustment coefficient to generate actual control commands to drive the motion execution mechanism and the spraying execution mechanism to complete the road marking construction.

[0006] Furthermore, in the visual input data generation module, the area of ​​interest covers the road area that the spraying mechanism is about to pass through, and the time window consists of four consecutive frames of road images.

[0007] Furthermore, in the line path data generation module, the average grayscale distribution of the time window is obtained by aligning multiple grayscale images within the same longitudinal sampling band in the time window frame by frame and then taking the average.

[0008] Furthermore, in the line path data generation module, the candidate center point is determined by establishing a grayscale profile along the transverse direction within each longitudinal sampling band and locating the midpoint of the left and right boundaries of the transition area from the main road body to the boundary or marker area.

[0009] Furthermore, in the line path data generation module, the continuous weighted update adopts a convex combination of the current candidate center point and the previously generated path point, wherein the weight of the convex combination is dynamically adjusted by the visual validity result.

[0010] Furthermore, in the line path data generation module, the local shape adjustment is achieved by applying a discrete Laplace smoothing operation to the path point sequence. The discrete Laplace smoothing operation corrects the intermediate point based on the discrete second-order difference term of the three adjacent points.

[0011] Furthermore, in the construction quality attribute determination module, the overall smoothness index is calculated in segments along the path point sequence according to a fixed-length sliding window, and stored in association with the corresponding path segment index.

[0012] Furthermore, in the construction quality attribute determination module, the local mutation risk index is calculated in segments along the path point sequence according to a fixed-length sliding window, and stored in association with the corresponding path segment index.

[0013] Furthermore, in the construction execution control module, the execution adjustment coefficient ranges from 0 to 1, and decreases monotonically as the linear risk increases.

[0014] Furthermore, in the construction execution control module, the actual control command includes the walking control component of the motion actuator and the spraying control component of the spraying actuator, both of which are synchronously scaled through the same execution adjustment coefficient.

[0015] The beneficial technical effects of the present invention are at least as follows: To address the aforementioned problems, this invention provides an integrated road marking robot control system combining visual detection and path planning. By deeply integrating visual input, path generation, and construction execution, a closed-loop adaptive mechanism of perception-planning-control is constructed. This system can dynamically generate high-precision marking paths based on real-time road images and adjust the walking and spraying rhythm in real time according to path quality, effectively avoiding marking deviation, discontinuity, or overlap. Visual validity results are introduced to continuously weight and update candidate center points, and combined with local morphology adjustment, significantly improving the stability and smoothness of path planning under complex road conditions. Innovatively, discrete second-order difference average amplitude and maximum value are used to quantify the overall smoothness index of the path. With the local mutation risk index, an objective and calculable assessment of construction quality is achieved. Based on this quality index, a linear risk quantity is generated by weighting, and an execution adjustment coefficient is obtained through monotonically decreasing mapping. The basic walking rhythm and spraying rhythm are scaled synchronously to ensure automatic speed reduction and matching of spraying parameters in high-risk road sections, balancing construction quality and work efficiency. At the same time, the longitudinal sampling zone adopts a structural design that divides along the robot's forward direction, is continuously adjacent, and fully covers the area of ​​interest. It does not require a fixed number, adapts to different working conditions, and the overall algorithm is based on finite window and local operation, which is computationally efficient and easy to deploy in real time on embedded platforms. This significantly improves the intelligence level, environmental adaptability, and engineering practicality of the road marking robot. Attached Figure Description

[0016] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0017] Figure 1 This is a flowchart of the road marking robot control system integrating visual inspection and path planning of the present invention. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0019] In one or more embodiments, such as Figure 1 As shown, a road marking robot control system integrating visual inspection and path planning is disclosed. The system includes the following: The visual input data generation module is used to acquire road image sequences, extract the area of ​​interest in the road ahead corresponding to the range of action of the spraying mechanism, perform grayscale mapping and normalization on continuous image frames in the area of ​​interest to obtain grayscale image sequences, calculate the visual effectiveness results based on the grayscale differences of adjacent frames within the time window, and output road visual input data. The road marking path data generation module is used to divide the region of interest into continuous and adjacent longitudinal sampling bands along the robot's forward direction based on the road visual input data. The longitudinal sampling bands are arranged sequentially along the horizontal direction to cover the entire length of the region of interest. Based on the average grayscale distribution of the time window, candidate center points of each longitudinal sampling band are determined, and the candidate center points are continuously weighted and updated in combination with the visual validity results to generate an initial path point sequence. Then, the initial path point sequence is locally morphologically adjusted to output road marking path data. The construction quality attribute determination module is used for the road marking path data. It calculates the average amplitude of the discrete second-order difference of the path point sequence in a sliding window manner as the overall smoothness index, and calculates the maximum value of the discrete second-order difference as the local mutation risk index. It outputs the construction quality attribute containing the overall smoothness index and the local mutation risk index. The construction execution control module is used to combine the overall smoothness index and the local sudden change risk index into a linear risk quantity based on the road marking path data and the construction quality attributes. It generates an execution adjustment coefficient through monotonically decreasing mapping, and synchronously scales the basic walking rhythm and the basic spraying rhythm based on the execution adjustment coefficient to generate actual control commands to drive the motion execution mechanism and the spraying execution mechanism to complete the road marking construction.

[0020] Specifically, in this embodiment, the implementation steps of this system are as follows: Step 1: Acquire road visual input data for road marking. This step generates road visual input data for road marking operations, providing a stable and construction-relevant environmental basis for subsequent road marking path planning. During its movement, the road marking robot continuously acquires road surface images through a vision acquisition device fixedly mounted on its body. This device faces the robot's direction of travel and covers the road area that the spraying mechanism will pass through. The acquired images form a road image sequence in chronological order. Based on the actual needs of road marking construction, the system selects the road area ahead corresponding to the spraying mechanism's operating range from the image sequence as the region of interest to characterize the road surface state where construction is about to take place.

[0021] At road construction sites, factors such as glare, water stains, shifting shadows, or dust obstruction can cause varying degrees of visual fluctuations on the road surface within a short period. To express these fluctuations in a calculable form, this step performs uniform processing on the images within the area of ​​interest. Specifically, each frame of the road image is converted into a grayscale map within the area of ​​interest, and the grayscale map results are normalized to fall within a uniform numerical range. Based on consecutive image frames that have undergone normalization, the system calculates the intensity of visual changes in the area of ​​interest within a short time window, thereby obtaining a visually valid result that reflects the stability of the road surface. This calculation process originates from the frame difference analysis concept in classical signal processing and computer vision, measuring signal fluctuations by statistically analyzing the amplitude of changes between adjacent observations. The calculation form is as follows: ; in, Indicates the number of times collected in chronological order The result of grayscale mapping and normalization of the frame road image within the region of interest. Indicates the relationship with the first The result of grayscale mapping and normalization of the road image of the previous frame in adjacent frames within the same region of interest. This indicates the number of consecutive frames involved in the calculation, used to limit the length of the time window for visual effectiveness evaluation. This represents the magnitude of the difference between two adjacent frames within the region of interest. This magnitude of difference is obtained by taking the absolute value of the pixel-by-pixel grayscale difference within the region of interest and then averaging the results. This represents the visual effectiveness of the road surface within that time window. Since the grayscale mapping results used in the calculation are all normalized values, the resulting... It has a consistent numerical scale and can be used for comparative analysis between different time windows or different road areas.

[0022] To illustrate the practical application of the above calculation method, four consecutive frames of road images can be selected to form a time series. In this case, the number of adjacent frame pairs is... After completing grayscale mapping and normalization, the difference magnitude within the region of interest for three adjacent frames was calculated, and the resulting differences were as follows: , and Substituting the above results into the formula and summing the three difference magnitudes, we obtain... Divide by To obtain visual effectiveness results When this calculation is repeated at different time windows or different road locations, the smaller The value corresponds to a situation where the road surface changes little in a short period of time and the visual state is relatively stable, while a larger value corresponds to a situation where the road surface changes little in a short period of time and the visual state is relatively stable. The values ​​correspond to cases with significant perturbations. Finally, this step will organize the grayscale sequences of the areas of interest and their corresponding visual effectiveness results in chronological order. The unified structure serves as the road vision input data, used to describe the availability of the road ahead of the robot in spatial and temporal dimensions, and as the input basis for subsequent road marking path generation steps.

[0023] Step Two: Generate road marking path data for marking planning based on road visual input data. This step generates road marking path data based on the road visual input data formed in Step One. The goal is to transform the "road direction reflected by the gray-scale sequence of the area of ​​interest" into a geometric path point sequence that the road marking robot can continuously execute for painting. The input data consists of two parts: first, a gray-scale sequence of the area of ​​interest organized in chronological order, which provides spatial structural clues to the road surface; second, the visual validity results. This reflects the stability of the grayscale sequence within the selected time window. This step uses both methods simultaneously for path generation, ensuring that the path follows the road structure while maintaining linear continuity despite visual fluctuations.

[0024] Path generation first constructs "longitudinal sampling bands" along the forward direction within the region of interest. Each sampling band corresponds to the lateral extent of the road that the spraying mechanism may cover within a short forward distance. For each sampling band, frame-by-frame alignment is performed within a time window of the grayscale sequence, and the average grayscale image is obtained to obtain the "time window average grayscale distribution" of the sampling band. This distribution can still maintain the statistical characteristics of the main road structure under transient reflections or occasional occlusion. Subsequently, a grayscale profile is established laterally within the sampling band: for each lateral position, the average grayscale value of that position within the sampling band and its neighborhood variation trend are statistically analyzed. By finding regions where the grayscale change transitions from the "main road structure" to "boundary / marker / texture abrupt change," the positions of the left and right boundaries are determined; the midpoint of the left and right boundaries is used as the candidate center point of the sampling band. . The statistical results from the time window of the input grayscale sequence, without introducing any additional data types, result in a set of candidate center points arranged by sampling index. It is used to characterize the main direction of a road in the forward direction.

[0025] Candidate center points are used to generate the initial path point sequence. At that time, the introduction and The related continuous weighted update originates from first-order recursive smoothing in classical signal processing (which can also be viewed as a discrete-time first-order low-pass recursion). Its prototype involves a convex combination of the current observation and the state from the previous time step to suppress observation noise and maintain state continuity. This application makes specific modifications to this prototype: the "observation" is concretized as candidate center points. The "smoothing coefficient" is concretized into visual validity results. This allows the weights to change according to the visual stability of the construction site, thus directly embedding the stability information obtained in step one into the path point generation process. The update formula is as follows: ; in, For the first One road marking path point; For the first Candidate center points determined by gray-scale sequence statistics within each longitudinal sampling band; This is the previous path point that has already been generated; This is the visual validity result output from step one. Both sides of the above formula represent points of the same type (points under the same coordinate system). and Since the coefficients are constant, this convex combination maintains consistency in numerical type and is logically sound. The logical relationship of the formula is: first, it is obtained from the grayscale sequence. , and then Adjusting the ratio of "fitting the current candidate center point" to "maintaining the continuation of the existing path" yields... .

[0026] To make the path more in line with the requirements of continuous spraying for smooth lines, after obtaining Then, a local shape adjustment is performed. The initial source of this adjustment is the discrete Laplacian smoothing commonly used in numerical analysis and computer graphics / robot path processing. Its prototype comes from minimizing the discrete second-order difference energy (corresponding to the discrete form of "bending energy" or "curvature energy"), which suppresses sharp local changes by applying a second-order difference term to the point sequence. This application has implemented a practical application of this prototype for road marking: only one local update is performed on the path point sequence to make the line shape smoother at abrupt changes, thereby reducing the risk of spraying edge jitter. The update formula is as follows: ; in, These are the smoothed path points; , , These are three adjacent points; The adjustment coefficient controls the smoothing intensity. The terms in parentheses are discrete second-order difference terms, and their results are still the same as... The same type of geometric point increment expression, therefore with The addition maintains consistent numerical data types and conforms to common sense. The logical relationship between this formula and the previous formula is as follows: first, the initial path is generated using the first formula. Then, through the second formula... The final path is obtained by performing local morphological adjustments. This achieves an engineering-feasible balance between "structural fit" and "linear smoothness".

[0027] Suppose that the visual validity result of step one within a certain time window is... Candidate center points are obtained on five consecutive longitudinal sampling bands (the center position is represented by an example of a lateral scalar in the same coordinate system). , , , , And set the initial path point. .

[0028] Substitute into the first equation and calculate point by point: ; ; ; ; .

[0029] Take again Smooth the midpoint once (using...) For example): the second-order difference term is .

[0030] Therefore Similarly, this can be applied to... Calculated This ultimately forms a smoothed path point sequence. .

[0031] During engineering commissioning, records can be made on reflective or water-stained surfaces. Changes over time, and observations Lateral jitter amplitude: when When it increases, the first type automatically enhances the effect. The first formula maintains the continuity of the path under local fluctuations; the second formula adjusts the shape of sharp local changes, making the spraying trajectory smoother, thus making it easier to obtain road markings with smooth edges in actual spraying. The output of this step is road marking path data, specifically a sequence of final path points sorted by the direction of travel. (The corresponding sampling band index order can be saved along with it) for subsequent steps to determine the construction quality attributes of the path and further for the generation of construction control instructions.

[0032] Step 3: Determine the construction quality attributes of the road marking path based on the road marking path data. This step determines the construction quality attributes of the path based on the road marking path data generated in Step 2, transforming the path from a geometrically executable trajectory into a quality profile usable for construction control decisions. The input is the path point sequence output from Step 2. The sequence consists of waypoints ordered in the forward direction, and the visual validity results from step one are incorporated during the generation process. The impact on path continuity, therefore This step also reflects the constraints of road spatial structure and visual stability on the alignment. Focusing on the quality requirements of "continuous spraying, stable edges, and consistent alignment" in road marking construction, this step transforms the local geometric changes of the path point sequence into calculable and comparable construction quality attributes to support the selection and adjustment of walking and spraying control strategies in subsequent execution phases.

[0033] There is a direct correlation between the spraying quality at the construction site and the degree of local bending of the spray path: bending and swaying of the path over short distances will cause overlapping or sparse coverage of the spray trajectory on the ground, manifested as fluctuating line width and rough edges; a smooth path over a longer interval is more likely to form a continuous and stable spray boundary. This step uses the classic idea of ​​discrete curve analysis to quantify the above relationship. In discrete curve analysis, the first difference of the path point sequence is used to characterize the variation amplitude between adjacent points, and the second difference is used to characterize the bending trend of the discrete curve; this idea can be traced back to the finite difference method in numerical analysis and the discrete curvature characterization in curve geometry. This step uses the "average amplitude of the discrete second difference" as the main index of overall smoothness, and the "average amplitude of the first difference" as an auxiliary term related to construction continuity, to reflect the impact of the overall variation amplitude of the path during execution on the spraying stability. Based on the above original idea, this step unifies the two types of difference terms into the same scoring framework. By adding a weighted term from the first-order difference term to the second-order difference term, a path-level construction quality score for road marking construction is formed, and its calculation formula is as follows: ; in, , , These are the adjacent path points output in step two; This represents the number of path points. It is a discrete second-order difference term, derived from the finite difference form, used to characterize the local bending trend of the path; It is a discrete first-order difference term used to characterize the magnitude of path variation between adjacent points; , which is a weighting coefficient used to adjust the contribution of the first-order difference term to the overall score; This is used to score the construction quality at the path level. The derivation of the formula is as follows: the second-order difference average amplitude is used as the main term to reflect the sensitivity to "spray unevenness caused by bending"; a weighted term of the first-order difference average amplitude is added to reflect the impact of "overall variation amplitude on continuous spraying stability"; finally, the number of path points is normalized to make paths of different lengths comparable. Because... In the same coordinate system, the second-order difference term and the first-order difference term are both numerical changes of the same type. Since the coefficients are constant, the summation and averaging operations maintain consistent numerical data types. Therefore, the left and right sides of the formula maintain consistency in numerical data types, which is reasonable. When the coordinates are two-dimensional or three-dimensional points, the symbol is... This indicates taking the Euclidean norm of the difference vector within the parentheses.

[0034] Road marking construction also exhibits a special sensitivity to "local anomalies." For example, areas such as manhole cover edges, patch seams, and localized reflective water stains can easily cause sudden shifts or reversals in the path point sequence within a short segment. These anomalies typically cause concentrated and significant damage to the spraying results. Discrete curve analysis often uses extreme value statistics to highlight peak behavior in the sequence. This step uses the maximum value of the discrete second-order difference as the intensity of local anomalies to describe the abrupt change risk at the most unfavorable location in the path. This form originates from the classic extreme value statistics idea of ​​taking the supremum of the difference sequence and is used in this application to characterize the local risk in road marking construction. The calculation formula is as follows: ; in, The maximum local bending strength in the path; , , These are the adjacent path points output in step two; the discrete second-order difference term within parentheses is consistent with the previous equation. This equation is... The logical relationship is as follows: Used to express the overall smoothness and continuity risk of a path in a statistical sense. Both are used to express the risk of the most significant local abrupt change points in the path; both are based on the same discrete second-order difference prototype, thus ensuring consistency in evaluation criteria, and in engineering, they correspond to two types of construction concerns: "overall spraying stability" and "local spraying defect trigger points." Because... Maintaining the same numerical type as the difference term, the maximum value operation does not change its numerical type. Therefore, the left and right sides of the formula are consistent in numerical type, which is reasonable. In path comparison applications, different candidate paths can be calculated separately. And perform sorting or threshold determination; in the segmented evaluation of the same path, a sliding window of fixed length can be used to... Divide the data into segments and calculate the values ​​for each segment. and This step aims to identify high-risk areas and implement more conservative walking and spraying instructions for those areas during subsequent control phases. The output of this step is road marking path data with construction quality attributes, specifically a sequence of path points. and the construction quality attribute results associated with each path segment index. ,in Slide the window edge according to a fixed length Calculate and save segment by segment. Provided directly from step two. and Through the The results are obtained through differential calculation, weighted summation, and extreme value extraction. Using this output, subsequent construction control can characterize the overall path smoothness risk and local abrupt change risk with unified, calculable quality attributes, and generate walking and spraying control instructions more suitable for road marking construction, thereby improving the continuity and edge consistency of the markings.

[0035] Step 4: Control the road marking robot to complete the construction based on the road marking path data with construction quality attributes. This step transforms the road marking path data with construction quality attributes output in Step 3 into the walking and spraying actions of the road marking robot. The input data contains a sequence of path points sorted by the direction of travel. and the construction quality attributes corresponding to this path. ,in Provides a geometric reference for "where to go and where to spray". Provides a comprehensive profile of the smoothness and risks of the entire path. This provides a risk profile of local mutations along a pathway. This step controls the risk at the control level. This is transformed into a unified execution adjustment coefficient, which is used to synchronously adjust the walking rhythm of the motion actuator and the spraying rhythm of the spraying actuator. This makes the execution of higher-risk sections more stable and lower-risk sections more efficient on the same path, thereby realizing the "quality attribute of the path" as a "differentiated strategy for execution actions".

[0036] The construction of the adjustment coefficient employs the gain scheduling and normalization techniques commonly used in control engineering: first, risk indicators are linearly combined into a single risk quantity, and then obtained through a monotonically decreasing normalization mapping. The adjustment coefficient is within a certain range. This idea can be traced back to gain scheduling in classical control and the normalization of uncertainty amplitude in robust design; this application concretizes it into the road marking scenario, to... and As a risk input, "execution parameter scaling" is taken as the output action. The linear risk quantity is defined as... Subsequently, in order to ensure that the adjustment coefficient decreases smoothly as the risk level increases, and at the same time... It maintains no scaling and uses a standard bounded normalization form. The formula for calculating the adjustment coefficient is as follows: ; in, To implement the adjustment coefficient; and Output from step three; and This is a weighting coefficient used to balance the contributions of overall smoothness risk and local mutation risk in performance regulation. Because , , , All participate in linear combinations using the same numerical type, and By normalizing the combination, the formula maintains a consistent numerical type on both sides and satisfies the common sense that "the higher the risk, the smaller the coefficient".

[0037] In practical implementation, the control system uses The geometric reference is used to calculate steering commands and issue them to the motion actuators; at the same time, a basic walking rhythm and a basic spraying rhythm are generated for the current path segment, which together constitute the basic execution commands. The controller reads the construction quality attributes associated with the current path segment index. Calculations yielded Afterwards, Scaling is performed to obtain the actual execution instructions issued. This allows the walking and spraying rhythms to be adjusted synchronously according to the level of risk. The actual control command is then obtained. ; in, Indicates the first The moment (or the first) The actual control commands issued to the actuators (corresponding to the control cycle of the segment path) include the traveling part of the motion actuator and the spraying part of the spraying actuator; Indicates by The generated basic control commands; For the reason The calculated adjustment coefficient. This formula reflects the logical relationship between the formulas: first from step three... From the previous equation, we obtain... Then by generate Finally, we obtain the result from this formula. And issued for implementation. Because Scaling factor and Maintaining a consistent numerical data type ensures that scaling operations are controlled and logical.

[0038] For example, the output of step three is... , ,Pick , First, calculate the linear risk. Substituting this into the first equation of this step, we get... Suppose that within a certain control cycle, the path... Generated basic control instructions Numerically represented as "walking rhythm component" Spraying rhythm component is (Both are command quantities under the same scale within the control system), then the actual control command is obtained from the second equation. The corresponding component is and When the other path corresponds to Smaller (e.g.) , Take the same When ), the same calculation can be performed. , This allows for an execution rhythm closer to the basic instructions, thereby improving construction efficiency on lower-risk paths; when Larger Further reduction slows down the execution pace, making the spraying trajectory more stable and suppressing edge fluctuations and local defects caused by unfavorable path geometry at the execution level.

[0039] The output of this step is a sequence of control commands for the road marking robot. And the actual construction execution results generated by this command sequence. Control command sequence Depend on and The decision is made jointly and sent to the motion actuator and the spraying actuator, causing the robot to move along... Complete the actual road marking spraying construction, and at the same time, pass the path quality attributes through This is implemented as an executable adjustment of walking and spraying actions.

[0040] This invention also provides a visual inspection and path planning integrated road marking robot control device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps as described in the above embodiments of the visual inspection and path planning integrated road marking robot control system; or, when the processor executes the computer program, it implements the functions of each module in the above system embodiments.

[0041] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the integrated vision detection and path planning road marking robot control device.

[0042] The integrated visual inspection and path planning road marking robot control device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The integrated visual inspection and path planning road marking robot control device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the integrated visual inspection and path planning road marking robot control device may also include input / output devices, network access devices, buses, etc.

[0043] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the integrated vision inspection and path planning road marking robot control device, connecting all parts of the device via various interfaces and lines.

[0044] The memory can be used to store the computer programs and / or modules. The processor, by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory, realizes various functions of the integrated visual inspection and path planning road marking robot control device. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the operation of the air conditioning controller, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0045] The integrated module of the vision detection and path planning road marking robot control device, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-described system embodiments can also be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various system embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0046] Those skilled in the art will understand that all or part of the processes in the systems described in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the embodiments of the above systems. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0047] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A road marking robot control system integrating visual inspection and path planning, characterized in that, The system includes: The visual input data generation module is used to acquire road image sequences, extract the area of ​​interest in the road ahead corresponding to the range of action of the spraying mechanism, perform grayscale mapping and normalization on continuous image frames in the area of ​​interest to obtain grayscale image sequences, calculate the visual effectiveness results based on the grayscale differences of adjacent frames within the time window, and output road visual input data. The road marking path data generation module is used to divide the region of interest into continuous and adjacent longitudinal sampling bands along the robot's forward direction based on the road visual input data. The longitudinal sampling bands are arranged sequentially along the horizontal direction to cover the entire length of the region of interest. Based on the average grayscale distribution of the time window, candidate center points of each longitudinal sampling band are determined, and the candidate center points are continuously weighted and updated in combination with the visual validity results to generate an initial path point sequence. Then, the initial path point sequence is locally morphologically adjusted to output road marking path data. The construction quality attribute determination module is used for the road marking path data. It calculates the average amplitude of the discrete second-order difference of the path point sequence in a sliding window manner as the overall smoothness index, and calculates the maximum value of the discrete second-order difference as the local mutation risk index. It outputs the construction quality attribute containing the overall smoothness index and the local mutation risk index. The construction execution control module is used to combine the overall smoothness index and the local sudden change risk index into a linear risk quantity based on the road marking path data and the construction quality attributes. It generates an execution adjustment coefficient through monotonically decreasing mapping, and synchronously scales the basic walking rhythm and the basic spraying rhythm based on the execution adjustment coefficient to generate actual control commands to drive the motion execution mechanism and the spraying execution mechanism to complete the road marking construction.

2. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the visual input data generation module, the area of ​​interest covers the road area that the spraying mechanism is about to pass through, and the time window consists of four consecutive frames of road images.

3. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the line path data generation module, the average grayscale distribution of the time window is obtained by aligning multiple grayscale images within the same vertical sampling band in the time window frame by frame and then taking the average.

4. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the line path data generation module, the candidate center point is determined by establishing a grayscale profile along the transverse direction within each longitudinal sampling band and locating the midpoint of the left and right boundaries of the transition area from the main road body to the boundary or marker area.

5. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the line path data generation module, the continuous weighted update adopts a convex combination of the current candidate center point and the previously generated path point, wherein the weight of the convex combination is dynamically adjusted by the visual validity result.

6. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the line path data generation module, the local shape adjustment is achieved by applying a discrete Laplace smoothing operation to the path point sequence. The discrete Laplace smoothing operation corrects the intermediate point based on the discrete second-order difference term of the three adjacent points.

7. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the construction quality attribute determination module, the overall smoothness index is calculated in segments along the path point sequence according to a fixed-length sliding window, and stored in association with the corresponding path segment index.

8. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the construction quality attribute determination module, the local mutation risk index is calculated in segments along the path point sequence according to a fixed-length sliding window, and stored in association with the corresponding path segment index.

9. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the construction execution control module, the execution adjustment coefficient ranges from 0 to 1, and decreases monotonically as the linear risk increases.

10. The integrated road marking robot control system for visual detection and path planning according to claim 1, characterized in that, In the construction execution control module, the actual control command includes the walking control component of the motion actuator and the spraying control component of the spraying actuator, both of which are synchronously scaled through the same execution adjustment coefficient.

Images