A visual detection road marking robot control method and system
By installing a visual inspection device at the front of the road marking robot, geometric and spraying state vectors are extracted and fused to construct construction completion constraints, solving the problem of uncontrollable construction quality caused by unstable visual inspection and achieving efficient construction quality control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FOSHAN DAOSHAN INTELLIGENT ROBOT CO LTD
- Filing Date
- 2026-02-05
- Publication Date
- 2026-06-09
Smart Images
Figure CN122172778A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of road engineering, and in particular relates to a control method and system for a vision-based road marking robot. Background Technology
[0002] In recent years, vision-based road marking robots have been gradually applied to actual construction scenarios. These robots acquire road surface images using cameras, identify existing or planned road markings, and control the robot's trajectory and spraying equipment accordingly to complete the marking operation. While this technology has improved the level of construction automation to some extent, its engineering applications still reveal a series of unavoidable problems. First, the road construction environment is complex and variable. Lighting conditions are significantly affected by day-night cycles, weather conditions, and road reflectivity. Existing markings, road surface pollution, and temporary obstructions can easily lead to unstable visual detection results. Current technologies typically assume that visual recognition results are sufficiently reliable, lacking an effective mechanism for evaluating the stability and reliability of the results. Consequently, even when visual information degrades, the robot may still follow the predetermined control strategy for marking, easily causing trajectory deviations or spraying anomalies. Secondly, existing road marking robots mostly rely on real-time visual deviations or trajectory errors for control, focusing on whether the line conforms to the target path at any given moment. This ignores the fact that road marking is an irreversible construction process. If problems such as uneven line width, broken lines, or insufficient coverage occur during construction, corrections are often only possible through rework or additional marking, increasing construction costs and potentially affecting normal traffic flow. Furthermore, current technologies primarily rely on offline inspection or manual verification after construction is completed. The lack of a direct link between the control system and the final acceptance standards leads to a disconnect between the control process and the project quality objectives. Against this backdrop, how to avoid substandard marking results under conditions of uncertainty in visual inspection, and how to proactively constrain control behaviors that may lead to quality problems during construction, have become critical technical issues that urgently need to be addressed in the field of road marking robots. Summary of the Invention
[0003] The purpose of this invention is to design a visual inspection road marking robot control method and system, which can effectively connect the control process with the project acceptance standards without adding extra construction procedures, and reduce the probability of unqualified marking due to visual degradation or control fluctuations.
[0004] To achieve the above objectives, a visual detection-based road marking robot control method is provided in a first aspect of the present invention, the method comprising: The road surface images, including the planned marking reference area and the already painted road marking area, are continuously collected by the vision detection device installed at the front of the road marking robot. Based on the road surface image, the geometric state vector representing the reference of the planned marking and the spraying state vector representing the actual construction quality of the painted road markings are extracted respectively. The geometric state vector and the spraying state vector are then normalized and fused to generate construction state information. Based on the construction status information of the current sampling period and the construction status information of the previous sampling period, the validity weight vector by dimension is calculated, and the construction status information is reweighted in combination with the spraying risk scalar to obtain the valid status description. Based on the effective state description, a construction completion constraint reflecting the current construction completion quality is constructed by the intersection of the weighted square term and the spraying-related component. Based on the construction completion constraints, control modulation coefficients are generated and applied to the basic construction instruction set to form the final construction control instructions, so as to synchronously adjust the travel speed, steering response coefficient and spraying execution rhythm of the road marking robot.
[0005] Furthermore, the area where the road markings have been painted is located behind the painting mechanism of the road marking robot, and the painting state vector is generated based on the actual width, continuity, and boundary stability of the painted road markings.
[0006] Furthermore, the geometric state vector includes a lateral deviation ratio, a direction change ratio, and a local curvature change ratio, and the spraying state vector includes a width deviation ratio, a continuity gap ratio, and a boundary instability ratio.
[0007] Furthermore, the normalization process scales the original feature values using a reference scale related to the desired datum width or the allowable lateral deviation threshold.
[0008] Furthermore, the spraying risk scalar is the maximum value of each component in the spraying state vector.
[0009] Furthermore, each dimension of the validity weight vector is obtained by exponentially decaying the absolute difference between the construction status information of the corresponding dimension of the current sampling period and the previous sampling period, and is further multiplied by the overall convergence factor determined by the spraying risk scalar.
[0010] Furthermore, in the construction of the construction completion constraint, the cross term is the sum of the products of all spraying-related components in the effective state description after pairing them.
[0011] Furthermore, the control modulation coefficient is obtained by inputting the construction completion constraint into an exponential decay function, and the value range of the control modulation coefficient is between 0 and 1.
[0012] Furthermore, the final construction control command reduces the travel speed of the road marking robot while simultaneously decreasing the steering response coefficient and extending the single action duration of the spraying execution rhythm, so as to maintain the continuity of the markings and the stability of the boundaries when the construction completion constraint indicates that there is a risk of spraying quality.
[0013] A second aspect of the invention provides a vision-based road marking robot control system, the system comprising: A visual inspection device is installed at the front of the road marking robot to continuously acquire images of the road surface, including the planned marking reference area and the already painted road marking area behind the robot. The status information processing module is used to extract geometric status vectors and spraying status vectors based on the road surface image, and then fuse the geometric status vectors and spraying status vectors after normalization to generate construction status information. The effective state description generation module is used to calculate the dimension-based effectiveness weight vector based on the construction state information of the current sampling period and the construction state information of the previous sampling period, and to reweight the construction state information in combination with the spraying risk scalar to obtain the effective state description. The construction completion constraint generation module is used to construct a construction completion constraint reflecting the current construction completion quality based on the effective state description by using the intersection of the weighted square term and the spraying-related component. The control command generation module is used to generate control modulation coefficients based on the construction completion constraints, and apply the control modulation coefficients to the basic construction command set to form the final construction control command, so as to synchronously adjust the travel speed, steering response coefficient and spraying execution rhythm of the road marking robot.
[0014] The beneficial technical effects of the present invention are at least as follows: To address the aforementioned problems, this invention provides a visual detection-based road marking robot control method and system. By installing a visual detection device at the front of the road marking robot, it simultaneously acquires road surface images containing both the planned road marking reference area ahead and the already painted road marking area behind. Geometric state vectors representing the geometric features of the planned reference area and painting state vectors representing the actual construction quality of the painted markings are extracted, respectively. These are then normalized and fused to generate construction state information. Furthermore, a dimensional validity weight vector is calculated by combining the current and previous sampling period's construction state information, and a painting risk scalar is introduced to reweight the construction state information, resulting in a valid state description. Based on this valid state description, a construction completion constraint containing weighted square terms and cross terms of painting-related components is constructed, and control modulation coefficients are generated accordingly to synchronously adjust the travel speed, steering response coefficient, and painting execution rhythm. Therefore, this invention achieves closed-loop quality feedback control of the construction process. In complex environments such as road surface reflection, interference from old markings, sudden changes in lighting, or partial obstruction, it can actively suppress control fluctuations caused by visual misjudgment, adaptively reduce system response sensitivity, ensure the consistency of marking width, edge stability and continuity, significantly improve the first-time forming pass rate, reduce rework and manual intervention, and effectively solve the technical problems of uncontrollable construction quality and poor robustness caused by blindly relying on forward vision in existing road marking robots. Attached Figure Description
[0015] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.
[0016] Figure 1 This is a flowchart of a vision-based road marking robot control method according to the present invention.
[0017] Figure 2 This is a framework diagram of a road marking robot control system based on visual detection according to the present invention. Detailed Implementation
[0018] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0019] In one or more embodiments, such as Figure 1 As shown, a visual detection-based road marking robot control method is disclosed, the method comprising the following: S1: The road surface image, including the planned marking reference area and the already painted road marking area behind the robot, is continuously acquired by the vision detection device installed at the front of the road marking robot; based on the road surface image, the geometric state vector representing the planned marking reference and the spraying state vector representing the actual construction quality of the painted road marking are extracted respectively, and the geometric state vector and the spraying state vector are normalized and then fused to generate construction state information; Specifically, this step transforms the visual inspection results acquired by the road marking robot at the construction site into construction state information that can be directly used for subsequent control and constraint calculations. During road marking construction, control decisions are simultaneously influenced by both the geometric state of "where the marking should be planned" and the apparent state of the marking, "what the actual marking looks like." Therefore, a method is needed to simultaneously represent both states in the same representation. To this end, a visual inspection device installed at the front of the robot, facing the road surface, continuously acquires images of the road surface at the construction site. The field of view covers both the planned marking reference area and the marking areas already marked by the robot. The images are acquired in consecutive frames to reflect the robot's real-time construction state during movement and marking, providing the initial perceptual basis for subsequent calculations.
[0020] In the processing, the first step is to analyze the areas in the image directly related to road marking construction, focusing on the specific task of road marking construction. This extracts the state elements of the target road marking reference and the already painted road markings. For the target road marking reference, by analyzing its positional and directional changes in continuous images, a geometric state vector is calculated to describe its lateral deviation trend, directional change trend, and local curvature change trend. For painted road markings, the width distribution, continuous segment features, and boundary stability of the markings in the image are analyzed to obtain a painting state vector describing the consistency of the painted appearance. In engineering implementation, and The features in each dimension are derived from directly computable geometric or statistical quantities in the image, and their numerical semantics all point to the current construction state. Since different features may have different scales in the original calculation process, to ensure that the geometric state and the spraying state can be combined in the same representation, reference quantity normalization is performed on each original feature dimension. For any one-dimensional original feature... Its normalized form is: ; in, This represents the characteristic of the normalized proportional quantity. This represents the feature value calculated directly from the current frame or a short time window. The reference scale corresponding to this feature is derived from construction task configuration or equipment calibration parameters, such as the expected marking width, allowable lateral deviation threshold, or scale parameters of the construction-related area. Through this processing, and Each component is converted into a proportional form, thus enabling it to be combined within the same mathematical space.
[0021] After completing the above processing, the normalized geometric state vector is... With the spraying state vector Integration to form construction status information The fusion method employs linear weighting, an idea derived from the classic approach of weighted combination of multi-source features. It balances the influence of different states on the final construction state through weight coefficients. The specific expression is as follows: ; in, This indicates the construction status information, which comprehensively reflects the overall status of the road marking robot at the current construction moment; This represents the geometric state vector calculated from the relative geometric relationship between the target road marking reference and the already painted markings; This represents the spraying state vector calculated from the appearance consistency characteristics of the sprayed markings. This represents the weighting coefficient, the value of which is determined by the construction task configuration and is used to reflect the importance of geometric state and spraying state in different construction scenarios. Through this fusion method, construction state information... The same scale space simultaneously contains information on both geometric deviations and coating appearance, facilitating direct use in subsequent steps. A set of calculation examples illustrates this, given the task configuration... And the geometric state vector Spraying state vector Substituting into the above formula, we get... and The construction status information is obtained by adding them together. The results, presented in a uniform scale, characterize the current construction status and can be directly used as input for subsequent steps to further define constraints related to road marking quality.
[0022] S2: Based on the construction status information of the current sampling period and the construction status information of the previous sampling period, calculate the validity weight vector by dimension, and combine it with the spraying risk scalar to reweight the construction status information to obtain the valid status description. Specifically, this step will output the construction status information from step one. Transform into an effective state description that can be used for construction quality constraint calculations The common state changes at road marking construction sites have two main characteristics: one is the instantaneous jump in local state components caused by factors such as lighting, reflection, and the superposition of old lines; the other is the accumulation of apparent risks in the painting process caused by spraying start / stop, cornering, acceleration / deceleration. The construction of an effective state description revolves around these two characteristics: first, the state differences between adjacent moments are transformed into "dimensional suppression intensity"; then, the painting risk is transformed into "overall convergence intensity"; finally, weights are applied to... Perform dimensional reorganization to obtain This allows subsequent constraint calculations to focus on more stable and construction-relevant state information. The input for this step is the construction state information output from step one in the current sampling period. It also reads the construction status information output and retained by step one from the controller buffer in the previous sampling period. This is used to characterize the state change amplitude between adjacent sampling periods; the controller or edge computing unit generates the current... Then, it is written to the buffer to update the data used in the next sampling period. The construction task configuration is given two scalar parameters. and It is used to adjust the "mutation suppression strength" and "risk convergence strength", both of which remain fixed within the same construction task cycle.
[0023] To ensure that subsequent constraint calculations remain mathematically feasible, The components in each dimension continue the proportional semantics of step one (state components within the same scale system), thus allowing direct use in dimensional multiplication and linear combination. The construction of the weights employs the classic idea of "exponentially decaying weighting," the original form of which is commonly found in exponential windows in signal processing and exponentially decaying gating in control: for a non-negative amplitude... ,use Map it to The attenuation factor of the interval satisfies the principle that "the larger the amplitude, the smaller the factor." This application makes two derivations and modifications to this original idea for road marking scenarios: First, the amplitude quantity is expanded from a single scalar to a state difference across dimensions. First, each state component in each dimension acquires an independent suppression strength; second, a spraying risk scalar is introduced. As a holistic convergence term, when the apparent risk of spraying increases, the weights of all dimensions converge simultaneously, reflecting the engineering logic of "prioritizing spraying risk" in irreversible road marking construction. Spraying risk scalar. The acquisition adopts a directly implementable aggregation method: from The maximum value or high quantile of the components contributed by the coating state is obtained. For example, the maximum value of the three components—"width deviation ratio," "continuity gap ratio," and "boundary instability ratio"—is taken as the [value]. Thus Always align with current coating appearance risks.
[0024] Weight by dimension The calculation is as follows (this formula is derived from the original form of exponential decay gating and superimposed with the overall convergence term of spraying risk; the trimming operator is used to limit the result to a preset range, which is convenient for engineering tuning and ensures numerical stability): ; in, To and Efficiency weight vectors of the same dimension; This is the construction status information (proportional vector) output from step one. The construction status information (proportional vector) is cached at the previous sampling time. The absolute difference by dimension (proportional vector); The mutation suppression coefficient is given by the construction task configuration and is used to adjust the sensitivity of exponential decay to the magnitude of difference. As a scalar measure of spraying risk, by The spraying-related components are obtained by aggregation, while maintaining the semantics of the proportional quantities; This is the risk convergence coefficient, given by the construction task configuration, used to adjust the intensity of the impact of spraying risk on the overall weight convergence; To prune the operator, the weights of each dimension are restricted to the endpoints of the interval. Because... , , Both are proportional semantics, and the numerical scaling system in which the independent variable and multiplicative convergence term of the exponential mapping are consistent, thus making... and Multiplication by dimension has a clear numerical meaning.
[0025] In obtaining Then, apply it to the construction status information. The above yields a valid state description. This form originates from the classic dimensional reweighting (commonly used in iterative reweighting, filtering gating, and feature selection for robust estimation). This application further couples the generation of weights with the risk of road marking and spraying, enabling the reweighting results to directly serve subsequent construction quality constraint calculations. ; in, This is a description of the valid state; The weight vector is calculated using the above formula; This is construction status information; This is an element-wise multiplication operation. and Consistent dimensions facilitate direct use in the next step. The input is used to construct construction quality constraints, and the overall state under abrupt changes and high-risk conditions is numerically converged. This step outputs a valid state description. This serves as the input for calculating the construction quality constraints in the next step; to ensure the continuity of the state sequence, the current construction state information is updated and cached for use in the next sampling time. .
[0026] S3: Based on the effective state description, a construction completion constraint reflecting the current construction completion quality is constructed by the intersection of the weighted square term and the spraying-related component; Specifically, the road marking construction completion constraint is described from the effective state description in step two. The deduction is based on the classic quadratic form concept in mathematics and control engineering for measuring the energy of multidimensional state deviations: weighting each state component according to its sensitivity and then performing squared convergence, so that components with larger deviations contribute more significantly to the overall constraint strength. In the road marking scenario, geometric deviations and apparent deterioration of the paint finish often jointly determine whether the final markings will face rework risks. Paint-related risks exhibit the engineering characteristic of "local deterioration followed by diffusion." For example, when uneven line width and continuity gaps rise simultaneously over a short distance, it often indicates that the paint system is in an unstable condition and may quickly evolve into visually identifiable substandard line segments. Based on this scenario characteristic, a second-order cross term is introduced as an extension term to the original framework of quadratic energy measurement: the pairwise product of paint-related components characterizes "risk clustering." When both types of paint risks increase simultaneously, the cross term will generate additional gain to the constraint strength, thereby transmitting control convergence pressure to subsequent control steps earlier. The extension is that the cross term only applies to the set of spray-related components, and its gain is controlled by the task configuration parameters to align it with the construction quality sensitivity points of road marking.
[0027] Construction completion constraints It is calculated by the following formula: ; in, Indicates the constraint quantity for construction completion; Indicates valid state description The Each component originates from the output of step two; express The number of dimensions; Indicates the first The sensitivity weight coefficients of each component are derived from the construction task configuration file or controller parameter table. During configuration, different weights are assigned to the geometric deviation-related components and the spraying appearance-related components to reflect the construction focus. This represents the penalty coefficient for spraying risk accumulation, derived from the construction task configuration, and used to adjust the impact of cross-items. The intensity of the impact; This represents a set of paired indices representing the spraying-related components, formed by first... The component index is labeled with a set of spray-related component indexes (e.g., those component indexes corresponding to spray appearance features such as "width deviation", "continuity gap", and "boundary instability"). Then, the index set is paired up to obtain... The two parts of this formula have a clear derivation relationship: the first part follows the classical quadratic form of energy convergence, ensuring that an increase in a single risk component can be achieved. The first part is amplified and reflected; the second part originates from the construction of second-order cross terms, used to characterize the aggregation effect when the spraying risk components increase simultaneously, thereby... It is more sensitive to the more dangerous "multi-risk concurrency" state in road marking scenarios. Because... The components maintain a consistent proportional semantic in step two, and the combination of the square term and the product term maintains a consistent numerical semantic. The constraint strength under the same proportional system can be directly used for the generation of subsequent control commands.
[0028] Valid state description ,in Corresponding geometric deviation components, and Corresponding components related to the sprayed coating appearance. Construction task configuration given. , , , and given Spraying paired index sets First, calculate the weighted squared convergence term: , , The sum of the three is: Next, calculate the spraying risk clustering term: multiplied by get Adding the two together gives During engineering debugging, data can be recorded at each sampling time. The numerical sequence is used as a debugging record to verify whether the cross term amplifies the constraint strength as expected when the spraying-related components show a synchronous increase. This provides a verifiable basis for subsequent control steps to trigger convergence actions earlier during the risk accumulation phase. The output is the construction completion constraint. This serves as the direct input for generating the road marking robot's construction control commands in the next step.
[0029] S4: Generate control modulation coefficients based on the construction completion constraints, and apply the control modulation coefficients to the basic construction instruction set to form the final construction control instructions, so as to synchronously adjust the travel speed, steering response coefficient and spraying execution rhythm of the road marking robot; Specifically, the road marking robot receives the construction completion constraint output in step three during the construction process. Based on this, construction control commands can be generated that can directly drive the travel and spraying actuators. This step employs the "constraint-driven amplitude modulation" approach, its mathematical starting point derived from the exponential decay mapping in classical control engineering and signal processing: mapping a non-negative constraint quantity to... The modulation factor within the interval smoothly decreases as the constraint increases, thus transforming the engineering intuition that "increased risk requires convergent construction intensity" into a calculable modulation relationship. The modification of this index mapping in this application lies in the modulation object and modulation semantics: the modulation object is the basic instruction set for road marking construction (including driving and spraying), and the modulation semantics revolve around completion constraints. The construction intensity is converged as a whole, ensuring that travel and spraying change synchronously under the same modulation factor, reducing line width instability or trajectory jitter caused by adjusting a single execution amount individually. (Base construction instructions) Given by the construction task configuration and the robot's underlying controller parameter table, these parameters typically include recommended travel speed settings, steering response coefficient settings, and spraying execution rhythm or valve opening settings. These settings are stored in the controller as structured data items and can be directly read to generate real-time control commands.
[0030] Completion Constraints To control the modulation coefficient The mapping adopts an exponential decay form: ; in, Indicates the control modulation coefficient; This indicates the construction completion constraint, which is calculated in step three based on the effective state description. This represents the modulation sensitivity coefficient, derived from the construction task configuration, used to adjust the influence of the constraint on the modulation factor attenuation rate. The derivation logic of this formula starts from the original form of exponential attenuation gating, treating the constraint as the independent variable of the exponential function, so that... When it increases Monotonically decreasing and remaining continuous, this avoids abrupt command changes due to segmented thresholds in engineering control. Because... It refers to the constraint strength formed under the same proportional semantic system. As a configuration factor, it works in conjunction with it to Having a consistent numerical scale, the output of the exponential function falls within... The range is convenient to use directly as an instruction scaling factor.
[0031] get Then, it is applied to the set of basic construction instructions. Forming final construction control instructions This proportional modulation method originates from classic gain modulation and soft-constraint execution strategies: maintaining the relative proportions between various execution quantities, and continuously scaling the overall amplitude only to avoid introducing new construction instabilities by unilaterally changing a single execution quantity. The command modulation relationship is as follows: ; in, This represents the final set of construction control instructions issued to the road marking robot's execution mechanism; This represents the set of basic construction instructions, derived from the construction task configuration and the underlying controller parameter table. Indicates completion constraint The control modulation coefficients obtained through mapping. The logical relationship between the two equations is: first, the constraint quantity is determined by the first equation. Convert to continuous modulation coefficients Then, using the second formula right Get by scaling the whole This allows the completion constraints to be directly converted into executable movement and spraying control commands. The data is sent to the travel control unit and the spraying execution unit via the controller communication bus or real-time control interface. The travel and steering settings are converted into control values for the drive motor, and the spraying execution unit will... The spraying-related settings are converted into valve opening, pumping rhythm, or nozzle execution rhythm to achieve actual line spraying on the road surface.
[0032] Let the constraint on the completion degree of the output in step three be: The construction task is configured with a given modulation sensitivity coefficient. Then, it is obtained by exponential mapping. Set a set of foundation construction instructions It contains three key setting components, denoted as The first component corresponds to the travel speed setpoint, the second component corresponds to the steering response coefficient setpoint, and the third component corresponds to the spraying execution rhythm setpoint, which is obtained from the command modulation. During engineering debugging, data can be recorded in each sampling period. The numerical sequence is used as a debugging record to verify the current situation. When rising Whether it declines according to an exponential pattern, and Is it correct? Synchronous scaling is performed while maintaining the relative proportions between components to verify whether the constraint-driven construction intensity convergence meets the stability requirements of road marking construction. The final output is the construction control command. The instruction, under the action of the actuator, completes the actual movement and spraying of the road surface.
[0033] In one or more embodiments, such as Figure 2 As shown, a vision-based road marking robot control system is disclosed, the system comprising: A visual inspection device is installed at the front of the road marking robot to continuously acquire images of the road surface, including the planned marking reference area and the already painted road marking area behind the robot. The status information processing module is used to extract geometric status vectors and spraying status vectors based on the road surface image, and then fuse the geometric status vectors and spraying status vectors after normalization to generate construction status information. The effective state description generation module is used to calculate the dimension-based effectiveness weight vector based on the construction state information of the current sampling period and the construction state information of the previous sampling period, and to reweight the construction state information in combination with the spraying risk scalar to obtain the effective state description. The construction completion constraint generation module is used to construct a construction completion constraint reflecting the current construction completion quality based on the effective state description by using the intersection of the weighted square term and the spraying-related component. The control command generation module is used to generate control modulation coefficients based on the construction completion constraints, and apply the control modulation coefficients to the basic construction command set to form the final construction control command, so as to synchronously adjust the travel speed, steering response coefficient and spraying execution rhythm of the road marking robot.
[0034] It is worth noting that the specific workflow of the road marking robot control system for visual detection provided in this embodiment of the invention is the same as that of the road marking robot control method for visual detection described in the above embodiment, and will not be repeated here.
[0035] This invention also provides a vision-based road marking robot control device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps described in the above-described vision-based road marking robot control method embodiment, for example... Figure 1 The steps S1 to S4 described above; or, when the processor executes the computer program, it implements the functions of each module in the above system embodiments.
[0036] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the vision-based road marking robot control device.
[0037] The vision-based road marking robot control device can be a desktop computer, laptop, handheld computer, or cloud server, among other computing devices. This device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the vision-based road marking robot control device may also include input / output devices, network access devices, and buses.
[0038] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the vision-based road marking robot control device, connecting all parts of the device via various interfaces and lines.
[0039] The memory can be used to store the computer program and / or modules. The processor implements various functions of the vision-based road marking robot control device by running or executing the computer program and / or modules stored in the memory, and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the operation of the air conditioning controller, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.
[0040] If the integrated module of the visual inspection road marking robot control device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0041] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0042] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A control method for a road marking robot based on visual detection, characterized in that, The method includes: The road surface images, including the planned marking reference area and the already painted road marking area, are continuously collected by the vision detection device installed at the front of the road marking robot. Based on the road surface image, the geometric state vector representing the reference of the planned marking and the spraying state vector representing the actual construction quality of the painted road markings are extracted respectively. The geometric state vector and the spraying state vector are then normalized and fused to generate construction state information. Based on the construction status information of the current sampling period and the construction status information of the previous sampling period, the validity weight vector by dimension is calculated, and the construction status information is reweighted in combination with the spraying risk scalar to obtain the valid status description. Based on the effective state description, a construction completion constraint reflecting the current construction completion quality is constructed by the intersection of the weighted square term and the spraying-related component. Based on the construction completion constraints, control modulation coefficients are generated and applied to the basic construction instruction set to form the final construction control instructions, so as to synchronously adjust the travel speed, steering response coefficient and spraying execution rhythm of the road marking robot.
2. The visual detection-based road marking robot control method according to claim 1, characterized in that, The area where the road markings have been painted is located behind the painting mechanism of the road marking robot, and the painting state vector is generated based on the actual width, continuity and boundary stability of the painted road markings.
3. The visual detection-based road marking robot control method according to claim 1, characterized in that, The geometric state vector includes a lateral deviation ratio, a direction change ratio, and a local curvature change ratio, while the spraying state vector includes a width deviation ratio, a continuity gap ratio, and a boundary instability ratio.
4. The visual detection-based road marking robot control method according to claim 1, characterized in that, The normalization process scales the original feature values using a reference scale related to the desired datum width or the allowable lateral deviation threshold.
5. The visual detection-based road marking robot control method according to claim 1, characterized in that, The spraying risk scalar is the maximum value of each component in the spraying state vector.
6. The visual detection-based road marking robot control method according to claim 1, characterized in that, Each dimension of the validity weight vector is obtained by exponentially decaying the absolute difference between the construction status information of the corresponding dimension of the current sampling period and the previous sampling period, and further multiplied by the overall convergence factor determined by the spraying risk scalar.
7. The visual detection-based road marking robot control method according to claim 1, characterized in that, In the construction of the construction completion constraint, the cross term is the sum of the products of all spraying-related components in the effective state description after pairing them.
8. The visual detection-based road marking robot control method according to claim 1, characterized in that, The control modulation coefficient is obtained by inputting the construction completion constraint into an exponential decay function, and the value of the control modulation coefficient is between 0 and 1.
9. The visual detection-based road marking robot control method according to claim 1, characterized in that, The final construction control command reduces the travel speed of the road marking robot, simultaneously decreases the steering response coefficient, and extends the single action duration of the spraying execution rhythm, so as to maintain the continuity of the marking and the stability of the boundary when the construction completion constraint indicates that there is a risk of spraying quality.
10. A road marking robot control system based on visual detection, characterized in that, The system includes: A visual inspection device is installed at the front of the road marking robot to continuously acquire images of the road surface, including the planned marking reference area and the already painted road marking area behind the robot. The status information processing module is used to extract geometric status vectors and spraying status vectors based on the road surface image, and then fuse the geometric status vectors and spraying status vectors after normalization to generate construction status information. The effective state description generation module is used to calculate the dimension-based effectiveness weight vector based on the construction state information of the current sampling period and the construction state information of the previous sampling period, and to reweight the construction state information in combination with the spraying risk scalar to obtain the effective state description. The construction completion constraint generation module is used to construct a construction completion constraint reflecting the current construction completion quality based on the effective state description by using the intersection of the weighted square term and the spraying-related component. The control command generation module is used to generate control modulation coefficients based on the construction completion constraints, and apply the control modulation coefficients to the basic construction command set to form the final construction control command, so as to synchronously adjust the travel speed, steering response coefficient and spraying execution rhythm of the road marking robot.