Real-time monitoring and evaluation method and system for road marking state based on video recognition

Abstract

The application discloses a road marking line state real-time monitoring and evaluation method and system based on video recognition. The method comprises the following steps: acquiring road video and vehicle driving information; pre-processing the image to obtain a pre-processing result; identifying the pre-processing result and classifying various road markings, and extracting the corresponding position, direction and shape to obtain key information; assigning a tracking ID to the road marking line, and combining the vehicle driving information to perform cross-frame tracking and spatial-time positioning to determine the continuous image sequence of the same road marking line; performing multi-dimensional health analysis on the road marking line to obtain a multi-dimensional health analysis result, wherein the multi-dimensional health analysis comprises reflection performance evaluation, structural integrity evaluation, color fading evaluation and geometric shape evaluation; calculating a marking line health index, and dividing the road marking line into different health grades. The method can improve the efficiency, objectivity, stability and comprehensiveness of the road marking line monitoring system.

Description

Technical Field

[0001] This invention relates to computer vision, and more specifically to a method and system for real-time monitoring and evaluation of road marking status based on video recognition. Background Technology

[0002] Road markings are an essential component of traffic organization, playing a crucial role in guiding lane travel, displaying traffic rules, and providing traffic safety warnings. The integrity and visibility of these markings directly impact the safety and efficiency of road traffic. However, over long-term use, road markings undergo varying degrees of performance degradation and structural damage due to external factors such as vehicle traffic, climate change (including temperature variations and ultraviolet radiation), rainwater erosion, and construction activities.

[0003] Specifically, these problems manifest themselves in the following ways: Over time, especially at night or in low-light conditions, the reflective material on the road markings wears down, reducing their reflectivity. This not only affects the driver's visibility but also reduces the accuracy of the autonomous driving system's recognition. The edges of the road markings gradually become blurred, and may even show cracks, gaps, or partial detachment, making lane boundaries less distinct and increasing the risk of traffic accidents. The color of the road markings fades over time and may be covered by mud, water, or oil, reducing visual contrast and affecting the visibility of the markings. Improper construction, road subsidence, or high-volume traffic pressure can cause slight displacement, bending, or abnormal shape of the road markings, further misleading drivers.

[0004] Current road marking inspection work mostly relies on manual inspection or single-frame image detection methods. This method is both time-consuming and labor-intensive, and it is difficult to achieve continuous monitoring and quantitative assessment of the health status of road markings.

[0005] Therefore, it is necessary to design a new method to solve the problems of low inspection efficiency, lack of unified objective evaluation standards, unstable recognition algorithms in complex environments, and inability to comprehensively evaluate the degradation of road markings in existing technologies, so as to improve the efficiency, objectivity, stability and comprehensiveness of road marking monitoring systems. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for real-time monitoring and evaluation of road marking status based on video recognition.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for real-time monitoring and evaluation of road marking status based on video recognition, comprising:

[0008] Acquire road video and vehicle driving information;

[0009] The images in the road video are preprocessed to obtain the preprocessing result;

[0010] The preprocessed results are identified and classified into various types of road markings, and their corresponding positions, directions and shapes are extracted to obtain key information;

[0011] The road markings are assigned tracking IDs, and cross-frame tracking and spatial-temporal positioning are performed in combination with vehicle driving information to determine the continuous image sequence of the same road markings.

[0012] Based on a continuous image sequence of the same road marking, a multidimensional health analysis is performed on the road marking to obtain the multidimensional health analysis results. The multidimensional health analysis includes reflectivity assessment, structural integrity assessment, color fading assessment, and geometric shape assessment.

[0013] Based on the results of the multidimensional health analysis, a road marking health index is calculated, and the road markings are divided into different health levels according to the road marking health index.

[0014] A further technical solution is as follows: Based on a continuous image sequence of the same road marking, a multidimensional health analysis is performed on the road marking to obtain the multidimensional health analysis results, including:

[0015] Based on a continuous image sequence of the same road markings, the retroreflection coefficient measurement is simulated using a local contrast algorithm to calculate the gray value difference between the marking area and the background road surface in order to obtain the reflectivity evaluation results.

[0016] The physical damage of the markings is quantified using pixel occupancy analysis, skeleton continuity detection, and edge roughness calculation methods to obtain structural integrity assessment results;

[0017] The image is converted to CIELAB or HSV color space, the color difference is calculated, and the aging trend analysis is used to determine whether the road markings have faded due to pollution or ultraviolet radiation, so as to obtain the color decay assessment result.

[0018] Curve fitting and residual analysis, lateral deviation calculation and logical consistency verification are used to detect the geometric distortion of the markings in order to obtain the geometric shape evaluation results;

[0019] The results of the reflectivity assessment, structural integrity assessment, color fading assessment, and geometric morphology assessment are combined to form a multidimensional health analysis result.

[0020] The further technical solution is as follows: Based on the continuous image sequence of the same road marking, a local contrast algorithm is used to simulate the retroreflection coefficient measurement, and the grayscale value difference between the marking area and the background road surface is calculated to obtain the reflectivity evaluation result, including:

[0021] Based on a continuous image sequence of the same road markings, the images are converted to grayscale space, and the average grayscale values ​​of the marking area and the background road surface are extracted respectively.

[0022] Calculate the Weber or Michelson contrast ratio based on the average gray value;

[0023] Based on the Weber or Michelson contrast ratio and a preset illumination grading model, the ambient illumination conditions are identified, and the reflectivity performance evaluation results are determined.

[0024] The further technical solution is as follows: the physical damage of the marking line is quantified using methods such as pixel occupancy analysis, skeleton continuity detection, and edge roughness calculation to obtain the structural integrity assessment result, including:

[0025] Based on the classification results of the road markings, a target mask is generated and compared with the actual binarized image to calculate the pixel occupancy rate.

[0026] When the pixel occupancy rate is lower than the preset standard, the skeleton of the road marking is extracted, the number of breakpoints of the solid line and the length variance of the dashed line are detected, unexpected damage is recorded, and the edge roughness is calculated using the edge detection operator to determine the degree of coating edge peeling and obtain the structural integrity assessment result.

[0027] The further technical solution is as follows: the image is converted to CIELAB or HSV color space, the color difference is calculated, and the aging trend analysis is used to determine whether the road markings have faded due to pollution or ultraviolet radiation, so as to obtain the color decay assessment result, including:

[0028] Based on a continuous image sequence of the same road markings, the images are converted to CIELAB or HSV color space for analysis.

[0029] Calculate the color difference between the current road marking color and the pre-stored standard chromaticity value;

[0030] By analyzing the color difference average change trend across multiple consecutive frames, it is determined whether the road markings have become dirty, covered by tire tracks, or faded due to ultraviolet radiation, thus obtaining a color fading assessment result.

[0031] The further technical solution is as follows: the method of using curve fitting and residual analysis, lateral deviation calculation and logical consistency verification to detect the geometric distortion of the markings in order to obtain the geometric shape evaluation result includes:

[0032] Based on a continuous image sequence of the same road markings, the center points of the lane markings are fitted using a cubic polynomial or spline curve to construct the lane model equation and obtain the fitted curve.

[0033] Calculate the lateral deviation between the actual detected center point of the road markings and the fitted curve;

[0034] When the lateral deviation variance of a certain section of road markings does not meet the requirements, a logical consistency check is performed by combining high-precision maps or historical data to confirm the consistency of road marking type and direction in order to obtain geometric shape evaluation results.

[0035] The further technical solution is as follows: Assigning a tracking ID to the road markings and combining it with vehicle driving information for cross-frame tracking and spatial-temporal positioning to determine a continuous image sequence of the same road markings includes:

[0036] The road markings are assigned tracking IDs, and combined with vehicle GPS and IMU data in the vehicle driving information, the spatial location of the road markings is synchronized with the time series to map each road marking to the actual road coordinates, forming spatiotemporal correlation data, and determining the continuous image sequence of the same road marking.

[0037] The further technical solution is as follows: the preprocessing results are identified and classified into various types of road markings, and the corresponding positions, directions, and shapes are extracted to obtain key information, including:

[0038] The preprocessing results are input into a deep convolutional neural network model to identify and classify various road markings, so as to obtain the position, category confidence score and semantic segmentation mask of each marking, and extract the contour, position and orientation;

[0039] The road markings are classified according to their shape, color, and width characteristics, and the edges are finely segmented to determine their location and shape.

[0040] Automatically identify and classify all road markings, and convert them into a structured data format that includes marking attributes and their health status to obtain key information.

[0041] This invention also provides a real-time monitoring and evaluation system for road marking status based on video recognition, including:

[0042] The acquisition unit is used to acquire road video and vehicle driving information;

[0043] A preprocessing unit is used to preprocess the images in the road video to obtain a preprocessing result;

[0044] The detection and classification unit is used to identify and classify various road markings based on the preprocessing results, and extract the corresponding location, direction and shape to obtain key information;

[0045] The tracking unit is used to assign a tracking ID to the road markings and combine vehicle driving information to perform cross-frame tracking and spatial-temporal positioning in order to determine the continuous image sequence of the same road markings.

[0046] A multidimensional analysis unit is used to perform multidimensional health analysis on the road markings based on a continuous image sequence of the same road markings to obtain multidimensional health analysis results. The multidimensional health analysis includes reflectivity assessment, structural integrity assessment, color fading assessment, and geometric shape assessment.

[0047] The grading unit is used to calculate the road marking health index based on the multidimensional health analysis results, and to divide the road markings into different health levels according to the road marking health index.

[0048] Its further technical solution is as follows: the multidimensional analysis unit includes:

[0049] The reflectivity performance evaluation subunit is used to simulate retroreflection coefficient measurement based on a continuous image sequence of the same road marking, calculate the difference in gray values ​​between the marking area and the background road surface, and obtain the reflectivity performance evaluation results.

[0050] The structural integrity assessment subunit is used to quantify the physical damage of road markings based on a continuous image sequence of the same road markings using pixel occupancy analysis, skeleton continuity detection, and edge roughness calculation methods, so as to obtain the structural integrity assessment results.

[0051] The color fading assessment subunit is used to convert the image to the CIELAB or HSV color space, calculate the color difference, and determine whether the road markings have faded due to pollution or ultraviolet radiation through aging trend analysis, so as to obtain the color fading assessment result.

[0052] The geometric shape evaluation subunit is used to detect the geometric distortion of the markings by means of curve fitting and residual analysis, lateral deviation calculation and logical consistency verification, so as to obtain the geometric shape evaluation results.

[0053] The combined subunit is used to combine the reflectivity assessment results, structural integrity assessment results, color fading assessment results, and geometric shape assessment results to form a multidimensional health analysis result.

[0054] The beneficial effects of this invention compared to existing technologies are as follows: This invention acquires road video and vehicle driving information, preprocesses, classifies, and extracts key information from the video images, and uses cross-frame tracking and spatiotemporal positioning technologies to assign tracking IDs to each road marking and generate continuous image sequences. Based on these sequences, it performs multi-dimensional health analysis, including reflectivity, structural integrity, color decay, and geometric shape, calculates the road marking health index, and classifies health levels. This effectively solves the problems of low inspection efficiency, lack of unified objective evaluation standards, unstable recognition algorithms in complex environments, and inability to comprehensively evaluate the decay status of road markings in existing technologies. It achieves improved efficiency, enhanced objectivity of evaluation, improved algorithm stability, and a comprehensive assessment of the health status of road markings in the road marking monitoring system.

[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating the real-time monitoring and evaluation method for road marking status based on video recognition provided in an embodiment of the present invention.

[0058] Figure 2 A schematic diagram of line tracking and spatiotemporal correlation provided in an embodiment of the present invention. Figure 1 ;

[0059] Figure 3 A schematic diagram of line tracking and spatiotemporal correlation provided in an embodiment of the present invention. Figure 2 ;

[0060] Figure 4 A schematic diagram illustrating multi-dimensional health analysis provided in an embodiment of the present invention;

[0061] Figure 5 The health index calculation and trend analysis chart provided in this embodiment of the invention;

[0062] Figure 6 A schematic block diagram of a real-time monitoring and evaluation system for road marking status based on video recognition provided in an embodiment of the present invention;

[0063] Figure 7 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0066] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0067] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0068] Please see Figure 1 , Figure 1 This is a flowchart illustrating the real-time monitoring and evaluation method for road marking status based on video recognition provided in this embodiment of the invention. This method, applied to a server, acquires road video and vehicle driving information, and performs preprocessing, identification, classification, and feature extraction on the video images to achieve automatic tracking and cross-frame tracking of road markings. The method utilizes multi-dimensional health analysis technology, including assessments of reflectivity, structural integrity, color decay, and geometric morphology, to comprehensively detect and quantify the decay status of road markings, thereby calculating a health index and classifying health levels. This method effectively solves the problems of low inspection efficiency, lack of unified objective evaluation standards, unstable recognition algorithms in complex environments, and the inability to comprehensively evaluate the decay status of road markings in existing technologies. By combining deep learning models and precise spatial-temporal positioning technology, it not only improves the efficiency and objectivity of the road marking monitoring system but also enhances the stability of the recognition algorithm in complex environments and enables a comprehensive and detailed evaluation of the road marking status, thereby improving the reliability and practicality of the entire monitoring system.

[0069] Figure 1This is a flowchart illustrating the real-time monitoring and evaluation method for road marking status based on video recognition provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S160.

[0070] S110, obtain road video and vehicle driving information.

[0071] In this embodiment, road video refers to a sequence of road surface images captured in real time by a camera mounted on a vehicle. This video data includes not only images of road markings but also other relevant elements on the road, such as lane boundaries and traffic signs. To ensure data accuracy and reliability, it is preferable to use 1080P (1920x1080 pixels) or higher resolution (such as 4K) video streams for acquisition. Furthermore, the camera should have a wide field of view and strong low-light performance to ensure stable operation under various lighting and environmental conditions. The camera is typically mounted at the front of the vehicle to ensure coverage of lane markings on both sides and to capture sufficient information for subsequent analysis.

[0072] Vehicle driving information refers to the relevant vehicle operating parameters recorded by the system simultaneously while acquiring the aforementioned road video. This mainly includes, but is not limited to, the following aspects:

[0073] Vehicle speed: This reflects the speed at which the vehicle moves during the shooting process, which is crucial for understanding the relative position of the lane markings and the vehicle.

[0074] GPS positioning information: provides precise location coordinates, allowing each video segment to be associated with a specific geographical location, which helps to create a spatial distribution map of the markings and achieve cross-frame tracking.

[0075] IMU (Inertial Measurement Unit) data includes information such as acceleration and angular velocity, which helps correct image jitter or shift caused by vehicle movement and improves the accuracy of video analysis.

[0076] Timestamp: Adding a precise time stamp to each video frame facilitates time alignment with other data sources (such as GPS tracks), thereby achieving spatiotemporal correlation.

[0077] Data is uploaded to a cloud-based analytics platform in real time via wireless networks (such as 4G / 5G). When the network is unstable, the system caches the data offline and automatically uploads it again when the network recovers, ensuring data integrity. The cloud platform is responsible for storing, managing, and further processing this data, providing support for line marking detection and health assessment.

[0078] By comprehensively utilizing the aforementioned road video and vehicle driving information, the method in this embodiment can automatically complete real-time monitoring and evaluation of road marking status during normal vehicle operation, greatly improving monitoring efficiency and coverage. Simultaneously, by combining deep learning algorithms and multi-dimensional health analysis technology, it achieves an objective quantitative assessment of road marking health status, providing more comprehensive and accurate detection results than traditional methods.

[0079] S120. Preprocess the images in the road video to obtain the preprocessing result.

[0080] In this embodiment, the preprocessing result refers to the optimized image data obtained after performing a series of processing operations on the images in the road video. These processing operations aim to improve the accuracy and efficiency of subsequent steps such as lane marking detection and classification, and health assessment. Specifically, the preprocessing process includes, but is not limited to, the following aspects:

[0081] Denoising: Use appropriate denoising algorithms to remove noise from the image. This step is crucial for improving image quality because noise can interfere with target recognition and feature extraction.

[0082] Contrast Enhancement: A contrast enhancement algorithm is applied to improve the visibility of the markings under different lighting conditions. The grayscale distribution of the image is adjusted to make the marking features more prominent, facilitating subsequent analysis.

[0083] Illumination compensation and color standardization: Illumination compensation techniques reduce the impact of ambient lighting variations on road marking colors. Furthermore, color standardization is necessary, converting the image to a standard color space (such as CIELAB or HSV) to more accurately analyze color fading of road markings. This step helps eliminate interference from factors such as weather and shadows on color judgment.

[0084] Edge detection and image sharpening: Edge detection operators and image sharpening techniques are used to extract and enhance the contour features of the markings. This process significantly improves the clarity of the marking boundaries, providing a high-quality data foundation for subsequent accurate segmentation and location determination.

[0085] Geometric correction: Considering the variations in the camera's shooting angle, geometric correction may be necessary to ensure that the shape and size of the markings accurately reflect the actual situation. This can be achieved through perspective transformation or other geometric transformation methods.

[0086] The image processed through the above steps is the preprocessed result. This not only improves the recognizability of road marking features but also provides more reliable data support for subsequent road marking detection, classification, and health status assessment. Such a preprocessing workflow is of great significance for building an efficient and accurate road marking health monitoring system.

[0087] S130. Identify and classify various road markings based on the preprocessing results, and extract their corresponding positions, directions, and shapes to obtain key information.

[0088] In this embodiment, key information refers to the detailed attributes of road markings extracted after the preprocessed image is identified and classified using a deep convolutional neural network model. These attributes include, but are not limited to, the location, orientation, category confidence score, semantic segmentation mask, shape features, color features, width features, and edge details of each type of marking. This key information provides a data foundation for subsequent multi-dimensional health analysis and helps the system more accurately assess the health status of road markings.

[0089] In one embodiment, step S130 described above may include steps S131 to S133.

[0090] S131. Input the preprocessing results into a deep convolutional neural network model to identify and classify various road markings, so as to obtain the position, category confidence score and semantic segmentation mask of each marking, and extract the contour, position and direction.

[0091] In step S131, the preprocessed image is input into a trained deep convolutional neural network model. This model is designed to recognize and classify various road markings. The output includes:

[0092] Location: The specific coordinates of each type of marking.

[0093] Category confidence score: The probability score of each detected line belonging to a specific category (e.g., solid line, dashed line, arrow, etc.).

[0094] Semantic segmentation mask: Precisely segment each type of line to obtain its specific contour region.

[0095] In addition, this step also extracts the outline, position, and direction information of the markings, which is crucial for understanding the actual shape and layout of the markings.

[0096] S132. Classify the road markings according to their shape, color, and width characteristics, and finely segment the edges to determine their position and shape.

[0097] Next, in step S132, the classification is further refined based on characteristics such as the shape, color, and width of the road markings. This step not only determines the basic type of the markings but also meticulously segments their edges to determine a more precise location and shape. For example:

[0098] Shape characteristics: such as straight lines, curves, or specific patterns (arrows).

[0099] Color characteristics: Variations in standard colors (white, yellow, etc.).

[0100] Width characteristics: The difference between the standard width and the actual width of different markings.

[0101] By analyzing these features, subtle changes and potential damage to the markings can be identified more accurately.

[0102] S133. Automatically identify and classify all road markings, and convert them into a structured data format containing marking attributes and their health status to obtain key information.

[0103] Finally, in step S133, the system automatically identifies and classifies all road markings and converts the relevant information into a structured data format. This structured data includes various attributes of the markings (such as location, direction, and category mentioned above), and also covers preliminary health status information. This is done to facilitate subsequent data processing and analysis, and to support the calculation of health indices.

[0104] In this embodiment, a specially trained deep convolutional neural network object detection model is used to identify and classify road markings. The model receives preprocessed images as input and outputs the bounding box location, class confidence score, and semantic segmentation mask for each identified road marking. This model is trained under supervision on a large dataset of road images labeled with different types of markings (such as solid lines, dashed lines, arrows, etc.).

[0105] First, the system uses this model to locate each road marking in the image and assigns a probability score to each marking belonging to a specific category. This includes not only determining the location of the marking but also identifying its contour, orientation, and other key attributes. For example: Contour: Precisely depicting the edges of the marking. Location: The specific coordinates of the marking in the image. Orientation: The angle or orientation of the marking relative to the direction of travel. This information is crucial for understanding the specific shape of the markings and their actual layout on the road.

[0106] Next, the system further classifies the identified markings based on features such as shape, color, and width. For example, solid lines, dashed lines, separator lines, and guide lines are labeled as different independent categories. Furthermore, to more accurately describe the state of each marking, the system performs detailed segmentation analysis on the marking edges. This process helps the system:

[0107] Precise location: Determine the exact location of the road markings, even in complex or changing road conditions.

[0108] Morphological characteristics: Capture subtle changes in the markings, such as wear or breakage.

[0109] Using this method, the system can not only identify various types of markings, but also generate detailed structured data containing the specific attributes of each marking.

[0110] Ultimately, the system automatically identifies and categorizes all road markings, converting them into a structured data format. This structured data includes all relevant attributes of the markings, such as type, location, direction, and morphological details. This data format provides a solid foundation for subsequent health assessments and monitoring, ensuring the efficiency and accuracy of the entire monitoring process.

[0111] In summary, through these three steps, the system can efficiently and accurately extract key information about road markings from vehicle-mounted video. This information not only provides a comprehensive record of the current state of the markings but also offers solid data support for subsequent health assessments. This method significantly improves the efficiency and accuracy of road marking monitoring, making maintenance work more targeted and timely.

[0112] S140. Assign a tracking ID to the road markings and combine it with vehicle driving information to perform cross-frame tracking and spatial-temporal positioning in order to determine the continuous image sequence of the same road markings.

[0113] In this embodiment, a tracking ID is assigned to the road markings. Combined with vehicle GPS and IMU data in the vehicle driving information, the spatial location of the road markings is synchronized with the time series to map each road marking to the actual road coordinates, forming spatiotemporal correlation data and determining the continuous image sequence of the same road marking.

[0114] Specifically, please refer to Figures 2 to 3 Step S140 focuses on assigning tracking IDs to the identified road markings and combining this with vehicle driving information (such as GPS and IMU data) to achieve cross-frame tracking and precise spatial and temporal positioning. This process aims to ensure that the same road markings in different frames can be accurately associated to form a continuous image sequence, thereby providing a stable data foundation for subsequent health analysis.

[0115] First, after completing the detection and classification of road markings, the system assigns a unique tracking ID to each identified road marking. This ID remains unchanged throughout the monitoring process, ensuring continuous tracking of the same marking regardless of vehicle movement or changes in environmental conditions. This step is crucial because it directly relates to the accurate assessment of subsequent changes in the marking's condition.

[0116] Next, the system will utilize GPS positioning information and inertial measurement unit (IMU) data recorded during vehicle movement to enhance cross-frame tracking performance. The specific steps are as follows:

[0117] Geometric consistency matching: By comparing the geometric features of datum lines (such as the position and shape of bounding boxes) in adjacent frames, the system can determine which datum lines are identical. This method helps to overcome the effects of viewpoint changes caused by vehicle movement.

[0118] Contour similarity analysis: In addition to location information, the system also compares the contour features of the marking edges. Even under different lighting conditions, contour features generally exhibit high stability, providing additional assurance for accurate matching.

[0119] Position prediction: Based on data from the previous few frames, the system can predict the possible location of the marking in the next few frames using a simple motion model. This helps reduce tracking errors caused by changes in vehicle speed.

[0120] To further improve tracking accuracy, the system combines the vehicle's real-time GPS coordinates with acceleration and angular velocity information provided by IMU sensors to achieve synchronous mapping of the spatial position of the markings with the time series. The advantages of this approach include:

[0121] Precise spatiotemporal correlation: The position of the road markings in each frame is not only associated with their two-dimensional coordinates in the image, but also with their three-dimensional geographic coordinates in the actual vehicle trajectory. This multi-dimensional correlation allows the system to more accurately reflect the changes in the road markings' state on the actual road.

[0122] Dynamic adjustment and compensation: When encountering situations such as poor network signal or GPS drift, IMU data can help the system perform temporary position correction to ensure data consistency and reliability.

[0123] Ultimately, by applying the aforementioned technologies, the system can generate continuous image sequences for each road marking. These sequences not only capture the changes in the appearance of the markings at different times and locations but also reflect their potential degradation over time. Such datasets are of great significance for in-depth analysis of the health status of road markings and the development of maintenance plans.

[0124] In summary, step S140 successfully constructed an efficient and reliable road marking tracking mechanism by assigning tracking IDs, combining vehicle driving information for cross-frame tracking, and achieving precise spatial and temporal positioning. This lays a solid foundation for subsequent multi-dimensional health analysis, ensuring the accuracy and practicality of the entire road marking health monitoring system.

[0125] S150. Based on a continuous image sequence of the same road marking, perform a multidimensional health analysis on the road marking to obtain the multidimensional health analysis results. The multidimensional health analysis includes reflectivity assessment, structural integrity assessment, color decay assessment, and geometric shape assessment.

[0126] In this embodiment, the multidimensional health analysis result refers to the comprehensive calculation of a quantitative index reflecting the health status of road markings by evaluating four dimensions: reflectivity, structural integrity, color decay, and geometric shape, based on a continuous image sequence of road markings.

[0127] In one embodiment, step S150 described above may include steps S151 to S155.

[0128] S151. Based on a continuous image sequence of the same road marking, the retroreflection coefficient is simulated using a local contrast algorithm to calculate the gray value difference between the marking area and the background road surface, so as to obtain the reflectivity evaluation result.

[0129] In this embodiment, the reflectivity evaluation result refers to the calculation of the visibility and reflectivity of the road marking by processing consecutive images of the same road marking, thereby determining whether it meets traffic safety requirements. This process not only relies on brightness values, but more importantly, considers the impact of ambient lighting conditions on the visibility of the road marking, providing a more accurate and practical evaluation standard.

[0130] In one embodiment, step S151 described above may include steps S1511 to S1513.

[0131] S1511. Based on a continuous image sequence of the same road marking, convert the image to grayscale space and extract the average grayscale value of the marking area and the background road surface respectively.

[0132] S1512. Calculate the Weber or Michelson contrast ratio based on the average gray value;

[0133] S1513. Based on the Weber or Michelson contrast ratio and the preset illumination grading model, identify the ambient illumination conditions and determine the reflectivity performance evaluation results.

[0134] In this embodiment, the reflectivity evaluation aims to simulate the measurement principle of the retroreflection coefficient. The system does not rely solely on brightness values, but instead employs a local contrast algorithm. Based on a continuous image sequence of the same road marking, the images are converted to grayscale space, and the average grayscale value G of the marking area is extracted respectively. mark The average gray value G of the background road surface road This step aims to provide foundational data for subsequent contrast calculations and ensure consistency under different lighting conditions.

[0135] The Weber or Michelson contrast ratio is calculated based on the average grayscale value. This step utilizes the data obtained in the previous step, using a mathematical formula (e.g., Weber contrast ratio C=(G...)). mark -G road ) / G roadThe specific contrast value is calculated to quantify the visibility of the road markings relative to the background road surface.

[0136] Based on the Weber or Michelson contrast ratio and a preset illumination grading model, the system identifies ambient lighting conditions and determines the reflectivity performance evaluation result. In this step, the system assesses whether the reflectivity of the road markings meets the standards based on the current ambient lighting conditions (such as strong light, cloudy daytime, or nighttime vehicle headlight mode) and the calculated contrast value. For example, in nighttime vehicle headlight mode, the system pays special attention to the brightness attenuation of the road markings in the vehicle headlight illumination area. If the brightness attenuation gradient is too large or the contrast ratio is lower than a set threshold, it may indicate that the reflective material has failed or detached.

[0137] Such detailed division and processing allows for a more accurate assessment of the health of road markings, ensuring driving safety.

[0138] S152. The physical damage of the marking line is quantified by methods such as pixel occupancy analysis, skeleton continuity detection and edge roughness calculation to obtain the structural integrity assessment results.

[0139] In this embodiment, the structural integrity assessment result refers to the process and analysis of continuous images of road markings to quantify their wear level, skeleton integrity, and edge condition, thereby determining whether the overall physical condition of the markings meets safety standards. The assessment results not only help maintenance departments promptly identify areas requiring repair or repainting but also provide data support for future marking design.

[0140] Step S152 aims to quantify the physical damage to the markings using methods such as pixel occupancy analysis, skeleton continuity detection, and edge roughness calculation to obtain structural integrity assessment results. This step is crucial for identifying and evaluating wear, breakage, and other physical damage to the markings during use.

[0141] In one embodiment, step S152 described above may include steps S1521 to S1522.

[0142] S1521. Generate an expected mask based on the classification result of the road markings, and compare it with the actual binarized image to calculate the pixel occupancy rate.

[0143] In this embodiment, a target mask is generated based on the classification result of the road markings, and then compared with the actual binary image to calculate the pixel occupancy rate. This step first generates an ideal marking shape as a reference based on the type of marking (such as solid or dashed lines), and then converts the actual captured marking image into a binary image (i.e., only black and white colors). The pixel occupancy rate is calculated by comparing the overlap between the two. If the occupancy rate is lower than a preset standard value (e.g., 85%), it indicates that the markings have a certain degree of wear or missing parts.

[0144] S1522. When the pixel occupancy rate is lower than the preset standard, the skeleton of the road marking is extracted, the number of breakpoints of the solid line and the length variance of the dashed line are detected, unexpected damage is recorded, and the edge roughness is calculated using the edge detection operator to determine the degree of coating edge peeling and obtain the structural integrity assessment result.

[0145] In this embodiment, when the pixel occupancy rate is lower than a preset standard, the skeleton of the road marking is extracted, the number of breakpoints in solid lines and the length variance of dashed lines are detected, unexpected damage is recorded, and edge roughness is calculated using an edge detection operator to determine the degree of coating edge peeling, thus obtaining a structural integrity assessment result. In this step, the system further examines the skeleton structure of the marking, especially for solid lines, checking for any unwanted breakpoints; for dashed lines, it checks whether the lengths and intervals between segments are consistent. Simultaneously, the system analyzes the smoothness of the marking edges, measuring the jaggedness or blurriness of the edges by calculating the root mean square error between edge pixels and the fitted straight line, thereby determining whether the coating is at risk of peeling.

[0146] These detailed analytical steps collectively constitute a comprehensive assessment of the structural integrity of road markings, ensuring the effectiveness and safety of traffic signs. Such meticulous analysis not only allows for the timely identification and repair of damaged road markings but also provides crucial feedback for future design and construction, ultimately improving road traffic safety.

[0147] S153. Convert the image to CIELAB or HSV color space, calculate the color difference, and determine whether the road markings have faded due to pollution or ultraviolet radiation through aging trend analysis to obtain the color decay assessment result.

[0148] In this embodiment, the color fading assessment result refers to the analysis of a series of consecutive image sequences of the same road markings through a series of steps to determine whether the color fading has occurred due to pollution or ultraviolet radiation.

[0149] In one embodiment, step S153 described above may include steps S1531 to S1533.

[0150] S1531. Based on a continuous image sequence of the same road markings, convert the images to CIELAB or HSV color space for analysis;

[0151] S1532. Calculate the color difference between the current road marking color and the pre-stored standard chromaticity value;

[0152] S1533. By analyzing the color difference average change trend of multiple consecutive frames, determine whether the road markings have become dirty, covered by tire tracks, or faded due to ultraviolet radiation, so as to obtain the color fading assessment result.

[0153] In this embodiment, firstly, based on a continuous image sequence of the same road markings, these images are converted to the CIELAB or HSV color space for analysis. These two color spaces were chosen because they are more resistant to changes in lighting conditions than the RGB color space, thus providing more stable colorimetric analysis.

[0154] Next, the color difference between the current road marking color and the pre-stored standard chromaticity value is calculated. The Euclidean distance formula (for the CIELAB color space) or a corresponding color difference calculation method (for the HSV color space) is used to calculate the color difference value ΔE. This step aims to quantify the degree of difference between the current road marking color and the ideal state. . Indicates differences in brightness. Indicates the differences along the red and green axes; This indicates the difference along the blue and yellow axes.

[0155] Then, the condition of the road markings is determined by observing the trend of the average color difference in multiple consecutive frames of images. This aging trend analysis can help identify the specific causes of marking degradation and obtain color decay assessment results accordingly.

[0156] In summary, the color fading assessment results are professional conclusions regarding the color status and changing trends of road markings, derived from in-depth analysis of road marking image sequences and utilizing methods such as color space conversion, color difference calculation, and aging trend analysis. These results help to facilitate timely measures for the maintenance and repair of road markings, ensuring traffic safety.

[0157] S154. Curve fitting and residual analysis, lateral deviation calculation and logical consistency verification are used to detect the geometric distortion of the markings in order to obtain the geometric shape evaluation results.

[0158] In this embodiment, the geometric shape evaluation result refers to the system's ability to identify and quantify the geometric distortion of road markings through curve fitting and residual analysis, lateral deviation calculation, and logical consistency verification, thereby assessing their consistency and accuracy. In short, the geometric shape evaluation result is an indicator of whether road markings conform to design specifications and their geometric integrity.

[0159] In one embodiment, step S154 described above may include steps S1541 to S1543.

[0160] S1541. Based on a continuous image sequence of the same road markings, the center point of the lane markings is fitted using a cubic polynomial or spline curve to construct the lane model equation and obtain the fitted curve.

[0161] S1542. Calculate the lateral deviation between the actual detected center point of the road markings and the fitted curve;

[0162] S1543. When the lateral deviation variance of a certain section of road marking does not meet the requirements, a logical consistency check is performed in conjunction with a high-precision map or historical data to confirm the consistency of the road marking type and direction in order to obtain the geometric shape evaluation result.

[0163] This step first processes the center points of road markings extracted from the continuous image sequence. By applying cubic polynomial or spline curve fitting techniques, the system can generate a smooth lane model equation, i.e., a fitted curve. This step is fundamental to identifying and quantifying the geometry of the road markings, ensuring that even with slight deformations or irregularities, an ideal reference model can be obtained for subsequent analysis.

[0164] After obtaining the fitted curve, the next step is to calculate the lateral distance deviation between the center point of each road marking actually detected and this fitted curve. This step helps to determine the difference between the actual position of the marking and its ideal position, thus providing a direct basis for assessing whether the marking has been distorted due to poor construction quality or road surface deformation.

[0165] When a significant increase in the lateral deviation variance of road markings is detected in a certain section, it indicates that there may be geometric distortion or other problems in that area. In this case, further logical consistency verification using high-precision maps or historical data is particularly important. This step mainly checks whether the current marking type (e.g., solid line, dashed line, etc.) is reasonable and consistent, and whether the traffic guidance direction is correct. Any illogical findings may indicate drawing errors or incomplete removal of old markings.

[0166] Geometric morphology assessment results refer to the system's ability to accurately identify geometric distortions on the road through a series of analytical methods (including curve fitting and residual analysis, lateral deviation calculation, and logical consistency verification), and to provide corresponding assessment conclusions. This not only helps improve road traffic safety and flow but also enables the early detection and resolution of potential problems, such as unclear road markings and misleading signs, thereby enhancing the overall quality and reliability of road infrastructure. Furthermore, this information is of significant guiding importance for subsequent maintenance work.

[0167] S155. Combine the reflectivity assessment results, structural integrity assessment results, color fading assessment results, and geometric shape assessment results to form a multidimensional health analysis result.

[0168] In this embodiment, by combining the above-mentioned reflective performance evaluation results, structural integrity evaluation results, color fading evaluation results, and geometric shape evaluation results, a result that comprehensively reflects the condition of road markings is obtained.

[0169] like Figure 4 As shown, various detection methods and technologies are used to comprehensively analyze and evaluate road markings from multiple perspectives. The main purpose is to ensure that traffic markings maintain good visibility, structural integrity, and color vibrancy under different environmental conditions, thereby guaranteeing traffic safety.

[0170] Lane markings: For these types of markings, the system pays special attention to reflectivity (i.e., visibility at night or in low light conditions) and potential structural damage (such as cracks, peeling, etc.). This is because lane markings directly affect the driver's choice of driving route, especially at high speeds.

[0171] Pedestrian crossing markings: Unlike lane markings, pedestrian crossing markings are required to have clear colors and be free from contamination. Therefore, the system will focus on assessing the degree of color fading and whether the surface is contaminated with oil, dust, or other substances.

[0172] The system can not only handle each type of pavement marking problem individually, but also identify comprehensive degradation phenomena caused by factors such as increased age, changes in traffic flow, and severe weather. This means it can provide a more comprehensive perspective to understand the overall condition of pavement markings and formulate maintenance plans accordingly.

[0173] The evaluation factors involved include:

[0174] Reflective properties: By measuring the reflectivity of the road marking material, we ensure that it provides sufficient visual guidance for drivers under various lighting conditions.

[0175] Shape and geometric offset: Check whether the actual position of the markings conforms to the design specifications to avoid offsets caused by construction errors or natural settlement.

[0176] Color: Regularly monitor the saturation and brightness of the marking colors to ensure they remain easily identifiable during the day.

[0177] Structural integrity: This includes tests on compressive strength, abrasion resistance, etc., to ensure that the road markings can withstand vehicle traffic for a long time without being damaged.

[0178] In summary, this multi-dimensional health analysis not only considers the physical characteristics of the road markings themselves but also incorporates the specific needs of real-world application scenarios. It provides traffic management departments with a scientific basis, helping to promptly identify and address potential safety hazards and improve road traffic safety. Simultaneously, it lays the foundation for the construction of future intelligent transportation systems, making traffic facility management more intelligent and precise.

[0179] S160. Calculate the road marking health index based on the multidimensional health analysis results, and classify the road markings into different health levels according to the road marking health index.

[0180] In this embodiment, as Figure 5 As shown, after completing the multi-dimensional health analysis of the markings, the system will integrate the evaluation results of various indicators and use a weighted fusion model to calculate a unified marking health index. This health index is a comprehensive score of the markings' health status, which can intuitively reflect the actual usage status of the markings.

[0181] The health index calculation relies on assessment results across multiple dimensions, each with different weights (such as reflectivity, structural integrity, and color fading). The system dynamically adjusts these weights based on actual needs and different road types (such as urban roads and highways). For example, the current frame's overall score is 54.5.

[0182] Reflective properties (reflection intensity, spot stability): 40%, below the threshold;

[0183] Integrity and structural damage (wear, fracture, overlay): 30%, wear is present;

[0184] Color and visual degradation (color difference change, brightness degradation): 20%, color difference increases;

[0185] Geometric shape consistency with road model (shape deviation, deformation): 10%, good shape;

[0186] Each dimension is scored based on the actual situation, typically using a rating range of 0-100, where 0 represents the worst state and 100 represents the healthiest state. For example:

[0187] Reflective performance: The reflective intensity of the markings is measured and compared with the standard value to obtain a reflective performance score. If the reflective intensity of the markings under strong light is 80% of the standard, the reflective performance score is 80.

[0188] Structural damage: The integrity of the markings is assessed based on characteristics such as notch rate, line width stability, and edge sharpness. A score of 90 is given for no obvious breakage; 60 for minor damage; and low for severe damage.

[0189] Color fading: The degree of color fading is calculated by analyzing the color difference change (ΔE) of the markings. If the color change of the markings is small, the score is 85; if the fading is significant, the score is lower (e.g., 50 or lower).

[0190] Geometric shape deviation: The degree of shape deviation is identified by comparing the shape of the road markings with the predetermined geometric model of the road. If the road markings show no obvious deformation, the score is higher (e.g., 90).

[0191] The score for each dimension is multiplied by its corresponding weight to obtain a weighted score for each dimension. Then, all weighted scores are summed to obtain the Health Index (HI). Based on the calculated Health Index, the system divides the baseline into multiple levels. Each level corresponds to a health state, from the healthiest to the most severely damaged. The following are common grading standards:

[0192] Health (HI≥85): All indicators of the road markings are in good condition, with no significant degradation in reflectivity, structural integrity, and color. Suitable for continued use, requiring no immediate maintenance.

[0193] Mild degradation (70≤HI<85): The markings show slight degradation, which may be due to a slight decrease in reflectivity or minor wear on some structures, but they still maintain basic function. Regular monitoring is recommended to plan for future repairs.

[0194] Moderate degradation (50≤HI<70): The road markings show significant degradation, with partial structural damage or severely reduced reflectivity. Repair or replacement is necessary to ensure safety.

[0195] Severe damage (HI<50): The reflectivity of the road markings has severely deteriorated, and there is obvious structural damage, which may affect traffic safety. Immediate repair or remarking is required to ensure road safety.

[0196] By using a weighted and aggregated health index, the health status of road markings can be accurately quantified, and they can be divided into multiple health levels based on different index ranges. This method not only provides road managers with clear decision-making support but also helps optimize marking maintenance and repair plans. Different marking types (such as lane lines, arrow markings, and pedestrian markings) can have their weights dynamically adjusted according to actual needs, allowing the entire system to adapt to different monitoring environments and usage scenarios. This not only improves the efficiency of traffic facility management but also enhances road traffic safety.

[0197] The method in this embodiment achieves automated data acquisition and detection through in-vehicle video, completing data acquisition and analysis during vehicle movement, significantly improving detection efficiency and coverage. By extracting structured color, brightness, and morphological features, an objective and quantitative evaluation index system is constructed, fundamentally reducing subjective bias. Cross-frame feature fusion and lane marking tracking strategies are introduced to enhance the model's temporal stability in complex scenes, thereby improving recognition reliability. Features such as reflectivity, color loss, structural integrity, and geometric changes are fused to form a unified health index, providing a more accurate comprehensive judgment.

[0198] Specifically, vehicle-mounted video acquisition technology enables real-time automatic monitoring of road markings, overcoming the efficiency and coverage limitations of traditional manual inspections. It allows for the real-time acquisition and analysis of marking data during vehicle movement. By combining multiple dimensions of indicators, including reflectivity, color fading, structural damage, and geometric morphology, a comprehensive road marking health assessment model is constructed, providing a more accurate and comprehensive health assessment than traditional methods. The introduction of cross-frame image tracking and temporal data association technology, through geometric consistency matching and contour similarity analysis of multiple frames, overcomes the limitations of single-frame image recognition, improving the accuracy and stability of road marking health assessments. Based on automated data analysis, a health index is calculated, and a road marking health assessment report is automatically generated, significantly reducing manual intervention and improving the efficiency and accuracy of the assessment process.

[0199] Real-time automatic acquisition and analysis of vehicle-mounted video enables coverage of more road areas, continuous monitoring, and significantly improved detection efficiency and coverage. A multi-dimensional comprehensive assessment, including indicators such as reflectivity, color change, and structural integrity, provides a more comprehensive and objective health assessment, avoiding errors and subjectivity inherent in manual judgment. By introducing cross-frame tracking and time-series analysis technology, single-frame data noise is eliminated, enhancing stability and robustness in complex environments. Automated generation of road marking health indices and assessment reports reduces the need for manual operation, shortens the time from detection to decision-making, and provides rapid and accurate road marking health data support, facilitating timely maintenance and management of road facilities.

[0200] The aforementioned real-time monitoring and evaluation method for road marking status based on video recognition acquires road video and vehicle driving information, preprocesses, classifies, and extracts key information from the video images, and uses cross-frame tracking and spatiotemporal positioning technology to assign tracking IDs to each road marking and generate continuous image sequences. Based on these sequences, a multi-dimensional health analysis is performed, including reflectivity, structural integrity, color decay, and geometric shape, to calculate the road marking health index and classify health levels. This effectively solves the problems of low inspection efficiency, lack of unified objective evaluation standards, unstable recognition algorithms in complex environments, and inability to comprehensively evaluate road marking decay in existing technologies. It achieves improved efficiency, enhanced objectivity, improved algorithm stability, and a comprehensive assessment of road marking health status in the road marking monitoring system.

[0201] Figure 6 This is a schematic block diagram of a real-time monitoring and evaluation system 300 for road marking status based on video recognition, provided in an embodiment of the present invention. Figure 6 As shown, corresponding to the above-described real-time monitoring and evaluation method for road marking status based on video recognition, the present invention also provides a real-time monitoring and evaluation system 300 for road marking status based on video recognition. This real-time monitoring and evaluation system 300 for road marking status based on video recognition includes a unit for executing the above-described real-time monitoring and evaluation method for road marking status based on video recognition, and the system can be configured in a server. Specifically, please refer to... Figure 6 The video recognition-based real-time monitoring and evaluation system for road marking status 300 includes an acquisition unit 301, a preprocessing unit 302, a detection and classification unit 303, a tracking unit 304, a multidimensional analysis unit 305, and a grading unit 306.

[0202] The system comprises: an acquisition unit 301 for acquiring road video and vehicle driving information; a preprocessing unit 302 for preprocessing images in the road video to obtain preprocessing results; a detection and classification unit 303 for identifying and classifying various road markings based on the preprocessing results, and extracting their corresponding positions, directions, and shapes to obtain key information; a tracking unit 304 for assigning tracking IDs to the road markings and performing cross-frame tracking and spatial-temporal positioning in conjunction with vehicle driving information to determine a continuous image sequence of the same road marking; a multidimensional analysis unit 305 for performing multidimensional health analysis on the road markings based on the continuous image sequence of the same road markings to obtain multidimensional health analysis results, including reflectivity assessment, structural integrity assessment, color decay assessment, and geometric shape assessment; and a grading unit 306 for calculating a road marking health index based on the multidimensional health analysis results and classifying the road markings into different health levels according to the road marking health index.

[0203] In one embodiment, the multidimensional analysis unit 305 includes:

[0204] The system comprises the following sub-units: a reflectivity evaluation sub-unit, a structural integrity evaluation sub-unit, and a geometric morphology evaluation sub-unit. The reflectivity evaluation sub-unit simulates retroreflection coefficient measurement using a local contrast algorithm based on a continuous image sequence of the same road marking, and calculates the grayscale difference between the marking area and the background road surface to obtain the reflectivity evaluation result. The structural integrity evaluation sub-unit quantifies the physical damage to the marking using methods such as pixel occupancy analysis, skeleton continuity detection, and edge roughness calculation to obtain the structural integrity evaluation result. The color fading evaluation sub-unit converts the image to the CIELAB or HSV color space, calculates the color difference, and determines whether the road marking has faded due to pollution or ultraviolet radiation through aging trend analysis to obtain the color fading evaluation result. The geometric morphology evaluation sub-unit detects the geometric distortion of the marking using curve fitting and residual analysis, lateral deviation calculation, and logical consistency verification to obtain the geometric morphology evaluation result. The combination sub-unit combines the reflectivity evaluation result, structural integrity evaluation result, color fading evaluation result, and geometric morphology evaluation result to form a multidimensional health analysis result.

[0205] In one embodiment, the reflectivity evaluation subunit includes:

[0206] The conversion module is used to convert images to grayscale space based on a continuous image sequence of the same road markings, and extract the average grayscale values ​​of the marking area and the background road surface respectively; the contrast calculation module is used to calculate the Weber or Michelson contrast based on the average grayscale value; the recognition and evaluation module is used to identify the ambient lighting conditions and determine the reflectivity evaluation result based on the Weber or Michelson contrast and a preset illumination grading model.

[0207] In one embodiment, the structural integrity assessment subunit includes:

[0208] The occupancy calculation module is used to generate an expected mask based on the classification result of the road markings and compare it with the actual binarized image to calculate the pixel occupancy. The evaluation calculation module is used to extract the skeleton of the road markings when the pixel occupancy is lower than the preset standard, detect the number of breakpoints of solid lines and the length variance of dashed lines, record unexpected damage, and use the edge detection operator to calculate the edge roughness to determine the degree of coating edge peeling and obtain the structural integrity evaluation result.

[0209] In one embodiment, the color fading assessment subunit includes:

[0210] The conversion analysis module is used to convert the images to CIELAB or HSV color space for analysis based on a continuous image sequence of the same road marking; the color difference calculation module is used to calculate the color difference between the current road marking color and the pre-stored standard chromaticity value; the judgment module is used to determine whether the road marking has become dirty, covered by tire tracks, or faded due to ultraviolet radiation by the trend of the average color difference change of multiple consecutive frames, so as to obtain the color decay assessment result.

[0211] In one embodiment, the geometry evaluation subunit includes:

[0212] The fitting module is used to fit the center point of the lane markings using a cubic polynomial or spline curve based on a continuous image sequence of the same road markings, and to construct the lane model equation to obtain the fitting curve; the deviation calculation module is used to calculate the lateral deviation between the actual detected center point of the road markings and the fitting curve; the verification module is used to perform logical consistency verification by combining high-precision maps or historical data when the lateral deviation variance of a certain section of road markings does not meet the requirements, to confirm the consistency of the road marking type and direction, and to obtain the geometric shape evaluation result.

[0213] In one embodiment, the tracking unit 304 is used to assign a tracking ID to the road markings, and combine the vehicle GPS and IMU data in the vehicle driving information to synchronize the spatial location of the road markings with the time series, so as to map each road marking to the actual road coordinates, form spatiotemporal correlation data, and determine the continuous image sequence of the same road marking.

[0214] In one embodiment, the detection and classification unit 303 includes:

[0215] The extraction subunit is used to input the preprocessed results into a deep convolutional neural network model to identify and classify various road markings, so as to obtain the position, category confidence score and semantic segmentation mask of each marking, and extract the contour, position and orientation; the fine segmentation subunit is used to classify the road markings according to their shape, color and width features, and to finely segment the edges to determine their position and shape; the classification subunit is used to automatically identify and classify all road markings, and convert them into a structured data format containing marking attributes and their health status to obtain key information.

[0216] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned video recognition-based real-time monitoring and evaluation system for road marking status 300 and its various units can be found in the corresponding descriptions in the aforementioned method embodiments. For the sake of convenience and brevity, these details will not be repeated here.

[0217] The aforementioned real-time monitoring and evaluation system 300 for road marking status based on video recognition can be implemented as a computer program, which can, for example... Figure 7 It runs on the computer device shown.

[0218] Please see Figure 7 , Figure 7 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0219] See Figure 7 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0220] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a method for real-time monitoring and evaluation of road marking status based on video recognition.

[0221] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0222] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for real-time monitoring and evaluation of road marking status based on video recognition.

[0223] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0224] The processor 502 is used to run the computer program 5032 stored in the memory to implement all the steps of the video recognition-based real-time monitoring and evaluation method for road marking status.

[0225] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0226] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0227] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform all the steps of the video recognition-based real-time monitoring and evaluation method for road marking status.

[0228] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0229] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0230] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each unit is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0231] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the system of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0232] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0233] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for real-time monitoring and evaluation of road marking status based on video recognition, characterized in that, include: Acquire road video and vehicle driving information; The images in the road video are preprocessed to obtain the preprocessing result; The preprocessed results are identified and classified into various types of road markings, and their corresponding positions, directions and shapes are extracted to obtain key information; The road markings are assigned tracking IDs, and cross-frame tracking and spatial-temporal positioning are performed in combination with vehicle driving information to determine the continuous image sequence of the same road markings. Based on a continuous image sequence of the same road marking, a multidimensional health analysis is performed on the road marking to obtain the multidimensional health analysis results. The multidimensional health analysis includes reflectivity assessment, structural integrity assessment, color fading assessment, and geometric shape assessment. Based on the results of the multidimensional health analysis, a road marking health index is calculated, and the road markings are divided into different health levels according to the road marking health index. The continuous image sequence based on the same road markings is used to perform multidimensional health analysis on the road markings to obtain multidimensional health analysis results, including: Based on a continuous image sequence of the same road markings, the retroreflection coefficient measurement is simulated using a local contrast algorithm to calculate the gray value difference between the marking area and the background road surface in order to obtain the reflectivity evaluation results. The physical damage of the markings is quantified using pixel occupancy analysis, skeleton continuity detection, and edge roughness calculation methods to obtain structural integrity assessment results; The image is converted to CIELAB or HSV color space, the color difference is calculated, and the aging trend analysis is used to determine whether the road markings have faded due to pollution or ultraviolet radiation, so as to obtain the color decay assessment result. Curve fitting and residual analysis, lateral deviation calculation and logical consistency verification are used to detect the geometric distortion of the markings in order to obtain the geometric shape evaluation results; The results of the reflectivity assessment, structural integrity assessment, color fading assessment, and geometric morphology assessment are combined to form a multidimensional health analysis result.

2. The method for real-time monitoring and evaluation of road marking status based on video recognition according to claim 1, characterized in that, The continuous image sequence based on the same road markings uses a local contrast algorithm to simulate retroreflection coefficient measurement and calculates the grayscale difference between the marking area and the background road surface to obtain reflectivity evaluation results, including: Based on a continuous image sequence of the same road markings, the images are converted to grayscale space, and the average grayscale values ​​of the marking area and the background road surface are extracted respectively. Calculate the Weber or Michelson contrast ratio based on the average gray value; Based on the Weber or Michelson contrast ratio and a preset illumination grading model, the ambient illumination conditions are identified, and the reflectivity performance evaluation results are determined.

3. The method for real-time monitoring and evaluation of road marking status based on video recognition according to claim 2, characterized in that, The method of quantifying the physical damage of the marking line using pixel occupancy analysis, skeleton continuity detection, and edge roughness calculation to obtain structural integrity assessment results includes: Based on the classification results of the road markings, a target mask is generated and compared with the actual binarized image to calculate the pixel occupancy rate. When the pixel occupancy rate is lower than the preset standard, the skeleton of the road marking is extracted, the number of breakpoints of the solid line and the length variance of the dashed line are detected, unexpected damage is recorded, and the edge roughness is calculated using the edge detection operator to determine the degree of coating edge peeling and obtain the structural integrity assessment result.

4. The method for real-time monitoring and evaluation of road marking status based on video recognition according to claim 2, characterized in that, The image is converted to the CIELAB or HSV color space, color difference is calculated, and aging trend analysis is used to determine whether the road markings have faded due to pollution or ultraviolet radiation, in order to obtain a color degradation assessment result, including: Based on a continuous image sequence of the same road markings, the images are converted to CIELAB or HSV color space for analysis. Calculate the color difference between the current road marking color and the pre-stored standard chromaticity value; By analyzing the color difference average change trend across multiple consecutive frames, it is determined whether the road markings have become dirty, covered by tire tracks, or faded due to ultraviolet radiation, thus obtaining a color fading assessment result.

5. The method for real-time monitoring and evaluation of road marking status based on video recognition according to claim 2, characterized in that, The method of using curve fitting and residual analysis, lateral deviation calculation, and logical consistency verification to detect the geometric distortion of the markings in order to obtain geometric shape evaluation results includes: Based on a continuous image sequence of the same road markings, the center points of the lane markings are fitted using a cubic polynomial or spline curve to construct the lane model equation and obtain the fitted curve. Calculate the lateral deviation between the actual detected center point of the road markings and the fitted curve; When the lateral deviation variance of a certain section of road markings does not meet the requirements, a logical consistency check is performed by combining high-precision maps or historical data to confirm the consistency of road marking type and direction in order to obtain geometric shape evaluation results.

6. The method for real-time monitoring and evaluation of road marking status based on video recognition according to claim 1, characterized in that, Assigning tracking IDs to the road markings and combining vehicle driving information for cross-frame tracking and spatial-temporal localization to determine consecutive image sequences of the same road marking includes: The road markings are assigned tracking IDs, and combined with vehicle GPS and IMU data in the vehicle driving information, the spatial location of the road markings is synchronized with the time series to map each road marking to the actual road coordinates, forming spatiotemporal correlation data, and determining the continuous image sequence of the same road marking.

7. The method for real-time monitoring and evaluation of road marking status based on video recognition according to claim 1, characterized in that, The preprocessing results are identified and classified into various types of road markings, and their corresponding positions, directions, and shapes are extracted to obtain key information, including: The preprocessing results are input into a deep convolutional neural network model to identify and classify various road markings, so as to obtain the position, category confidence score and semantic segmentation mask of each marking, and extract the contour, position and orientation; The road markings are classified according to their shape, color, and width characteristics, and the edges are finely segmented to determine their location and shape. Automatically identify and classify all road markings, and convert them into a structured data format that includes marking attributes and their health status to obtain key information.

8. A real-time monitoring and evaluation system for road marking status based on video recognition, implementing the real-time monitoring and evaluation method for road marking status based on video recognition as described in any one of claims 1-7, characterized in that, include: The acquisition unit is used to acquire road video and vehicle driving information; A preprocessing unit is used to preprocess the images in the road video to obtain a preprocessing result; The detection and classification unit is used to identify and classify various road markings based on the preprocessing results, and extract the corresponding location, direction and shape to obtain key information; The tracking unit is used to assign a tracking ID to the road markings and combine vehicle driving information to perform cross-frame tracking and spatial-temporal positioning in order to determine the continuous image sequence of the same road markings. A multidimensional analysis unit is used to perform multidimensional health analysis on the road markings based on a continuous image sequence of the same road markings to obtain multidimensional health analysis results. The multidimensional health analysis includes reflectivity assessment, structural integrity assessment, color fading assessment, and geometric shape assessment. The grading unit is used to calculate the road marking health index based on the multidimensional health analysis results, and to divide the road markings into different health levels according to the road marking health index.

9. The real-time monitoring and evaluation system for road marking status based on video recognition according to claim 8, characterized in that, The multidimensional analysis unit includes: The reflectivity performance evaluation subunit is used to simulate retroreflection coefficient measurement based on a continuous image sequence of the same road marking, calculate the difference in gray values ​​between the marking area and the background road surface, and obtain the reflectivity performance evaluation results. The structural integrity assessment subunit is used to quantify the physical damage of the markings using pixel occupancy analysis, skeleton continuity detection, and edge roughness calculation methods to obtain the structural integrity assessment results. The color fading assessment subunit is used to convert the image to the CIELAB or HSV color space, calculate the color difference, and determine whether the road markings have faded due to pollution or ultraviolet radiation through aging trend analysis, so as to obtain the color fading assessment result. The geometric shape evaluation subunit is used to detect the geometric distortion of the markings by means of curve fitting and residual analysis, lateral deviation calculation and logical consistency verification, so as to obtain the geometric shape evaluation results. The combined subunit is used to combine the reflectivity assessment results, structural integrity assessment results, color fading assessment results, and geometric shape assessment results to form a multidimensional health analysis result.

Images