Traffic marking quality status assessment method, equipment and storage medium

Abstract

The present invention provides a method for evaluating the quality of traffic markings, the method comprising S1: preprocessing a first vehicle-mounted image set acquired based on a vehicle-mounted camera to obtain a fifth vehicle-mounted image set that meets the requirements of traffic sign line detection; S2: setting a region of interest of the vehicle-mounted image based on the fifth vehicle-mounted image set, and clipping the vehicle-mounted images in the fifth vehicle-mounted image set based on the region of interest of the vehicle-mounted image to obtain a processed sixth vehicle-mounted image set; S3: performing traffic marking target detection and semantic segmentation processing on the vehicle-mounted images in the sixth vehicle-mounted image set; S4: calculating the peak signal-to-noise ratio to determine the quality of traffic markings. By preprocessing the images taken by the vehicle-mounted camera to the recognition and quality evaluation of traffic markings, the entire process is highly automated, reducing manual intervention, improving the processing yield and efficiency, and realizing low-cost, high-efficiency, and accurate evaluation of the quality of traffic markings.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic safety, and in particular to a method for evaluating the quality of traffic markings, electronic equipment and a computer-readable storage medium. Background Art

[0002] Traffic marking quality status assessment refers to the quality inspection and status assessment of various traffic markings (such as lane lines, pedestrian crossing lines, arrow markings, etc.) marked on urban roads. The main purpose is to promptly detect and deal with marking wear, fading and other quality problems to ensure the safety and standardization of road traffic, maintain the normal traffic order of urban roads and improve the effectiveness of traffic management.

[0003] Initially, the quality assessment of traffic markings mainly relied on manual inspections, where inspectors drove inspection vehicles to inspect traffic markings along the road and took photos to record problematic markings. However, this method of relying on manual inspections has problems such as fuzzy classification, missing information, and incorrect locations, which further increases the difficulty of assessment and processing, especially in urban environments with heavy traffic.

[0004] In recent years, technicians in this field have developed various traffic marking quality detection solutions and devices, the most common of which are based on LiDAR sensors. Relying on the high precision, high resolution and real-time characteristics of LiDAR sensors, they emit laser pulses to obtain information such as the spatial position, posture and speed of target objects, and realize the inspection of traffic signs. However, LiDAR sensors have high equipment costs and large amounts of data to process, which increases the complexity and operating costs of traffic marking quality assessment work; at the same time, in actual maintenance projects, they also involve a large amount of investment in material costs and engineering costs, which increases the maintenance and management costs of the equipment.

[0005] Therefore, how to reduce the cost of traffic marking quality status assessment and achieve efficient and accurate traffic marking quality status assessment is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0006] The object of the present invention is to provide a method for evaluating the quality status of traffic markings, an electronic device and a computer-readable storage medium, which can reduce the cost of evaluating the quality status of traffic markings and achieve efficient and accurate evaluation of the quality status of traffic markings.

[0007] In order to solve the above technical problems, the present invention provides a method for evaluating the quality of traffic markings, which comprises the following steps:

[0008] S1: preprocessing a first vehicle-mounted image set acquired based on a vehicle-mounted camera to obtain a fifth vehicle-mounted image set satisfying traffic sign line detection;

[0009] S2: setting a region of interest of the vehicle image based on the fifth vehicle image set, and cropping the vehicle image in the fifth vehicle image set based on retaining the region of interest of the vehicle image, so as to obtain a processed sixth vehicle image set;

[0010] S3: performing traffic marking target detection and semantic segmentation processing on the vehicle images in the sixth vehicle image set;

[0011] S4: According to the processing results of traffic marking target detection and semantic segmentation, the peak signal-to-noise ratio is calculated to determine the quality status of the traffic marking.

[0012] Optionally, in the method for evaluating the quality of traffic markings, step S1 includes the following sub-steps:

[0013] S1.1: using a Gaussian filtering algorithm to smooth the vehicle images in the first vehicle image set to obtain a second vehicle image set after noise is removed;

[0014] S1.2: using a median filtering algorithm to remove salt and pepper noise from the vehicle images in the second vehicle image set, so as to obtain a third vehicle image set after the noise is removed;

[0015] S1.3: Processing the vehicle images in the third vehicle image set using an adaptive histogram equalization algorithm to obtain a fourth vehicle image set;

[0016] S1.4: Use the Laplace operator to process the vehicle images in the fourth vehicle image set to obtain a fifth vehicle image set.

[0017] Optionally, in the method for evaluating the quality of traffic markings, step S2 includes the following sub-steps:

[0018] S2.1: Randomly select from the fifth vehicle image set K Vehicle images; among them, K is an integer greater than 2;

[0019] S2.2: Determine the extracted K The ordinate of the horizontal line where the vanishing point generated by the lane marking of the vehicle-borne image is located is obtained by counting the ordinate area of ​​the horizontal line where the vanishing point is located; the horizontal line where the vanishing point generated by the lane marking is the boundary between the sky and the road;

[0020] S2.3: taking the lower limit of the ordinate area of ​​the horizontal line where the vanishing point is located as the upper boundary of the region of interest of the vehicle image, and dividing the vehicle image into a sky part and a road part based on the upper boundary;

[0021] S2.4: determining a road surface portion in the vehicle-borne image as a region of interest of the vehicle-borne image;

[0022] S2.5: All vehicle images in the fifth vehicle image set are cropped based on the region of interest of the vehicle image to obtain a processed sixth vehicle image set.

[0023] Optionally, in the method for evaluating the quality of traffic markings, step S3 includes the following sub-steps:

[0024] S3.1: Build an image deep learning model based on a multi-task learning architecture;

[0025] S3.2: Perform traffic marking target detection and semantic segmentation processing on the vehicle images in the sixth vehicle image set according to the image deep learning model.

[0026] Optionally, in the method for evaluating the quality of traffic markings, step S3.2 includes the following sub-steps:

[0027] S3.2.1: Network architecture for shared feature extraction based on image deep learning model Extract low-level features of input vehicle images;

[0028] S3.2.2: Based on the convolutional layer and feature pyramid network module, predict the category probability and detection box boundary of each anchor box of the input vehicle image to determine the geometric information of each traffic marking object; the geometric information includes the coordinates of the upper left corner, width and height of the traffic marking;

[0029] S3.2.3: Based on the convolutional layer and upsampling layer modules, a segmentation mask of the same size as the input vehicle image is predicted, where each pixel represents the category to which it belongs, to obtain pixels with traffic marking categories.

[0030] Optionally, in the method for evaluating the quality of traffic markings, step S4 includes the following sub-steps:

[0031] S4.1: For each traffic marking, perform vehicle-mounted image cropping according to geometric information of the traffic marking to obtain an initial target area image of the traffic marking;

[0032] S4.2: assigning a value of 255 to all pixels of the traffic marking category in the initial target area image, and assigning a value of 0 to the remaining pixels, so as to obtain a first target area image;

[0033] S4.3: Obtain the boundary contour of all pixels of the traffic marking category in the first target area image, assign all pixels in the closed boundary contour to 255, and assign the remaining pixels to 0, so as to obtain the second target area image;

[0034] S4.4: Calculate the peak signal-to-noise ratio based on the following formula;

[0035] ;

[0036] Among them, PSNR is the peak signal-to-noise ratio; I 1 is the first target area image; I 2 is the second target area image; ||·|| F Represents the matrix Frobenius Norm operations;

[0037] S4.5: Compare the calculated peak signal-to-noise ratio with a preset traffic marking quality assessment threshold to determine a grade of the corresponding traffic marking quality condition; the grade is: excellent, good, fair or poor.

[0038] Optionally, the method for evaluating the quality of traffic markings further includes the following steps:

[0039] S5: Build an archive of abnormal traffic marking quality conditions and track maintenance status.

[0040] Optionally, in the method for evaluating the quality of traffic markings, step S5 includes the following sub-steps:

[0041] S5.1: Establishing an archive library and initializing the abnormal data of the quality status of the road traffic markings, wherein the abnormal data at least includes: the original image of the vehicle, the abnormal highlight mark, the peak signal-to-noise ratio value, the GPS positioning data, the timestamp information and the vehicle direction;

[0042] S5.2: When the vehicle-mounted image with the quality status of the traffic markings being of medium level is transmitted, an early warning message is sent to the backend to prompt the staff to conduct further inspection and processing;

[0043] When a vehicle-mounted image showing that the quality of traffic markings is poor is transmitted, an emergency signal is sent to the backend to prompt the staff to take emergency measures to repair the road markings.

[0044] S5.3: For each new incoming abnormal data, according to the vehicle direction, GPS Locate data and time for duplicate detection:

[0045] S5.4: Based on GPS positioning data and vehicle orientation information, automatically capture data of vehicles passing the same location at different time periods for tracking and monitoring; in order to track the implementation of maintenance measures.

[0046] The present invention also provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;

[0047] When the processor executes the computer program instructions, the method for evaluating the quality status of traffic markings as described above is implemented.

[0048] The present invention also provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method for evaluating the quality status of traffic markings as described above is implemented.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1) Provide basic data for evaluating the quality of traffic markings based on images taken by vehicle-mounted cameras, effectively reducing evaluation costs;

[0051] 2) Perform a series of preprocessing on the images captured by the on-board camera to effectively reduce noise and retain lane line details. It has strong robustness, can effectively cope with environmental changes and noise interference, and improve the accuracy of the evaluation object;

[0052] 3) From image preprocessing to traffic marking recognition and quality assessment, the entire process is highly automated, reducing manual intervention and improving the efficiency of traffic marking quality assessment;

[0053] 4) Using the region of interest (ROI) extraction technology of the image, the key parts of the image are automatically identified and processed, thus improving the efficiency of the evaluation;

[0054] 5) Using the peak signal-to-noise ratio to quantitatively evaluate the usage status of traffic markings can accurately reflect the integrity and quality of traffic markings and improve the accuracy of the assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0056] Figure 1 is a flow chart of a method for evaluating the quality status of traffic markings in the present invention;

[0057] Figure 2 It is a schematic diagram of the location of the vanishing point of the lane line;

[0058] Figure 3 This is a schematic diagram of the lane line segmentation and extraction effect. DETAILED DESCRIPTION

[0059] The following is a further detailed description of the traffic marking quality assessment method, electronic device and computer-readable storage medium proposed in the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer according to the following description and claims. It should be noted that the drawings are all in a very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0060] The following is an explanation of the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the present embodiment only illustrate the basic concept of the present invention in a schematic manner, and only the components related to the present invention are shown in the figure instead of being drawn according to the number, shape and size of the components in the actual implementation. The type, quantity and proportion of each component in the actual implementation can be a random change, and the component layout type may also be more complicated.

[0061] Certain terms are used throughout the specification and claims to refer to specific system components. As will be appreciated by those skilled in the art, different companies may refer to a component by different names. It is not intended herein to distinguish between components that have different names but the same function. In the specification and claims, the terms "include" and "comprising" are used in an open-ended manner and, therefore, should be interpreted as "including, but not limited to ...".

[0062] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0063] In addition, each embodiment of the following description has one or more technical features, but this does not mean that the user of the present invention must implement all the technical features in any embodiment at the same time, or can only implement part or all of the technical features in different embodiments separately. In other words, under the premise that implementation is possible, those skilled in the art can selectively implement part or all of the technical features in any embodiment according to the disclosure of the present invention and according to the design specifications or implementation requirements, or selectively implement a combination of part or all of the technical features in multiple embodiments, thereby increasing the flexibility of the implementation of the present invention.

[0064] In order to make the purpose and features of the present invention more obvious and understandable, the specific implementation methods of the present invention are further described below in conjunction with the accompanying drawings. However, the present invention can be implemented in different forms and should not be considered to be limited to the described embodiments.

[0065] In the first aspect, the present invention provides a method for evaluating the quality of traffic markings. Figures 1 to 3 , the method for evaluating the quality of traffic markings of the present invention is described in detail. The method for evaluating the quality of traffic markings includes:

[0066] First, execute step S1 to obtain a first vehicle image set based on the vehicle camera. P 1 ={pic 1 _1, pic 1 _2, …, pic 1 _n} Preprocessing is performed to obtain the fifth vehicle image set that meets the traffic sign line detection requirements. P 5 ;

[0067] Step S1 includes the following sub-steps:

[0068] S1.1: Use Gaussian filtering algorithm to filter the first vehicle image set P 1 The vehicle images in the image are smoothed to obtain a second vehicle image set after noise removal. P 2 ={pic 2 _1, pic 2 _2, …, pic 2 _n} ;

[0069] S1.2: Use the median filter algorithm to remove the second vehicle image set P 2 The salt and pepper noise of the vehicle images in the image is removed to obtain the third vehicle image set after the noise is removed. P 3 ={pic 3 _1, pic 3 _2, …, pic 3 _n} ;

[0070] S1.3: Use adaptive histogram equalization ( CLAHE, Contrast Limited Adaptive Histogram Equalization ) algorithm for the third vehicle image set P3 The vehicle images in are processed to obtain a fourth vehicle image set P 4 ={ pic 4 _1, pic 4 _2, …, pic 4 _n};

[0071] S1.4: Use the Laplacian operator to calculate the fourth vehicle image set P 4 The vehicle images in are processed to obtain the fifth vehicle image set P 5 = { pic 5 _1, pic 5 _2, …, pic 5 _n}.

[0072] Next, step S2 is performed, based on the fifth vehicle-mounted image set P 5 Set the region of interest for the vehicle image ROI , and based on retaining the region of interest of the vehicle image, the fifth vehicle image set P 5 The vehicle images in the image are cropped to obtain the processed sixth vehicle image set P 6 ;

[0073] Please refer to Figure 2 , step S2 includes the following sub-steps:

[0074] S2.1: From the fifth vehicle image collection P 5 Randomly select K Vehicle images; among them, K is an integer greater than 2;

[0075] S2.2: Determine the extracted K The vertical coordinate of the horizontal line where the vanishing point of the lane markings generated by the vehicle image is located T , get the vertical coordinate area of ​​the horizontal line where the vanishing point is located by counting R , R ∈( T_min, T_max ); The horizontal line where the vanishing point of the lane marking is located is the boundary between the sky and the road;

[0076] S2.3: The vertical coordinate area of ​​the horizontal line where the vanishing point is located R The lower limit T_min As the upper boundary of the region of interest ROI of the vehicle image, the vehicle image is divided into a sky part and a road part based on the upper boundary;

[0077] S2.4: Determine a road surface portion in the vehicle-borne image as a region of interest ROI of the vehicle-borne image;

[0078] S2.5: Crop the fifth vehicle image set based on the region of interest of the vehicle image P 5 All vehicle images in the image are processed to obtain the sixth vehicle image set P 6 ={ pic 6 _1, pic 6 _2, …, pic 6 _n}.

[0079] Next, step S3 is executed to collect the sixth vehicle-mounted image set P 6 Car images in the vehicle are used for traffic marking target detection and semantic segmentation;

[0080] Step S3 includes the following sub-steps:

[0081] S3.1: Build an image deep learning model based on a multi-task learning architecture;

[0082] S3.2: Based on the image deep learning model, the sixth vehicle image set P 6 The vehicle images in the image are subjected to traffic marking target detection and semantic segmentation processing. Among them, step S3.2 includes the following sub-steps: S3.2.1: Extract low-level features (such as edges, textures, etc.) of the input vehicle image based on the network architecture of shared feature extraction of the image deep learning model; S3.2.2: Based on the convolution layer, feature pyramid network FPN The module predicts the category probability and detection box boundary of each anchor box of the input vehicle image to determine the geometric information of each traffic marking object; the geometric information includes the coordinates of the upper left corner of the traffic marking ( x, y ),width w and height h ; S3.2.3: Based on the convolution layer and upsampling layer modules, a segmentation mask of the same size as the input vehicle image is predicted, where each pixel represents the category to which it belongs, to obtain pixels with traffic marking categories.

[0083] Next, step S4 is executed to calculate the peak signal-to-noise ratio (PSNR) according to the processing results of traffic marking target detection and semantic segmentation to determine the quality status of the traffic marking.

[0084] Specifically, PSNR is used to measure the ratio between useful signals and useless noise. The larger the signal-to-noise ratio, the less noise there is in the image, indicating that the traffic markings have high integrity and good quality. Conversely, the smaller the signal-to-noise ratio, the poorer the quality of the traffic markings.

[0085] Wherein, step S4 includes the following sub-steps:

[0086] S4.1: For each traffic marking, according to the geometric information of the traffic marking (i.e., the coordinates of the upper left corner of the traffic marking object ( x_i, y_i ),width w_i ,high h_i ) to crop the vehicle image to obtain the initial target area image of the traffic marking I ;

[0087] S4.2: Initial target area image I The pixel values ​​of all traffic marking categories are assigned to 255, and the remaining pixels are assigned to 0 (as background pixels) to obtain the first target area image. I 1 ;

[0088] S4.3: Acquire the first target area image I 1 The boundary contour of all pixels of the traffic marking category within the closed boundary contour is assigned a value of 255, and the remaining pixels are assigned a value of 0 (as background pixels) to obtain the second target area image. I 2 ;

[0089] S4.4: Calculate the peak signal-to-noise ratio (PSNR) based on the following formula;

[0090] ;

[0091] Among them, PSNR is the peak signal-to-noise ratio; I 1 is the first target area image; I 2 is the second target area image; ||·|| F Represents the matrix Frobenius Norm operations;

[0092] S4.5: Compare the calculated peak signal-to-noise ratio (PSNR) with a preset traffic marking quality assessment threshold to determine a grade of the corresponding traffic marking quality condition; the grade is: excellent, good, fair or poor.

[0093] The quality of traffic markings is classified into four levels: excellent, good, fair and poor. PSNRTraffic marking quality assessment thresholds of values ​​and grades K 0 、K 1 、K 2 For comparison, the specific traffic marking quality status is evaluated as follows:

[0094] when PSNR>K 0 When , the quality of the traffic marking is "excellent";

[0095] when K 0 > PSNR>K 1 When , the quality of the traffic marking is "good";

[0096] when K 1 > PSNR>K 2 When , the quality status of the traffic marking is "medium";

[0097] when K 2 > PSNR , the quality status of the traffic marking is "poor".

[0098] Next, step S5 is executed to construct an archive of abnormal traffic marking quality conditions and track the maintenance status.

[0099] Specifically, step S5 includes the following sub-steps:

[0100] S5.1: Establishing an archive library and initializing the abnormal data of the quality status of the road traffic markings, wherein the abnormal data at least includes: the original image of the vehicle, the abnormal highlight mark, the peak signal-to-noise ratio value, the GPS positioning data, the timestamp information and the vehicle direction;

[0101] S5.2: When the vehicle-mounted image with the quality status of the traffic markings being of medium level is transmitted, an early warning message is sent to the backend to prompt the staff to conduct further inspection and processing;

[0102] When a vehicle-mounted image showing that the quality of traffic markings is poor is transmitted, an emergency signal is sent to the backend to prompt the staff to take emergency measures to repair the road markings.

[0103] S5.3: For each new incoming abnormal data, according to the vehicle direction, GPS Locate data and time for duplicate detection;

[0104] Among them, for the abnormal image data continuously transmitted in the same time period, its GPS Positioning information, if the adjacent position is less than the threshold S, then it is determined that the continuously passed in exception information here describes the only exception situation;

[0105] For abnormal image data transmitted in different time periods, determine its GPS Location information: If the location is adjacent to the existing abnormal data in the archive and is greater than the threshold S , then this abnormal data is a new abnormal situation; otherwise, continue to judge the vehicle direction. If the vehicle direction is different from the vehicle direction recorded in the abnormal data already existing in the archive, then this abnormal data is a new abnormal situation. Otherwise, this abnormal data is an abnormality already recorded in the archive. The newly detected abnormal data is associated with the original abnormal data.

[0106] S5.4: Based on GPS positioning data and vehicle orientation information, automatically capture data of vehicles passing the same location at different time periods for tracking and monitoring; in order to track the implementation of maintenance measures.

[0107] Among them, if no abnormality is found after detection of the captured data, it means that the abnormality has been successfully handled, the abnormal record is backed up and output, and deleted from the archive; if the captured data is still found to be abnormal after detection, or there are continuous abnormalities related to this location, an unprocessed prompt will be sent to the background, requiring relevant personnel to take corresponding measures.

[0108] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0109] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0110] In a second aspect, an embodiment of the present application provides an electronic device, including a processor and a memory storing computer program instructions;

[0111] When the processor executes the computer program instructions, the method for evaluating the quality status of traffic markings as described in any one of the first aspects of the embodiments of the present application is implemented.

[0112] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, a method for evaluating the quality status of traffic markings as described in any one of the first aspect of the embodiments of the present application is implemented.

[0113] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0114] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0115] Those skilled in the art will appreciate that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0116] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0119] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0120] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0121] 1) Provide basic data for evaluating the quality of traffic markings based on images taken by vehicle-mounted cameras, effectively reducing evaluation costs;

[0122] 2) Perform a series of preprocessing on the images captured by the on-board camera to effectively reduce noise and retain lane line details. It has strong robustness, can effectively cope with environmental changes and noise interference, and improve the accuracy of the evaluation object;

[0123] 3) From image preprocessing to traffic marking recognition and quality assessment, the entire process is highly automated, reducing manual intervention and improving the efficiency of traffic marking quality assessment;

[0124] 4) Using the region of interest (ROI) extraction technology of the image, the key parts of the image are automatically identified and processed, thus improving the efficiency of the evaluation;

[0125] 5) Using the peak signal-to-noise ratio to quantitatively evaluate the usage status of traffic markings can accurately reflect the integrity and quality of traffic markings and improve the accuracy of the assessment.

[0126] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A method for evaluating the quality of traffic markings, characterized in that: The following steps are involved: S1: A first vehicle image set obtained based on a vehicle camera P 1 Preprocessing is performed to obtain the fifth vehicle image set that meets the traffic sign line detection requirements. P 5 ; S2: Based on the fifth vehicle image collection P 5 Set the region of interest ROI of the vehicle image, and perform the fifth vehicle image set based on the region of interest of the retained vehicle image. P 5 The vehicle images in the image are cropped to obtain a processed sixth vehicle image set P6; S3: the sixth vehicle image set P 6 Traffic marking target detection and semantic segmentation processing are performed on the vehicle-mounted image in S3, and S3 includes the following sub-steps: S3.1: Build an image deep learning model based on a multi-task learning architecture; S3.2: Based on the image deep learning model, the sixth vehicle image set P 6 Car images in the vehicle are used for traffic marking target detection and semantic segmentation; S3.2 includes the following sub-steps: S3.2.1: Network architecture for shared feature extraction based on image deep learning model Extract low-level features of input vehicle images; S3.2.2: Based on convolutional layer and feature pyramid network FPN The module predicts the category probability and detection box boundary of each anchor box of the input vehicle image to determine the geometric information of each traffic marking object; the geometric information includes the coordinates of the upper left corner of the traffic marking ( x, y ),width w and height h ; S3.2.3: Based on the convolutional layer and upsampling layer modules, a segmentation mask of the same size as the input vehicle image is predicted, where each pixel represents the category to which it belongs, to obtain pixels with traffic marking categories; S4: According to the processing results of traffic marking target detection and semantic segmentation, the peak signal-to-noise ratio is calculated to determine the quality status of the traffic marking. S4 includes the following sub-steps: S4.1: For each traffic marking, perform vehicle-mounted image cropping according to the geometric information of the traffic marking to obtain the initial target area image of the traffic marking I ; S4.2: Initial target area image I The pixel values ​​of all traffic marking categories are assigned to 255, and the remaining pixels are assigned to 0 to obtain the first target area image. I 1 ; S4.3: Acquire the first target area image I 1 The boundary contour of all the pixels of the traffic marking category in the closed boundary contour is assigned a value of 255 to all the pixels in the closed boundary contour, and the remaining pixels are assigned a value of 0 to obtain the second target area image I 2 ; S4.4: Calculate the peak signal-to-noise ratio based on the following formula; ； Among them, PSNR is the peak signal-to-noise ratio; I 1 is the first target area image; I 2 is the second target area image; ||·|| F Represents the matrix Frobenius Norm operations; S4.5: Compare the calculated peak signal-to-noise ratio with a preset traffic marking quality assessment threshold to determine a grade of the corresponding traffic marking quality condition; the grade is: excellent, good, fair or poor.

2. The method for evaluating the quality of traffic markings according to claim 1, characterized in that: Step S1 includes the following sub-steps: S1.1: Use Gaussian filtering algorithm to filter the first vehicle image set P 1 The vehicle images in the image are smoothed to obtain a second vehicle image set after noise removal. P 2 ; S1.2: Use the median filter algorithm to remove the second vehicle image set P 2 The salt and pepper noise of the vehicle images in the image is removed to obtain the third vehicle image set after the noise is removed. P 3 ; S1.3: Use the adaptive histogram equalization CLAHE algorithm to perform the third vehicle image set P 3 The vehicle images in are processed to obtain a fourth vehicle image set P 4 ; S1.4: Use the Laplacian operator to calculate the fourth vehicle image set P 4 The vehicle images in are processed to obtain the fifth vehicle image set P 5 .

3. The method for evaluating the quality of traffic markings according to claim 1, characterized in that: Step S2 includes the following sub-steps: S2.1: From the fifth vehicle image collection P 5 Randomly select K Vehicle images; in, K is an integer greater than 2; S2.2: Determine the extracted K The vertical coordinate of the horizontal line where the vanishing point of the lane markings generated by the vehicle image is located T , get the vertical coordinate area of ​​the horizontal line where the vanishing point is located by counting R ; The horizontal line where the vanishing point generated by the lane marking is located is the boundary between the sky and the road; S2.3: The vertical coordinate area of ​​the horizontal line where the vanishing point is located R The lower limit T_min As the upper boundary of the region of interest ROI of the vehicle image, the vehicle image is divided into a sky part and a road part based on the upper boundary; S2.4: Determine a road surface portion in the vehicle-borne image as a region of interest ROI of the vehicle-borne image; S2.5: Crop the fifth vehicle image set based on the region of interest of the vehicle image P 5 All vehicle images in the image are processed to obtain the sixth vehicle image set P 6 .

4. The method for evaluating the quality of traffic markings according to claim 1, characterized in that: The following steps are also included: S5: Build an archive of abnormal traffic marking quality conditions and track maintenance status.

5. The method for evaluating the quality of traffic markings according to claim 1, characterized in that: Step S5 includes the following sub-steps: S5.1: Establish an archive and initialize the abnormal data of the quality status of the road traffic markings, wherein the abnormal data at least includes: the original image of the vehicle, the abnormal highlight mark, the peak signal-to-noise ratio value, the GPS positioning data, the timestamp information and the vehicle direction; S5.2: When the vehicle-mounted image with the quality status of the traffic markings being of medium level is transmitted, an early warning message is sent to the backend to prompt the staff to conduct further inspection and processing; When a vehicle-mounted image showing that the quality of traffic markings is poor is transmitted, an emergency signal is sent to the backend to prompt the staff to take emergency measures to repair the road markings. S5.3: For each new incoming abnormal data, according to the vehicle direction, GPS Locate data and time for duplicate detection: S5.4: Based on GPS positioning data and vehicle orientation information, automatically capture data of vehicles passing the same location at different time periods for tracking and monitoring; in order to track the implementation of maintenance measures.

6. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for evaluating the quality status of traffic markings as described in any one of claims 1 to 5 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method for evaluating the quality status of traffic markings as described in any one of claims 1 to 5 is implemented.

Images