Road disease detection and ledger updating method and device, equipment and storage medium

Abstract

The invention discloses a road disease detection and standing book updating method and device, equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a to-be-detected road image and the position information of a road region corresponding to the to-be-detected image; extracting a marking mask image of the to-be-detected road image, performing skeletonization and topological repair processing on the marking mask image, and determining a marking center line; based on the position information of the road to be detected and the position information corresponding to each ledger image in the traffic marking ledger, determining a plurality of candidate ledger images to be updated from the traffic marking ledger; and based on the comprehensive similarity between the to-be-detected road image and the candidate to-be-updated standing book image, determining a standing book updating strategy. According to the invention, intelligent, real-time and refined maintenance management of the road marking and the associated facility ledger is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a road disease detection and account updating method and device, equipment and a storage medium. BACKGROUND

[0002] With the rapid development of intelligent transportation and digital maintenance system, road markings, raised road signs and vertical signs play a fundamental role in road traffic safety, order control and asset management. In recent years, highway and urban road management departments have gradually introduced video inspection and vehicle-mounted sensing equipment to replace traditional manual inspection methods in order to realize intelligent identification and dynamic maintenance of facility status.

[0003] In related technologies, single image feature method lacks unified data standard and geometric description system, and it is difficult to handle problems such as broken markings, residual old markings or non-standard settings in complex environments. In addition, the account updating lacks spatial indexing and change detection mechanism, and the inspection results are difficult to automatically align with the asset database and dynamically maintain. SUMMARY

[0004] The main purpose of the present application is to provide a road disease detection and account updating method, device, equipment, storage medium and program product, which aims to solve the technical problem of low reliability of related technology road disease detection and account updating.

[0005] To achieve the above purpose, the present application provides a road disease detection and account updating method, which comprises: obtaining the position information of the road image to be detected and the road region corresponding to the image to be detected; extracting the marking mask image of the road image to be detected and performing skeletonization and topological repair processing on the marking mask image to determine the marking center line; Based on the position information of the road to be detected and the position information corresponding to each account image in the traffic marking account, a plurality of candidate account images to be updated are determined from the traffic marking account; Based on the comprehensive similarity between the road image to be detected and the candidate account image to be updated, the account updating strategy is determined; the comprehensive similarity is determined based on the marking mask feature difference, the marking center line distance, the position information and the semantic consistency score between the road image to be detected and the account image to be updated.

[0006] In an embodiment, the step of extracting the marking mask image of the road image to be detected and performing skeletonization and topological repair processing on the marking mask image to determine the marking center line comprises: performing edge detection on the road image to be detected to obtain a binary edge image of the road image to be detected; performing Hough transform on the binary edge image to extract a line segment set related to the traffic lane marking in the binary edge image, wherein the line segment set comprises a number of line segments, an average line segment length and a direction variance; determining a complexity of the traffic lane marking in the to-be-detected road image based on the line segment set; if the complexity of the traffic lane marking is low, performing skeletonization processing on the lane marking mask image by using a Zhang-Suen thinning algorithm to extract a lane marking skeleton; if the complexity of the traffic lane marking is high, performing skeletonization processing on the lane marking mask image by using a Guo-Hall algorithm to extract a lane marking skeleton; performing topological repair processing on the lane marking skeleton to determine a lane marking center line.

[0007] In an embodiment, the step of performing topological repair processing on the lane marking skeleton to determine a lane marking center line comprises: traversing the lane marking skeleton and identifying all discontinuous breakpoints; determining a slidable local repair window, and in the local repair window region, using a shortest path algorithm and / or a morphological dilation operation to connect lane marking segments at both ends of the breakpoint to obtain continuous lane marking; based on a preset branch length threshold, determining and removing burrs in the continuous lane marking; performing smoothing processing on the lane marking skeleton after the burrs are removed to determine a lane marking center line.

[0008] In an embodiment, the step of determining an account updating strategy based on the comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image comprises: completing disease area determination in a lane marking buffer zone of the to-be-detected road image to determine a disease type in the disease area; determining a disease severity of the disease type based on a disease severity calculation model; updating the traffic lane marking account based on the disease severity and the comprehensive similarity.

[0009] In an embodiment, the step of updating the traffic lane marking account based on the disease severity and the comprehensive similarity comprises: determining a lane marking mask feature difference based on a lane marking mask intersection ratio between the to-be-detected road image and the candidate to-be-updated account image; determining a lane marking center line distance based on a distance measurement between a road center line in the to-be-detected road image and a road center line in the candidate to-be-updated account image; performing weighted combination on the lane marking mask feature difference, the lane marking center line distance, the position information difference and the semantic consistency score to obtain the comprehensive similarity; If the comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image is not less than the similarity threshold value, and the change degree of the disease severity of the to-be-detected road image compared with the disease severity of the candidate to-be-updated account image is not less than the preset threshold value, the candidate to-be-updated account image with the maximum comprehensive similarity is replaced by the to-be-detected road image. Otherwise, the to-be-detected road image is added to the traffic marking account.

[0010] In an embodiment, the step of determining the plurality of candidate to-be-updated account images from the traffic marking account based on the position information of the to-be-detected road and the position information corresponding to each account image in the traffic marking account comprises: constructing an R-tree spatial index based on the minimum bounding rectangle of each account image; defining a spatial query window based on the spatial coordinates of the to-be-detected road, performing an overlap query operation, and obtaining an initial candidate image set; converging the initial candidate image set based on the name information, lane number, and mileage post interval overlap degree of the to-be-detected road, removing images that are spatially adjacent but not logically continuous, and obtaining the candidate to-be-updated account image.

[0011] In an embodiment, after the step of determining the account updating strategy, the method further comprises the step of: based on the model prediction results generated in the disease judgment and account matching process, evaluating the object-level prediction uncertainty; if the object-level prediction uncertainty exceeds the uncertainty threshold value or conflicts with the historical account record, generating and outputting a manual review task.

[0012] In a second aspect, to achieve the above object, the present application further provides a road disease detection and account updating device, the device comprising: an image acquisition module configured to acquire a to-be-detected road image and position information of a corresponding inspection area of the to-be-detected image; a feature extraction module configured to extract a marking mask image of the to-be-detected road image and perform skeletonization and topological repair processing on the marking mask image to determine a marking center line; a candidate account determination module configured to determine a plurality of candidate to-be-updated account images from a traffic marking account based on position information of the to-be-detected road and position information corresponding to each account image in the traffic marking account; an account updating module configured to determine an account updating strategy based on a comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image; the comprehensive similarity is determined based on marking mask feature differences, marking center line distances, position information, and semantic consistency scores between the to-be-detected road image and the to-be-updated account image.

[0013] In a third aspect, to achieve the above object, the present application further provides a road disease detection and account updating device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the road disease detection and account updating method.

[0014] In a fourth aspect, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the road disease detection and account updating method.

[0015] The one or more technical solutions provided by the present application have at least the following technical effects: By introducing the reticle mask skeletonization and topology repair mechanism, a unified and stable geometric representation is constructed, which effectively alleviates the identification unreliable problems caused by reticle fracture, residual interference and non-standard setting in complex road environment; at the same time, based on the position information, a spatial index is constructed and the multi-dimensional similarity indexes such as reticle feature difference, reticle center line distance and semantic consistency are fused, so as to realize the accurate matching and change determination of the inspection image and the existing account, and avoid the false update and missed update caused by relying on single image feature or coarse-grained spatial matching; in addition, by combining the disease severity evaluation and prediction uncertainty analysis, the automatic updating of the account is supported, and manual review is triggered when the result exists conflict or credibility is insufficient, so as to significantly improve the accuracy, reliability and engineering usability of road disease detection and traffic marking account updating. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the accompanying drawings needed to be used in the embodiment or related art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0018] Figure 1 It is a schematic diagram of the overall process of the road disease detection and account updating method embodiment of the present application.

[0019] Figure 2 It is a flowchart of the implementation details of the road disease detection and account updating method embodiment of the present application.

[0020] Figure 3 It is a schematic diagram of the reticle center line extraction process of the embodiment of the present application.

[0021] Figure 4 The schematic diagram of the account matching process of the embodiment of the present application is shown.

[0022] Figure 5 The schematic diagram of the structure of the road disease detection and account updating device of the present application is shown.

[0023] Figure 6 The schematic diagram of the structure of the road disease detection and account updating device of the present application is shown.

[0024] The purpose implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0025] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and do not limit the present application.

[0026] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below in conjunction with the drawings and the specific embodiments.

[0027] The main solution of the embodiment of the present application is: extracting the marking center line through skeletonization and topological repair, determining the disease type and quantifying the severity in the buffer zone, using R-tree index to realize efficient spatial matching, and calculating the comprehensive similarity based on mask difference, marking center line distance, positioning deviation and semantic consistency, so as to intelligently judge the update or addition of road diseases.

[0028] Specifically, the embodiment of the present application provides a road disease detection and account updating method, referring to Figure 1 and Figure 2 , Figure 1 The schematic diagram of the overall process of the road disease detection and account updating method embodiment of the present application is shown. Figure 2 The flowchart of the implementation details of the road disease detection and account updating method embodiment of the present application is shown. In the present embodiment, the road disease detection and account updating method comprises steps S10-S40: Step S10, acquiring the to-be-detected road image and the position information of the road region corresponding to the to-be-detected image.

[0029] It should be noted that the to-be-detected road image can be acquired from the road video stream collected in real time by the front or overhead camera of the vehicle-mounted terminal system, and the positioning and attitude information of the GNSS and IMU sensors are recorded simultaneously to confirm the position information of the corresponding road region.

[0030] Further, in order to ensure the stability of data processing, the to-be-detected road image can also be pre-processed. For example, the video frame is subjected to image stabilization, denoising, exposure and white balance normalization, glare / rain and mist suppression, motion blur compensation and the like, so as to improve the image quality and ensure the time sequence continuity, thereby providing clear and stable input for subsequent disease detection.

[0031] Specifically, the image stabilization is performed by using the optical flow method, the optical flow window size is 15x15, and the maximum displacement threshold is 10 pixels; the denoising is performed by using the Gaussian filter, the filter kernel size is 5x5, and the standard deviation is 1.5; the exposure is performed by using the HDR processing, 3-5 images with different exposures are fused, and the exposure times are 1 / 60s, 1 / 125s and 1 / 250s respectively; the de-fogging is performed by using the dark channel prior algorithm, the window size is 15x15, and the atmospheric light value estimation window size is 0.1 times the image size; the de-blurring is performed by using the Wiener filter for motion blur compensation, and the point spread function (PSF) length is estimated to be 5-10 pixels.

[0032] Step S20, extracting a lane marking mask image of the to-be-detected road image and performing skeletonization and topological repair processing on the lane marking mask image to determine a lane marking center line.

[0033] Specifically, the lane marking mask image can be extracted by a semantic segmentation model or an instance segmentation model, and then the lane marking mask image is refined by a skeletonization algorithm and topological repair is performed, redundant pixels are deleted, and the skeleton structure of the object is retained, and finally a single-pixel-width skeleton image is generated to realize lane marking center line extraction.

[0034] It can be understood that after obtaining the high-quality image, this step aims to solve a core cognitive deepening problem: how to convert the unstructured pixel information "seen" by the deep learning model into structured geometric features that meet the engineering definition and can be used for accurate measurement and analysis. This is a key step that distinguishes it from traditional disease detection methods and lays the foundation for all subsequent quantitative analysis.

[0035] In a feasible implementation, step S20 includes steps A10-A60: Step A10, performing edge detection on the to-be-detected road image to obtain a binary edge image of the to-be-detected road image.

[0036] Step A20, performing Hough transform on the binary edge image to extract a line segment set related to the traffic lane marking in the binary edge image; the line segment set includes the number of line segments, the average line segment length and the direction variance.

[0037] Step A30, determining the complexity of the traffic lane marking in the to-be-detected road image based on the line segment set.

[0038] Step A40, if the complexity of the traffic marking is low, the skeletonization processing is performed on the marking mask image by Zhang-Suen thinning algorithm to extract the marking skeleton.

[0039] Step A50, if the complexity of the traffic marking is high, the skeletonization processing is performed on the marking mask image by Guo-Hall algorithm to extract the marking skeleton.

[0040] Step A60, the topological repair processing is performed on the marking skeleton to determine the marking center line.

[0041] Among them, step A60 includes steps A61-A64: Step A61, all discontinuous breakpoints are identified by traversing the marking skeleton. Step A62, a slidable local repair window is determined, and in the local repair window region, the shortest path algorithm and / or morphological dilation operation are used to connect the marking segments at both ends of the breakpoint to obtain continuous marking. Step A63, based on the preset branch length threshold, burrs in the continuous marking are determined and removed.

[0042] Step A64, the marking skeleton after removing the burrs is smoothed to determine the marking center line.

[0043] Specifically, the marking is extracted by a semantic segmentation model (such as DeepLabV3+, PSPNet) or an instance segmentation model (such as MaskR-CNN, YOLACT), and the pixel mask generated is discrete and unstructured "pixel group". Although it visually identifies the marking area, it cannot be directly used to accurately calculate the length, curvature, continuous interruption distance and other key geometric parameters of the marking. Therefore, with reference to Figure 3 The embodiment introduces subsequent skeletonization and topological repair processing: First, by pre-judging the marking complexity, the optimal algorithm is adaptively selected to ensure the robustness of the skeleton extraction.

[0044] Exemplarily, the specific steps based on the pre-judgment of the marking complexity include: Edge detection is performed on the road image to be detected, and a binary edge image is generated by using Canny edge detection algorithm (low threshold is set to 50, high threshold is set to 150, and Sobel kernel size is 3x3) or Sobel operator. This step focuses on the ROI (region of interest) of the road area, for example, the lower half of the image, to avoid irrelevant background interference.

[0045] Apply Hough transform (e.g., probabilistic Hough transform, accumulator threshold set to 50, minimum line segment length set to 20 pixels, maximum line segment gap set to 5 pixels) to the edge image, detect and extract the line segment set related to the marking line, including the number of line segments , average line segment length , and direction variance (the direction variance is obtained by calculating the standard deviation of all line segment angles).

[0046] The complexity index can be expressed as: ; This index reflects the fragmentation degree and direction diversity of the marking line: simple marking lines (such as straight solid or dashed lines) usually have fewer line segments, longer lengths, and consistent directions (small ); complex marking lines (such as arrows, guiding text, or cross marks) have more line segments, shorter lengths, and diverse directions (large ).

[0047] If the complexity index is less than a preset threshold (e.g. , calibrated according to typical road image dataset experience, such as urban road straight line marking , arrow marking ), it is determined that the marking line complexity is low, and the Zhang-Suen thinning algorithm is selected to ensure fast processing and good connectivity; otherwise, it is determined that the marking line complexity is high, and the Guo-Hall algorithm is selected to better preserve the geometric features of complex shapes and avoid distortion.

[0048] The preset threshold can be calibrated and verified through an offline dataset (such as an image set containing simple lane lines vs. complex arrow samples). This pre-judgment method has low computational overhead (edge detection , Hough transform , , and the like), improves the robustness of skeleton extraction in complex road environments, and does not rely on the mask output of the semantic segmentation model, avoiding the propagation of segmentation errors.

[0049] Experiments show that this pre-judgment mechanism improves the skeleton integrity rate by more than 12% in complex marking line scenarios (such as arrows, numbers), and the overall processing efficiency is comparable to non-adaptive methods.

[0050] For marking line mask images with low complexity, the Zhang-Suen thinning algorithm is used to iteratively remove boundary pixels to well maintain the topological structure and connectivity of the marking line. The Zhang-Suen thinning algorithm is suitable for continuous and morphologically simple marking lines.

[0051] wherein the iteration termination condition of the Zhang-Suen thinning algorithm is that the difference in the number of pixels removed in two consecutive iterations is less than 0.1% of the total number of pixels, or the number of iterations reaches the maximum number of iterations (recommended value 100 times). The algorithm is suitable for processing continuous markings and can maintain the connectivity and endpoint position of the markings.

[0052] For high complexity marking mask images, the Guo-Hall thinning algorithm uses template matching method, and the processing effect of the marking with complex shapes such as arrows and characters is better, which can better preserve the geometric features. Specifically, the Guo-Hall algorithm uses 8-neighbor template matching, and the template library contains 256 possible neighborhood configurations. The algorithm is suitable for processing complex shape markings such as arrows and guide marks, and can better preserve the geometric features of the markings.

[0053] Further, due to factors such as light, occlusion, and stains in the real scene, and possible errors in the segmentation model itself, the extracted original skeleton often has defects such as breakpoints, burrs (redundant branches), and jagged edges. These defects do not conform to the geometric continuity and smoothness that road markings as physical entities should have. Therefore, topological repair must be performed, and the process includes: Connectivity check: traverse the skeleton and identify all discontinuous breakpoints. Among them, the traversal of breakpoints can be achieved by depth / breadth first search algorithm. Illustratively, the breakpoint judgment threshold is 3 pixels, that is, when the distance between two connected components is less than 3 pixels, it is judged as a breakpoint Breakpoint repair: In the local area, use the shortest path algorithm (such as Dijkstra) and / or morphological dilation operation to intelligently connect the broken lines caused by minor segmentation errors and restore the continuity of the markings. Illustratively, by setting a branch length threshold, the branch length threshold is set to 5% of the total length of the markings, that is, when the branch length is less than 5% of the total length of the markings, it is judged as a redundant branch and deleted, and the short "burr" produced by image noise or edge mis-detection is pruned, and the main trunk center line is preserved. The search radius of the shortest path algorithm is 10 pixels, and the kernel size of the morphological operation is 3x3.

[0054] Geometric smoothing: Smooth the repaired center line using methods such as Gaussian filtering to eliminate jagged edges and make it more consistent with the smooth curve in engineering practice. Illustratively, Gaussian filtering is used for smoothing, and the filter kernel size is 5x5 and the standard deviation is 1.0.

[0055] It can be understood that through the complete process of "segmentation-skeletonization-topology repair", the original output of computer vision (pixel mask) is successfully converted into a high-quality, standardized, and vectorized geometric primitive "marking center line" by the embodiment. The center line has a precise mathematical definition and complete topological structure, and it is not only a visual presentation, but also a computable and analyzable engineering data.

[0056] In step S30, based on the position information of the to-be-detected road and the position information corresponding to each account image in the traffic marking account, a plurality of candidate to-be-updated account images are determined from the traffic marking account.

[0057] Specifically, after the implementation of single detection "recognition" and "quantization", the next core problem to be solved by the embodiment is how to effectively associate the current detection result with a large amount of historical account data. This is the key bottleneck of realizing facility life cycle management, because inefficient and inaccurate matching will make any cross-time state tracking and analysis impossible. Therefore, the embodiment introduces a candidate account determination method divided into two steps of "coarse screening" and "fine selection".

[0058] Specifically, referring to Figure 4 For coarse screening, in order to improve the efficiency of account updating, candidate to-be-updated account images need to be retrieved from the traffic marking account to reduce the subsequent matching update range and improve the update efficiency.

[0059] In a feasible implementation, step S30 includes steps AB10 to B30: Step B10, based on the minimum bounding rectangle of each account image, an R-tree spatial index is constructed.

[0060] Step B20, based on the spatial coordinates of the to-be-detected road, a spatial query window is defined, an overlap query operation is performed, and an initial candidate image set is obtained.

[0061] Step B30, based on the name information, lane number and mileage post interval overlap degree of the to-be-detected road, the initial candidate image set is converged, images that are spatially adjacent but logically discontinuous are removed, and candidate to-be-updated account images are obtained.

[0062] Specifically, the embodiment first reconstructs the unorganized account data which needs to be compared one by one through the spatial hierarchical R-tree. As a highly optimized spatial data structure, the R-tree can organize two-dimensional spatial information (the geographical position range of the marking line) into a tree hierarchy, so that the efficiency of range query is proportional to the logarithm of the total amount of data, rather than linearly related. This fundamentally solves the problem of slow system response when the scale of account data expands.

[0063] Subsequently, through the two-step strategy of "R-tree space fast screening" and "logical information selection", the system can reduce the candidate matching objects from millions of data to dozens or even a few within milliseconds. Experiments show that after using R-tree index, the average number of candidates is reduced from 128 to 23, and the matching delay is reduced from 38.2ms / object to 12.6ms / object, with an efficiency improvement of about 3 times. This not only greatly reduces the computational burden of subsequent precise matching, but more importantly, it provides technical feasibility for the high-frequency and quasi-real-time dynamic tracking and management of road facility assets, and is an indispensable first step in building a dynamic life cycle management system.

[0064] Illustratively, based on the geographic location information (such as WGS84 coordinates, road ID, lane number and milepost interval) of each image or object in the traffic marking account, an efficient spatial index structure, R-tree, is constructed. R-tree can store the minimum bounding rectangle (MBR) of the object according to the spatial region level, supporting fast range query and neighborhood retrieval.

[0065] Specifically, the minimum node capacity M_min of R-tree is recommended to be set to 2, and the maximum node capacity M_max is recommended to be set to 50. When the node capacity exceeds M_max, a node splitting strategy is adopted, two seed nodes are selected, and the remaining nodes are allocated to the nearest seed node, and this process is repeated until all nodes are allocated. This strategy can effectively balance the depth and node utilization of the index tree. Experiments show that with this parameter setting, the query efficiency of R-tree index is more than 100 times higher than linear search.

[0066] Through this index structure, the account entries with similar geographic location to the road area to be detected can be quickly locked in the massive account data, greatly reducing the computational complexity and retrieval time required for subsequent matching, and improving the overall system efficiency. At the same time, R-tree index has dynamic updating capability, which can automatically maintain the spatial hierarchical relationship as the account content changes, thereby ensuring the long-term scalability and stability of the index structure.

[0067] Specifically, the GNSS positioning information corresponding to the road image to be detected and the road reference line are used as the query window to perform spatial overlap search in R-tree, and the account images that intersect or are adjacent to the to-be-detected area in spatial range are quickly screened out. Subsequently, combined with the milepost interval, lane number and driving direction of the road and other constraint conditions, the preliminary results are converged again to exclude objects that are adjacent in space but not continuous in logic, forming a simplified candidate account set. In this way, the system not only ensures the spatial relevance of the matching data, but also avoids the redundant calculation brought by full comparison, effectively improving the real-time performance and accuracy of the account updating process, thereby realizing the rapid response and accurate update of road marking changes.

[0068] Step S40, determining the account updating strategy based on the comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image; the comprehensive similarity is determined based on the marking mask feature difference, the marking center line distance, the position information, and the semantic consistency score between the to-be-detected road image and the to-be-updated account image.

[0069] After the coarse screening, the to-be-updated account image needs to be determined through fine screening, and the fine screening aims to solve another core problem in the life cycle management: how to accurately determine “this one” as “that one” in history. Only relying on the position (GPS) or appearance (traditional image features) for matching is prone to errors in the complex urban environment with dense markings, high building obstructions, and variable lighting, leading to “misattribution”, making the account record chaotic, and failing to track the life cycle. Therefore, the embodiment innovatively proposes a multi-dimensional comprehensive similarity calculation method, which decomposes the matching problem into comprehensive consideration of four dimensions of geometric shape, linear structure, spatial position, and logical attribute, thereby realizing unprecedented matching robustness.

[0070] Specifically, in a feasible implementation, step S40 includes steps C10-C30: Step C10, determining the disease type in the disease area within the marking buffer zone of the to-be-detected road image.

[0071] Step C20, determining the disease severity of the disease type based on a disease severity calculation model.

[0072] Step C30, updating the traffic marking account based on the disease severity and the comprehensive similarity.

[0073] Wherein, the road disease type includes marking loss, old marking not removed, and / or marking setting not standardized, the disease severity of the marking loss is determined based on marking coverage loss rate, maximum interruption length, and edge roughness, the disease severity of the old marking not removed is determined based on brightness contrast attenuation degree, and the disease severity of the marking setting not standardized is determined based on rule compliance degree.

[0074] Step C30 includes steps D10-D0: Step D10, determining the marking mask feature difference based on the marking mask intersection ratio between the to-be-detected road image and the candidate to-be-updated account image.

[0075] Step D20, determining the marking center line distance based on the distance measurement between the road center line in the to-be-detected road image and the road center line in the candidate to-be-updated account image. Step D30, weighting and combining the marking mask feature difference, the marking center line distance, the position information difference, and the semantic consistency score to obtain the comprehensive similarity.

[0076] Step D40, if the comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image is not less than the similarity threshold, and the change degree of the disease severity of the to-be-detected road image compared with the disease severity of the candidate to-be-updated account image is not less than the preset threshold, then the candidate to-be-updated account image with the maximum comprehensive similarity is replaced by the to-be-detected road image.

[0077] Step D50, otherwise, the to-be-detected road image is added to the traffic marking account.

[0078] The core of the embodiment is to calculate a comprehensive similarity score S match, which integrates the information of the following four dimensions: Geometric shape similarity (difference in marking mask features), which is obtained by calculating the intersection over union (IoU) between the to-be-detected image and the candidate account image. This dimension mainly answers whether the overall shape and coverage of the two markings are similar. IoU is sensitive to large-scale wear, addition, and other shape changes of markings, and is the basis for judging identity.

[0079] Linear structure similarity (based on marking center line distance): Based on the extracted structured marking center line, the Fréchet distance or Hausdorff distance between the two center lines can be calculated to determine the marking center line distance. This dimension answers whether the direction and twisted shape of the two lines are consistent. Compared with the mask IoU, the marking center line distance is more sensitive to small displacement, rotation or local deformation of the marking. It focuses on the internal structure of the marking as a "line", can effectively distinguish markings with similar positions but different directions, and greatly improves the recognition degree in dense areas.

[0080] Spatial position similarity (based on GNSS distance): The Haversine distance between the GNSS positioning coordinates of the two can be calculated. This dimension answers whether they are close enough in geographical space. This is a basic constraint for matching, and the application ensures high-precision positioning input by integrating IMU data and post-processing.

[0081] Logical attribute similarity (based on semantic consistency score): Compare the types (such as solid / dashed lines), colors, and lane ownership of the markings, etc. High-level semantic information. This dimension answers whether their identity information is consistent. Semantic information provides strong logical constraints for matching, which can effectively exclude markings that are similar in space and shape but completely different in nature.

[0082] Specifically, after extracting the road marking mask image and the road marking centerline, the first step is to perform defect area detection within the road marking buffer zone. In this embodiment, road defect types include missing road markings, old road markings not being removed, and / or improperly installed road markings.

[0083] The determination of defect areas combines characteristics such as holes, breaks, abrupt width changes, and brightness differences in the pavement marking mask to identify types of defects, including missing pavement markings, uncleared old markings, and non-standard pavement marking installation. Missing pavement markings are determined based on characteristics such as mask holes, break lengths, and abrupt width changes. Uncleared old markings are determined by combining historical map sheet differences, directional consistency, and brightness thresholds. Non-standard installation is addressed by using a rule base to verify width, spacing, arrow positions, and pedestrian crossing strips. Raised pavement markers and vertical markings are determined to be missing or non-standard by checking the required grid points and spacing / tilt thresholds.

[0084] Next, based on the severity calculation model, various types of defects are quantified. For defects in road markings, a comprehensive score is calculated based on the coverage loss rate, the proportion of the maximum interruption length, and the edge roughness index. For old road markings that have not been removed, attenuation analysis is performed using a brightness contrast attenuation metric. For non-standard settings, a rule base is used to check the compliance of width, spacing, arrow position, and pedestrian crossing strip spacing, and a rule compliance score is output based on the inverse index of rule compliance. Based on the comprehensive analysis of various defect types, a defect severity calculation model is confirmed, which can be expressed as: in: The weights of each indicator can be adjusted according to the scenario, and the sum of all weights is 1.

[0085] R loss =A missing / A total Missing area A missing Total area A total It mainly reflects the overall wear and tear of the road marking materials and is the basis for assessing whether the road marking functions (such as nighttime reflectivity and rainy night markings) meet the standards.

[0086] Maximum interrupt length Total length of the markings The proportion. The deeper meaning of this indicator lies in assessing whether the continuity of the lane marking guidance function has been disrupted. A short and wide wear ( R loss A higher fracture might be less desirable than a thin but long one. R gap(Higher) lines pose a greater risk to the driver's visual guidance. This is especially true for solid lines. R gap This value indicates a serious driving safety hazard.

[0087] It can be calculated using methods such as contour curvature variance or fractal dimension. It quantifies the "roughness" of the road marking edges, which not only affects the aesthetics of the city, but more importantly, blurry edges reduce the visibility of the road markings under complex lighting conditions (such as backlighting and wet, reflective surfaces), and is an early sign of road marking quality deterioration.

[0088] : Indicates that the brightness of the new markings is calculated. L new brightness compared to old markings L old Differences between ( R lum = 1 - L old / L new This is used to assess the severity of old road markings that have not been removed. It is directly related to the reflectivity and contrast of the road markings; too low a contrast will render the road markings ineffective at night or in bad weather.

[0089] : An inverse indicator of rule compliance, where To assign a score to a rule, for example, the degree of irregularity in the markings' geometric dimensions, spacing, and position can be quantified by comparing them with a rule base K containing more than 20 rules, thus obtaining a rule score. It reflects the engineering quality and compliance of the road marking construction; non-compliant road markings may convey incorrect or misleading information to drivers.

[0090] Understandably, through the above multi-dimensional quantitative assessment model, this application transforms the traditional, experience-based, vague, and inconsistent disease assessment in maintenance work into an automated, precise, and standardized scientific assessment. The output severity score... S sev It can be directly used for prioritizing maintenance work orders, accurately budgeting maintenance costs, and conducting macro-analysis and prediction of the health status of regional road networks, thereby realizing intelligent and scientific road maintenance decision-making.

[0091] Then, the comprehensive similarity between the to-be-detected road image and the candidate account image is calculated, which includes calculating the intersection over union (IoU) of the lane mask between the to-be-detected image and the candidate account image, which is used to reflect the geometric shape difference. The Fréchet or Hausdorff distance of the center line of the two is calculated, which reflects the degree of linear displacement. The lane mask feature difference, lane center line distance, GNSS positioning deviation and semantic consistency score are combined by weighting to form the comprehensive similarity.

[0092] Specifically, first, the intersection over union (IoU) of the lane mask between the to-be-detected image and the candidate account image is calculated to reflect the geometric shape difference. The calculation method of the lane mask intersection over union is as follows: Wherein, M t is the lane mask of the to-be-detected image, M l is the lane mask of the candidate account image, ∩ represents intersection, and ∪ represents union. The larger the IoU value, the more similar the geometric shape, and the value range is [0, 1].

[0093] Then, the distance measure between the lane center line in the to-be-detected image and the lane center line in the candidate account image is calculated to reflect the degree of linear displacement.

[0094] In an example, the lane center line distance is calculated by Fréchet distance, which considers the directionality of the curve and is more suitable for evaluating the similarity of the lane center line. The specific calculation method is as follows: For two center lines C t and C l , which contain n and m points respectively, the Fréchet distance is calculated by dynamic programming method, and the distance matrix D is defined, wherein D i represents the Fréchet distance between the first i points of C t and the first j points of C l . Its recursive formula is as follows: Wherein, is the Euclidean distance between two points. The final Fréchet distance is D n . The time complexity of this algorithm is O(n×m), and the space complexity is O(n×m). For long lane, segmented calculation or sampling method can be used to reduce the calculation complexity.

[0095] In another example, the distance between the center lines of the markings is calculated using Hausdorff distance: Hausdorff distance calculates the maximum and minimum distance between two curves, regardless of the directionality of the curves. For two center lines... C t and C l The formula for calculating the Hausdorff distance is: d H ( C t , C l ) = max(sup {x e C t } inf {y∈ C l} ||xy||, sup {y e C l } inf {x∈ C t} ||xy||) in, d H ( C t , C l ( ) represents two center lines C t and C l The Hausdorff distance, where x represents the centerline. C t A point in the center line, where y represents the center line. C l One of the points, sup {x e C t } inf {y∈ C l} ||xy|| indicates from C t point to C l Maximum and minimum distance h( C t , C l ), sup {y e C l } inf {x∈ C t} ||xy|| indicates from C l point to C tthe maximum minimum distance h C l , C t )max(sup {x e C t } inf {y∈ C l} ||x-y||, sup {y e C l } inf {x∈ C t} ||x-y||) represents the maximum value.

[0096] The specific calculation steps are as follows: for each point x in C t , calculate the minimum distance from it to all points in C l , and take the maximum value to obtain h C t , C l ); for each point y in C l , calculate the minimum distance from it to all points in C t , and take the maximum value to obtain h C l , C t ); the Hausdorff distance is max(h C t , C l ), h C l , C t )). The time complexity of this algorithm is O(n×m), which can be optimized to O(n×log(m)) through space index structures such as KD trees.

[0097] Subsequently, the centerline distance of the reticle d F is normalized, and finally the distance is normalized: d F_norm = d F / d max , where d F is the centerline distance, and d max is the normalization constant, with a recommended value of 10 meters. For different road types, dmax Dynamic adjustment: urban road recommendation value 5-8 meters, highway recommendation value 10-15 meters. The normalized distance value is used for comprehensive similarity calculation to ensure the dimensional consistency of different dimension indicators.

[0098] Then, the GNSS positioning distance between the to-be-detected road image and the candidate account image is calculated to reflect the spatial position similarity. The calculation method is: d gps = Haversine( lat t , lon t , lat l , lon l ) Wherein,( lat t , lon t ) is the GPS coordinate of the to-be-detected image,( lat l , lon l ) is the GPS coordinate of the candidate account image, and Haversine is the formula for calculating the great circle distance between two points. Normalize the distance: d gps_norm = d gps / r Wherein, r is the positioning normalized radius, and the recommended value is 3-10 meters, which can be dynamically adjusted according to the road type.

[0099] Then, the semantic consistency score is calculated to measure whether the object type and attribute match. The semantic consistency score S sem Through the following dimensions: object type consistency (such as whether the marking type and disease type are consistent), attribute consistency (such as whether the marking color, material and other attributes are consistent), rule compliance consistency (such as whether it conforms to the same rule constraints).

[0100] The calculation formula is: S sem = w type × S type + w attr × S attr + w rule × Srule ,in, S type , S attr , S rule The scores are for consistency in type, attribute, and rule compliance, respectively. w type , w attr , w rule For the corresponding weights.

[0101] Finally, the overall similarity can be represented as follows: in, The weighted coefficient of the overall similarity reflects the importance of geometric, location, and semantic elements.

[0102] Based on this, a similarity threshold is set. With severity change threshold When the overall similarity and (Severity of damage to the new ledger images) Difference in the severity of damage compared to old ledger images Update the ledger when the severity change threshold is exceeded; otherwise, keep it. Add new objects when the similarity threshold is not reached.

[0103] When the overall similarity is not less than the similarity threshold and the change in the severity of the disease exceeds the set threshold, the system considers that there is a significant change in the road section and automatically replaces the original ledger object with the new detection results. When the overall similarity or change rate does not reach the threshold, the system determines it to be a newly added or suspected newly added object and adds it to the ledger as a new entry.

[0104] Experiments show that, compared with using only IoU matching (success rate 81.67%, false match rate 18.33%), the comprehensive similarity matching method of this application can achieve a success rate of 95.0%, reduce the false match rate to 5.0%, and improve the matching accuracy by about 13.33 percentage points, which significantly improves the accuracy and reliability of ledger updates.

[0105] Furthermore, to improve the reliability of the results, the system can introduce uncertainty assessment before updating. Specifically, steps S40 is followed by steps S50-S60: Step S50: Based on the model prediction results generated during the process of performing disease identification and ledger matching, assess the uncertainty of the output object-level prediction.

[0106] Step S60, if the object-level prediction uncertainty exceeds the uncertainty threshold, or conflicts with the historical records, an artificial review task is generated and output.

[0107] Specifically, in key fields such as road asset management, any automated system must address the core issue of "trustworthiness". Traditional AI models are like a "black box", with opaque decision-making processes and no ability to judge the reliability of their own results. Once a false positive or false negative occurs, it will directly affect the correctness of maintenance decisions, and even pose a safety risk. To solve this problem, the present application prospectively designs a complete closed loop from "uncertainty evaluation" to "artificial review" and then to "model self-optimization", aiming to upgrade the system from a static "black box tool" to a dynamic, trustworthy, and self-evolving "learning partner".

[0108] The present embodiment uses uncertainty quantification technology to allow the model to output a companion "uncertainty score U" along with the detection or matching results.

[0109] In an example, through the Monte Carlo Dropout method, during the model inference (prediction) stage, the Dropout layer in the network is kept active, and the same input is randomly forward propagated T times (e.g., T = 30). Since the random inactivation path is different each time, T slightly different prediction results are obtained. The variance between these results is quantified as the "uncertainty" of the model for that prediction. The greater the variance, the lower the "consensus" within the model for that result, and the less reliable the result.

[0110] In another example, through the Deep Ensemble method, M (e.g., M = 5-10) models with the same structure but slightly different initializations or training data are independently trained to form an "expert committee". During inference, all models simultaneously predict the input, and the degree of disagreement (variance) between the M prediction results represents the uncertainty of the system.

[0111] Subsequently, the system does not blindly accept all automated results, but sets up an intelligent "sentinel". When any of the following conditions are met, the system automatically marks the event as high risk and generates a structured artificial review task to push to maintenance experts: Automatically generate review tasks when U exceeds the threshold or conflicts with historical records, and the artificial review results are fed back to the model training. The specific manifestations are: High uncertainty: the uncertainty score U of the prediction result exceeds the preset threshold (e.g., U ≥ 0.3). This indicates that the model itself does not have sufficient confidence in this judgment.

[0112] Logical conflict: the current detection result is obviously contradictory to the historical record. The application designs a conflict detection mechanism, including: Position conflict: the current position of the matched object deviates too much from the historical position (e.g. more than 5 meters).

[0113] Type conflict: the same object has a dramatic change in disease type that is not logical (e.g. from "minor wear and tear" to "non-standard installation").

[0114] Severity conflict: the severity score jumps unreasonably (e.g. from 0.9 to 0.1 in a short period of time).

[0115] It can be understood that the result of manual review is not only the correction of a single event. The "correct" or "incorrect" label confirmed by the review expert will be automatically packaged into high-quality training samples along with the corresponding image, uncertainty score and other data, and fed back to the training database of the model. The system can periodically or after accumulating a certain number of samples, use these valuable expert knowledge for incremental training or retraining. This "detection-evaluation-review-learning" closed-loop mechanism enables the technical solution of the application to have continuous self-evolution ability. The system will become more "intelligent" in actual use, be more "alert" to scenes where it has made mistakes, and its overall performance and reliability will continue to improve over time and with the accumulation of data, ultimately achieving the perfect combination of automation and high reliability.

[0116] Finally, the updated structured account data is output in GeoJSON or Shapefile format and pushed to the GIS platform, work order system and asset management platform, realizing the automatic closed-loop management of road facility inspection, maintenance and rectification.

[0117] It can be understood that by integrating image segmentation, geometric skeleton extraction, spatial index retrieval, disease identification and severity quantitative evaluation, a full-process automation system from road disease detection to account dynamic updating is constructed. Not only can it realize efficient candidate account retrieval based on R-tree spatial index, greatly reducing the matching calculation amount and improving the updating efficiency; but also through the comprehensive similarity judgment of the center line and mask features of the marking line, it can realize accurate identification and quantitative evaluation of the disease area, ensuring the accuracy and reliability of the updating result. At the same time, combined with the uncertainty evaluation and manual review closed-loop mechanism, the adaptive learning ability and long-term running stability of the system are significantly improved, thereby realizing intelligent, real-time and fine maintenance and management of road marking and related facility accounts.

[0118] In order for those skilled in the art to better understand the present embodiment, the technical solutions of the present embodiment are explained below through specific implementation examples in specific application scenarios. It can be understood that the following examples are only used to explain the present application, and are not used to limit the protection scope of the claims of the present application.

[0119] The present example proposes an end-to-end integrated machine solution, integrating vehicle-mounted imaging, GNSS / IMU, edge computing, communication and review linkage modules, aiming at defects such as missing marking, old marking not removed, non-standard setting, missing protruding road marking, non-standard facade marking, etc., to realize the full-process automation closed loop of "detection - evaluation - positioning - matching - updating - linkage". The system realizes accurate identification, quantitative evaluation and dynamic account maintenance through the cooperation of multiple source sensors, center line / geometric topology constraints, rule compliance reasoning, multi-index severity scoring and cross-time and space account matching.

[0120] Specifically, with reference to Figure 4 , the method flow of the present example is: Data acquisition and preprocessing: The acquisition module synchronously acquires vehicle-mounted camera video stream, GNSS / IMU positioning data and vehicle-mounted attitude information; The preprocessing module performs image stabilization, denoising, exposure / white balance normalization, glare / rain and fog suppression, motion blur compensation and time sequence alignment.

[0121] Marking segmentation and facility detection: The semantic / instance segmentation output marking mask is used , the center line is obtained through skeleton thinning (such as Zhang-Suen / Guo-Hall) and topological repair is performed; The disease area determination is completed within the marking buffer zone; protruding road markings and facade markers are identified and classified through target detection.

[0122] Disease type determination: Marking missing according to mask void, fracture length, width mutation, etc. Old marking not removed, combined with historical map difference, direction consistency, brightness threshold joint determination; Non-standard setting calls rule library K to check width, spacing, arrow position, pedestrian crossing strip, etc. Protruding road markings and facade markers are determined to be missing or non-standard by the required grid points, spacing / angle threshold. The rule library contains: lane line width, virtual and real line segment length / spacing, number of pedestrian crossing strips, distance of arrow / guide marker to stop line or intersection, protruding road marking spacing / missing rate threshold, facade marker spacing and angle tolerance.

[0123] Spatial mapping and indexing: Using the external road reference line, the defect objects are mapped to geographic line segments / points / areas, and the WGS84 coordinates are output with the road ID, lane, and mileage marker interval attached. Construct spatial indexes such as R trees to support fast retrieval.

[0124] Ledger matching and difference update: Set similarity threshold With severity change threshold When the overall similarity and (Severity of damage to the new ledger images) Severity of damage to old ledger images The difference is not less than the severity change threshold. Update the ledger when the similarity threshold is reached, otherwise keep it; add new objects when the similarity threshold is not reached.

[0125] Uncertainty assessment and review closed loop: Output pixel / object level uncertainty using methods such as MC Dropout / Deep Ensemble ; when When the threshold is exceeded or there is a conflict with the historical record, a review task is automatically generated, and the results of the manual review are fed back into the model training.

[0126] Output and platform integration: Structured records (GeoJSON / Shapefile) and evidence image slices are generated and pushed to GIS, work order system and asset management platform to realize closed-loop scheduling of inspection and rectification.

[0127] The implementation results of this example are shown in Tables 1, 2, and 3: As can be seen from Table 1, the comprehensive similarity calculation matching method (IoU+GNSS+) of this example scheme D is as follows. d F + S sem The matching success rate reached 95.0%, and the false matching rate dropped to 5.0%. Compared with Scheme A, which only uses IoU (matching success rate 81.67%, false matching rate 18.33%), the matching success rate increased by 13.33 percentage points and the false matching rate decreased by 13.33 percentage points, significantly improving the accuracy and reliability of ledger matching. Compared with the lack of a ledger update mechanism in Reference Document 1, this application achieves efficient cross-temporal and spatial ledger matching, providing important technical support for road facility management.

[0128] As can be seen from Table 2, by using the R-tree retrieval, the number of candidate account images can be effectively reduced, the average number of candidates is reduced from 128 to 23, reduced by about 82%, the matching delay is reduced from 38.2ms / object to 12.6ms / object, the efficiency is improved by about 3 times, the matching efficiency problem when the account data is large is effectively solved, and the real-time account updating is realized.

[0129] As can be seen from Table 3, when the uncertainty threshold is set to 0.3, 53 samples are reviewed, the manual review hit rate is 50.94%, and the missed detection rate is 10.0%, which can effectively identify high-risk results and perform manual review; when the uncertainty threshold is set to 0.4, 9 samples are reviewed, the manual review hit rate is 55.56%, but the missed detection rate is as high as 83.33%, which is relatively high. Considering the hit rate and the missed detection rate, it is recommended that the uncertainty threshold be set to 0.3-0.35, which can effectively control the missed detection rate while ensuring the review hit rate.

[0130] Table 1: Cross-period matching ablation comparison table Table 2: Acceleration effect table of tree candidate set convergence Table 3: Uncertainty trigger threshold sensitivity It should be noted that all the above examples are only for understanding the present application and do not constitute a limitation on the road disease detection and account updating method of the present application. More simple transformations based on this technical concept are within the protection scope of the present application.

[0131] The present application also provides a road disease detection and account updating device, please refer to Figure 5 The road disease detection and account updating device comprises: An image acquisition module 10 is configured to acquire a to-be-detected road image and position information of a to-be-detected image corresponding to a patrol area.

[0132] A feature extraction module 20 is configured to extract a marking mask image of the to-be-detected road image and perform skeletonization and topological repair processing on the marking mask image to determine a marking center line.

[0133] A candidate account determination module 30 is configured to determine a plurality of candidate to-be-updated account images from a traffic marking account based on position information of the to-be-detected road and position information corresponding to each account image in the traffic marking account.

[0134] The account updating module 40 is configured to determine an account updating strategy based on a comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image, and the comprehensive similarity is determined based on a marking mask feature difference, a marking center line distance, position information, and a semantic consistency score between the to-be-detected road image and the to-be-updated account image.

[0135] The road disease detection and account updating device provided in the present application can solve the technical problem of low reliability of road disease detection and account updating in the related art by using the road disease detection and account updating method in the above embodiments. Compared with the related art, the road disease detection and account updating device provided in the present application has the same beneficial effects as the road disease detection and account updating method provided in the above embodiments, and other technical features in the road disease detection and account updating device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0136] The present application provides a road disease detection and account updating device, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the road disease detection and account updating method in the above embodiments.

[0137] Reference will be made to the following Figure 6 which shows a structural schematic diagram of a road disease detection and account updating device suitable for implementing the embodiments of the present application. The road disease detection and account updating device in the embodiments of the present application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Multimedia Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 6 The road disease detection and account updating device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0138] As Figure 6As shown, the road disease detection and ledger updating device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 (ROM) or loaded from a storage device 1003 into a random access memory 1004 (RAM). In the random access memory 1004, various programs and data required for the operation of the road disease detection and ledger updating device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output interface 1006 (I / O interface) is also connected to the bus 1005. Generally, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the road disease detection and ledger updating device to communicate wirelessly or wired with other devices to exchange data. Although the road disease detection and ledger updating device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0139] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0140] The road disease detection and ledger updating device provided by the present disclosure adopts the road disease detection and ledger updating method in the above embodiments, and can solve the technical problem of low reliability of road disease detection and ledger updating in related technologies. Compared with related technologies, the road disease detection and ledger updating device provided by the present disclosure has the same beneficial effects as the road disease detection and ledger updating method provided by the above embodiments, and other technical features in the road disease detection and ledger updating device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0141] It should be understood that various parts of the present disclosure can be realized in hardware, software, firmware, or a combination thereof. In the description of the above-described embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0142] The above merely provides the specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present disclosure, which should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

[0143] The present disclosure provides a computer readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to execute the road disease detection and account updating method in the above-described embodiments.

[0144] The computer readable storage medium provided by the present disclosure may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to: electric wires, optical cables, RF (Radio Frequency: radio frequency), etc., or any suitable combination of the above.

[0145] The above computer readable storage medium can be contained in the road disease detection and account updating device; or can exist separately without being assembled into the road disease detection and account updating device.

[0146] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the road disease detection and account updating device, the road disease detection and account updating device: acquires a to-be-detected road image and position information of a road region corresponding to the to-be-detected image; extracts a marking mask image of the to-be-detected road image and performs skeletonization and topological repair processing on the marking mask image to determine a marking center line; determines a plurality of candidate to-be-updated account images from the traffic marking account based on the position information of the to-be-detected road and position information corresponding to each account image in the traffic marking account; and determines an account updating strategy based on comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image.

[0147] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0148] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified functions. It should also be noted that, in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0149] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not constitute a limitation on the modules themselves.

[0150] The readable storage medium provided in the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the road disease detection and account updating method described above, and can solve the technical problem of low reliability of road disease detection and account updating in the related art. Compared with the related art, the computer readable storage medium provided in the present application has the same beneficial effects as the road disease detection and account updating method provided in the above embodiments, and will not be described here.

[0151] The present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the road disease detection and account updating method described above are implemented.

[0152] The computer program product provided in the present application can solve the technical problem of low reliability of road disease detection and account updating in the related art. Compared with the related art, the computer program product provided in the present application has the same beneficial effects as the road disease detection and account updating method provided in the above embodiments, and will not be described here.

[0153] The above is only some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation, direct / indirect application in other related technical fields made by using the content of the present application specification and drawings, or the like, are included in the patent protection scope of the present application.

Claims

1. A method for detecting road defects and updating records, characterized in that, The method comprises: obtaining a to-be-detected road image and position information of a road region corresponding to the to-be-detected image; extracting a marking mask image of the to-be-detected road image and performing skeletonization and topological repair processing on the marking mask image to determine a marking center line; determining a plurality of candidate to-be-updated account images from the traffic marking account based on the position information of the to-be-detected road and the position information corresponding to each account image in the traffic marking account; determining an account updating strategy based on the comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image; the comprehensive similarity is determined based on the marking mask feature difference, the marking center line distance, the position information and the semantic consistency score between the to-be-detected road image and the to-be-updated account image.

2. The road defect detection and ledger updating method of claim 1, wherein, The step of extracting the marking mask image of the to-be-detected road image and performing skeletonization and topological repair processing on the marking mask image to determine the marking center line comprises: performing edge detection on the to-be-detected road image to obtain a binary edge image of the to-be-detected road image; performing Hough transformation on the binary edge image to extract a line segment set related to the traffic marking in the binary edge image; the line segment set comprises the number of line segments, the average line segment length and the direction variance; determining the complexity of the traffic marking in the to-be-detected road image based on the line segment set; if the complexity of the traffic marking is low, performing skeletonization processing on the marking mask image by a Zhang-Suen thinning algorithm to extract a marking skeleton; if the complexity of the traffic marking is high, performing skeletonization processing on the marking mask image by a Guo-Hall algorithm to extract a marking skeleton; performing topological repair processing on the marking skeleton to determine the marking center line.

3. The method of road defect detection and record updating of claim 2, wherein, The step of performing topological repair processing on the marking skeleton to determine the marking center line comprises: traversing the marking skeleton and identifying all discontinuous breakpoints; determining a locally repairable window that can slide, and in the locally repairable window region, using a shortest path algorithm and / or a morphological dilation operation to connect the marking segments at both ends of the breakpoint to obtain continuous marking; based on a preset branch length threshold, determining and removing burrs in the continuous marking; performing smoothing processing on the marking skeleton after removing the burrs to determine the marking center line.

4. The method of road defect detection and record updating of claim 1, wherein, The step of determining an account updating strategy based on the comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image comprises: completing disease area determination in a marking buffer zone of the to-be-detected road image to determine the disease type in the disease area; determining the disease severity of the disease type based on a disease severity calculation model; updating the traffic marking account based on the disease severity and the comprehensive similarity.

5. The method of road defect detection and record updating of claim 4, wherein, The step of updating the traffic marking account based on the disease severity and the comprehensive similarity comprises: determining the marking mask feature difference based on the marking mask intersection ratio between the to-be-detected road image and the candidate to-be-updated account image; determine the target line center line distance based on a distance metric between a road center line in the to-be-detected road image and a road center line in the candidate to-be-updated account image; combine the target line mask feature difference, the target line center line distance, the position information difference, and the semantic consistency score by weighting, to obtain a comprehensive similarity; if the comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image is not less than a similarity threshold and a change degree of disease severity of the to-be-detected road image compared to disease severity of the candidate to-be-updated account image is not less than a preset threshold, replace the candidate to-be-updated account image with the maximum comprehensive similarity with the to-be-detected road image; otherwise, add the to-be-detected road image to the traffic marking account.

6. The method of road defect detection and asset record updating of claim 1, wherein, The step of determining the plurality of candidate to-be-updated account images from the traffic marking account based on the position information of the to-be-detected road and position information corresponding to each account image in the traffic marking account comprises: constructing an R-tree spatial index based on a minimum bounding rectangle of each account image; defining a spatial query window based on a spatial coordinate of the to-be-detected road, performing an overlap query operation, and obtaining an initial candidate image set; converging the initial candidate image set based on name information, lane number, and mileage post interval overlap degree of the to-be-detected road, removing images that are spatially adjacent but not logically continuous, and obtaining the candidate to-be-updated account image.

7. The method of road defect detection and asset record updating of claim 1, wherein, After the step of determining the account update strategy, the method further comprises the steps of: evaluating an object-level prediction uncertainty based on a model prediction result generated in the disease determination and account matching process; if the object-level prediction uncertainty exceeds an uncertainty threshold or conflicts with a historical account record, generating and outputting a manual review task.

8. A road disease detection and ledger updating apparatus characterized by comprising: The device comprises: an image acquisition module configured to acquire a to-be-detected road image and position information of a patrol area corresponding to the to-be-detected image; a feature extraction module configured to extract a target line mask image of the to-be-detected road image and perform skeletonization and topological repair processing on the target line mask image to determine a target line center line; a candidate account determination module configured to determine a plurality of candidate to-be-updated account images from a traffic marking account based on position information of the to-be-detected road and position information corresponding to each account image in the traffic marking account; an account update module configured to determine an account update strategy based on a comprehensive similarity between the to-be-detected road image and the candidate to-be-updated account image, wherein the comprehensive similarity is determined based on a target line mask feature difference, a target line center line distance, position information, and a semantic consistency score between the to-be-detected road image and the to-be-updated account image.

9. A road distress detection and ledger updating apparatus, characterized by, The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the road disease detection and account update method according to any one of claims 1 to 7.

10. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the road disease detection and ledger updating method in any one of claims 1 to 7.

Images