Road and municipal road barrier-free facility intelligent inspection and management and control system based on AI visual identification

Abstract

The invention relates to the technical field of intelligent traffic and urban management, and particularly discloses an intelligent inspection and management and control system for barrier-free facilities of roads and municipal roads based on AI visual identification. The system comprises a multi-source data acquisition module, a geographic information fusion and specification adaptation module, a multi-modal AI visual analysis engine, a dynamic decision and control instruction generation module and a full-process closed-loop management module. By dynamically matching geographic information and normative standards, fusing multi-modal data to carry out robustness analysis, and automatically generating a hierarchical management and control instruction and a block chain evidence storage closed loop based on an abnormal level, high-precision and self-adaptive intelligent inspection and full-process traceable management and control of the barrier-free facility state of the highway and the municipal road are realized. According to the system, a traditional fixed threshold judgment mode is thoroughly changed by constructing a multi-level normative knowledge graph and dynamically fusing the multi-level normative knowledge graph with real-time geographic positioning.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation and urban management technology, specifically relating to an intelligent inspection and control system for barrier-free facilities on highways and municipal roads based on AI visual recognition. Background Technology

[0002] In the field of smart city and digital municipal construction, intelligent inspection and management of infrastructure is a key link in improving urban governance and ensuring public safety and service quality.

[0003] The integrity and standardization of accessibility facilities on highways and municipal roads are directly related to the travel rights and safety of people with disabilities and other special groups. Their daily inspection and maintenance work constitutes an important part of the city's refined management.

[0004] The intelligent inspection system based on computer vision and artificial intelligence technology aims to replace the traditional manual inspection mode by automating image acquisition and analysis, so as to achieve rapid identification and assessment of facility status.

[0005] The basic principle of this type of system is to acquire information about the appearance of facilities by using image acquisition equipment deployed on inspection vehicles or at fixed locations, and to use trained algorithm models to automatically detect defects or violations.

[0006] Existing technologies typically use preset fixed thresholds or rules to analyze the acquired images. For example, image measurement technology is used to calculate the slope of an accessible ramp and compare it with a fixed standard value to determine its compliance.

[0007] This static rule-based approach has significant limitations: First, different countries, regions, and even cities have different specific standards for accessibility facilities (such as ramp slope and width). A fixed set of judgment thresholds cannot adapt to diverse local regulations, resulting in a significant increase in the misjudgment rate when the system is applied across regions.

[0008] Existing methods lack effective integration with dynamic data such as geographic information systems, and cannot automatically call upon the corresponding compliance standard library based on the specific geographical location of the facility, resulting in insufficient intelligence and adaptability. In addition, traditional visual inspection methods are not robust to changes in the on-site environment such as complex lighting conditions and partial occlusion of facilities, further affecting the accuracy and reliability of the inspection.

[0009] How to build an accessible facility inspection and control system that can intelligently adapt to multi-source standards, dynamically integrate geographic information, and has strong environmental adaptability has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0010] The purpose of this invention is to provide an intelligent inspection and control system for barrier-free facilities on highways and municipal roads based on AI visual recognition, so as to solve the problems in the existing technology that cannot be adapted to diverse local regulations due to the use of fixed judgment rules, lack of dynamic integration with geographic information, and insufficient robustness to complex on-site environments.

[0011] The technical solution of this invention is an intelligent inspection and control system for barrier-free facilities on highways and municipal roads based on AI visual recognition. The system includes a multi-source data acquisition module, a geographic information fusion and standard adaptation module, a multimodal AI visual analysis engine, a dynamic decision-making and control instruction generation module, and a full-process closed-loop management module.

[0012] The multi-source data acquisition module is used to acquire raw data of the inspection target in real time. This module integrates a high-resolution optical imaging unit, a lidar point cloud acquisition unit, and an inertial measurement unit. The optical imaging unit acquires color images and infrared thermal imaging data of the obstacle-free facilities along the inspection route at a rate of 30 frames per second. The lidar point cloud acquisition unit synchronously emits a 905-nanometer wavelength laser beam, acquiring three-dimensional spatial point cloud data of the facilities at a density of 200,000 points per second. The inertial measurement unit continuously records the acceleration and angular velocity changes of the inspection vehicle in three-dimensional space, with a data output frequency of 100 Hz. The above three units achieve strict alignment of data acquisition times through a hardware time synchronization mechanism, with a time synchronization accuracy better than 1 millisecond.

[0013] The Geographic Information Fusion and Standard Adaptation module provides precise geographic context and compliance judgment benchmarks for visual analysis. This module incorporates a multi-level standard knowledge graph, organized in a tree structure. The root node is the national-level general accessibility design standard, followed by detailed standard provisions at the provincial, municipal, and even specific administrative division levels.

[0014] Each regulation clause is stored in a structured manner, including the regulation entry number, the polygon boundary coordinates of the applicable geographical area, the facility type code, and specific quantitative technical parameter thresholds. This module receives real-time positioning data from the global navigation satellite system (GNSS) attached to the multi-source data acquisition module and uses a spatial geocoding engine to quickly calculate the spatial inclusion relationship between the latitude and longitude coordinates of the current inspection location and the geographical area polygons in the regulation knowledge graph. After the calculation is complete, the module dynamically loads and activates all relevant regulation clauses applicable to the current geographical location, forming a temporary, location-specific set of compliance judgment rules, which is then pushed to the multimodal AI visual analysis engine in real time.

[0015] The multimodal AI visual analysis engine is the core computing unit of this system, used for fusion analysis and state determination of the collected raw data. This engine adopts a cascaded multi-task neural network architecture, specifically including a data preprocessing and alignment submodule, a feature extraction and fusion submodule, and a compliance determination submodule. The data preprocessing and alignment submodule first performs spatiotemporal alignment correction on the received image data, point cloud data, and inertial measurement unit (IMU) data, uses IMU data to compensate for image blurring caused by carrier motion, and assigns precise three-dimensional coordinates to each pixel in the image based on the point cloud data.

[0016] The feature extraction and fusion submodule contains two parallel deep convolutional neural network branches. One branch processes aligned 2D image data to extract color, texture, and 2D morphological features; the other branch processes 3D point cloud data to extract spatial geometric features through point cloud voxelization and 3D convolution operations.

[0017] Subsequently, the feature fusion network uses a cross-modal attention mechanism to weightedly fuse feature vectors from the 2D and 3D branches, generating a unified feature descriptor that includes appearance and geometric information. The compliance determination submodule receives this unified feature descriptor along with a set of proprietary rules from the geographic information fusion and standardization adaptation module. This submodule does not perform simple threshold comparisons but instead executes a multi-stage inference process.

[0018] In the first stage, key metric parameters are parsed from the unified feature descriptor based on the facility type code. For example, for accessible ramps, the slope angle, net width, and platform size are parsed.

[0019] In the second stage, the parsed parameter values ​​are compared with the corresponding dynamic thresholds in the rule set.

[0020] In the third stage, an environmental confidence assessment mechanism is introduced. This mechanism calculates the overall confidence score of the detection based on the image's illumination uniformity, occlusion area ratio, and point cloud data integrity. Only when the measurement parameters exceed the dynamic threshold and the overall confidence score is higher than the preset 0.85 will an "anomaly" judgment result and a detailed parameter deviation report be generated.

[0021] The dynamic decision-making and control instruction generation module generates differentiated handling instructions based on AI analysis results. This module receives judgment results from a multimodal AI visual analysis engine. When the judgment result is "normal," the module generates a facility status record, attaching a timestamp, geographical location, and detection confidence level, and archives it to the facility's historical database. When the judgment result is "abnormal," the module initiates tiered decision-making logic.

[0022] First, based on the severity of parameter deviations, anomalies are categorized into three levels: Level 1 is minor deviation, Level 2 is moderate non-compliance, and Level 3 is severe defect or omission. Second, the module connects to the facility asset database to query the ownership unit, maintenance responsibility entity, and historical maintenance records of the anomaly facility. Based on the anomaly level, responsibility entity information, and preset maintenance response time limits, the module automatically generates clearly targeted control instructions.

[0023] For Level 1 anomalies, an "Early Warning Notification" containing specific deviation data is generated and pushed to the responsible entity's daily inspection terminal via the government messaging platform. For Level 2 anomalies, a "Rectification Order within a Time Limit" is generated, clearly requiring the responsible entity to conduct on-site verification and provide a response plan within 72 hours. This order is also copied to the superior regulatory unit. For Level 3 anomalies, an "Emergency Response Order" and an "Administrative Enforcement Recommendation" are immediately generated. The "Emergency Response Order" requires the responsible entity to take temporary protective measures within 24 hours, while the "Administrative Enforcement Recommendation" is automatically pushed to the urban management administrative enforcement platform, recommending the initiation of an investigation.

[0024] The end-to-end closed-loop management module ensures traceability, supervision, and evaluability throughout the entire process from problem discovery to handling and acceptance. This module constructs a blockchain-based work order storage and transfer chain. All work orders and instructions issued by the dynamic decision-making and control instruction generation module have their generation time, content, and issuance logic hash value recorded on the blockchain as initial blocks. Every status update, on-site photo feedback, and repair result report between the responsible party, maintenance unit, supervision unit, and regulatory unit is added to the blockchain as new transaction data after consensus verification, forming an immutable process log.

[0025] The module also provides a visual monitoring interface, displaying the real-time status of all accessibility facilities in the form of a map layer, with different colors marking statuses such as normal, warning, rectification in progress, and pending acceptance. For work orders that have not been processed within the time limit, the system automatically triggers an escalation reminder program and accumulates the credit score of the responsible party according to preset rules. This credit score will serve as a reference for future allocation of maintenance resources and performance evaluation.

[0026] Furthermore, the environmental confidence assessment mechanism in the multimodal AI visual analysis engine calculates the comprehensive confidence score as follows: the illumination uniformity index of the current frame image, the visible area ratio of the target facility area, and the coverage completeness of the three-dimensional point cloud on the key geometric feature points of the facility are calculated respectively. After normalizing these three indicators to the range of 0 to 1, they are assigned weights of 0.3, 0.4, and 0.3, respectively. The original confidence score is obtained by weighted summation. Finally, a calibration factor dynamically adjusted based on historical test data is introduced to fine-tune the original confidence score to output the final comprehensive confidence score.

[0027] Furthermore, the spatial geocoding engine in the geographic information fusion and standardization adaptation module uses the ray method to calculate spatial inclusion relationships. The engine uses the current latitude and longitude coordinates as a test point, and emits a ray horizontally to the right from the point. It calculates the number of intersections between the ray and the boundary of each geographic region polygon in the standard knowledge graph. If the number of intersections is odd, it is determined that the test point is located within the polygon region. This calculation process is performed in parallel for all leaf node regions in the knowledge graph until a matching administrative division level is found.

[0028] Furthermore, when the dynamic decision-making and control instruction generation module generates control instructions, its maintenance response time limit rule is a configurable matrix; the horizontal axis of the matrix represents the anomaly level from 1 to 3, and the vertical axis represents the high, medium, and low pedestrian traffic levels in the area where the facility is located; each cell in the matrix defines the standard response time limit for that level of anomaly in that pedestrian traffic area; the pedestrian traffic level is obtained by mapping real-time data provided by the urban population thermal big data platform.

[0029] Furthermore, the credit score accumulation rules in the full-process closed-loop management module are as follows: an initial credit score of 100 points is set for each responsible entity; 2 points are deducted for each day a work order is not processed within the time limit; 1 point is added back for each work order that passes inspection and rectification; if a work order is generated repeatedly for the same facility and the same problem, the deduction for the second work order is doubled; when the credit score is below 80 points, the system automatically marks the entity as a key supervision target, and any new work orders generated by the entity will be directly upgraded to an abnormal level for processing.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention fundamentally changes the traditional fixed-threshold judgment mode by constructing a multi-level normative knowledge graph and dynamically integrating it with real-time geolocation. The system can automatically and accurately invoke and apply the specific regulatory standards applicable to that geographical location based on the real-time location of the inspection vehicle or equipment, achieving intelligent dynamic adaptation of compliance judgment benchmarks. This design fundamentally solves the problem of high misjudgment rate caused by inconsistent standards when the system is deployed across regions, significantly improving the system's universality and regulatory compliance accuracy.

[0031] 2. This invention innovatively employs a multimodal data fusion analysis mechanism combining optical images and LiDAR point clouds, and introduces environmental confidence assessment. Through spatiotemporal alignment, cross-modal feature fusion, and comprehensive confidence calculation based on illumination, occlusion, and point cloud integrity, the system exhibits strong robustness to complex and variable real-world road environments. This effectively overcomes the shortcomings of single-vision detection, which suffers from a sharp performance decline under adverse lighting conditions and partial occlusion, ensuring the reliability and stability of state detection and analysis results.

[0032] 3. This invention constructs a complete technology chain from intelligent detection to hierarchical decision-making and closed-loop management. The system not only detects anomalies but also automatically generates differentiated and executable control instructions based on the severity of the anomaly and the responsible party for the facility, placing the entire process under the supervision of blockchain-based evidence storage. This design upgrades the traditional "detection-reporting" model to an intelligent closed-loop management system of "perception-analysis-decision-execution-supervision," significantly improving the automation level, efficiency, and transparency of municipal facility management, and achieving refined and intelligent management of the entire lifecycle of barrier-free facility operation and maintenance. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the overall technical solution architecture of the intelligent inspection and control system for barrier-free facilities on highways and municipal roads based on AI visual recognition proposed in this invention. Figure 2 This is a schematic diagram of the core principle framework of the multimodal AI visual analysis engine in this invention; Figure 3 This is a logical flowchart of the geographic information fusion and standard adaptation module in this invention; Figure 4 This is a schematic diagram of the hierarchical decision-making logic of the dynamic decision-making and control instruction generation module in this invention; Figure 5 This is a flowchart of the work order storage and supervision process of the closed-loop management module in this invention. Detailed Implementation

[0034] This invention provides an intelligent inspection and control system for barrier-free facilities on highways and municipal roads based on AI visual recognition. Please refer to the appendix. Figures 1 to 5 This system is a comprehensive technology system integrating data acquisition, geographic information fusion, artificial intelligence analysis, intelligent decision-making, and closed-loop management. It aims to achieve automated and intelligent inspection and operation management of various barrier-free facilities within highway and municipal road areas, such as curb ramps, tactile paving, and pedestrian crossing audio prompts. The following, in conjunction with the accompanying diagrams, provides a detailed explanation of each component of the system and its collaborative workflow.

[0035] The multi-source data acquisition module is the input terminal for the system to perceive the physical world, responsible for acquiring raw multimodal data of the inspected target facilities in real time and synchronously. This module is usually integrated on a mobile inspection platform, which can be a dedicated inspection vehicle, inspection robot, or drone equipped with relevant equipment. The module integrates three core sensing units: a high-resolution optical imaging unit, a lidar point cloud acquisition unit, and an inertial measurement unit.

[0036] The high-resolution optical imaging unit includes at least one visible light camera and one infrared thermal imaging camera. The visible light camera continuously acquires a sequence of color images along the inspection route at a fixed frame rate of 30 frames per second, with a resolution of no less than 1920 x 1080 pixels, clearly capturing visual information such as the texture, color, and surface markings of the facilities. The infrared thermal imaging camera operates synchronously to provide supplementary thermal radiation information under low-light conditions or when inspecting facilities made of specific materials. The lidar point cloud acquisition unit uses a 905-nanometer wavelength laser emitter, emitting and receiving hundreds of thousands to millions of laser points per second into the surrounding environment through mechanical rotation or solid-state scanning, forming high-density three-dimensional spatial point cloud data.

[0037] Each point cloud data point contains its three-dimensional coordinates in the lidar's own coordinate system and reflection intensity information. The inertial measurement unit consists of a three-axis accelerometer and a three-axis gyroscope, continuously measuring and outputting the linear acceleration of the inspection vehicle in three orthogonal directions and its angular velocity around the three axes at a frequency of 100 Hz. The data acquisition times of these three units must be strictly aligned, which is the basis for subsequent multimodal data fusion analysis. To this end, the system adopts a time synchronization mechanism based on a precision hardware clock.

[0038] All sensing units are connected to a unified high-precision clock source, which is periodically calibrated using timing signals from the Global Navigation Satellite System. Each data packet is stamped with a microsecond-level timestamp from this clock source upon generation. The data preprocessing stage first checks the consistency of the timestamps of each data stream. Data frames with a deviation exceeding 1 millisecond are marked or time alignment correction is performed using interpolation algorithms to ensure that image frames, point cloud frames, and inertial measurement unit data acquired at the same time correspond completely in the time dimension.

[0039] The geographic information fusion and standards adaptation module provides the entire system with precise geographic context and dynamically changing compliance judgment benchmarks. Please refer to the appendix. Figure 3 The core of this module is a structured, multi-level, canonical knowledge graph. Logically, this knowledge graph presents itself as a tree-like hierarchical structure.

[0040] The root node of the tree represents nationally promulgated general accessibility design standards, such as the basic principles and general requirements in the "Accessibility Design Standard". Downward from the root node, the first level of child nodes corresponds to each provincial-level administrative division. Each provincial-level node is associated with the implementation rules or supplementary regulations formulated by that province based on the national standards. Provincial-level nodes are further subdivided into city-level nodes, and city-level nodes can even be associated with the personalized management requirements of specific administrative divisions such as districts and sub-districts. Each specific clause of the standard, as a leaf node or attribute node in the knowledge graph, is stored in a structured data format.

[0041] A complete record of a standard provision includes the following key fields: a unique identifier for the standard provision, which typically consists of the standard code, chapter number, and provision number; a sequence of polygonal boundary coordinates for the applicable geographical area, which defines the geographical scope in which this standard applies, usually represented by a closed polygon formed by connecting a series of latitude and longitude coordinates in sequence; a facility type code, a standardized code used to identify the type of facility to which this standard applies, for example, code A01 represents a curb ramp and A02 represents a tactile paving; and specific quantitative technical parameter thresholds, which is a data structure containing multiple key-value pairs, for example, for a curb ramp, the key-value pairs might include "maximum slope angle: 8 degrees", "minimum net width: 1200 mm", and "minimum platform depth: 1500 mm".

[0042] During system operation, this module continuously receives positioning data streams from the Global Navigation Satellite System (GNSS) attached to the multi-source data acquisition module. The positioning data includes at least the current latitude and longitude coordinates, altitude, and positioning accuracy factor. The spatial geocoding engine is the core computational component within the module, responsible for performing rapid spatial inclusion relationship calculations. Upon receiving the current latitude and longitude coordinates, the engine treats them as a test point on a two-dimensional plane. The calculation employs the classic ray casting algorithm. The algorithm emits an infinitely extending ray horizontally to the right from the test point. Then, the algorithm calculates in parallel the number of intersections between this ray and the boundaries of the geographic region polygons associated with all leaf nodes in the canonical knowledge graph.

[0043] When calculating the number of intersections, it is necessary to accurately handle special cases where rays pass through polygon vertices and coincide with polygon edges. If the calculated number of intersections for a specific polygon is odd, the test point is logically determined to be inside the polygon; if it is even, it is determined to be outside. Through this parallel computation, the engine can quickly locate the highest-precision administrative division level of the current inspection location, such as the jurisdiction of a specific street. After successful location, the module dynamically loads and activates all applicable normative clauses related to the current inspection task from the knowledge graph.

[0044] These activated clauses and their parameter thresholds together constitute a temporary, location-specific set of compliance determination rules. This set of rules is encapsulated in real time into a standardized data exchange format, such as JSON or Protocol Buffers, and pushed to the multimodal AI vision analysis engine via an internal message bus as a dynamic benchmark for its compliance determination.

[0045] The multimodal AI visual analysis engine is the intelligent core of this system, responsible for analyzing facility status from raw multimodal data and making compliance determinations. Please refer to the appendix. Figure 2The engine employs a meticulously designed cascaded multi-task neural network architecture, whose data processing flow can be divided into three logically interconnected sub-modules: a data preprocessing and alignment sub-module, a feature extraction and fusion sub-module, and a compliance determination sub-module. The data preprocessing and alignment sub-module first receives the raw data stream from the multi-source data acquisition module. Its primary task is to complete the spatiotemporal alignment and enhancement of the multi-source data.

[0046] For image data, the submodule utilizes synchronously received inertial measurement unit (IMU) data to compensate for image blur caused by carrier movement using a digital image stabilization algorithm. The algorithm estimates the image sensor's trajectory during exposure based on the angular velocity measured by the gyroscope and performs reverse motion compensation in the image domain, resulting in a clearer image. For LiDAR point cloud data, the submodule similarly utilizes IMU data and a point cloud motion distortion correction algorithm to correct distortions within a single frame of point cloud caused by carrier movement.

[0047] After completing their respective time synchronization corrections, the submodules perform the crucial sensor fusion and alignment operation. Using a pre-calibrated camera-LiDAR extrinsic parameter matrix, it projects the 3D point cloud data onto a 2D image plane, assigning each pixel in the image its corresponding 3D spatial coordinates, thus establishing a precise correspondence between image pixels and 3D point cloud points. This forms spatiotemporally aligned, pixel-point cloud-associated data pairs.

[0048] The feature extraction and fusion submodule receives the aligned data pairs described above and initiates two parallel deep neural network branches for feature learning. The two-dimensional image branch employs a deep convolutional neural network, for example, based on... The backbone network of the architecture. This network takes an aligned color image as input and extracts visual features from low to high levels through multiple layers of convolution, pooling, and nonlinear activation functions. These features encode the edges, textures, color distribution, and two-dimensional morphological contours of the facility. The network ultimately outputs a high-dimensional feature vector, denoted as... .

[0049] The 3D point cloud branch processes the aligned 3D point cloud data. Since point clouds are unordered and unstructured, this branch first uses a voxelization method to divide the 3D space into a regular voxel grid, and then performs statistical feature aggregation on the point cloud falling within each voxel, such as calculating the average coordinates and density of points within the voxel, thereby converting the irregular point cloud into a regular 3D voxel tensor.

[0050] Subsequently, a 3D convolutional neural network processes the voxel tensor to extract the facility's 3D geometric features, such as surface curvature, plane normals, and overall spatial dimensions. This branch ultimately outputs another high-dimensional feature vector, denoted as... Next, the feature fusion network begins to work. It receives... and As input, the fusion is not a simple concatenation, but rather employs a cross-modal attention mechanism. This mechanism first calculates the correlation matrix between the two feature vectors, and then assigns appropriate weights to feature channels from different modalities based on the correlation.

[0051] Specifically, for tasks sensitive to appearance details, the weight of image features is adaptively increased; for tasks sensitive to geometric dimensions and spatial relationships, the weight of point cloud features dominates. After weighted fusion, a unified feature descriptor rich in multimodal information is generated, denoted as... This descriptor combines the facility's appearance attributes with its precise geometric attributes.

[0052] The compliance determination submodule is the final step in making the judgment. This submodule receives unified feature descriptors from both the feature extraction and fusion submodules. The system also includes a dynamic rule set from the geographic information fusion and standardization adaptation module. Its decision-making process is a rigorous multi-stage reasoning process. The first stage is key parameter parsing. The submodule contains a series of parameter parsing heads for different facility types. Based on the facility type code specified in the rule set, the corresponding parsing head is activated. This parsing head is essentially a small, fully connected neural network. Using the input as input, the system directly regresses to output the key metric values ​​of the facility.

[0053] For example, for a curb ramp, the parsing head outputs three floating-point numbers, representing the parsed slope angle, net width, and platform depth, respectively. The second stage is dynamic threshold comparison. The submodule compares the parsed parameter values ​​one by one with the corresponding dynamic thresholds in the rule set. For example, it compares the parsed slope angle with the "maximum slope angle" threshold in the rule. At this point, the comparison result is a Boolean value or a deviation. However, the system does not immediately draw a conclusion based on this Boolean value, but instead proceeds to the third stage: environmental confidence assessment. The environmental confidence assessment mechanism aims to quantify the reliability of this detection to address interference from complex on-site environments.

[0054] The mechanism calculates three independent metrics: image illumination uniformity, which is evaluated by calculating the ratio of the standard deviation to the mean of the grayscale image of the current image frame. The more uniform the illumination, the closer the value of this metric is to 0; the visible area ratio of the target facility, which is calculated by semantic segmentation of the facility area to the total number of pixels in the facility area; and point cloud coverage integrity, which is calculated by checking whether the key geometric feature points of the facility are effectively captured in the point cloud, such as checking whether there is point cloud coverage on the slope of a ramp.

[0055] These three indicators were normalized to a range of 0 to 1, where 1 represents the optimal condition. Then, the system assigned fixed weights to these three indicators: illumination uniformity (0.3), visible area percentage (0.4), and point cloud integrity (0.3). The original confidence scores were calculated by weighted summation. The system also maintains dynamic calibration factors based on historical data. This factor is periodically adjusted based on the degree of agreement between manual verification results and system judgment results under similar environmental conditions over a past period. The final overall confidence score... The result is calculated using the formula: ; in It is a floating-point number close to 1, used for fine-tuning the confidence scale. The submodule will only generate an "anomaly" determination result if the following two conditions are met: First, the resolved value of at least one key metric parameter exceeds the range allowed by the dynamic threshold; second, the final comprehensive confidence score... The value is higher than the system's preset rigid threshold of 0.85. If it is determined to be "normal", a normal status record is generated; if it is determined to be "abnormal", an abnormal report is generated, which details the type of abnormal facility, its location, the parameters of deviation and their specific values, the amount exceeding the threshold, and the overall confidence score of this detection.

[0056] The dynamic decision-making and control instruction generation module is responsible for transforming AI analysis results into executable management actions. Please refer to the appendix. Figure 4 This module receives judgment reports from a multimodal AI visual analysis engine. Its internal operation follows a clear hierarchical decision-making logic. When a report is judged as "normal," the module automatically generates a standardized facility status log. This log includes the facility's unique identifier, detection timestamp, precise geographical coordinates, the code clause number used during detection, values ​​of various analytical parameters, and a detection confidence score. This log is compressed and encrypted before being transmitted to the backend facility history database for permanent archiving and for facility health trend analysis. When a report is judged as "abnormal," the module immediately initiates anomaly handling procedures.

[0057] The first step in the process is anomaly classification. The module has a pre-defined anomaly classification matrix, which takes the severity of parameter deviations as input. For example, for a slope angle, a deviation within 1 degree might be classified as Level 1 (Slight Deviation), a deviation between 1 and 3 degrees as Level 2 (Moderate Deviation), and a deviation exceeding 3 degrees or complete missing facilities as Level 3 (Severe Defect). The system synthesizes the levels of all anomaly parameters and takes the highest level as the final classification for the anomaly event.

[0058] The second step is to associate the responsible parties. The module queries the facility asset database and matches the ownership unit and maintenance responsibility entity information based on the geographical location of the abnormal facility. The asset database records the ownership unit, contact person, contact number, and historical maintenance records for each facility segment. The third step is to generate differentiated control instructions. The module has a configurable maintenance response time limit rule matrix. This matrix is ​​a two-dimensional lookup table, with the horizontal axis representing the abnormality level (1 to 3) and the vertical axis representing the real-time pedestrian traffic level (high, medium, and low) of the area where the facility is located.

[0059] The pedestrian flow level is not a fixed value, but rather dynamically mapped from real-time data provided by the city's population heat map big data platform via an interface, based on the surrounding mobile phone signaling density or video surveillance pedestrian flow statistics. Each cell in the matrix defines the standard response time limit for that scenario. For example, for a Level 2 anomaly, high pedestrian flow area, the response time limit might be 48 hours; for the same level, low pedestrian flow area, the time limit might be extended to 96 hours. Based on the anomaly level, responsible party, and dynamic time limit, the module automatically generates specific instructions.

[0060] For Level 1 anomalies, an "Early Warning Notification" is generated, containing details of the anomaly and a self-inspection reminder, and is pushed to the mobile terminal of the designated contact person in the responsible unit via government WeChat or SMS platform. For Level 2 anomalies, a mandatory "Rectification Order within a Time Limit" is generated, clearly requiring the responsible party to conduct on-site verification, formulate a plan, and submit feedback through the system within a specified time limit. This order is also automatically copied to the business system of the district-level municipal facilities supervision unit. For Level 3 anomalies, the module generates two documents in parallel: one is an "Emergency Response Instruction," requiring the responsible party to arrive at the scene within 24 hours to set up temporary warning signs or take physical isolation and other protective measures, and upload on-site photos through the system; the other is an "Administrative Enforcement Recommendation," detailing the violation and applicable legal provisions, and is automatically pushed to the digital case-handling platform of the Urban Management Administrative Enforcement Bureau, recommending that it initiate a formal case investigation procedure. All generated instructions have a unique serial number and digital signature.

[0061] The end-to-end closed-loop management module ensures that the entire management chain, from problem discovery to final acceptance, is transparent, traceable, and subject to supervision. Please refer to the appendix. Figure 5 The core innovation of this module lies in the construction of a work order storage and circulation chain based on blockchain technology. When the dynamic decision-making and control instruction generation module issues any type of work order or instruction, the complete content of the instruction, the generation time, the hash value of the issuance logic, the hash value of the associated exception report, and the initial status "issued" are packaged together to form the initial data block.

[0062] After verification by the pre-set consensus nodes within the module, the block is added to the permissioned blockchain specifically built for this system, completing the initial, tamper-proof notarization. Following this, the work order enters its lifecycle. The responsible entity receives the work order through its terminal application, verifies it on-site, and updates the status to "accepted," while also uploading on-site photos as evidence. This operation generates a new transaction, which is then signed and uploaded to the blockchain. After completing the repair, the repair unit submits a repair report and completion photos, updating the status to "pending acceptance." The supervision or regulatory unit conducts on-site or online acceptance; if it confirms compliance, the status is updated to "accepted"; otherwise, it is returned for "rectification."

[0063] Every status change, every file upload, and every operator signature and timestamp is permanently recorded as an on-chain transaction, forming a complete and reliable process log. The module also provides a panoramic visual monitoring interface. This interface uses an electronic map as its base, displaying information through overlaying different layers. All accessibility facilities are rendered as different colored pins or blocks based on their latest status: green represents normal, yellow represents Level 1 warning, orange represents Level 2 rectification in progress, red represents Level 3 emergency anomaly, and blue represents pending acceptance. Supervisors can easily grasp the overall situation. For any work order, if its current status fails to progress within the response time limit specified by the rule matrix, the system automatically triggers an escalation reminder procedure.

[0064] The initial alert is sent to the directly responsible entity, the second alert is copied to its superior authority, and the third alert is reported to the highest regulatory level. Simultaneously, the system strictly enforces the credit score accumulation rule. Each responsible entity registered in the system has an initial credit score account with an initial value of 100 points. For each day a work order remains unprocessed, the system automatically deducts 2 points from its account. If the work order ultimately passes acceptance, the system adds 1 point as a positive incentive.

[0065] If the same facility generates multiple work orders for the same issue within a short period, the point deduction rule will double starting from the second work order; for example, 4 points will be deducted for each day of delay. When a responsible entity's credit score falls below 80 points, the system automatically marks it as a "key monitoring target." Subsequently, any new anomalies generated by this entity will have their anomaly level automatically upgraded by one level before a control instruction is generated; for example, a level 1 anomaly will be processed according to the level 2 anomaly procedure, thus applying stronger management pressure. All credit score data is publicly available and transparent, and serves as an important reference for annual performance evaluations and the allocation of maintenance funds.

[0066] The environmental confidence assessment mechanism in the multimodal AI visual analysis engine calculates its comprehensive confidence score through a multi-indicator fusion quantitative evaluation. Specifically, the system first calculates three basic indicators. The image illumination uniformity indicator is calculated by converting the current frame's color image to a grayscale image and calculating the standard deviation of the intensity of all pixels in the entire grayscale image. with the mean The ratio of the two values, i.e., the coefficient of variation, is an index of light uniformity. The saturation function value is defined as 1 minus the coefficient of variation, ensuring it falls between 0 and 1. The calculation of the visible area percentage of the target facility relies on a semantic segmentation model, which accurately segments the target facility area in the image and identifies the portion of that area occluded by other objects, thus determining the visible area percentage. The completeness of the 3D point cloud coverage is calculated by dividing the number of pixels in the unobstructed area of ​​the facility by the total number of pixels in the facility area. Calculating the completeness of the 3D point cloud coverage requires a 3D keypoint template of the facility, such as a 3D corner model of a standard curb ramp. It is equal to the ratio of the number of key points that can be successfully matched in the point cloud scanned at the current location to the total number of key points in the template.

[0067] These three indicator values , , Normalization was performed to ensure they were on the same scale. Then, the system assigned them fixed weight coefficients of 0.3, 0.4, and 0.3 respectively. Original confidence scores. Calculated using the weighted summation formula: ; in, Indicators representing the uniformity of illumination. This indicates the percentage of the target facility's visible area. Indicates the completeness of 3D point cloud coverage. This represents the original confidence score.

[0068] Finally, the system introduces a dynamic calibration factor. This factor is obtained through continuous monitoring of historical data: the system periodically extracts a batch of test records, compares and analyzes the original system confidence scores of these records with their corresponding manual verification results, and calculates the deviation correction coefficient using linear regression or moving average methods. . It typically fluctuates slightly around 1.0, such as 0.98 or 1.02, to ensure that the system confidence assessment aligns with human experience. The final output is the overall confidence score. That is and The product of.

Claims

1. An AI vision recognition-based intelligent inspection and management system for barrier-free facilities on highways and municipal roads, characterized in that, Comprise: A multi-source data acquisition module for real-time acquisition of original data of inspection targets; A geographic information fusion and specification adaptation module for providing accurate geographic context and compliance determination benchmarks for visual analysis; A multi-modal AI visual analysis engine for fusion analysis and state determination of collected original data; A dynamic decision-making and control instruction generation module for generating differentiated treatment instructions based on AI analysis results, and a full-process closed-loop management module for ensuring traceability, supervision, and evaluation of the entire process from problem discovery to treatment acceptance.

2. The AI vision recognition-based highway and municipal road barrier-free facility intelligent inspection and management system of claim 1, wherein The multi-source data acquisition module integrates a high-resolution optical imaging unit, a laser radar point cloud acquisition unit, and an inertial measurement unit. The optical imaging unit acquires color images and infrared thermal imaging data of barrier-free facilities on the inspection route at a rate of 30 frames per second. The laser radar point cloud acquisition unit synchronously emits a laser beam with a wavelength of 905 nanometers and acquires three-dimensional spatial point cloud data of the facility at a density of 200,000 points per second. The inertial measurement unit continuously records the acceleration and angular velocity changes of the inspection carrier in three-dimensional space, and its data output frequency is 100 Hz. The optical imaging unit, laser radar point cloud acquisition unit, and inertial measurement unit achieve strict alignment of data acquisition times through a hardware time synchronization mechanism. 3.The AI vision-recognized highway and municipal road barrier-free facility intelligent inspection and management system of claim 2, characterized in that, The geographic information fusion and specification adaptation module has a multi-level specification knowledge graph organized in a tree structure. The root node of the multi-level specification knowledge graph is the national universal accessibility design specification, and below it are the provincial, municipal, and even specific administrative division specification provisions. Each specification provision is structured and stored, including specification item number, polygon boundary coordinates of applicable geographic area, facility type code, and specific quantitative technical parameter threshold. The geographic information fusion and specification adaptation module receives global satellite navigation system positioning data from the multi-source data acquisition module in real time, and through a spatial geographic coding engine, calculates the spatial inclusion relationship between the latitude and longitude coordinates of the current inspection location and the geographic area polygons in the specification knowledge graph to dynamically load and activate all relevant specification provisions applicable to the current geographic location, forming a location-specific compliance determination rule set.

4. The AI vision recognition-based highway and municipal road barrier-free facility intelligent inspection and management system of claim 3, characterized in that, The multi-modal AI visual analysis engine adopts a cascaded multi-task neural network architecture and includes a data preprocessing and alignment submodule, a feature extraction and fusion submodule, and a compliance judgment submodule. The data preprocessing and alignment submodule performs spatio-temporal alignment correction on received image data, point cloud data, and inertial measurement unit data, compensates for image blur caused by carrier motion using inertial measurement unit data, and assigns accurate three-dimensional coordinates to each pixel in the image based on the point cloud data. The feature extraction and fusion submodule includes two parallel deep convolutional neural network branches. One branch processes the aligned two-dimensional image data to extract color, texture, and two-dimensional morphological features. The other branch processes three-dimensional point cloud data and extracts spatial geometric structure features through point cloud voxelization and three-dimensional convolution operations. Subsequently, a feature fusion network performs weighted fusion of feature vectors from the two-dimensional and three-dimensional branches through a cross-modal attention mechanism to generate a unified feature descriptor containing appearance and geometric information. 5.The AI vision-recognized highway and municipal road barrier-free facility intelligent inspection and management system according to claim 4, characterized in that, The compliance judgment submodule receives the unified feature descriptor and a set of compliance judgment rules from the geographic information fusion and specification adaptation module, and performs a multi-stage reasoning process. The multi-stage reasoning process includes a first stage of resolving key metric parameters from the unified feature descriptor according to facility type encoding, a second stage of comparing the resolved parameter values with corresponding dynamic thresholds in the compliance judgment rule set, and a third stage of introducing an environmental confidence evaluation mechanism to calculate the comprehensive confidence score of this detection. The compliance judgment submodule generates an abnormality judgment result and a detailed parameter deviation report when the metric parameters exceed the dynamic thresholds and the comprehensive confidence score is higher than the preset 0.

85. 6.The AI vision-recognized highway and municipal road barrier-free facility intelligent inspection and management system according to claim 5, characterized in that, The dynamic decision and control instruction generation module receives the judgment result from the multi-modal AI visual analysis engine. When the judgment result is normal, it generates a facility state record with a timestamp, geographic location, and detection confidence and archives it to the facility history database. When the judgment result is abnormal, it starts a hierarchical decision logic. The hierarchical decision logic includes steps of dividing the abnormality into three levels according to the severity of the parameter deviation, querying the property unit and maintenance responsibility subject of the abnormal facility and the historical maintenance records by associating the facility asset database, and automatically generating a control instruction with a clear direction based on the abnormality level, responsibility subject information, and preset maintenance response time limit rules. 7.The AI vision-recognized highway and municipal road barrier-free facility intelligent inspection and management system according to claim 6, characterized in that, The full-process closed-loop management module constructs a work order storage and circulation chain based on blockchain. The generation time, content, and signature logic hash value of all work orders and instructions issued by the dynamic decision and control instruction generation module are stored as initial blocks. Each state update, on-site photo feedback, and maintenance result reporting operation of the work order between the responsibility subject, maintenance unit, supervision unit, and supervision unit are added to the chain as new transaction data after consensus verification to form an unalterable process log. 8.The AI vision-recognized highway and municipal road barrier-free facility intelligent inspection and management system according to claim 1, characterized in that, The process of calculating the comprehensive confidence score by the environment confidence evaluation mechanism is as follows: the illumination uniformity index of the current frame image, the area proportion of the visible part of the target facility area, and the coverage completeness of the three-dimensional point cloud to the facility key geometric feature points are calculated respectively; After normalizing the three indexes to the interval of 0 to 1, weights of 0.3, 0.4, and 0.3 are respectively assigned; the original confidence score is calculated by weighted summation; Finally, a calibration factor based on historical detection data is introduced to fine-tune the original confidence score to output the final comprehensive confidence score. 9.The AI vision-recognized highway and municipal road barrier-free facility intelligent inspection and management system according to claim 1, characterized in that, The spatial geocoding engine adopts a ray method to realize spatial containment relationship calculation; The spatial geocoding engine constructs a test point from the current latitude and longitude coordinates, emits a ray horizontally to the right from the point, and calculates the number of intersection points of the ray with the polygon boundary of each geographic area in the standard knowledge graph; If the number of intersection points is odd, it is determined that the test point is located in the polygon area; The calculation process is executed in parallel for all leaf node areas in the knowledge graph until the matching administrative division level is found. 10.The AI vision-recognized highway and municipal road barrier-free facility intelligent inspection and management system according to claim 1, characterized in that, The maintenance response time limit rule is a configurable matrix; The horizontal axis of the matrix is the abnormality level 1-3 level, and the vertical axis is the passenger flow level of the area where the facility is located, high, medium, and low; Each cell in the matrix defines the standard response time limit of the abnormality of this level in this passenger flow area; The passenger flow level is obtained by mapping real-time data provided by the city population heat data platform.

Images