A slope disaster collaborative perception method integrating edge side anomaly capture and cloud vision and language semantic verification

Abstract

The application discloses a slope disaster collaborative perception method integrating edge side anomaly capture and cloud vision and language semantic verification, comprising the following steps: S1, acquiring a monitoring image with position information of an easy-disaster area in a highway section by using an edge side computing device; S2, performing key feature extraction and enhancement on the acquired image to generate a structure-level saliency graph; S3, based on the generated structure-level saliency graph, identifying the slight geometric deviation and morphological mutation of the saliency structure on the time axis by using a key feature displacement tracking, deformation fitting and target point coordinate drift analysis method, and generating an abnormal monitoring result with position information; S4, uploading the generated monitoring result with position information to a server carrying a cloud large model through communication setting of the edge side computing device and the cloud server to determine the disaster event; and S5, reporting the determination result to an external management center according to the type and emergency degree of the event.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation, specifically to a collaborative perception method for slope disasters that integrates edge-side anomaly capture with cloud-based visual and linguistic semantic verification. Background Technology

[0002] In today's rapidly developing transportation era, road safety has become a critical issue concerning the lives of every traveler. With the acceleration of urbanization and the continuous increase in the number of motor vehicles, the complexity of road conditions is constantly increasing, and the incidence of traffic accidents remains high, bringing irreparable pain to countless families. According to relevant statistics, hundreds of thousands of people die each year globally due to traffic accidents, and countless more are injured. Behind these accidents are often a complex interplay of factors, including poor road conditions, chaotic traffic flow, the impact of severe weather, and the disregard for traffic rules. For example, road surface cracks can lead to loss of vehicle control, and speeding increases the risk of accidents due to traffic safety issues; disorderly traffic flow and sudden abnormal events constantly threaten the safety of road users. To achieve rapid detection of road damage events, we need to build a high-precision real-time road condition recognition system that integrates multi-source data, including visual (camera) and IoT sensors. Summary of the Invention

[0003] The purpose of this application is to provide a collaborative perception method for slope disasters that integrates edge-side anomaly detection with cloud-based visual and semantic verification. The specific technical solution is as follows:

[0004] A collaborative perception method for slope disasters integrating edge-side anomaly detection and cloud-based visual and semantic verification includes: S1, acquiring monitoring images with location information of disaster-prone areas in highway sections using edge-side computing devices; S2, extracting and enhancing key features from the images acquired in S1 to generate a structural saliency map; S3, based on the structural saliency map generated in S2, identifying minute geometric shifts and morphological changes in salient structures over time and generating anomaly monitoring results with location information through key feature displacement tracking, deformation fitting, and target point coordinate drift analysis; S4, uploading the monitoring results with location information generated in S3 to a server carrying a large cloud model for disaster event determination through communication settings between the edge-side computing device and the cloud server; S5, classifying and reporting the determination results in S4 according to the type and urgency of the event to an external management center to initiate corresponding emergency response procedures and record and provide feedback on the event handling process.

[0005] S1 includes: S1.1, delineating disaster-prone areas within the highway section based on historical disaster information; S1.2, planning the installation locations of high-definition cameras according to the characteristics of the highway section, the location of curves, and the distribution of entrances and exits, so that the camera's field of view covers the disaster-prone areas in S1.1; S1.3, marking significant structures within the disaster-prone areas in S1.2; S1.4, adjusting the camera view, establishing a high-speed video transmission link based on a wired network, configuring the corresponding network equipment and network parameters, and continuously collecting road video image data at a preset frame rate.

[0006] S2 includes: S2.1, based on the road video image data output in S1.4, a visual saliency detection network is used to extract salient structures from complex backgrounds in the image data; S2.2, a target detection model is used to handle global target initialization, inter-frame correlation, and state determination, solving the problems of target occlusion, out-of-view, and re-identification, and providing global constraints for continuous monitoring; S2.3, adaptive feature aggregation is used to dynamically weight and fuse multi-level and multi-scale texture / contour / semantic features according to changes in target scale, illumination, and viewpoint, retaining effective features and filtering redundant noise; S2.4, a spatial attention fusion mechanism is used to strengthen the feature weights of key spatial regions of salient structures, suppress background interference, and focus on the details of the structure itself.

[0007] S3 includes: S3.1, obtaining the anchor point location of salient structures based on the structural saliency map generated in S2; S3.2, continuously tracking the anchor point location of salient structures, calculating the horizontal and vertical coordinate drift of a single target point and the relative positional deviation between target points, thereby obtaining the time-series change curve of the coordinate drift of each target point; S3.3, based on the structural saliency map generated in S2 and the time-series change curve of the coordinate drift of each target point in S3.2, identifying the small geometric shifts and morphological changes of salient structures on the time axis; S3.4, based on the small geometric shifts and morphological changes of salient structures on the time axis in S3.3, performing anomaly judgment according to a preset threshold and generating anomaly monitoring results with location information.

[0008] The cloud-based large model in S4 adopts the Transformer architecture. Through a self-attention mechanism, it comprehensively analyzes the significant structural anchor points, deformation, and traffic scene context of the slope globally, and performs semantic-level discrimination of disaster events to improve the accuracy of identification of complex scenes and low-frequency disaster events. When the large model confirms that the abnormal event is a real road disaster or traffic anomaly, it generates a final monitoring report.

[0009] The beneficial effect of this application lies in addressing the common problems in existing highway slope disaster monitoring: lightweight models offer strong real-time performance but low accuracy, while multimodal large language models offer high accuracy and semantic understanding but poor real-time performance. This application proposes a collaborative slope disaster perception method integrating edge-side anomaly capture and cloud-based visual-linguistic semantic verification. At the edge, a small neural network model is deployed, leveraging its lightweight and low-latency characteristics to monitor traffic data in real time and provide rapid early warnings, capturing potential slope disasters at the earliest opportunity. At the terminal, a finely tuned traffic large language model is introduced. After receiving early warning information from the small model, it performs in-depth analysis and confirmation of complex and ambiguous traffic events, comprehensively understanding the essence of the event by combining multimodal information. This edge-end collaborative architecture, with its complementary advantages of small and large models, ensures both the timeliness of slope disaster detection and improves the accuracy and reliability of the detection results. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the application process. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of this application. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0012] like Figure 1 As shown, a collaborative perception method for slope disasters integrating edge-side anomaly detection with cloud-based visual and linguistic semantic verification includes:

[0013] S1. Use edge computing devices to acquire monitoring images with location information of disaster-prone areas in the highway segment. Specifically, this includes: S1.1. Delineating disaster-prone areas in the highway segment based on historical disaster information; S1.2. Planning the installation positions of high-definition cameras according to the characteristics of the highway segment, the location of curves, and the distribution of entrances and exits, so that the camera's field of view covers the disaster-prone areas in S1.1; S1.3. Marking significant structures within the disaster-prone areas in S1.2; S1.4. Adjusting the camera view, establishing a high-speed video transmission link based on a wired network, configuring the corresponding network equipment and network parameters, and continuously acquiring road video image data at a preset frame rate. In this application, the installation locations of high-definition cameras are rationally planned based on factors such as the characteristics of highway sections, the location of curves, and the distribution of entrances and exits, ensuring that the camera's field of view covers key areas prone to disasters. To effectively monitor dynamic changes in slopes, the presence of prominent structures such as drainage channels, guardrails, and protective structures is first observed. If present, these are considered prominent slope structures. If absent, targets are set in areas such as high slopes and high roadbed ends, serving as anchor points for monitoring. High-definition cameras are deployed at the planned locations, and their installation angles and focal lengths are adjusted to ensure stable and clear image capture. A high-speed video transmission link based on a wired network is established, and corresponding network equipment and parameters are configured to ensure stable and low-latency transmission of video data to edge computing devices. Road video data is continuously collected at a set frame rate and transmitted in real time to edge computing devices via the data transmission link, providing data support for subsequent road disaster and anomaly detection.

[0014] S2. Extract and enhance key features from the images acquired in S1 to generate a structural saliency map. Specifically, this includes: S2.1. Based on the road video image data output in S1.4, a visual saliency detection network is used to isolate salient structures from complex backgrounds in the image data; S2.2. A target detection model is used to handle global target initialization, inter-frame correlation, and state determination, solving problems such as target occlusion, out-of-view issues, and re-identification, providing global constraints for continuous monitoring; S2.3. Adaptive feature aggregation is used to dynamically weight and fuse multi-level, multi-scale texture / contour / semantic features according to changes in target scale, illumination, and viewpoint, preserving effective features and filtering redundant noise; S2.4. A spatial attention fusion mechanism is used to strengthen the feature weights of key spatial regions of salient structures, suppressing background interference and focusing on the details of the structure itself. In the application, through target monitoring models, visual saliency detection networks, adaptive feature aggregation, and spatial attention fusion mechanisms, key area features are accurately separated and enhanced in complex backgrounds. Key salient structural anchor points such as slope targets, roadbed slopes, guardrails, and drainage channels are located and identified, generating structural saliency maps.

[0015] S3. Based on the structural saliency map generated in S2, the system identifies minute geometric shifts and morphological changes of salient structures on the time axis and generates anomaly monitoring results with location information through key feature displacement tracking, deformation fitting, and target point coordinate drift analysis. Specifically, this includes: S3.1. Obtaining the anchor point location of salient structures based on the structural saliency map generated in S2; S3.2. Continuously tracking the anchor point location of salient structures, calculating the horizontal and vertical coordinate drift of a single target point and the relative positional deviation between target points, thereby obtaining the time-series change curve of the coordinate drift of each target point; S3.3. Based on the structural saliency map generated in S2 and the time-series change curve of the coordinate drift of each target point in S3.2, identifying minute geometric shifts and morphological changes of salient structures on the time axis; S3.4. Based on the minute geometric shifts and morphological changes of salient structures on the time axis in S3.3, performing anomaly judgment according to a preset threshold and generating anomaly monitoring results with location information. In this application, after locating the anchor points of saliently damaged structures, the system performs temporal evolution modeling and change detection of salient visual cues. Through methods such as reference object displacement tracking, guardrail deformation fitting, and target point coordinate drift analysis, it identifies minute geometric shifts and morphological abrupt changes in the facility over time. This system correlates salient visual cues with prior structural information, enabling key algorithms such as slope stability anomaly perception based on salient structural anchor point displacement, achieving full-chain perception from "visually salient anomalies—geometric structural abrupt changes—potential disaster morphology." A lightweight small model is loaded onto an edge computing device to preprocess and perform inference analysis on real-time transmitted video data, enabling rapid capture of slope disaster anomalies and generating detection results.

[0016] S4. Through the communication settings between the edge computing devices and the cloud server, the monitoring results with location information generated in S3 are uploaded to the server equipped with the cloud-based large-scale model for disaster event determination. Specifically, the cloud-based large-scale model adopts a Transformer architecture and uses a self-attention mechanism to comprehensively analyze the salient structural anchor points, deformation, and traffic scene context of the slope globally. It performs semantic-level discrimination of disaster events to improve the accuracy of identifying complex scenes and low-frequency disaster events. When the large-scale model confirms that the abnormal event is a real road disaster or traffic anomaly, it generates a final monitoring report. In application, the location information of the event is collected by the edge devices, and video clips containing the slope disaster are extracted from the video stream. The video clips and location information are then transmitted to the cloud-based large-scale model, which performs visual-linguistic semantic verification and deep analysis on the abnormal event to complete the accurate confirmation of the disaster event. Specifically, the Transformer architecture is adopted, which uses a self-attention mechanism to comprehensively analyze the context of significant slope structural anchor points, deformation, traffic scenarios, etc., in a global scope, and performs semantic-level discrimination of disaster events to improve the accuracy of identification of complex scenarios and low-frequency disaster events. When the large model confirms that the abnormal event is a real road disaster or traffic anomaly, a final detection report is generated and the results are fed back to the event fusion and confirmation service module.

[0017] S5. Report the judgment results in S4 to the external management center according to the type and urgency of the event, so as to initiate the corresponding emergency response process, and record and provide feedback on the event handling process.

[0018] In practical applications, S1-S3 are executed through small neural network models deployed in edge computing devices. Leveraging their lightweight and low-latency characteristics, they perform real-time monitoring and rapid early warning of traffic data, capturing potential slope damage anomalies at the earliest possible moment. S4-S5 transmit the anomaly information captured by the edge computing devices to the terminal. The terminal then incorporates a finely tuned traffic language model, receiving early warning information from the small models and performing in-depth analysis and confirmation of complex and ambiguous traffic events, comprehensively understanding the nature of the event by combining multimodal information. This edge-end collaborative architecture, with small and large models complementing each other, ensures both the timeliness of slope damage detection and improves the accuracy and reliability of the detection results.

Claims

1. A collaborative perception method for slope disasters integrating edge-side anomaly detection with cloud-based visual and linguistic semantic verification, characterized in that, include: S1. Acquire monitoring images with location information of disaster-prone areas in highway sections and send them to edge computing devices; S2. Extract and enhance key features from the image obtained in S1 to generate a structural saliency map; S3. Based on the structural saliency map generated in S2, the system identifies minute geometric shifts and morphological changes of saliency structures on the time axis and generates anomaly monitoring results with location information by using key feature displacement tracking, deformation fitting, and target point coordinate drift analysis methods. S4. Through the communication settings between the edge computing device and the cloud server, the monitoring results with location information generated in S3 are uploaded to the server carrying the cloud-based large model for disaster event determination. S5. The judgment results in S4 are classified and reported to the external management center according to the type and urgency of the event, so as to initiate the corresponding emergency response process and record and provide feedback on the event handling process.

2. The slope disaster collaborative perception method integrating edge-side anomaly capture and cloud-based visual and linguistic semantic verification as described in claim 1, characterized in that, S1 includes: S1.1 Based on historical disaster information of highway sections, delineate disaster-prone areas within the highway sections; S1.

2. Based on the characteristics of the highway section, the location of the curves, and the distribution of entrances and exits, plan the installation location of the high-definition camera so that the camera's field of view covers the disaster-prone areas in S1.

1. S1.3 Mark prominent structures in the disaster-prone areas of S1.2; S1.4 Adjust the camera view, establish a high-speed video transmission link based on the wired network, configure the corresponding network equipment and network parameters, continuously collect road video image data according to the preset frame rate and send it to the edge computing device.

3. The slope disaster collaborative perception method integrating edge-side anomaly capture and cloud-based visual and linguistic semantic verification as described in claim 2, characterized in that, S2 includes: S2.1 Based on the road video image data output in S1.4, a visual saliency detection network is used to extract the salient structures in the image data from the complex background; S2.

2. The target monitoring model is used to be responsible for global target initialization, inter-frame correlation and state determination, and to solve the problems of target occlusion, out of view and re-identification, and to provide global constraints for continuous monitoring. S2.

3. Adaptive feature aggregation is used to dynamically weight and fuse multi-level and multi-scale texture / contour / semantic features according to changes in target scale, illumination, and viewpoint, preserving effective features and filtering redundant noise. S2.4 Employ a spatial attention fusion mechanism to enhance the feature weights of key spatial regions of salient structures, suppress background interference, and focus on the details of the structure itself.

4. The slope disaster collaborative perception method integrating edge-side anomaly capture and cloud-based visual and linguistic semantic verification as described in claim 3, characterized in that, S3 includes: S3.

1. Based on the structural saliency map generated in S2, obtain the location of salient structural anchor points; S3.2 Continuously track the anchor point location of significant structures, calculate the horizontal and vertical coordinate drift of a single target point and the relative positional deviation between target points, and obtain the time-series change curve of the coordinate drift of each target point; S3.3 Based on the structural saliency map generated in S2 and the time-series change curves of the coordinate drift of each target point in S3.2, identify the minute geometric shifts and morphological abrupt changes of the saliency structure on the time axis; S3.4 Based on the minute geometric shifts and morphological abrupt changes of the salient structure on the time axis in S3.3, anomaly determination is performed according to a preset threshold, and anomaly monitoring results with location information are generated.

5. The slope disaster collaborative perception method integrating edge-side anomaly capture and cloud-based visual and linguistic semantic verification as described in claim 4, characterized in that, The cloud-based large model in S4 adopts the Transformer architecture. Through a self-attention mechanism, it comprehensively analyzes the significant structural anchor points, deformation, and traffic scene context of the slope globally, and performs semantic-level discrimination of disaster events to improve the accuracy of identification of complex scenes and low-frequency disaster events. When the large model confirms that the abnormal event is a real road disaster or traffic anomaly, it generates a final monitoring report.

Images