Municipal facility early warning method based on video intelligent perception and AI diagnosis

Abstract

The invention relates to the technical field of urban infrastructure monitoring, and discloses a municipal facility early warning method based on video intelligent perception and AI diagnosis, which comprises the following steps: in a reference time period, extracting micro-vibration signals of a well lid and a road surface area through video perception, screening event data with a remarkable cooperative formant by using AI, and determining a pre-warning result of the well lid and the road surface area; in real-time monitoring, micro-vibration signals of a current event are extracted for spectral analysis, a current spectrum and the reference fingerprints are compared through AI, a characteristic asymmetry index representing the dynamic response decoupling degree of a well lid and a road surface is calculated, and when MAI exceeds a threshold value, recessive instability risk early warning is generated. And finally, through a closed-loop processing step, verifying the state stability after maintenance, and automatically closing the work order. According to the method, the problem that the hidden instability of the well lid cannot be recognized in an early stage in the prior art is solved, the structure instability risk is warned in advance from a conventional monitoring video, and the safety management level of municipal facilities is remarkably improved.

Description

Technical Field

[0001] This application relates to the field of urban infrastructure monitoring technology, and in particular to a method for early warning of municipal facilities based on video intelligent perception and AI diagnosis. Background Technology

[0002] Instability of municipal infrastructure, such as manhole covers and storm drain grates, is a common safety hazard. Among them, the instability of municipal manhole covers, such as sinking, tilting, or loosening, is a significant threat to urban public safety.

[0003] Existing monitoring technologies mainly rely on manual inspections or video image analysis, which have the following technical shortcomings: Existing video analysis technology, such as the "Method for Monitoring the Status of Manhole Covers Based on Video Images," primarily identifies faults by comparing the static position or visible deformation of the manhole cover. This is a "post-event alarm" mechanism and cannot identify the "hidden instability" risk caused by structural stiffness decay before visible displacement of the manhole cover occurs. Furthermore, environmental interference such as camera shake and vehicle traffic easily leads to a large number of false alarms in traditional video analysis, while simply setting a displacement threshold for filtering presents a dilemma in threshold setting.

[0004] Secondly, the existing technical solution, "a manhole cover displacement detection system based on feature point matching", usually treats the manhole cover as an isolated rigid target and fails to consider the coupling relationship between the manhole cover and the surrounding road surface in dynamic response, thus failing to capture early dynamic signs of structural instability.

[0005] Therefore, how to extract dynamic physical attributes that reflect the health status of manhole cover structures from conventional monitoring videos, and thereby achieve early and accurate warnings of their hidden instability, has become an urgent technical problem to be solved. Summary of the Invention

[0006] This application provides a municipal facility early warning method based on video intelligent perception and AI diagnosis, which can improve the accuracy and robustness of municipal facility early warning methods in complex environments. The technical solution is as follows: A method for early warning of municipal facilities based on video intelligent perception and AI diagnosis includes: During the benchmark establishment period, based on the monitoring video stream, the micro-vibration signals of the manhole cover area and the road surface area around the manhole cover under multiple external excitation events are extracted by video intelligent perception technology. Based on the AI ​​diagnostic algorithm, event data with significant resonance peaks in the vibration response spectra of the manhole cover area and the road surface area around the manhole cover within the preset frequency range are selected. The resonance peak frequency and amplitude ratio in the selected event data are statistically aggregated, and the aggregated results are used to establish a benchmark feature fingerprint characterizing the co-resonance state of the two. In response to external excitation events detected in real-time monitoring video streams, the micro-vibration signals of the current manhole cover area and the surrounding road surface area are extracted through video intelligent perception technology, and the current spectrum analysis is performed based on AI diagnosis. Based on AI diagnostics, the current spectrum is compared with the baseline feature fingerprint to calculate a feature asymmetry index (MAI) that characterizes the degree of decoupling between the manhole cover and the dynamic response of the surrounding road surface. When the Feature Asymmetry Index (MAI) exceeds a preset threshold, a warning of hidden instability risk of manhole cover is generated. In response to the early warning of hidden instability risk, a handling work order is generated. After receiving the handling completion signal, a continuous monitoring period is entered. The work order is automatically closed only when all characteristic asymmetry indicators (MAI) calculated by AI diagnosis meet the stability conditions during the continuous monitoring period.

[0007] Furthermore, the video intelligent perception technology uses an AI-based phase-based motion amplification algorithm to extract micro-vibration signals through a deep learning model.

[0008] Furthermore, the AI ​​diagnostic algorithm obtains the vibration response spectrum based on power spectral density analysis, and uses a convolutional neural network model and a target detection model to screen event data that have significant resonance peaks within a preset frequency range.

[0009] Furthermore, the statistical aggregation includes: calculating the median and interquartile range for the formant frequency and amplitude ratio, respectively, and applying the Tukey Fences criterion to remove outliers; calculating the average value of the data after removing outliers, which serves as the baseline frequency F0 and baseline amplitude ratio R0 in the baseline feature fingerprint.

[0010] Furthermore, the detection of external stimulus events is based on AI-driven frame difference method, combined with target detection model to confirm that the vehicle has entered the manhole cover area.

[0011] Furthermore, after detecting an external excitation event, video clips of 2 seconds before and after the event are automatically captured for micro-vibration signal extraction and spectrum analysis.

[0012] Furthermore, the calculation of the Characteristic Asymmetry Index (MAI) includes: Calculate the frequency decoupling degree ΔF = |Fc-current - Fr-current|; Where Fc-current is the resonant peak frequency of the current manhole cover area. Fr-current is the resonant peak frequency of the road surface area surrounding the current manhole cover; Calculate the amplitude decoupling degree ΔA = (Ac-current / Ar-current) / R0; Where Ac-current is the resonance peak amplitude of the current manhole cover area, Ar-current is the resonance peak amplitude of the road surface area surrounding the current manhole cover, and R0 is the reference amplitude ratio; Calculate MAI = WF × ΔF + WA × |ΔA - 1|, where WF and WA are weighting coefficients, and WF + WA = 1; The weighting coefficients are dynamically adjusted based on the frequency of resonance peaks and / or the type of excitation events in the baseline feature fingerprint, using a lightweight machine learning model for AI diagnosis.

[0013] Furthermore, in the latent instability risk warning step, when the characteristic asymmetry index (MAI) exceeds the preset threshold, the frequency of MAI exceeding the standard within the preset time period is further checked, and a latent instability risk warning for the manhole cover is generated only when the frequency reaches the preset threshold.

[0014] Furthermore, the stability conditions include: the average value of all characteristic asymmetry indices (MAI) calculated during the continuous monitoring period is lower than a preset threshold, and its standard deviation is less than a preset stability threshold.

[0015] Furthermore, a computer-readable storage medium is characterized in that it stores at least one computer program, which is loaded and executed by a processor to implement the aforementioned method for early warning of municipal facilities based on video intelligent perception and AI diagnosis.

[0016] On the one hand, a computer-readable storage medium is provided, in which at least one computer program is stored, which is loaded and executed by a processor to implement a platform-based municipal facility early warning method.

[0017] On the one hand, a computer program product or computer program is provided, which includes program code stored in a computer-readable storage medium. The processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the aforementioned platform municipal facility early warning method. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is a flowchart of a municipal facility early warning method based on video intelligent perception and AI diagnosis provided in an embodiment of this application.

[0020] Figure 2 This is a partial flowchart of a municipal facility early warning method based on video intelligent perception and AI diagnosis provided in an embodiment of this application.

[0021] Figure 3 This is a partial flowchart of a municipal facility early warning method based on video intelligent perception and AI diagnosis provided in an embodiment of this application.

[0022] Figure 4 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0024] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0025] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain better results.

[0026] Machine Learning (ML) is a multidisciplinary field that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge sub-models to continuously improve their performance.

[0027] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0028] Figure 1 This is a schematic diagram of the implementation environment of a municipal facility early warning method based on video intelligent perception and AI diagnosis provided in an embodiment of this application; S1. Baseline Feature Fingerprint Establishment Steps: During the baseline establishment period, based on the monitoring video stream, micro-vibration signals of the manhole cover area and the road surface area around the manhole cover under multiple external excitation events are extracted using video intelligent perception technology. Based on the AI ​​diagnostic algorithm, event data with significant resonance peaks in the vibration response spectra of the manhole cover area and the road surface area around the manhole cover within a preset frequency range are selected. The resonance peak frequency and amplitude ratio in the selected event data are statistically aggregated, and the aggregation results are used to establish a baseline feature fingerprint characterizing the co-resonance state of the two. S2. Real-time feature analysis steps: In response to external excitation events detected in the real-time monitoring video stream, the micro-vibration signals of the current manhole cover area and the surrounding road surface area are extracted through video intelligent perception technology, and the current spectrum analysis is performed based on AI diagnosis. S3. Feature Asymmetry Calculation Steps: Based on AI diagnosis, compare the current spectrum with the benchmark feature fingerprint to calculate a feature asymmetry index (MAI) that characterizes the degree of decoupling between the manhole cover and the dynamic response of the surrounding road surface. S4. Latent instability risk warning step: When the Feature Asymmetry Index (MAI) exceeds the preset threshold, a latent instability risk warning for the manhole cover is generated. S5. Closed-loop handling steps: In response to the latent instability risk warning, a handling work order is generated. After receiving the handling completion signal, a continuous monitoring period is entered. The work order is automatically closed only when all feature asymmetry indices (MAI) calculated by AI diagnosis meet the stability conditions during the continuous monitoring period. The present invention will now be described in detail with reference to the accompanying drawings and embodiments. The following embodiments are for illustrative purposes only and do not constitute a limitation on the scope of protection of the present invention.

[0029] Example 1 This embodiment uses the monitoring of manhole covers on a main road in a city as an example to illustrate the invention in detail.

[0030] like Figure 1 As shown, the implementation of the system of the present invention includes the following steps: During the benchmark establishment period, based on the monitoring video stream, the micro-vibration signals of the manhole cover area and the road surface area around the manhole cover under multiple external excitation events are extracted by video intelligent perception technology. Based on the AI ​​diagnostic algorithm, event data with significant resonance peaks in the vibration response spectra of the manhole cover area and the road surface area around the manhole cover within the preset frequency range are selected. The resonance peak frequency and amplitude ratio in the selected event data are statistically aggregated, and the aggregated results are used to establish a benchmark feature fingerprint characterizing the co-resonance state of the two. Specifically, after the manhole cover is installed and accepted, video data is continuously collected for 24 hours during periods of stable traffic flow. The video intelligent perception technology is based on the AI-based phase-based motion amplification algorithm to process the video sequences of the manhole cover area (Rc) and the road surface area around the manhole cover (Rr) and extract their micro-vibration waveform signals.

[0031] It should be noted that after the manhole cover is installed and accepted, and during periods of stable traffic flow, the main purpose is to ensure that the monitoring starting point is in good condition. At the same time, in order to obtain effective and representative samples of external excitation events, data deviations caused by overload during peak periods or insufficient excitation during off-peak periods should be avoided. The AI-based phase-based motion amplification algorithm uses a deep learning model to more accurately estimate the motion phase of each pixel in a video. The AI ​​model is trained to understand subtle pixel change patterns in the video sequence, enabling it to more robustly separate the vibration signal of interest and filter out irrelevant noise such as lighting changes and shadows. This ensures greater sensitivity and accuracy to minute, sub-pixel-level motion. By using AI diagnostic analysis to amplify the motion through minute phase changes in the video sequence, it can extract vibration waveforms that are imperceptible to the human eye from ordinary surveillance videos. Preferably, the deep learning model adopts the CNN (Convolutional Neural Network) model.

[0032] The extracted micro-vibration signals are screened using an AI diagnostic algorithm. The AI ​​diagnostic algorithm is based on power spectral density analysis. That is, the extracted micro-vibration signals are first analyzed by power spectral density to obtain a standard vibration response spectrum. Then, the AI ​​diagnostic algorithm is based on a convolutional neural network model and a target detection model to screen time data with effective resonance peaks. Specifically, power spectral density analysis converts non-stationary time-domain vibration signals into stable spectrum diagrams of energy distribution in the frequency domain, and the convolutional neural network model is trained here as an intelligent "spectrum reader". Then, the stable spectrum maps of the manhole cover area and the road surface area are used as dual-channel inputs. The convolutional neural network model automatically learns and extracts the features of the "significant resonance peaks" at multiple scales (such as peak sharpness, prominence relative to background noise, waveform smoothness, etc.) through its convolutional layers. Finally, it outputs a classification decision to determine whether there are significant resonance peaks in the spectrum maps of the two areas that conform to the characteristics of a healthy state within the preset frequency range. For example, under healthy conditions, there is a significant co-resonance peak between the manhole cover area and the road surface area within a preset frequency range (e.g., around 35Hz). The center frequency F0 of this resonance peak (e.g., F0=35Hz) and the amplitude ratio R0 of the manhole cover and the road surface at this frequency are recorded, for example, R0=Ac / Ar=1.2. The resonance peak frequency and amplitude ratio are statistically aggregated, and the aggregated result is used to establish a benchmark feature fingerprint characterizing the co-resonance state of the two, which is also stored in the database as the benchmark feature fingerprint of the monitoring point.

[0033] It should be noted that the healthy state of a manhole cover refers to the manhole cover being tightly coupled with the road surface, and their vibration response being coordinated. When the manhole cover becomes unstable (unhealthy state), the coordination will be disrupted. Power spectral density analysis is a mathematical tool that converts signals from the time domain to the frequency domain. In this step, power spectral density analysis transforms the seemingly chaotic micro-vibration waveforms extracted from the video into key features that can clearly reflect the structural health status of the manhole cover and road surface system. The key features are the resonant frequency and amplitude.

[0034] The purpose of using AI diagnostics to filter out event data that have significant resonance peaks within a preset frequency range is that not all vehicles passing by will excite effective vibration modes that can be used for analysis. For example, excessively fast or slow vehicle speeds, non-perpendicular crushing, and instantaneous sensor (camera) vibrations may all result in the absence of clear resonance peaks in the spectrum or the appearance of noise peaks.

[0035] For significant co-resonance peaks, in practice, AI diagnostics can be used to set signal-to-noise ratio and peak value thresholds for judgment. For example, a resonance peak is considered "significant" only if its height exceeds three times the average level of the background noise and is sufficiently sharp within the half-power bandwidth. The above definition of significant resonance peaks is merely an example; further details are not elaborated here. It should be noted that in order to accurately extract micro-vibration signals (such as the 35Hz resonance peak), the frame rate of the video acquisition device should be at least twice the highest frequency of the signal.

[0036] In a preferred embodiment, a high frame rate acquisition device is used for the monitoring equipment to avoid frequency aliasing and signal distortion caused by insufficient frame rate. Generally, a frame rate greater than 60fps is sufficient to cover the typical vibration frequency range of municipal facilities (20-50Hz).

[0037] As a specific explanation of this step, the frequency and amplitude ratio of the resonance peaks are statistically aggregated to establish a baseline feature fingerprint characterizing the coordinated resonance state of the two. This fingerprint is then stored in the database as the baseline feature fingerprint of the monitoring point. The operation method is as follows: like Figure 2As shown, (1): Preparation of valid dataset The input comes from the output of the data filtering step, which is all the individual event data that are determined to have "significant resonance peaks within a preset frequency range".

[0038] Each valid event data point contains the following core parameters: Fci: Resonant frequency of the manhole cover area in this event Fri: Resonance peak frequency of the road surface area surrounding the manhole cover in this incident. Aci: Resonance peak amplitude in the manhole cover area during this event Ari: The resonance peak amplitude of the road surface area surrounding the manhole cover in this incident A(i): The magnitude ratio of the event, A(i) = Aci / Ari (2): Statistical aggregation of resonant frequencies Statistical analysis was performed on the resonance peak frequency Fci of the manhole cover area in all valid events: Calculate the median M-Fc and interquartile range IQR-Fc of the frequency dataset. At the same time, the Tukey Fences criterion is applied to identify and remove outliers: any data point outside the range [Q1 - 1.5×IQR, Q3 +1.5×IQR] is considered an outlier; For the clean dataset after removing outliers, calculate its average value μ-Fc as a candidate value for the reference frequency; For the resonance peak frequency Fri of the road surface area around the manhole cover, the same statistical analysis process was repeated to obtain μ-Fr. It's important to note that the Tukey fences criterion is an empirical statistical method used to identify outliers in a dataset. It determines outlier limits by calculating the interquartile range (IQR). Specifically, the first quartile (Q1) and third quartile (Q3) are first calculated, then IQR = Q3 - Q1. Next, the lower and upper limits for outliers are determined using Q1 - kIQR and Q3 + kIQR, where k is typically 1.5 or 3. A data point below the lower limit or above the upper limit is considered an outlier.

[0039] (3): Statistical aggregation of amplitude ratio Statistical analysis was performed on the amplitude ratio A(i) of all valid events: Calculate the median M-ra and interquartile range IQR-ra of the amplitude ratio dataset; Similarly, the Tukey Fences criterion is applied to identify and remove outliers; For the clean dataset after removing outliers, calculate its average value μ-ra as the baseline amplitude ratio.

[0040] (4): Final determination of baseline feature fingerprint Based on the above statistical aggregation results, a complete baseline feature fingerprint is established: Reference frequency: F0 = (μ-Fc + μ-Fr) / 2, which is based on the physical assumption that the frequency of the manhole cover and the road surface are the same under healthy conditions; Reference amplitude ratio: R0 = μ-ra Simultaneously record the quality indicators for benchmark establishment: Total number of valid events N; The coefficient of variation (CV-Fre) of frequency data; The coefficient of variation (CV-ra) of the amplitude ratio data; {F0, R0, N, CV-Fre, CV-ra} are stored in the database as complete baseline feature fingerprints.

[0041] S2. Real-time feature analysis steps: In response to external excitation events detected in the real-time monitoring video stream, the micro-vibration signals of the current manhole cover area and the surrounding road surface area are extracted through video intelligent perception technology, and the current spectrum analysis is performed based on AI diagnosis. The system runs continuously, detecting events of vehicles passing through the manhole cover area using AI-driven frame difference method. When an event is detected, video clips of 2 seconds before and after the event are automatically captured. The micro-vibration signal extraction and AI diagnosis based on power spectrum analysis in S1 are also performed on the clip to obtain the vibration response spectrum of the current manhole cover and road surface. This step is relative to (1) effective dataset preparation performed in step S1.

[0042] Specifically, the operation steps of the AI-driven frame difference method are as follows: (1) Motion perception based on traditional frame difference method: Take two (or three) frames of images from the video stream, convert them into grayscale images and perform difference; static backgrounds (such as roads and buildings) are close to zero (black) after difference, while moving objects (such as vehicles) will produce obvious white areas (foreground) in the difference image.

[0043] The system then calculates the total area of ​​this "white area." If the area exceeds a preset threshold, it means that a sufficiently large object is moving, and it is initially determined to be a "possible event," triggering the next step of AI object detection.

[0044] AI-based target detection: used for target identification and confirmation. After a "possible event" occurs in step (1), the system inputs the video segment corresponding to the "possible event" into a pre-trained target detection model. Then, the model is used to perform object recognition and object localization. Object recognition is used to determine whether there is a vehicle in the picture. If a vehicle appears, it is used to output a bounding box to mark the position of the vehicle. Finally, the system compares the vehicle bounding box output by AI with the predefined "manhole cover area". Only when the vehicle actually enters or is very close to the manhole cover area is it finally confirmed as a valid "external stimulus event". The AI-driven frame difference method has the characteristics of high efficiency and high accuracy. Preferably, the target detection model uses YOLO, SSD, etc.

[0045] It's important to note that capturing two seconds of video footage before and after the event in this step is to capture a complete, high-quality vibration signal sample. The first two seconds of the event capture the vibration build-up phase. When the vehicle approaches but hasn't yet run over the manhole cover, the vibration it generates is transmitted through the soil and road surface structure; the last two seconds of the event capture the vibration decay phase. After the vehicle leaves, the "aftershocks" from the manhole cover and road surface don't stop immediately but gradually decay. This complete decay process helps the algorithm more accurately determine the vibration frequency and damping characteristics.

[0046] S3. Feature Asymmetry Calculation Steps: Based on AI diagnosis, compare the current spectrum with the benchmark feature fingerprint to calculate a feature asymmetry index (MAI) that characterizes the degree of decoupling between the manhole cover and the dynamic response of the surrounding road surface. The calculation of the Feature Asymmetry Index (MAI) is based on the frequency drift and amplitude change of the coupled resonance peak corresponding to the baseline feature fingerprint in the current vibration response spectrum between the manhole cover area and the road surface area surrounding the manhole cover. Specifically, frequency drift refers to the weakening of the rigid connection between the manhole cover and the road surface when the manhole cover becomes unstable (e.g., the base is loose or a cavity appears underneath). The manhole cover gains a greater "degree of freedom" relative to the road surface, and its own vibration characteristics change. As a result, under external excitation, the resonant frequency of the manhole cover area and the resonant frequency of the road surface area are no longer consistent, but "drift" and separate from each other. Amplitude change refers to the alteration of damping characteristics when a manhole cover becomes loose. Generally, loose parts are more "sensitive" to vibration energy, leading to an abnormally increased vibration amplitude. Simultaneously, the energy transfer path becomes less smooth. This causes a significant change in the vibration intensity ratio Ac-current / Ar-current between the manhole cover and the road surface, no longer equal to the baseline ratio R0 under healthy conditions. "Current" refers to the current time. Under healthy conditions, vibration energy is transferred and distributed between the manhole cover and the road surface according to a stable ratio, reflected in the ratio of their resonance peak amplitudes, R0 = Ac / Ar. Among them, frequency drift is quantified by frequency decoupling degree ΔF, amplitude change is quantified by amplitude decoupling degree ΔA, and characteristic asymmetry index MAI is a weighted combination of ΔF and ΔA; The weighting coefficients of the weighted combination are dynamically adjusted by AI diagnosis based on the frequency of the formants in the baseline feature fingerprint and / or the type of external excitation events. Specifically, the AI ​​diagnostic dynamic adjustment involves an intelligent module within the system that assigns appropriate weights WF and WA (WA+WF=1) to ΔF and ΔA based on the current situation. This weight allocation is based on a lightweight machine learning model: the system uses a lightweight regression or classification model in the background, inputting features such as the baseline frequency F0, the identified vehicle type (usually a numerical code), time, and temperature. The model then learns and analyzes these features to output the optimal weight coefficients WF and WA for the MAI calculated for this specific event. Simultaneously, the model learns the optimal weight allocation strategy by training on historical data, with the training objective of maximizing the discriminative power of the MAI index against the "true unstable state." Preferably, lightweight regression or classification models employ gradient boosting trees (GBDT) or small neural networks; The specific process for calculating the Feature Asymmetry Index (MAI) is as follows: Obtain the resonant peak frequency Fc-current and amplitude Ac-current of the manhole cover area from the current vibration response spectrum; Obtain the resonant peak frequency Fr-current and amplitude Ar-current of the road surface area surrounding the manhole cover from the current vibration response spectrum; Calculate the frequency decoupling degree ΔF = |Fc-current - Fr-current|; Calculate the amplitude decoupling degree ΔA = (Ac-current / Ar-current) / R0; The characteristic asymmetry index MAI is calculated as: MAI = WF × ΔF + WA × |ΔA - 1|.

[0047] S4. Early warning of hidden instability risk: When the Feature Asymmetry Index (MAI) exceeds the preset threshold, an early warning of hidden instability risk of manhole cover is generated. Specifically as follows: Based on the single MAI value calculated in S3, the MAI value is compared with a preset threshold. If MAI is less than or equal to the preset threshold, the event is logged and the system remains in normal status. If MAI > preset threshold, the system enters the early warning counting stage, and then checks the frequency of MAI exceeding the standard within a certain period of time. If the frequency threshold is not reached, the system will return to normal monitoring status. If the frequency threshold is reached: it is confirmed as a persistent risk, an early warning is generated, an early warning work order containing complete information is created, and then the work order automatically notifies maintenance personnel through multiple channels; the system enters a waiting response state, preparing to enter the S5 closed-loop handling process.

[0048] like Figure 3 As shown, S5: Closed-loop processing S51: Handling completion signal trigger and monitoring period start Signal input: After completing the inspection and tightening of the manhole cover on site, maintenance personnel submit a "processing completed" signal to the system via mobile terminal.

[0049] Special monitoring initiated: Upon receiving this signal, the system does not immediately close the work order, but automatically enters a continuous monitoring period of a preset duration (e.g., 24 hours, 48 ​​hours, etc.). During this period, the system's monitoring and analysis priority for the manhole cover remains unchanged.

[0050] S52: Continuous Monitoring and Stability Data Acquisition Real-time MAI calculation: During the continuous monitoring period, the system fully executes the S2-S3 process for each newly detected vehicle excitation event passing through the manhole cover, that is, extracting micro-vibration signals and performing spectrum analysis to calculate a new characteristic asymmetry index (MAI).

[0051] Data set formation: The system continuously collects and records newly generated MAI values ​​until the end of the monitoring period or until a preset number of samples (e.g., 10 valid events) is reached.

[0052] S53: Stability Condition Verification and Intelligent Decision-Making Dual-condition judgment: At the end of the continuous monitoring period, the system automatically performs statistical analysis on all MAI values ​​collected during this period (e.g., the last N=10 times) and verifies whether they simultaneously meet the following two stability conditions: Average value condition: The arithmetic mean of N MAI values ​​must be lower than a preset warning threshold (e.g., 0.5). This condition indicates that the overall abnormality of the manhole cover has returned to a safe range.

[0053] Standard Deviation Condition: The standard deviation of these N MAI values ​​must be less than a preset stability threshold (e.g., 0.1). This condition ensures that the manhole cover's condition is stable and consistent, rather than fluctuating significantly around a critical value, thus verifying the durability of the repair. S54: Closed-Loop Decision Execution: Stability Condition Satisfaction: If the above average and standard deviation conditions are both satisfied, the system determines that the latent instability risk of the manhole cover has been successfully eliminated; subsequently, the system automatically closes the warning work order and records the entire handling process, forming a complete management closed loop.

[0054] Stability conditions not met: If any condition is not met (e.g., the MAI average is still higher than the threshold, or the standard deviation is greater than or equal to the preset stability threshold), the system determines that the repair has not achieved the expected results. In this case, the system will not close the work order, but will automatically trigger a work order escalation notification or regenerate an early warning work order, prompting the need to start a new round of more in-depth repair procedures; In this context, a work order escalation notification typically refers to notifying higher-level managers to handle the issue.

[0055] It should be noted that the S5 step, by introducing a data-based automated verification mechanism, upgrades the traditional "human reporting and human repair" open-loop management model to an intelligent closed-loop management model of "human repair and machine verification," effectively eliminating "false repairs" and recurring problems, and significantly improving the scientific nature, reliability, and management efficiency of municipal facility operation and maintenance.

[0056] It should be noted that the above-described municipal facility early warning system based on video intelligent perception and AI diagnosis is only illustrated by the division of the above functional modules when performing municipal facility detection. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer equipment can be divided into different functional modules to complete all or part of the functions described above.

[0057] Figure 4This is a schematic diagram of a server structure provided in an embodiment of this application. The server 40 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 41 and one or more memories 42. The one or more memories 42 store at least one computer program, which is loaded and executed by the one or more processors 41 to implement the methods provided in the above-described method embodiments. Of course, the server 40 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 40 may also include other components for implementing device functions, which will not be elaborated upon here.

[0058] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a computer program that can be executed by a processor to perform the monitoring of the manhole cover road surface in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0059] In an exemplary embodiment, a computer program product or computer program is also provided, which includes program code stored in a computer-readable storage medium. The processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the aforementioned platform municipal facility early warning method.

[0060] In some embodiments, the computer program involved in the present application embodiments may be deployed and executed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network may constitute a blockchain system.

[0061] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0062] The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for early warning of municipal facilities based on video intelligent perception and AI diagnosis, characterized in that, The method includes: During the baseline establishment period, based on the monitoring video stream, the micro-vibration signals of the manhole cover area and the road surface area around the manhole cover under multiple external excitation events are extracted by video intelligent perception technology. Based on the AI ​​diagnostic algorithm, event data in which the vibration response spectra of the manhole cover area and the road surface area around the manhole cover have significant resonance peaks in the preset frequency range are selected. The resonance peak frequency and amplitude ratio in the selected event data are statistically aggregated, and the aggregation result is used to establish a baseline feature fingerprint characterizing the co-resonance state of the two. In response to external excitation events detected in real-time monitoring video streams, the micro-vibration signals of the current manhole cover area and the surrounding road surface area are extracted through video intelligent perception technology, and the current spectrum analysis is performed based on AI diagnosis. Based on AI diagnostics, the current spectrum is compared with the benchmark feature fingerprint to calculate a feature asymmetry index (MAI) that characterizes the degree of decoupling between the manhole cover and the dynamic response of the surrounding road surface. When the Feature Asymmetry Index (MAI) exceeds a preset threshold, a warning of hidden instability risk of the manhole cover is generated. In response to the latent instability risk warning, a handling work order is generated, and after receiving the handling completion signal, a continuous monitoring period is entered. The work order is automatically closed only when all characteristic asymmetry indicators (MAI) calculated by AI diagnosis meet the stability conditions during the continuous monitoring period.

2. The municipal facility early warning method based on video intelligent perception and AI diagnosis according to claim 1, characterized in that, The video intelligent perception technology is based on the phase-based motion amplification algorithm of AI, which extracts micro-vibration signals through a deep learning model.

3. The municipal facility early warning method based on video intelligent perception and AI diagnosis according to claim 1, characterized in that, The AI ​​diagnostic algorithm obtains the vibration response spectrum based on power spectral density analysis, and uses a convolutional neural network model and a target detection model to screen event data that have significant resonance peaks within a preset frequency range.

4. The municipal facility early warning method based on video intelligent perception and AI diagnosis according to claim 1, characterized in that, The statistical aggregation includes: calculating the median and interquartile range for the formant frequency and amplitude ratio, respectively, and applying the Tukey Fences criterion to remove outliers; calculating the average value of the data after removing outliers, which serves as the base frequency F0 and base amplitude ratio R0 in the base feature fingerprint.

5. The municipal facility early warning method based on video intelligent perception and AI diagnosis according to claim 1, characterized in that, The detection of the external stimulus event is based on AI-driven frame difference method, combined with target detection model to confirm that the vehicle has entered the manhole cover area.

6. The municipal facility early warning method based on video intelligent perception and AI diagnosis according to claim 5, characterized in that, After an external excitation event is detected, video clips of 2 seconds before and after the event are automatically captured for micro-vibration signal extraction and spectrum analysis.

7. The municipal facility early warning method based on video intelligent perception and AI diagnosis according to claim 1, characterized in that, The calculation of the Characteristic Asymmetry Index (MAI) includes: Calculate the frequency decoupling degree ΔF = |Fc-current - Fr-current|; Where Fc-current is the resonant peak frequency of the current manhole cover area. Fr-current is the resonant peak frequency of the road surface area surrounding the current manhole cover; Calculate the amplitude decoupling degree ΔA = (Ac-current / Ar-current) / R0; Where Ac-current is the resonance peak amplitude of the current manhole cover area, Ar-current is the resonance peak amplitude of the road surface area surrounding the current manhole cover, and R0 is the reference amplitude ratio; Calculate MAI = WF × ΔF + WA × |ΔA - 1|, where WF and WA are weighting coefficients, and WF + WA = 1; The weighting coefficients are dynamically adjusted through AI diagnostics based on the frequency of the resonance peaks in the baseline feature fingerprint and / or the type of the excitation event. The AI ​​diagnostics employs a lightweight machine learning model.

8. The municipal facility early warning method based on video intelligent perception and AI diagnosis according to claim 1, characterized in that, In the hidden instability risk warning step, when the characteristic asymmetry index (MAI) exceeds a preset threshold, the frequency of MAI exceeding the standard within a preset time period is further checked, and a hidden instability risk warning for the manhole cover is generated only when the frequency reaches the preset threshold.

9. The municipal facility early warning method based on video intelligent perception and AI diagnosis according to claim 1, characterized in that, The stability conditions include: the average value of all characteristic asymmetry indices (MAI) calculated during the continuous monitoring period is lower than a preset threshold, and their standard deviation is less than a preset stability threshold.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the municipal facility early warning method based on video intelligent perception and AI diagnosis as described in any one of claims 1-9.

Images