Engineering anomaly monitoring method and system based on video frame difference analysis

Abstract

The invention provides an engineering anomaly monitoring method and system for video frame difference analysis, and relates to the technical field of image processing, and the method comprises the steps: obtaining multi-source video data of a monitoring region; video frame images in the multi-source video data are preprocessed; determining a region of interest in the preprocessed video frame image; a difference image of the current video frame image and a reference frame image in the region of interest is calculated, and the reference frame image comprises a dynamic reference frame and a benchmark reference frame; determining a deformation region in the difference image through an adaptive threshold segmentation algorithm; extracting characteristic parameters of the deformation area; according to the characteristic parameters, engineering anomaly recognition analysis is carried out through a dual-drive recognition model based on a rule base and machine learning; and according to an engineering anomaly identification analysis result, triggering an early warning signal of a corresponding level. The engineering monitoring method and system based on the video frame difference can achieve high-precision and low-false-alarm monitoring, reduce cost, are easy to deploy and are suitable for multiple geotechnical engineering scenes.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an engineering anomaly monitoring method and system for video frame difference analysis. Background Technology

[0002] Stability monitoring of geotechnical engineering structures is a core component in ensuring safe operation and preventing geological disasters. Video monitoring technology, due to its advantages such as non-contact and visualization, has become an important means of engineering safety monitoring. Currently, video surveillance equipment is widely deployed at various engineering sites, providing the hardware foundation and application prospects for intelligent monitoring methods based on video analytics.

[0003] Currently, mainstream engineering monitoring technologies mainly include traditional contact measurement and video analysis methods. Traditional methods, such as GNSS, inclinometers, and radar interferometry, suffer from high equipment costs, complex deployment, difficulty in achieving full coverage, and inability to provide real-time early warnings. Existing video monitoring technologies are mostly based on the fixed threshold frame difference method, which, while capable of identifying obvious displacements, has weak ability to capture subtle deformations, lacks intelligent early warning capabilities, and is susceptible to environmental interference such as changes in lighting and weather conditions, resulting in a high false alarm rate.

[0004] Existing video monitoring methods have significant drawbacks in engineering applications: First, the algorithms have poor adaptability and are difficult to cope with complex working conditions such as nighttime, backlight, and dust. Second, they rely on manual delineation of monitoring areas and adjustment of parameters, resulting in low automation. Third, the system integration is low, and the data processing and early warning functions are weak, making it difficult to meet the engineering monitoring needs of real-time, accurate, and low false alarm. Summary of the Invention

[0005] To address the technical problems of poor adaptability, low automation, and limited early warning reliability of existing video monitoring methods under complex working conditions, this invention provides an engineering anomaly monitoring method and system based on video frame difference analysis.

[0006] The technical solutions provided by the embodiments of the present invention are as follows:

[0007] First aspect:

[0008] This invention provides a method for monitoring engineering anomalies using video frame difference analysis, comprising:

[0009] S1: Acquire multi-source video data of the monitored area;

[0010] S2: Preprocess the video frame images in the multi-source video data;

[0011] S3: Determine the region of interest in the preprocessed video frame image;

[0012] S4: Calculate the difference image between the current video frame image and the reference frame image within the region of interest, where the reference frame image includes the dynamic reference frame and the baseline reference frame;

[0013] S5: Determine the deformed regions in the difference image using an adaptive threshold segmentation algorithm;

[0014] S6: Extract feature parameters of the deformed region;

[0015] S7: Based on feature parameters, perform engineering anomaly identification and analysis using a dual-drive identification model based on rule base and machine learning;

[0016] S8: Based on the results of engineering anomaly identification and analysis, trigger the corresponding level of early warning signal.

[0017] The second aspect:

[0018] An engineering anomaly monitoring system based on video frame difference analysis provided in this embodiment of the invention includes:

[0019] processor;

[0020] A memory storing computer-readable instructions, which, when executed by the processor, implement the engineering anomaly monitoring method for video frame difference analysis as described in the first aspect.

[0021] Third aspect:

[0022] The present invention provides a computer-readable storage medium storing the present invention on a computer, which, when executed by a processor, implements the engineering anomaly monitoring method for video frame difference analysis as described in the first aspect.

[0023] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0024] This invention significantly improves adaptability under complex working conditions by integrating improved image preprocessing and dual-reference frame difference analysis. Automatic ROI delineation and multi-dimensional feature extraction achieve full automation and precision in the monitoring process. A dual-driven recognition model combining rule base and machine learning effectively distinguishes between normal deformation and disaster precursors. Furthermore, a highly reliable, low-false-alarm intelligent monitoring system is constructed through a tiered early warning mechanism and integrated data management, significantly enhancing real-time early warning and decision support capabilities for engineering safety, and meeting the needs for real-time, accurate, and low-false-alarm engineering monitoring. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating an engineering anomaly monitoring method based on video frame difference analysis, provided in an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of the structure of an engineering anomaly monitoring system for video frame difference analysis provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0029] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0030] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0031] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0032] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0033] Reference manual attached Figure 1 The diagram shows a flowchart of an engineering anomaly monitoring method based on video frame difference analysis provided by an embodiment of the present invention.

[0034] Reference manual attached Figure 2 The diagram shows a structural schematic of an engineering anomaly monitoring system for video frame difference analysis provided in an embodiment of the present invention.

[0035] This invention provides a method for monitoring engineering anomalies using video frame difference analysis. This method can be implemented using a video frame difference analysis-based engineering anomaly monitoring device, which can be a terminal or a server. The processing flow of the video frame difference analysis-based engineering anomaly monitoring method may include the following steps:

[0036] S1: Acquire multi-source video data of the monitored area.

[0037] Among them, the multi-source video data is the video stream of the monitoring area (such as slope, tunnel, etc.) collected by the dual-mode camera (visible light + infrared, compatible with mainstream protocols such as USB, RTSP, ONVIF, etc.) driven by the present invention.

[0038] Furthermore, the present invention can support multiple input sources, including local files (MP4, AVI format), network streams, and cloud streams, through a network transmission unit (supporting 4G / 5G / Wi-Fi), and automatically parse video parameters (resolution, frame rate, encoding format) to adapt to different hardware environments at engineering sites (such as outdoor solar-powered cameras and wired monitoring equipment in tunnels).

[0039] Specifically, when acquiring multi-source video data, the dual-mode camera's visible light mode resolution is ≥1920×1080@30fps, with a minimum illumination of 0.01Lux. The infrared mode wavelength is 850nm, with an effective monitoring distance ≥50m. The edge computing terminal uses an NVIDIA Jetson Nano or equivalent device, with an image processing latency <50ms / frame.

[0040] For example, one dual-mode camera (visible light + infrared) is deployed in each of the three key areas of the slope. The cameras are mounted on fixed brackets 20m away from the slope and connected to an edge computing terminal (NVIDIA Jetson Nano) via a 4G network. The terminal is then connected to the project's monitoring center server via fiber optic cable.

[0041] S2: Preprocess the video frame images in the multi-source video data.

[0042] Optionally, preprocessing is deployed on an edge computing terminal to perform denoising, enhancement, and region of interest (ROI) extraction, and output optimized video frames and ROI data.

[0043] In one possible implementation, S2 specifically includes sub-steps S201 to S203:

[0044] S201: The video frame image is filtered using a bilateral filtering algorithm.

[0045] Specifically, a bilateral filtering algorithm is used. By setting dynamic thresholds for "spatial domain weight" and "grayscale domain weight", key details such as rock fissures and textures are preserved while removing dust, raindrops, and sensor noise.

[0046] S202: When in a low-light scene, the video frame image is enhanced using an adaptive gamma correction algorithm.

[0047] Specifically, for low-light scenarios (such as inside tunnels or on slopes at night), an adaptive gamma correction algorithm is automatically activated to adjust the gamma value (range 0.3-2.0) based on the image brightness histogram, thereby improving the clarity of dark areas.

[0048] S203: When in a strong backlight scene, the Retinex enhancement algorithm is used to enhance the video frame image.

[0049] Specifically, for strong backlighting scenes, the Retinex enhancement algorithm is used to suppress overexposure in strong light areas and restore the true texture of the rock surface.

[0050] It should be noted that this invention has high adaptability across all scenarios, supports multiple video input sources, and can operate under different lighting conditions. Combined with the ROI function, it effectively focuses on key areas. This invention supports multi-source video input and optimizes preprocessing algorithms for engineering scenarios such as low light, backlight, and dust, achieving stable monitoring in all weather conditions (day / night) and under all working conditions, adapting to complex field environments.

[0051] For example, the monitoring program of this invention is deployed on the monitoring center server with the following configuration parameters: video frame rate 30fps, ROI automatic generation mode, dynamic reference frame N=5, warning threshold (attention level: deformation rate 0.05mm / h, warning level: 0.2mm / h, danger level: 0.5mm / h), data storage period of 1 year, and synchronized to Alibaba Cloud object storage in the cloud.

[0052] S3: Determine the region of interest in the preprocessed video frame image.

[0053] Optionally, the determination of the Region of Interest (ROI) supports two modes: user-interactive mode and automatic mode.

[0054] The user interaction mode allows users to manually draw regions of interest through a visual interface. These regions of interest include key areas such as the toe of the slope and tunnel lining joints.

[0055] The automatic mode uses the Canny edge detection operator to identify the boundary between rock mass and non-rock mass and automatically generate the region of interest. The boundary of non-rock mass includes vegetation, sky, and roads.

[0056] For example, this invention drives a camera to capture real-time video streams. For scenarios with high dust levels during the rainy season and backlighting in the early morning and evening, it automatically activates bilateral filtering for noise reduction and Retinex enhancement to clearly preserve the surface texture of the slope. The Canny operator automatically identifies the boundaries between the slope and vegetation, generating ROIs for three key areas, reducing computation by 45% and CPU usage by ≥40%.

[0057] In this embodiment of the invention, the Region of Interest (ROI) is determined through both manual drawing and automatic identification, allowing for flexible adaptation to different monitoring needs. In user-interactive mode, key areas, such as slope toes and tunnel joints, can be precisely specified to ensure the focus of monitoring. In automatic mode, the Canny edge detection algorithm efficiently identifies the boundaries between rock and non-rock masses, automatically generating ROIs, reducing manual intervention and improving processing efficiency. The combination of these two methods not only enhances the accuracy and flexibility of monitoring but also reduces the complexity of manual operation, enabling the system to operate stably under various working conditions.

[0058] S4: Calculate the difference image between the current video frame image and the reference frame image within the region of interest, where the reference frame image includes the dynamic reference frame and the baseline reference frame.

[0059] Specifically, the present invention automatically sets a "baseline reference frame" (initial stable frame) and a "dynamic reference frame" (average frame of the previous N frames, where N can be configured by the parameters of the present invention, with a default of 5 frames). The difference image is calculated between the current frame and the two reference frames respectively, avoiding misjudgment caused by a single reference frame due to gradual changes in the environment (such as slow changes in illumination).

[0060] In one possible implementation, S4 specifically includes sub-steps S401 and S402:

[0061] S401: Determine the binarization threshold of the difference image using an adaptive threshold segmentation algorithm.

[0062] Specifically, the adaptive thresholding algorithm automatically calculates the binarization threshold of the frame difference image based on the Otsu algorithm.

[0063] S402: Determine the deformed regions in the difference image based on the binarization threshold.

[0064] For example, in a mountainous highway slope monitoring project, the slope height is 30m, and the slope body is composed of alternating layers of gravel and sandstone. Small-scale landslides have occurred in the past. It is necessary to monitor three key areas: the slope toe, the platform in the middle of the slope, and the intercepting ditch at the top of the slope, to prevent the risk of collapse during the rainy season.

[0065] In this embodiment of the invention, based on an efficient frame difference algorithm, real-time or near-real-time deformation analysis and early warning can be achieved, supporting automatic ROI selection and automatic threshold adjustment, reducing manual intervention.

[0066] S5: Determine the deformed regions in the difference image using an adaptive threshold segmentation algorithm.

[0067] Specifically, the adaptive threshold segmentation algorithm introduces "regional variance weights" to set differentiated thresholds for different regions within the ROI (such as intact rock mass areas and suspected fracture areas) to accurately extract deformed regions.

[0068] S6: Extract feature parameters of the deformed region.

[0069] Optionally, the feature parameters include: displacement features, morphological features, and dynamic features.

[0070] Among them, the displacement feature is calculated by sub-pixel level template matching algorithm (normalized cross-correlation coefficient) to calculate the change in center coordinates of the deformation area (X / Y direction) with an accuracy of 0.05mm (based on target calibration).

[0071] Among them, the morphological characteristics are the area, perimeter, and roundness of the deformed region (to determine whether it is a "linear crack" or a "surface bulge"), and the crack length / width (for linear deformation).

[0072] Among them, dynamic characteristics include the calculation of deformation rate (displacement per unit time), deformation acceleration (rate change), and abrupt changes in characteristic parameters (such as crack length increasing by more than 5 cm within 10 minutes).

[0073] Specifically, the program runs a dual-reference frame difference, adaptive threshold segmentation, and sub-pixel matching procedure to output multi-dimensional feature parameters.

[0074] For example, from 8:00 to 12:00 on June 15, 2024, linear deformation was found in the platform area of ​​the slope through the frame difference of the two reference frames. The characteristic parameters were extracted as follows: initial crack length 20cm, deformation rate 0.15mm / h (attention level), crack width 2mm, and roundness 0.2 (typical linear crack).

[0075] It should be noted that the "dual reference frame difference + sub-pixel matching" algorithm achieves a micro-deformation recognition accuracy of 0.05mm. Combined with the "rule + CNN" anomaly recognition model, the false alarm rate is reduced to below 5%, solving the problem of "missed detection and false alarm". Alarms are graded according to the severity of deformation, improving the scientific validity and reliability of early warning systems.

[0076] S7: Based on feature parameters, an engineering anomaly identification and analysis is performed using a dual-drive identification model based on rule base and machine learning.

[0077] Specifically, by integrating a rule base with a CNN model, this invention outputs anomaly detection results and physical deformation amounts.

[0078] In one possible implementation, S7 specifically includes sub-steps S701 to S703:

[0079] S701: Based on feature parameters, identify engineering anomalies in routine scenarios using anomaly detection rules in the rule base.

[0080] Specifically, the anomaly detection rules in the rule base include "deformation rate > 0.2 mm / h and lasts for 1 hour", "crack length growth rate > 1 cm / 10 min" and "area of ​​planar deformation region > 1 m²", and the rule parameters can be modified through the configuration file.

[0081] S702: Based on feature parameters, a lightweight CNN model is used to identify engineering anomalies in complex scenes.

[0082] Specifically, for complex scenarios (such as rock mass with multiple fractures and cross deformation), a trained lightweight CNN model (MobileNet architecture) is used to input multi-dimensional feature parameters and output three types of results: "normal creep", "suspected anomaly" and "definite anomaly". The model inference time is <100ms / frame, which is suitable for real-time monitoring requirements.

[0083] S703: Converts pixel-level displacement into physical deformation through three-dimensional coordinate mapping, generating a time-deformation curve.

[0084] Specifically, by using three-dimensional coordinate mapping (which requires prior calibration of the camera's internal and external parameters via a target), pixel-level displacement is converted into physical deformation, generating a "time-deformation" curve to visually display the rock mass deformation trend.

[0085] In this embodiment of the invention, the dual-drive model achieves complementary fast rule matching and intelligent learning, reducing the false alarm rate to below 5% and the inference latency to <100ms / frame, thus meeting real-time requirements.

[0086] S8: Based on the results of engineering anomaly identification and analysis, trigger the corresponding level of early warning signal.

[0087] Specifically, it supports users to customize three-level warning thresholds through a visual interface. The default thresholds are: attention level (deformation rate 0.05-0.2mm / h), warning level (0.2-0.5mm / h), and danger level (>0.5mm / h). It also supports integration with third-party engineering management platforms (through API interface).

[0088] In one possible implementation, S8 specifically includes sub-steps S801 to S803:

[0089] S801: When the deformation rate is 0.05-0.2 mm / h or the crack length increases by 0.5-1 cm / 10 min in the engineering anomaly identification and analysis results, a yellow warning box will pop up on the interface.

[0090] S802: When the deformation rate is 0.2-0.5 mm / h, or the crack length increases by 1-2 cm / 10 min, or the area of ​​planar deformation is 0.5-1 m² in the engineering anomaly identification and analysis results, an audible and visual alarm will be triggered, and a warning message will be pushed to the management personnel via SMS / email.

[0091] S803: When the deformation rate is greater than 0.5 mm / h, the crack length increases by more than 2 cm / 10 min, or the area of ​​planar deformation is greater than 1 m² in the engineering anomaly identification and analysis results, the highest level early warning will be activated, the on-site equipment will be linked, and emergency response suggestions will be generated.

[0092] For example, from 12:00 to 14:00 on June 15, 2024, the rainfall increased. The present invention detected that the crack length increased to 35cm (the growth rate was 1.25cm / 10min) and the deformation rate increased to 0.25mm / h, triggering a "warning level" warning. The present invention immediately popped up a red warning box in the monitoring center, activated the audible and visual alarm, and pushed the warning information to the project manager and supervising engineer via SMS.

[0093] It should be noted that this invention can set multi-level warning thresholds (attention level, alert level, danger level) according to the proportion of the changing area, dynamically display the warning status and prompt the user. Based on deformation quantification results and engineering safety standards, this invention sets a three-level warning mechanism and supports multi-channel alarms: Warning threshold configuration: This invention provides a visual interface that allows users to customize thresholds according to the type of project (such as highway slopes, mine tunnels).

[0094] In embodiments of the present invention, a multi-level mechanism is matched with engineering safety standards to form a closed loop of "monitoring-early warning-response", with a response delay of <100ms, thereby improving the timeliness of disaster prevention and control.

[0095] In one possible implementation, step S8 is followed by:

[0096] S9: Store and visualize early warning signals and monitoring data.

[0097] Specifically, the monitoring data management includes a local SQLite database and a cloud storage interface, enabling data classification and storage, automatic backup (local + cloud dual backup), historical data retrieval, data export (Excel, CSV format), and integration with third-party systems (such as engineering management platforms, via API interface).

[0098] Optionally, a web / PC-based visual interface program is provided, supporting parameter configuration, real-time monitoring, and historical data review.

[0099] It should be noted that this invention supports real-time display of the original video, current frame, reference frame, and analysis results. It also supports dragging a progress bar to retrieve historical frames, and data is stored both locally and in the cloud. This invention supports local storage (SQLite database, saving feature parameters, alert events, and keyframe images) and cloud synchronization (connecting to Alibaba Cloud / Huawei Cloud object storage, automatically uploading original video clips (5 minutes before and after abnormal periods) and deformation curve reports). The data retention period is configurable (default 1 year). This invention provides a web / PC monitoring interface for real-time display.

[0100] Specifically, the visualization supports overlaying the original video with frame difference analysis (with labeled deformation areas and feature parameters), trend curves of "time-deformation amount" and "time-deformation rate", displacement flow field diagram of ROI area (arrows indicate deformation direction and magnitude), list of early warning events (sorted by level, with support for filtering and exporting to Excel), and allows for dragging along the timeline and filtering by date to retrieve historical data. It can also replay the linked screen of "original video-frame difference analysis-early warning status" for any time period, facilitating post-event analysis.

[0101] For example, after receiving a report, management personnel went to the site for inspection and found new cracks in the platform in the middle of the slope (consistent with the monitoring results of this invention). They immediately took measures of "surcharge counterpressure at the toe of the slope + grouting reinforcement". Afterwards, by reviewing the historical data from 8:00 to 14:00 on the 15th through this invention, they viewed the linked screen of "original video - frame difference analysis - deformation curve" to verify the deformation development process and provide a basis for optimizing the subsequent support scheme.

[0102] S10: Predict the future deformation trend of each region of interest using LSTM.

[0103] Among these methods, predicting future deformation trends involves updating the CNN model through manual labeling feedback and adaptively adjusting algorithm parameters based on historical data.

[0104] Specifically, by integrating time series prediction models such as LSTM, future deformation trends can be predicted based on historical deformation data, thus expanding the prediction model. The program also incorporates a closed-loop optimization mechanism to continuously improve recognition accuracy.

[0105] Furthermore, the program's feedback learning is reflected in the following: Managers can label "false alarm / missed alarm" events through the interface, and the program automatically adds the event's feature parameters and actual labels (normal / abnormal) to the training set, periodically (configurable period, weekly by default) to retrain the CNN model and update the model parameters. The program's adaptive threshold adjustment is reflected in the following: The program statistically analyzes historical warning data, assesses false alarm rates under different seasons and weather conditions, and automatically optimizes frame difference thresholds and warning thresholds (e.g., automatically increasing the filtering intensity corresponding to "raindrop interference" on rainy days).

[0106] It should be noted that this invention provides a visual operation interface, supports parameter customization, historical data backtracking, and multi-terminal access. The generated deformation curves and early warning reports directly meet the needs of engineering management and can be operated without professional algorithm knowledge.

[0107] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0108] In this invention, from automatic ROI delineation and adaptive threshold adjustment to early warning triggering and data backup, the entire process requires no manual intervention, achieving unattended monitoring and reducing labor costs. Relying on ordinary network cameras and edge computing terminals (single terminal cost < 5000 RMB), compared to radar and GNSS systems, it significantly reduces system deployment and maintenance costs, lowering overall deployment costs by more than 60%, and supports upgrades and modifications to existing monitoring systems, making it widely applicable. This invention is not only suitable for slope engineering but can also be extended to the field of structural health monitoring for civil infrastructure such as bridges, dams, and tunnels.

[0109] Reference manual attached Figure 2 The diagram shows a structural schematic of an engineering anomaly monitoring system for video frame difference analysis provided by the present invention.

[0110] The present invention also provides an engineering anomaly monitoring system 20 for video frame difference analysis, applied to the above-mentioned engineering anomaly monitoring method for video frame difference analysis, comprising:

[0111] Processor 201.

[0112] The memory 202 stores computer-readable instructions, which, when executed by the processor 201, implement the engineering anomaly monitoring method of video frame difference analysis as described in the method embodiment.

[0113] The video frame difference analysis engineering anomaly monitoring system 20 provided by the present invention can execute the above-described video frame difference analysis engineering anomaly monitoring method and achieve the same or similar technical effects. To avoid duplication, the present invention will not elaborate further.

[0114] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0115] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0116] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer product of the present invention. The computer product of the present invention includes one or more computer instructions or the computer invention itself. When the computer instructions or the computer invention are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0117] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0118] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0119] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0120] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0122] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0125] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing the code of this invention, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] This invention provides a computer-readable storage medium storing the present invention, which, when executed by a processor, implements the engineering anomaly monitoring method for video frame difference analysis as described in the method embodiment.

[0127] The present invention provides a computer-readable storage medium that can implement the steps and effects of the engineering anomaly monitoring method for video frame difference analysis in the above-described method embodiments. To avoid repetition, the present invention will not repeat the details.

[0128] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0129] The following points need to be explained:

[0130] (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0131] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0132] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0133] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of engineering anomaly monitoring by video frame difference analysis, characterized in that, The method comprises the following steps: S1: acquiring multi-source video data of a monitoring area; S2: pre-processing video frame images in the multi-source video data; S3: determining a region of interest in the pre-processed video frame images; S4: calculating a difference image of a current video frame image and a reference frame image in the region of interest, wherein the reference frame image comprises a dynamic reference frame and a benchmark reference frame; S5: determining a deformation region in the difference image through an adaptive threshold segmentation algorithm; S6: extracting feature parameters of the deformation region; S7: performing engineering anomaly identification analysis through a dual-drive identification model based on a rule base and machine learning according to the feature parameters; S8: triggering a corresponding level of early warning signals according to the engineering anomaly identification analysis result.

2. The method of engineering anomaly monitoring through video frame differencing of claim 1, wherein, The S2 specifically comprises: S201: filtering the video frame images through a bilateral filtering algorithm; S202: when in a low-illumination scene, enhancing the video frame images through an adaptive gamma correction algorithm; S203: when in a strong back light scene, enhancing the video frame images through a Retinex enhancement algorithm.

3. The method of engineering anomaly monitoring through video frame differencing of claim 1, wherein, The determination mode of the region of interest supports two modes: a user interaction mode and an automatic mode; The user interaction mode manually draws the region of interest through a visual interface; The automatic mode automatically generates the region of interest through a Canny edge detection operator to identify the boundary between rock mass and non-rock mass.

4. The method of engineering anomaly monitoring through video frame differencing of claim 1, wherein, The S4 specifically comprises: S401: determining a binary threshold of the difference image through an adaptive threshold segmentation algorithm; S402: determining the deformation region in the difference image according to the binary threshold.

5. The method of engineering anomaly monitoring through video frame differencing of claim 1, wherein, The feature parameters comprise displacement features, morphological features and dynamic features.

6. The method of engineering anomaly monitoring through video frame differencing of claim 1, wherein, The S7 specifically comprises: S701: performing engineering anomaly identification of a regular scene through an abnormality judgment rule in the rule base according to the feature parameters; S702: performing engineering anomaly identification of a complex scene through a lightweight CNN model according to the feature parameters; S703: converting pixel-level displacement into physical deformation through three-dimensional coordinate mapping to generate a time-deformation curve.

7. The method of engineering anomaly monitoring through video frame differencing of claim 1, wherein, The S8 specifically comprises: S801: when the deformation rate in the engineering anomaly identification analysis result is 0.05-0.2 mm / h, or the crack length growth is 0.5-1 cm / 10 min, a yellow early warning box is popped up on the interface; S802: when the deformation rate in the engineering anomaly identification analysis result is 0.2-0.5 mm / h, or the crack length growth is 1-2 cm / 10 min, or the planar deformation area is 0.5-1 m², a sound-light alarm is triggered, and early warning information is pushed to the management personnel through a short message / email; S803: when the deformation rate in the engineering anomaly identification analysis result is >0.5 mm / h, or the crack length growth is >2 cm / 10 min, or the planar deformation area is >1 m², the highest level of early warning is started, the on-site equipment is linked, and an emergency disposal suggestion is generated.

8. The method of engineering anomaly monitoring through video frame differencing of claim 1, wherein, The step S8 further comprises: S9: store and visualize the early warning signals together with the monitoring data.

9. The method of engineering anomaly monitoring through video frame differencing of claim 1, wherein, After step S8, further comprising: S10: predict the future deformation trend of each region of interest by LSTM.

10. An engineered anomaly monitoring system for video frame difference analysis, characterized in that, Comprising: a processor; a memory, on which computer readable instructions are stored, which, when executed by the processor, implement the engineering anomaly monitoring method of video frame difference analysis according to any one of claims 1 to 9.

Images