Outdoor box video monitoring method, device and system based on multi-dimensional fusion

Abstract

The invention discloses an outdoor box video monitoring method, device and system based on multi-dimensional fusion. The system has the advantages that the visible light camera adopted by the main lens collects the physical state of the surface of the box body and the peripheral visible dynamic state, and the short-wave infrared equipment is adopted by the auxiliary lens to collect invisible characteristics and temperature distribution; feature points are extracted to calculate similarity, pixel-level matching is realized through a constructed perspective transformation matrix, and a correlation database is constructed; based on the multispectral data, three types of interaction events of biological interference, natural action and environmental transaction are identified in combination with deep learning, a fault and event association data set is constructed, a weight is assigned, a baseline threshold is set, and early warning is triggered when the threshold is exceeded; after early warning, the starting time and position of an event are positioned, a risk network is constructed to calculate the contribution degree of the event to determine a core risk source, three levels of risks are divided, a traceability map is generated, a differentiated operation and maintenance scheme is generated, monitoring is optimized, and the accuracy and operation and maintenance efficiency of outdoor box monitoring are improved.

Description

Technical Field

[0001] This invention relates to the field of video surveillance technology, and in particular to an outdoor enclosure video surveillance method, device, and system based on multi-dimensional fusion. Background Technology

[0002] With the booming development of the information industry, outdoor video surveillance cabinets are becoming an increasingly important part of ensuring public safety and facility operation, with applications ranging from city streets and industrial parks to transportation hubs. The market has placed higher demands on the real-time performance, clarity, and stability of multi-device collaborative monitoring of these surveillance images.

[0003] With the increasing demand for outdoor enclosure video surveillance, monitoring systems need to simultaneously process multiple types of data, including real-time video streams, environmental parameter monitoring data layers, and alarm information display layers. However, the complex outdoor environment and the technical characteristics of different monitoring scenarios pose numerous challenges to video surveillance processing, making efficient multi-dimensional fusion monitoring a key requirement for industry development. While existing technologies can meet basic monitoring needs, they have significant shortcomings:

[0004] Currently, multi-dimensional data processing relies heavily on microcontrollers or general-purpose processors. Microcontrollers have limited computing power and are prone to latency and frame drops when dealing with multiple channels of high-definition video data. While general-purpose processors offer better performance, they suffer from high power consumption and heat dissipation issues in the complex environment of outdoor enclosures, making it difficult to meet the low-power and high-reliability requirements of outdoor scenarios. Although processing solutions based on some dedicated chips have improved in recent years, most have not achieved fine-grained fusion and scheduling of multi-dimensional data. They employ a single data processing mechanism, requiring full processing even when only some dimensions of data need updating, resulting in resource waste. Furthermore, the format differences between different types of data make data mismatches prone to occur during fusion. In multi-enclosure cluster monitoring scenarios, the errors relying on traditional synchronization mechanisms often exceed expectations, and the lack of dynamic adjustment in load distribution can easily lead to local overload when multiple channels of high-definition video are transmitted centrally, resulting in poor synchronization and low processing efficiency in outdoor enclosure video monitoring. Summary of the Invention

[0005] The purpose of this invention is to solve the problems of poor synchronization and low processing efficiency, and to propose an outdoor box-type video surveillance method, device and system based on multi-dimensional fusion.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] Outdoor enclosure video surveillance methods based on multi-dimensional fusion include:

[0008] S1. Multispectral video acquisition and feature alignment: Through the coordinated acquisition of visible light and infrared spectra, the visible and latent features of the box surface and surrounding environment are captured in all dimensions, and spatiotemporal correlation is established through precise registration.

[0009] S2. Interactive event semantic parsing and weight modeling: Based on the multispectral video data obtained in S1, identify the interactive events between the enclosure and the surrounding environment, and construct an event risk weight database;

[0010] S3, Multi-dimensional Fusion Decision Making and Risk Source Tracing Analysis: When S2 triggers an early warning, it calls up the multispectral video clips from the same period for in-depth analysis, locates the starting point of the interactive event and constructs the risk transmission path;

[0011] S4, Operation and Maintenance Response Execution and Health Closed-Loop Management: Based on the core risk sources, risk levels and source tracing visualization maps output by S3, differentiated response strategies are generated to handle risks, and process data is fed back into the historical database to achieve continuous optimization of outdoor enclosure monitoring.

[0012] As a further improvement of the present invention, the specific implementation process of step S1 includes:

[0013] Video images are acquired by deploying a main and secondary collaborative lens group. The main lens uses a visible light camera to capture changes in the physical state of the box surface and visible dynamic visual features of the surrounding environment.

[0014] The secondary lens is coaxially mounted with the main lens and equipped with a short-wave infrared device to simultaneously collect the hot spots of moisture distribution on the surface of the box, the infrared reflection trajectory of vegetation, and the temperature distribution differences.

[0015] Extract geometric feature points from the corner points of the box edge and the vertices of the surrounding fixed reference objects in the visible light image, and thermal feature points of the corresponding areas in the infrared image;

[0016] Let the descriptor vector of any geometric feature point in a visible light image be... The descriptor vector of any thermal feature point in the infrared image is The descriptor vector contains the grayscale value changes, pixel gradient direction, and texture density within a 3×3 pixel region surrounding the current feature point;

[0017] Through formula The feature similarity between the geometric features of the visible light image and the thermal features of the infrared image is obtained, where, Represents the first in a visible light image The descriptor vector of the geometric feature point is the first Each component value; Indicates the first in the infrared image The descriptor vector of the i-th thermal feature point Each component value; , This represents the total number of feature points;

[0018] When the feature similarity is greater than a preset threshold, it is determined to be a preliminary matching point pair. Based on the preliminary matching point pair, a 3×3 perspective transformation matrix from infrared image to visible light image is obtained, and the coordinates of each pixel in the infrared image are mapped to the coordinate system of the visible light image to construct a database linking visual scene and hidden features.

[0019] As a further improvement of the present invention, the specific operation steps of step S2 are as follows:

[0020] A framework for recognizing three core interactive events—biological interference, natural effects, and environmental anomalies—is established, and multispectral video is analyzed in real time using deep learning algorithms.

[0021] Identification of biological interference events, including:

[0022] Rodents gnawing on the bottom of the enclosure: The baseline of the bottom of the enclosure is located based on the geometric feature points of the visible light image. The area 50cm above the bottom edge and extending 30cm to each side is selected as the key monitoring area. The boundary is corrected by combining the temperature gradient of the infrared image.

[0023] Extract the damaged contour of the box surface, and calculate the perimeter, area and average pixel gray value of the damaged area to establish a damage feature vector;

[0024] Perform inter-frame difference analysis on consecutive frames. If the overlap of the damaged areas is greater than a preset threshold and the expansion direction is consistent, and the distance between the hot spot and the damaged area is ≤3 pixels and the duration exceeds a preset threshold, it is determined to be a rodent bite.

[0025] Through formula The daily expansion area of ​​the bite marks was obtained, among which, Indicates the first The area of ​​the newly added biting area, This represents the number of newly detected areas within 24 hours.

[0026] As a further improvement of the present invention, the specific identification process for the natural action type event identification is as follows:

[0027] Natural action event identification, including:

[0028] Rainwater runoff trajectory: Identification is based on brightness changes in visible light images and temperature gradient changes in infrared images; the duration of fixed rainwater runoff trajectories is calculated using trajectory repeatability, and then... (The sentence is incomplete and requires more context to translate accurately.) The trajectory repeatability is obtained, where and These are the sets of flow paths for two consecutive days. When R ≥ 0.8 and the duration is ≥ 3 days, the trajectory is considered fixed.

[0029] Leaf accumulation is identified through color space analysis and texture feature detection and compared with a preset threshold. When the accumulation exceeds 10% of the preset threshold, it is determined to be leaf accumulation.

[0030] Snow melting is identified by infrared temperature distribution and changes in the area of ​​visible white regions.

[0031] As a further improvement of the present invention, the specific implementation process of the environmental anomaly event identification is as follows:

[0032] Dust adhesion: A multispectral image of the clean state of the container is selected as the reference image template; the gray value of the image pixels is linearly corrected based on the ratio of the average gray value of the real-time image and the reference image.

[0033] The high-texture area on the surface of the enclosure is selected as the detection area for sharpness testing; the formula is used... The clarity index is obtained, among which This represents the total number of pixels in the detection area. Represents the coordinates within the detection area Pixel edge sharpness; This represents the average edge sharpness within the detection area; the sharpness degradation rate is obtained based on the sharpness index of real-time and reference images;

[0034] When the rate of decrease in sharpness of 5 consecutive frames is greater than a preset threshold, and the rate of decrease in a single frame does not fluctuate by more than 5%, the decrease in sharpness is determined to be caused by dust.

[0035] Strong wind impacts are identified by the box's swaying angle and the impact audio characteristics, and the number of impacts within a period of time is counted.

[0036] Let the continuous video frame sequence be Each frame of the image is preprocessed with Gaussian filtering and the weighted difference between adjacent frames is calculated. A 3×3 cross-shaped structuring element is used for opening operation to denoise the image, and a 7×7 square structuring element is used for closing operation to fill the gaps. The box failure records and corresponding video surveillance data within the period are collected to construct a fault and event association dataset, and the interference type events are weighted.

[0037] As a further improvement of the present invention, the specific operation steps of step S3 are as follows:

[0038] When an alert is triggered, the 30 minutes before and after the alert are set as the analysis window. If all data points in the window exceed the baseline threshold, the start time of the window is taken as the start time of the event. Extract the centroid coordinates of the motion area of ​​the first frame in which the alert appears, and combine them with the dual-spectral registration results to determine the physical location, including the bottom gap of the box where rodents first appear, the box seam where rainwater first flows, and the accurate area on the top of the box where birds build nests.

[0039] Extract interference events within the window to construct a set, including the interference event type, weight value, and occurrence timestamp, using a formula. Obtaining interference events and The relationship between the values; where, Indicates interference event and The time interval between occurrences; These represent interference events. and The corresponding weights are assigned.

[0040] If the relation metric exceeds the preset threshold and meets the time-series conditions, it is determined to be... for The preceding causes; using interfering events as nodes, transmission relationships as directed edges, and relationship metrics as edge weights, a multi-event linkage risk network is constructed;

[0041] Combining the weight values ​​of interfering events with the relationship metrics, using the formula Receive event contribution points;

[0042] The core risk sources are identified by sorting them in descending order of contribution. Based on the contribution of the core risk sources and the transmission range of related events, three levels of risk are divided, and a source tracing visualization map is generated. Different colors are used to mark event types and line thickness to reflect the degree of correlation.

[0043] As a further improvement of the present invention, the specific operation process of step S4 is as follows:

[0044] Generate differentiated operation and maintenance solutions:

[0045] Level 1 risks are assigned the highest response priority, with on-site maintenance teams carrying specialized tools to carry out on-site repairs, prioritizing power interfaces and signal modules.

[0046] Level 2 risks are assigned the second highest priority and are handled by the regional operations and maintenance team for non-core areas such as side ventilation holes and bottom support structures.

[0047] Level 3 risks are assigned a general priority and do not require on-site handling. Multispectral video data is extracted at 8-hour intervals to analyze the event's contribution and core risk sources in real time and execute corresponding solutions.

[0048] A primary and secondary collaborative camera group was used to collect video of the entire operation and maintenance process, and the visible light geometric feature points and infrared thermal feature points before and after the operation and maintenance were compared; the operation and maintenance personnel uploaded operation records through mobile terminals, including operation and maintenance timestamps, tool models and descriptions of the status of core risk sources, and synchronously linked them to the event risk weight database.

[0049] The operation and maintenance data is fitted to the fault and event association dataset. The event baseline threshold is calibrated based on 3 months of fault-free multispectral data. The recurrence rate of events after 6 months of operation and maintenance is calculated, and the event weight value is adjusted.

[0050] A second aspect of the present invention provides an outdoor enclosure video surveillance device based on multi-dimensional fusion, comprising:

[0051] The multispectral acquisition device includes a main and secondary collaborative lens group. The main lens is a high-definition visible light camera, and the secondary lens is equipped with a short-wave infrared device to acquire multispectral data of the outdoor enclosure and its surroundings, and communicates with the data processing equipment.

[0052] The data processing equipment is used to receive data from the multispectral acquisition equipment, extract feature points and calculate feature similarity, achieve pixel-level matching, build a feature association database, identify three types of core interactive events, build a fault and event association dataset and assign weights, set baseline thresholds, and communicate with the decision analysis equipment.

[0053] The decision analysis equipment is used to receive early warning signals from the data processing equipment, determine the start time and location of events, build a multi-event linkage risk network, identify core risk sources and classify risk levels, generate a source tracing visualization map, and communicate with the operation and maintenance management equipment.

[0054] Operation and maintenance management equipment is used to generate differentiated operation and maintenance plans based on risk levels, monitor the operation and maintenance process, record operation data, and feed back into the database to calibrate baseline thresholds and adjust event weight values.

[0055] A third aspect of the present invention provides an outdoor enclosure video surveillance system based on multi-dimensional fusion, comprising:

[0056] Multispectral video acquisition and feature alignment module: The main lens uses a high-definition visible light camera to capture changes in the physical state of the box surface and visible dynamic visual features of the surrounding environment; the secondary lens is coaxially mounted with the main lens to synchronously acquire the invisible features and temperature distribution differences on the box surface, extract geometric feature points from the visible light image and thermal feature points from the infrared image, calculate feature similarity, achieve pixel-level matching through perspective transformation matrix, and build a database linking visual scenes and invisible features.

[0057] Interactive event semantic parsing and weight modeling module: Based on multispectral video data, combined with deep learning algorithms, it identifies three types of core interactive events: biological interference, natural effects, and environmental anomalies. It constructs a fault and event association dataset, assigns weights to various events, sets baseline thresholds, and triggers an early warning when the event feature parameters continuously exceed the threshold.

[0058] Multi-dimensional integrated decision-making and risk tracing analysis module: After an early warning is triggered, the analysis window is set for 30 minutes before and after the trigger. The module determines the start time and physical location of the event, extracts interference events to construct a set, calculates the relationship measurement between events, determines the pre-existing causes and constructs a multi-event linkage risk network, calculates the contribution of events to determine the core risk source, classifies the risk into three levels, and generates a source tracing visualization map.

[0059] Operation and maintenance response execution and health closed-loop management module: Generate differentiated operation and maintenance plans based on risk level. During operation and maintenance, the main and secondary collaborative camera groups monitor and associate operation records, feed the operation and maintenance data back to the database, calibrate baseline thresholds and adjust event weight values.

[0060] The beneficial effects of this invention are:

[0061] This invention deploys a primary and secondary collaborative lens group to capture visible physical features such as rust and loose screws on the surface of outdoor enclosures, as well as dynamic visual information such as people approaching and leaves falling, using a high-definition visible light camera. It also uses a short-wave infrared device to simultaneously collect hidden features such as water hotspots from water seepage in the enclosure joints and differences in surface temperature distribution. Based on geometric and thermal feature point extraction, feature similarity calculation, and perspective transformation matrix operation, pixel-level feature alignment is achieved and a database linking visual scenes and hidden features is constructed, significantly improving the completeness and efficiency of outdoor enclosure status monitoring.

[0062] This invention constructs a cross-modal recognition framework for three types of core interactive events, and combines deep learning algorithms to automatically analyze biological interference, natural effects, and environmental anomalies. It establishes a fault association dataset through quantitative judgment rules and event weight assignment. After an early warning is triggered, it can locate the starting position of the event, construct a multi-event linkage risk network, divide the risk into three levels based on the event contribution, and generate a source tracing visualization map. At the same time, it matches differentiated operation and maintenance solutions based on risk levels, and feeds operation and maintenance data back into the database to calibrate baseline thresholds and adjust event weights, significantly improving the accuracy of risk identification and the efficiency of operation and maintenance response. Attached Figure Description

[0063] Figure 1 This is a flowchart of the method of the present invention; Detailed Implementation

[0064] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0065] like Figure 1 As shown, the present invention is an outdoor box-type video surveillance method based on multi-dimensional fusion, including multispectral video acquisition and feature alignment, semantic parsing of interactive events and weight construction, multi-dimensional fusion decision-making and risk tracing analysis.

[0066] S1. Multispectral Video Acquisition and Feature Alignment: Through the coordinated acquisition of visible and infrared spectra, the system achieves full-dimensional capture of both explicit and implicit features of the enclosure surface and its surrounding environment. Precise registration establishes spatiotemporal correlations, providing high-quality data support for subsequent interactive event analysis. The specific implementation process is as follows:

[0067] Video images are acquired by deploying a primary and secondary coordinated camera group, including:

[0068] The main lens uses a high-definition visible light camera to cover the outdoor enclosure surface and a visual scene within a 3-meter radius around it. It is used to capture the visible dynamic visual features of changes in the physical state of the enclosure surface and the surrounding environment, including: the expansion of rust area on the enclosure surface, the trajectory of paint peeling, and the visual features of loose screws; the approaching behavior of people or machinery in the vicinity, the falling trajectory of fallen leaves or debris, and the real-time flow path of rainwater on the enclosure surface.

[0069] The secondary lens is coaxially mounted with the main lens on the same bracket and equipped with a short-wave infrared device to simultaneously collect the invisible features on the surface of the enclosure. The invisible features include water distribution hot spots and the infrared reflection trajectory of vegetation. Water distribution hot spots refer to the irregular high-reflectivity areas in the infrared image caused by hidden water seepage from the enclosure seams or heat dissipation holes. Changes in the area of ​​the high-reflectivity areas can reflect the degree of water seepage. At the same time, the infrared device captures the temperature distribution differences on the surface of the enclosure.

[0070] Geometric feature points are extracted from the box edge corners and the vertices of the surrounding fixed reference objects in the visible light image. At the same time, thermal feature points of the corresponding area are extracted from the infrared image, such as the infrared radiation difference line of the box edge and the infrared contour vertices of the reference objects. The number of geometric feature points in the visible light image and the number of thermal feature points in the infrared image are both ≥20 and are evenly distributed in the entire image area.

[0071] Let the descriptor vector of any geometric feature point in a visible light image be... The descriptor vector of any thermal feature point in the infrared image is The descriptor vector includes the grayscale value variation, pixel gradient direction, and texture density within a 3×3 pixel region surrounding the current feature point.

[0072] Through formula The similarity between the geometric features of the visible light image and the thermal features of the infrared image is calculated. Represents the first in a visible light image The descriptor vector of the geometric feature point is the first Each component value; Indicates the first in the infrared image The descriptor vector of the i-th thermal feature point Each component value; , This represents the total number of feature points;

[0073] When the feature similarity is greater than a preset threshold, it is determined to be a preliminary matching point between the geometric features of the visible light image and the thermal features of the infrared image. ,in, Represents the coordinates of geometric feature points in a visible light image. The coordinates of the thermal feature points in the infrared image are given; a 3×3 perspective transformation matrix from the infrared image to the visible light image is obtained based on the initial matching point pairs.

[0074] By performing perspective transformation matrix operations, the coordinates of each pixel in the infrared image are accurately mapped to the coordinate system of the visible light image, achieving pixel-level matching and constructing a database linking visual scenes and hidden features. Pixel-level matching means that the coordinate deviation between the feature point in the infrared image and the corresponding physical location in the visible light image is ≤1 pixel.

[0075] S2. Interactive Event Semantic Analysis and Weight Modeling: Based on the multispectral video data acquired in S1, interactive events between the enclosure and its surrounding environment are identified, and an event risk weight database is constructed to provide quantitative basis for subsequent time-series analysis and early warning decision-making. The specific implementation process is as follows:

[0076] A framework for recognizing three types of core interactive events was established, and multispectral videos were analyzed in real time using deep learning algorithms. The three types of core interactive events include biological interference events, natural effects events, and environmental anomaly events.

[0077] Biological interference events include behavioral characteristics such as rodents gnawing on the bottom of the enclosure, birds nesting on the top of the enclosure, and small mammals climbing the enclosure; among which:

[0078] Rodent bites are identified by detecting the continuous damage and expansion trajectory of the bottom edge area of ​​the enclosure: the baseline of the bottom of the enclosure is located by geometric feature points in the visible light image, and a rectangular area extending 50cm upward from the bottom edge of the enclosure and 30cm to each side is selected as the key monitoring area; the boundary of the monitoring area is corrected by combining the temperature gradient changes of the area where the enclosure is in contact with the ground in the infrared image.

[0079] An edge detection algorithm is used to extract the damaged contour of the box surface within the monitoring area. The perimeter, area and average pixel gray value of the damaged area in each frame image are calculated to establish a damage feature vector.

[0080] Abnormal thermal signals within the monitoring area are detected using a 3×3 sliding window. When there are continuous hot spots of 38-42℃ and an area ≥5×5 pixels, they are marked as suspicious activity areas.

[0081] Inter-frame difference calculation is performed on continuous video frames. If the overlap of the damaged areas between adjacent frames is greater than a preset threshold and the expansion direction is consistent, it is determined to be a continuous damaged trajectory. The spatial correlation between infrared hot spots and damaged areas is verified simultaneously. When the distance between the center of the hot spot and the damaged outline is ≤3 pixels and the duration of the hot spot is greater than a preset threshold, it is determined to be a causal relationship between biological activity and the formation of damage.

[0082] When the duration of continuous damage and hot spots is greater than a preset threshold, it is determined to be rodent bite.

[0083] Through formula The daily expansion area of ​​the bite marks was calculated, where, Indicates the first The area of ​​the newly added biting area, This represents the number of newly detected areas within 24 hours.

[0084] Bird nesting behavior is determined by identifying the material accumulation pattern in the top area of ​​the box and the duration of bird stay. When the volume of nesting material accumulation is greater than a preset threshold and the bird stays for ≥5 minutes, it is determined to be bird nesting.

[0085] Small mammal climbing is identified by detecting the movement trajectory characteristics on the vertical surface of the box, with the climbing speed range set at 0.1 to 0.8 m / s and the verticality of the movement trajectory ≥70°;

[0086] Natural event recognition includes changes in the flow trajectory of rainwater along the surface of the enclosure, accumulation of fallen leaves covering the ventilation holes, and the melting rate of snow on the top of the enclosure; among these,

[0087] Rainwater runoff trajectories are identified by combining brightness changes in visible light images and temperature gradient changes in infrared images; the duration of fixed rainwater runoff trajectories is calculated using the trajectory repeatability formula. The trajectory repeatability is calculated, where and These are the sets of flow paths for two consecutive days. When R ≥ 0.8 and the duration is ≥ 3 days, the trajectory is considered fixed.

[0088] Leaf accumulation is identified by color space analysis and texture feature detection and compared with a preset threshold. When the detected value exceeds 10% of the preset threshold, it is determined to be leaf accumulation.

[0089] Snow melting is identified by infrared temperature distribution and changes in the area of ​​white regions in visible light;

[0090] Environmental anomaly event identification includes the rate at which construction dust adheres to the enclosure, the frequency of collisions between the enclosure and surrounding objects caused by strong winds, and the impact of changes in ambient light intensity on the enclosure's surface temperature; among which:

[0091] Dust adhesion is detected by the decrease in image sharpness and the change in the reflectivity of the enclosure surface:

[0092] A multispectral image of the clean state of the enclosure was selected as the reference image template; Gaussian filtering was used to denoise the real-time acquired image, smoothing high-frequency noise and preserving the macroscopic features of the enclosure surface; based on the ratio of the average gray value of the real-time image to the reference image, the gray value of each pixel in the real-time image was linearly corrected.

[0093] The high-texture areas on the cabinet surface are selected as the detection areas for sharpness testing. The high-texture areas include the metal nameplate, the screw arrangement area, and the cabinet splicing seams.

[0094] Through formula The clarity index is calculated, where This represents the total number of pixels in the detection area. Represents the coordinates within the detection area Pixel edge sharpness; This represents the average edge sharpness within the detection area;

[0095] The sharpness reduction rate is obtained by comparing the sharpness index of the real-time image with the sharpness index of the corresponding reference image.

[0096] When the rate of decrease in sharpness of 5 consecutive frames is greater than a preset threshold, and the rate of decrease in a single frame does not fluctuate by more than 5%, it is determined that the sharpness is significantly reduced due to dust.

[0097] Strong wind collisions are identified by the box's swaying angle and collision audio characteristics. The swaying angle threshold is set at 2°, and the peak value of the collision audio spectrum characteristics is ≥60dB. The frequency of collision events is obtained by statistically analyzing the number of collisions within a period of time.

[0098] The influence of light intensity was identified through temporal variation analysis of infrared temperature distribution, with the temperature fluctuation range set at ±3℃;

[0099] Let the continuous video frame sequence be Each frame of the image is preprocessed with Gaussian filtering and the weighted difference between adjacent frames is calculated; morphological filtering is performed on the moving regions, using 3×3 cross-shaped structuring elements for opening operations to remove noise and 7×7 square structuring elements for closing operations to fill holes; an interactive event weight assignment database based on historical fault data is constructed.

[0100] Collect enclosure failure records and corresponding video surveillance data within a specified period to construct a failure and event association dataset; extract interaction event features from the 30 days prior to each failure case, including event frequency, event intensity, and event duration.

[0101] Weights are assigned to interference-type events based on fault and correlation datasets combined with interaction event features; where:

[0102] The weight of the rodent biting event is 9, the weight of the rainwater track fixation event is 8, the weight of the bird nesting event is 7, the weight of the strong wind collision event is 6, the weight of the snow melting anomaly is 5, the weight of the fallen leaf accumulation event is 4, the weight of the dust attachment event is 3, the weight of the small animal climbing event is 3, and the weight of the light intensity change is 2.

[0103] Baseline thresholds for various events are set based on historical data from a 12-month period of no failures; an early warning is triggered when the characteristic parameters of one or more types of events in the interference type exceed the baseline threshold for N consecutive days.

[0104] S3. Multi-dimensional Fusion Decision-Making and Risk Source Tracing Analysis: When S2 triggers an early warning, it calls upon concurrent multispectral video clips for in-depth analysis to locate the starting point of the interaction event and construct the risk transmission path, providing decision support for precise operation and maintenance. The specific implementation process is as follows:

[0105] When an alert is triggered, the 30 minutes before and after the alert is triggered will be set as the analysis window. When all data points in the window exceed the baseline threshold, the window start time will be used as the event start time.

[0106] Extract the centroid coordinates of the motion region in the first frame of the warning, and combine them with the dual-spectral registration results established by S1 to determine the physical location; the specific positioning process includes the location of the bottom gap of the box where rodents first appear, the location of the box seam where rainwater first flows, and the accurate area of ​​the top of the box where birds build nests.

[0107] Extract interference type events within the analysis window to construct an event set, where, , Indicates the type of interference event; Indicates the weight value of the interfering event; Represents the timestamp of the interference event; using the formula The interference events were calculated. and The relationship between the values; where, Indicates interference event and The time interval between occurrences; These represent interference events. and The corresponding weights are assigned; the higher the relational metric, the stronger the correlation between the interfering events.

[0108] When the relationship metric value is greater than the preset threshold, and At that time, it was determined to be for Precursor causes, marked as ;

[0109] A multi-event linkage risk network is constructed using interference events as nodes, transmission relationships as directed edges, and relationship metrics as edge weights.

[0110] Based on the weight values ​​and relationship metrics of interfering events, using the formula... The contribution of the event is calculated.

[0111] All events are sorted in descending order based on their contribution. The top two events are taken as interference events and core risk sources. If there are events with the same contribution value, the number of related events of the interference event is further compared. Events with more related events are marked as core risk sources. If the contribution value of all events is less than the preset threshold, it is determined that there is no single core risk source. The top three interference events in terms of event contribution are merged and marked as joint core risk sources.

[0112] Based on the contribution of core risk sources to events and the transmission range of related events, three levels of risk are defined:

[0113] Level 1 Risk: The contribution of the core risk source events exceeds the preset threshold, and the associated events cover the core functional areas of the enclosure, including the power interface and signal module.

[0114] Level 2 risk: The contribution of the core risk source events is within the preset range and the related events cover the non-core areas of the enclosure, including the side ventilation holes and the bottom support structure.

[0115] Level 3 risk: The contribution of the core risk source event is less than the preset threshold, and there is no related event transmission;

[0116] For core risk sources and transmission paths, a source tracing visualization map is generated: starting from the core risk source and ending with the final warning event, different colors are used to mark event types, including orange for biological interference, blue for natural effects, and gray for environmental anomalies. The thickness of the lines corresponds to the measurement value, where the thicker the line, the stronger the correlation.

[0117] S4. Operation and Maintenance Response Execution and Health Closed-Loop Management: Based on the core risk sources, risk levels, and source tracing visualization maps output by S3, differentiated response strategies are generated to handle risks, and process data is fed back into the historical database to achieve continuous optimization of outdoor enclosure monitoring. The specific implementation process is as follows:

[0118] S401, Generation of Differentiated Operation and Maintenance Solutions:

[0119] Level 1 Risk Response: Set the response priority to the highest level; based on the core risk sources and the types of interference events, the on-site maintenance team will carry specialized processing tools to carry out on-site repairs, prioritizing the core functional areas;

[0120] Level 2 Risk Response: Set the response priority to the second highest level: Based on the core risk sources and the types of interference events, the regional operations and maintenance team will handle non-core areas.

[0121] Level 3 Risk Response: Set the response priority to general level: No on-site handling required. Monitor the abnormal area in real time, extract multispectral video data at fixed time intervals of 8 hours, analyze the contribution of the event and the core risk sources in real time, determine the risk level and execute the corresponding operation and maintenance plan.

[0122] S402. Dynamic Monitoring and Recording of Operation and Maintenance Processes: During the operation and maintenance personnel's processing, the primary and secondary collaborative camera group deployed in S1 is activated to collect video of the entire process, enabling visual traceability of operation and maintenance behavior and secondary risk avoidance.

[0123] Based on the visual scene and hidden feature association database established by S1, the visible light geometric feature points and infrared thermal feature points before and after operation and maintenance are compared in real time to ensure that the operation and maintenance operation does not cause physical damage to the surface of the enclosure or abnormality of the hidden features.

[0124] Operation records are uploaded in real time by maintenance personnel via mobile terminals. The records include: start and end timestamps of maintenance, model of the handling tool, and a description of the status of the core risk source before and after handling. The records are also synchronously linked to the event risk weight database built by S2 to form a one-to-one correspondence between early warning events and maintenance operations.

[0125] S403, Historical Database Iteration:

[0126] All data from the operation and maintenance process, including response records, effect verification results, and feature change curves, are fitted to the S2 fault and event association dataset:

[0127] Baseline threshold update: The event baseline threshold of step S2 is calibrated based on multispectral data from a fault-free period within 3 months.

[0128] Weighting adjustment: Statistically analyze the recurrence rate of different interactive events after maintenance within 6 months and adjust the event weight values ​​of step S2 accordingly; for example, if the recurrence rate of snow melting abnormal events decreases from 30% to 5% after maintenance, the corresponding weight value can be reduced from 5 to 4 to adapt the weight value to the current container conditions.

[0129] This invention relates to an outdoor enclosure video surveillance device based on multi-dimensional fusion, comprising:

[0130] Multispectral acquisition equipment: Deploy a main and secondary collaborative lens group. The main lens adopts a high-definition visible light camera to cover the outdoor box surface and the surrounding 3-meter visual scene, which is used to capture the physical state changes of the box surface and the visible dynamic visual characteristics of the surrounding environment.

[0131] The secondary lens is coaxially mounted on the same bracket as the main lens, equipped with a short-wave infrared device to simultaneously collect invisible features such as water distribution hot spots and infrared reflection trajectories of vegetation on the surface of the box, and capture the temperature distribution differences on the surface of the box. It also communicates with the data processing equipment to transmit the collected multispectral video data in real time.

[0132] Data processing equipment: used to receive data transmitted by multispectral acquisition equipment, extract geometric feature points of box edge corners and surrounding reference vertices in visible light images, as well as thermal feature points of corresponding areas in infrared images, calculate feature similarity and achieve pixel-level matching through perspective transformation matrix, and build a database linking visual scenes and latent features.

[0133] Simultaneously, deep learning algorithms are used to identify three types of core interactive events: biological interference, natural effects, and environmental anomalies. Historical fault records and corresponding video data are collected to construct a fault and event association dataset. Weights are assigned to various events and baseline thresholds are set. At the same time, communication is established with decision analysis equipment to send early warning signals when event characteristic parameters continuously exceed the threshold.

[0134] Decision analysis equipment: It is used to receive early warning signals from data processing equipment, set the 30 minutes before and after the early warning as the analysis window, determine the start time and physical location of the event, extract interference events to construct a set and calculate the relationship metric between events, construct a multi-event linkage risk network, determine the core risk source based on the contribution of events, divide the risk into three levels, generate a source tracing visualization map with different colors to mark event types and line thickness to reflect the degree of correlation, and communicate with operation and maintenance management equipment to transmit risk analysis results;

[0135] Operation and maintenance management equipment: used to generate differentiated operation and maintenance solutions based on risk level: Level 1 risks are handled by the on-site team with dedicated tools in the core area; Level 2 risks are handled by the regional team in the non-core area; Level 3 risks are monitored and analyzed at 8-hour intervals; during operation and maintenance, multi-spectral acquisition equipment is used to monitor the process, receive operation records uploaded by operation and maintenance personnel through mobile terminals, feed the operation and maintenance data back into the fault and event correlation dataset, calibrate the baseline threshold based on 3 months of fault-free data, and adjust the weight value based on the 6-month event recurrence rate to achieve system optimization.

[0136] This invention is an outdoor enclosure video surveillance system based on multi-dimensional fusion, comprising:

[0137] Multispectral video acquisition and feature alignment module: Data is acquired by deploying a main and secondary collaborative lens group. The main lens uses a high-definition visible light camera to cover the outdoor box surface and a 3-meter radius around it, capturing changes in the physical state of the box surface and visible dynamic visual features of the surrounding environment.

[0138] The secondary lens is coaxially mounted with the main lens and equipped with a short-wave infrared device to simultaneously acquire the invisible features on the surface of the box and capture the temperature distribution differences. It extracts the geometric feature points of the visible light image and the thermal feature points of the infrared image, calculates the feature similarity, and achieves pixel-level matching through the perspective transformation matrix to build a database that associates visual scenes with invisible features.

[0139] Interactive event semantic parsing and weight modeling module: Based on multispectral video data, it uses deep learning algorithms to identify three types of core interactive events; among which:

[0140] Biological disturbance events are identified, including behaviors such as rodent biting, bird nesting, and small mammal climbing; natural effects events are identified, including rainwater flow paths, fallen leaf accumulation, and snow melting; environmental anomalies are identified, including dust accumulation, strong wind impacts, and the impact of light intensity; at the same time, a fault and event association dataset is constructed, weights are assigned to various events, baseline thresholds are set, and an early warning is triggered when the event feature parameters continuously exceed the threshold.

[0141] Multi-dimensional integrated decision-making and risk tracing analysis module: After an early warning is triggered, the 30 minutes before and after the trigger are set as the analysis window to determine the start time and physical location of the event; extract interfering events to construct a set, calculate the relationship metric between events, determine the pre-existing causes and construct a multi-event linkage risk network, calculate the event contribution to determine the core risk source; based on the event contribution of the core risk source and the transmission range of related events, classify the risk levels into first-level, second-level, and third-level, generate a source tracing visualization map, mark the event type with different colors, and use the thickness of the lines to represent the degree of correlation;

[0142] Operation and Maintenance Response Execution and Health Closed-Loop Management Module: Generates differentiated operation and maintenance solutions based on risk level. Level 1 risks are repaired on-site by the on-site operation and maintenance team, prioritizing core functional areas; Level 2 risks are handled by the regional operation and maintenance team for non-core areas; Level 3 risks do not require on-site handling, but are monitored in real time and analyzed periodically.

[0143] During operation and maintenance, the system monitors through a primary and secondary collaborative camera group. Operation and maintenance personnel upload operation records, and the operation and maintenance data is fed back into the database. Based on three months of fault-free data, the baseline threshold is calibrated, and the weight assignment is adjusted by statistically analyzing the event recurrence rate over six months, thereby achieving continuous optimization of the monitoring system.

[0144] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. An outdoor box video monitoring method based on multi-dimensional fusion, characterized in that, Comprise: S1, multispectral video acquisition and feature alignment: through the cooperative acquisition of visible light and infrared spectrum, the dominant and recessive features of the box surface and the surrounding environment are captured in all dimensions, and the space-time correlation is established through accurate registration; S2, interactive event semantic analysis and weight modeling: based on the multispectral video data obtained by S1, the interactive events of the box and the surrounding environment are identified, and the event risk weight database is constructed; S3, multi-dimensional fusion decision and risk traceability analysis: when S2 triggers the alarm, the synchronous multispectral video segment is called for in-depth analysis, the starting point of the interactive event is located, and the risk transmission path is constructed; S4, operation and maintenance response execution and health closed-loop management: based on the core risk source, risk level and traceability visualization atlas output by S3, a differentiated response strategy is generated to handle the risk, and the process data is fed back to the historical database to realize the continuous optimization of outdoor box monitoring. 2.The multi-dimension fusion based outdoor cabinet video monitoring method according to claim 1, characterized in that, The specific implementation process of the S1 step comprises: Video images are collected by deploying a main and auxiliary cooperative lens group, the main lens uses a visible light camera to capture the physical state changes of the box surface and the visible dynamic visual features of the surrounding environment; The auxiliary lens is coaxially installed with the main lens and carries a short-wave infrared device to synchronously collect the water distribution hot spots on the box surface, the infrared reflection trajectory of vegetation and the temperature distribution difference; The geometric feature points of the box edge corner points and the surrounding fixed reference point vertices in the visible light image are extracted, and the thermal feature points in the corresponding area of the infrared image are extracted; Let the descriptor vector of any geometric feature point in the visible light image be , and the descriptor vector of any thermal feature point in the infrared image be , where the descriptor vector contains the gray value variation, pixel gradient direction, and texture density in the 3x3 pixel region around the current feature point. The feature similarity between the geometric features of the visible light image and the thermal features of the infrared image is obtained by the formula , wherein represents the value of the i-th component of the descriptor vector of the j-th geometric feature point in the visible light image; represents the value of the i-th component of the descriptor vector of the j-th thermal feature point in the infrared image; , is the total number of feature points;​​​​ When the feature similarity is greater than a preset threshold, it is determined that it is a preliminary matching point pair, a 3x3 order perspective transformation matrix of the infrared image to the visible light image is obtained based on the preliminary matching point pair, and each pixel coordinate of the infrared image is mapped to the visible light image coordinate system to construct a visual scene and hidden feature correlation database. 3.The multi-dimension fusion based outdoor cabinet video monitoring method according to claim 1, characterized in that, The specific operation steps of the S2 step are as follows: A three-class core interactive event recognition framework including biological interference event recognition, natural action event recognition and environmental abnormal event recognition is established, and the multispectral video is analyzed in real time through a deep learning algorithm; The biological interference event recognition comprises: Rodent gnawing the bottom of the box: based on the visible light image geometric feature point positioning box bottom baseline, the bottom edge upward 50cm, the two sides each extending 30cm area is selected as the key monitoring area, combined with the infrared image temperature gradient correction boundary; The box surface damage contour is extracted, and the perimeter, area and pixel gray mean value of the damaged area are calculated to establish a damage feature vector; If the overlap degree of the damaged area is greater than a preset threshold and the expansion direction is consistent, and the distance between the hot spot and the damaged area is less than or equal to 3 pixels and the duration exceeds the preset threshold, it is determined that it is a rodent gnawing; The daily expansion area of the bite traces is obtained by the formula wherein, represents the area of the nth newly added bite region, is the number of newly added regions detected within 24 hours.​ 4.The multi-dimension fusion based outdoor cabinet video monitoring method according to claim 3, characterized in that, The specific identification process of the natural action event recognition is as follows: The natural action event recognition comprises: Rainwater flow trajectory: based on the brightness change of visible light image and the temperature gradient change of infrared image to identify; the fixation duration of rainwater flow trajectory is calculated by trajectory repetition degree, and the formula is obtained, wherein and are the flow path sets of adjacent two days, when R≥0.8 and the duration is ≥3, it is determined that the trajectory is fixed. Leaf accumulation is identified by color space analysis and texture feature detection and compared with a preset threshold, when more than 10% of the preset threshold is exceeded, it is determined that it is leaf accumulation; Snowmelt is identified by infrared temperature distribution and visible light white area area change. 5.The multi-dimension fusion based outdoor cabinet video monitoring method according to claim 1, characterized in that, The specific implementation process of the environmental abnormal event recognition is as follows: Dust adhesion: Select the multispectral image of the clean state of the box as the reference image template; based on the average gray value ratio of the real-time image and the reference image, linearly correct the gray value of the image pixel; A high-texture region on the surface of the box is selected as a detection region for the definition detection; a definition index is obtained through a formula wherein N represents the total number of pixel points in the detection region, Xi represents the coordinate of a pixel in the detection region, and the edge definition of the pixel; Xi represents the average value of the edge definition in the detection region; a definition decrease rate is obtained based on the definition indexes of the real-time and reference images; When the sharpness reduction rate of 5 consecutive images is greater than the preset threshold, and the single-frame reduction rate fluctuation is not more than 5%, it is determined that the sharpness reduction is caused by dust; Strong wind collision is identified by box swing angle and collision audio features, and the number of collisions in the cycle time is counted; Let the sequence of consecutive video frames be Gaussian filter preprocessing is performed on each frame of image and the weighted difference value of adjacent frames is calculated; Open operation denoising is performed using a 3x3 cross-shaped structure element, and closed operation hole filling is performed using a 7x7 square structure element; collect the box fault records in the cycle time and the video monitoring data in the corresponding period, construct the fault and event association data set, and assign weights to the interference type events. 6.The multi-dimension fusion based outdoor cabinet video monitoring method according to claim 1, characterized in that, The specific operation steps of the S3 step are as follows: When the warning is triggered, set 30 minutes before and after the warning as the analysis window, if all data points in the window are above the baseline threshold, use the start time of the window as the event start time; extract the motion region centroid coordinates of the first occurrence frame, and determine the physical location based on the dual-spectrum registration result, including the first occurrence of the rodent at the bottom gap of the box, the first fixed flow of the rainwater at the joint of the box, and the accurate area of the bird nest at the top of the box; The interference event set in the extraction window is constructed, including the interference event type, weight value and occurrence time stamp, and the relationship measure value between the interference events is obtained through the formula ​​​​​​​​​ If the relationship metric value exceeds the preset threshold and the timing condition is met, it is determined that To The predisposing cause; with the interference event as the node, the conduction relationship as the directed edge, and the relationship metric value as the edge weight, a multi-event linkage risk network is constructed; Combining the interference event weight value and the relationship measure value, an event contribution degree is obtained through a formula ​ Based on the descending order of contribution degree, the core risk source is determined; based on the contribution degree of the core risk source and the transmission range of the associated events, three risk levels are divided, and a traceable visualization map is generated, with different colors marking the event type and line thickness representing the correlation degree. 7.The multi-dimensional fusion based outdoor cabinet video monitoring method according to claim 1, characterized in that, The specific operation process of the S4 step is as follows: Generate a differentiated operation and maintenance scheme: The first level risk sets the highest response priority, and the on-site operation and maintenance team carries out on-site maintenance with special tools, and prioritizes the power interface and signal module; The second level risk sets the second highest priority, and the regional operation and maintenance team handles the non-core areas of the side heat dissipation hole and the bottom support structure; The third level risk sets a general priority, which does not require on-site disposal, and 8-hour interval multispectral video data is extracted for real-time analysis of event contribution and core risk source and execution of corresponding scheme; Enable the main and auxiliary cooperative lens group to collect the whole process video of operation and maintenance, compare the visible light geometric feature points and infrared thermal feature points before and after operation and maintenance; operation and maintenance personnel upload operation records through mobile terminal, including operation timestamp, tool model and core risk source state description, and synchronize associated event risk weight database; Fit the operation and maintenance data to the fault and event association data set, calibrate the event baseline threshold based on 3 months of fault-free multispectral data, count the recurrence rate after 6 months of event operation and maintenance, and adjust the event weight value.

8. The device for applying the outdoor box video monitoring method based on multi-dimensional fusion according to any one of claims 1-7, comprising: The multispectral acquisition device includes a main and auxiliary cooperative lens group, the main lens is a high-definition visible light camera, and the auxiliary lens is equipped with a short-wave infrared device for acquiring multispectral data of the outdoor box and the surrounding area, and is in communication connection with the data processing device; The data processing device is used for receiving data of the multispectral acquisition device, extracting feature points and calculating feature similarity, realizing pixel-level matching, constructing a feature correlation database, identifying three types of core interaction events, constructing a fault and event correlation dataset and assigning weights, setting a baseline threshold, and communicating with a decision analysis device; The decision analysis device is used for receiving early warning signals from the data processing device, determining event start time and location, constructing a multi-event linkage risk network, determining core risk sources and dividing risk levels, generating a traceable visualization map, and communicating with an operation and maintenance device; The operation and maintenance device is used for generating a differentiated operation and maintenance scheme based on risk levels, monitoring the operation and maintenance process, recording operation data, and feeding back the database to calibrate the baseline threshold and adjust the event weight value.

9. The system for the outdoor box video monitoring method based on multi-dimensional fusion according to any one of claims 1-7, comprising: a multispectral video acquisition and feature alignment module: a main lens adopts a high-definition visible light camera to capture physical state changes of the box surface and visible dynamic visual features of the surrounding environment; a sub-lens is coaxially installed with the main lens to synchronously acquire invisible features and temperature distribution differences of the box surface, extract visible light image geometric feature points and infrared image thermal feature points, calculate feature similarity, realize pixel-level matching through a perspective transformation matrix, and construct a visual scene and invisible feature correlation database; an interaction event semantic analysis and weight modeling module: based on multispectral video data, combined with a deep learning algorithm, three types of core interaction events, i.e., biological interference, natural action, and environmental abnormality, are identified, a fault and event correlation dataset is constructed, weights are assigned to each type of event, a baseline threshold is set, and early warning is triggered when event feature parameters continuously exceed the threshold; a multi-dimensional fusion decision and risk traceability analysis module: used for setting 30 minutes before and after triggering as an analysis window after early warning is triggered, determining event start time and physical location, extracting interference events to construct a set, calculating relationship measurement values between events, determining predisposing causes and constructing a multi-event linkage risk network, calculating event contribution to determine core risk sources, dividing three risk levels, and generating a traceable visualization map; an operation response execution and health closed-loop management module: based on risk levels, a differentiated operation and maintenance scheme is generated, operation and maintenance are monitored through a main and sub-cooperative lens group, operation records are associated, operation and maintenance data are fed back to the database, the baseline threshold is calibrated, and the event weight value is adjusted.

Images