Multi-scene internet-of-things off-site dynamic inspection self-adaptive monitoring system

Abstract

The invention discloses a multi-scene internet of things off-site dynamic inspection self-adaptive monitoring system, and relates to the technical field of intelligent inspection and path planning, the system is composed of a plurality of functional modules, and the system comprises a moving object detection module for detecting a moving object by adopting a Gaussian mixture background modeling and optical flow method; meanwhile, a light-weight MobileViT-XXS model is deployed, and a moving object target is tracked; the moving object classification and identification module is used for judging the type of a moving object through a multi-modal classification decision tree: when the moving object is judged to be a non-living body, carrying out ROI extraction and optimal frame interception on the image saliency of the moving object, and generating EXIF metadata through super-resolution reconstruction; when the judgment result is a biological object, biological category judgment is carried out: when the judgment result is an animal, geo-fencing data matching is carried out, and a spatial-temporal feature fusion model is adopted; and when human beings are judged, face quality evaluation is carried out through multi-frame fusion super-resolution.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent inspection and path planning, and specifically to a multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system. Background Art

[0002] In recent years, the cross-integration of artificial intelligence and computer vision technologies has spawned a new generation of technological innovation in intelligent monitoring systems. The iterative upgrades of deep learning-based target detection algorithms, such as YOLOv5, EfficientDet, multimodal feature extraction frameworks, and trajectory prediction models, have promoted the large-scale implementation of intelligent monitoring in security, smart cities, traffic management, industrial inspections, and other fields. Through the combination of real-time video stream analysis and edge computing capabilities, modern monitoring systems have initially achieved dynamic perception of people, vehicles, animals, and specific objects, and have demonstrated significant value in scenarios such as abnormal behavior warning, traffic flow optimization, and wildlife protection.

[0003] Existing technologies often rely on pre-trained models for static feature extraction, but lack the ability to adapt to dynamic scenarios. For trajectory analysis and prediction, traditional methods based on Kalman filtering or LSTM modeling fail to fully integrate map topology information, resulting in a disconnect between trajectory predictions and the actual physical space. Furthermore, multi-camera collaboration mechanisms often employ fixed scheduling rules and lack the ability to predict dynamic paths. This makes it difficult to achieve efficient cross-device resource allocation and pre-adjust cameras to verified angles to capture object movement along the predicted path. Summary of the Invention

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0005] Multi-scenario IoT off-site dynamic inspection adaptive monitoring system, including:

[0006] The mobile object detection module uses mixed Gaussian background modeling and optical flow method to detect moving objects. At the same time, it uses the lightweight MobileViT-XXS model to track moving objects.

[0007] The moving object classification and recognition module uses a multimodal classification decision tree to identify the type of moving objects:

[0008] When it is judged as a non-living object, ROI extraction is performed on the saliency of the moving object image, the optimal frame is captured, and EXIF ​​metadata is generated through super-resolution reconstruction; when it is judged as a living object, the biological category is determined:

[0009] When it is determined to be an animal, geo-fence data matching is performed through a spatiotemporal feature fusion model;

[0010] When the image is judged to be human, multi-frame fusion super-resolution is used to evaluate facial quality and extract cross-camera ReID features.

[0011] The spatial modeling and trajectory prediction module optimizes camera calibration parameters and aligns with the map through multi-view SFM point cloud reconstruction; and predicts the trajectory of moving objects through the spatiotemporal graph neural network.

[0012] The intelligent scheduling module intelligently distributes bitstreams based on Voronoi diagram division of viewshed analysis and PID control of target motion vectors, and predictively adjusts camera positions based on moving object trajectory prediction.

[0013] Furthermore, the process of detecting a moving object is as follows:

[0014] S101: Dual-engine motion detection for moving objects is performed using mixed Gaussian background modeling and optical flow. A scene background model is established using mixed Gaussian background modeling. The static background in the scene of a moving object is modeled and synthesized to distinguish the moving object from the surrounding background. A weighted fusion strategy is used for moving objects, integrating multi-source data and assigning dynamic weights. Parts of the moving object with rapidly changing pixels are cropped as motion regions.

[0015] S102: Based on the improved YOLOv8 detection network, it accurately locates moving objects. It introduces GSConv gradient-stopped convolution, divides the inference process into normal mode and high-speed mode, and switches the resolution according to the moving speed of the moving object.

[0016] S103: Continuously track the target; during the target continuous tracking stage, the appearance feature extractor is used to extract a 512-dimensional feature vector with cross-domain generalization capability from the target area, and a cascade matching strategy is adopted, combined with spatiotemporal consistency constraints, for long-term tracking.

[0017] Furthermore, the process of distinguishing the type of the moving object is as follows:

[0018] Classify detected moving objects as non-living or living, and perform specific recognition tasks for different categories. Use a multimodal classification decision tree to identify the type of moving objects, combining visual features, motion patterns, and contextual information to form a hierarchical classification logic.

[0019] The color histogram, texture features and shape outline of the object are extracted based on MobileViT-XXS; the speed, acceleration and movement direction are extracted based on the DeepSORT tracking results; by inputting the detection results, bounding box, motion trajectory and appearance features of the moving object, the category is judged by multi-feature fusion, and the biological and non-biological judgments are output.

[0020] Furthermore, the process of generating EXIF ​​metadata through super-resolution reconstruction is as follows:

[0021] Based on the saliency map, the minimum bounding box of the moving object is cropped to obtain a low-resolution ROI image. Through perceptual loss and adversarial training, a high-resolution image is output. The center part of the moving object image is located through visual saliency detection.

[0022] Embed the moving object image file header in JSON format while being compatible with the traditional EXIF ​​standard. Use EXIF ​​tools to parse the raw EXIF ​​data, create a new tag in the Exif sub-IFD, encode the JSON string into ASCII, write the corresponding tag field, record the timestamp of the target's first detection, map the target's pixel coordinates in the moving object image to real-world map coordinates, and quickly locate the event occurrence point by combining the timestamp and world map coordinates.

[0023] Furthermore, the process of determining the biological category is as follows:

[0024] Based on EfficientNet-B4, animal appearance features are extracted, LSTM is used to encode the motion patterns of adjacent frames, the temporal characteristics of moving objects are captured, and biological categories are classified and determined based on deep learning.

[0025] Furthermore, when it is determined to be an animal, the geo-fence data is matched, and the process of the spatiotemporal feature fusion model is as follows:

[0026] When an animal is identified, geo-fence data matching is performed, connecting to the local biological database to obtain common animal species in the monitoring area. If the target is located in a known animal habitat, the species list of the area is matched first. At the same time, the species probability is adjusted according to seasonal changes. The Transformer architecture is used to splice multimodal features, and the self-attention mechanism is used to learn the correlation between features and output the species probability distribution. When a new species is detected, the system automatically downloads the characteristic data of the species from the cloud.

[0027] Furthermore, when the image is judged to be human, the process of performing facial quality assessment and cross-camera ReID feature extraction through multi-frame fusion super-resolution is as follows:

[0028] Multi-frame fusion super-resolution is used to repair the video. Multiple consecutive frames of blurred face images are input, and a clear face is generated through the VSR model. The inter-frame motion is aligned using the optical flow method. The clarity, occlusion level, and lighting uniformity of the face image are evaluated through facial quality assessment. A quality score is output, and images below the threshold are not recognized. At the same time, ArcFace and PCB models are used to extract cross-camera ReID features.

[0029] Furthermore, the map registration process is as follows:

[0030] Through multi-view SFM point cloud reconstruction, SIFT / SURF is used to extract key points in the scene image. The camera pose is calculated by matching key points from different viewpoints. The 3D point cloud coordinates are calculated based on the disparity of the matched point pairs. By inputting images of moving objects in the same scene captured by multiple cameras, the point cloud of the objects in the scene is output.

[0031] The reconstructed point cloud is aligned with the known map to achieve the unification of spatial coordinates; the nearest neighbor matching of the point cloud and the map is iteratively calculated, and the transformation matrix is ​​optimized by minimizing the Euclidean distance; semantic information is introduced based on ICP, and the alignment process is accelerated through semantic constraints.

[0032] Furthermore, the process of predicting the trajectory of a moving object is as follows:

[0033] Spatial dependencies are captured through graph neural networks, and neighbor node features are aggregated based on temporal dynamics modeled by time series models and graph attention mechanisms. The current state of the object and scene semantic information are input; LSTM is used to encode historical trajectories and extract long-term dependencies; spatial features are spliced ​​with temporal features, input into the fully connected layer to generate trajectory prediction results, and output motion patterns.

[0034] Furthermore, the process of predictively adjusting the camera position based on the moving object trajectory prediction is as follows:

[0035] Adjust encoding parameters according to the type of moving object and scene complexity, enable adaptive frame rate and dynamic quantization parameters in scenes requiring high resolution, use low-resolution encoding for non-critical areas, obtain the target's speed and direction from the optical flow method or DeepSORT tracking results, calculate the deviation between the target and the camera center point, dynamically adjust the camera's steering speed and focal length based on the deviation, dynamically adjust the PID gain according to the target motion pattern, and smooth the camera motion through a low-pass filter. Predict the future position of the moving object based on STGCN, and adjust the camera angle in advance according to the predicted trajectory to cover the area where the target may appear, allocating additional computing resources to high-probability target paths.

[0036] The multi-scenario IoT off-site dynamic inspection adaptive monitoring system provided by the present invention has the following beneficial effects:

[0037] (1) The present invention realizes dynamic target detection and continuous tracking in complex scenes through a dual-engine detection mechanism of mixed Gaussian background modeling and optical flow method, combined with the lightweight MobileViT-XXS model and OSNet feature extractor; effectively distinguishes moving objects from background noise such as illumination changes and leaf shaking, reducing the false detection rate; at the same time, it adopts pattern reasoning and adaptive ROI sampling strategy to reduce computing resource consumption and improve detection speed; through cascade matching, it realizes long-term stable tracking of target ID and reduces tracking interruptions caused by occlusion or loss.

[0038] (2) The present invention adopts a multimodal classification decision tree to integrate visual features, motion patterns and scene semantics, geographic fences and seasonal changes to hierarchically discriminate between living and non-living things; combines EfficientNet-B4 and LSTM to extract spatiotemporal features, improves the accuracy of animal / human identification, and supports dynamic updating of cloud data of unknown species; geo-fence matching and seasonal weight adjustment optimize regional species identification, for example, prioritizing the identification of birds during migratory bird season; through super-resolution reconstruction and EXIF ​​metadata embedding, key details are restored in low-resolution images, and multi-dimensional information such as timestamps and geographic coordinates are recorded to facilitate rapid event backtracking.

[0039] (3) The present invention adopts lightweight model designs such as MobileViT-XXS and GSConv gradient-stopped convolution, which are compatible with EXIF ​​standards and JSON metadata formats; the lightweight model is adapted to edge computing devices, and XIF metadata is compatible with traditional standards, supporting multi-tool parsing and cloud-based collaborative analysis, which is convenient for integration with existing monitoring systems. Through spatiotemporal feature fusion, semantically constrained point cloud registration, and dynamic adjustment of PID parameters, real-time response to complex scenes is achieved, and semantic information is introduced into multi-view SFM point cloud reconstruction to accelerate registration and improve the accuracy of spatial modeling in complex scenes; optical flow alignment and low-pass filter are used to smooth camera motion to avoid image blur or jitter caused by rapid target movement. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the system flow of the present invention. DETAILED DESCRIPTION

[0041] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0042] Example 1

[0043] See also Figure 1, Example 1 of the present application provides a multi-scenario IoT off-site dynamic inspection adaptive monitoring system, the system comprising:

[0044] The mobile object detection module uses mixed Gaussian background modeling and optical flow method to detect moving objects. At the same time, it uses the lightweight MobileViT-XXS model to track moving objects.

[0045] Moving object detection:

[0046] S101: Dual-engine motion detection of moving objects using mixed Gaussian background modeling and optical flow.

[0047] S101-A: Multimodal motion perception: Using mixed Gaussian background modeling to build a scene background model, static backgrounds in scenes with moving objects are modeled and synthesized to distinguish moving objects from the surrounding environment, supporting scene understanding, analysis, and applications. Three to five Gaussian distributions are used per camera pixel, and background parameters are updated in real time to output a foreground mask to filter out interference such as lighting changes and swaying leaves in camera images. Using dense optical flow, pixel displacement vectors between consecutive video frames are calculated to construct a motion energy map to capture moving objects against dynamic backgrounds.

[0048] S101-B: For moving objects, a weighted fusion strategy is used. By integrating multiple data sources, such as optical flow, frame difference, and depth information, and assigning dynamic weights, this strategy enhances the continuity of moving object contours and suppresses background noise interference, providing more accurate initial screening results for moving regions for subsequent morphological processing. After morphological processing, holes are filled through dilation operations, and connected domain analysis is used to remove noise. Furthermore, areas of moving objects with sharp pixel changes are cropped as moving regions.

[0049] S102: Accurately locate moving objects based on the improved YOLOv8 detection network;

[0050] S102-A: Replace the existing Backbone with MobileViT-XXS and introduce GSConv gradient-stopped convolution to reduce parameter redundancy. The inference process is divided into normal mode and high-speed mode. In normal mode, the input pixel resolution is 640×640. In high-speed mode, the pixel resolution switches to 320×320 when the moving target is greater than 15 pixels / frame, otherwise the normal mode is used.

[0051] Backbone: The backbone network in a deep learning model, which is responsible for extracting features from input data, such as images; these features are then used for various tasks, such as classification, detection, or segmentation;

[0052] MobileViT-XXS: A lightweight visual transformer in the MobileViT family, MobileViT-XXS reduces computational effort in resource-constrained environments. It demonstrates superior performance on many visual recognition tasks and is more flexible in capturing long-range dependencies than traditional convolutional neural networks. MobileViT can significantly improve efficiency and accuracy in resource-constrained environments.

[0053] S102-B: Adaptive detection enhancement;

[0054] Based on the ROI focus of the motion region prior, the fusion mask area is sampled 3 times, and the non-interest area is downsampled by 1 / 4; multi-scale detection fusion is performed, and the feature pyramid output P3 (80×80), P4 (40×40), P5 (20×20) is used to weight the fusion detection results;

[0055] Fusion mask region: By combining the motion region detection results with the ROI focus mask (based on scene priors or motion regions), the high-confidence region of the dynamic target is separated from the background to form a binary mask for refined sampling;

[0056] S103: Continuous tracking of the target;

[0057] During the continuous target tracking phase, the system uses a lightweight appearance feature extractor, based on the OSNet network, to extract a 512-dimensional feature vector with cross-domain generalization capabilities from the target area. It also uses a cascade matching strategy combined with spatiotemporal consistency constraints to achieve highly robust long-term tracking of the target.

[0058] OSNet: A lightweight Re-ID model that extracts appearance features of objects from video streams. It is particularly suitable for cross-camera object tracking and ensures a certain level of recognition accuracy in different scenarios.

[0059] The output generates a 512-dimensional feature vector, which represents the unique appearance characteristics of moving objects in complex environments such as different lighting conditions and viewing angles. Compared with traditional convolutional neural networks, OSNet enhances the model's adaptability to objects of different scales by introducing a multi-scale feature learning mechanism.

[0060] Level 1: Mahalanobis distance matching; a distance metric based on the covariance matrix that measures the similarity between the distributions of two moving objects; calculating the difference between the detected target in the current frame and the predicted trajectory position in the previous frame; by considering the distribution characteristics of direction and scale, a more accurate matching result is obtained than the Euclidean distance;

[0061] Second level: cosine similarity matching; compare the similarity between the target feature vectors extracted by OSNet, and measure the angle between the two vectors using cosine similarity. The cosine similarity ranges from [-1, 1], with larger values ​​indicating more similarity between the two vectors.

[0062] Level 3: Trajectory prediction and compensation: Use algorithms such as Kalman filtering or particle filtering to predict the target's position in the next frame. Based on historical motion information such as speed and acceleration, the target's future position is estimated and the matching strategy is adjusted accordingly. When the target is temporarily obscured or lost, its possible position can be predicted to remedy the situation, reducing the probability of ID switching and improving tracking continuity.

[0063] The moving object classification and recognition module uses a multimodal classification decision tree to identify the type of moving objects:

[0064] When it is judged as a non-living object, ROI extraction is performed on the saliency of the moving object image, the optimal frame is captured, and EXIF ​​metadata is generated through super-resolution reconstruction; when it is judged as a living object, the biological category is determined:

[0065] When it is determined to be an animal, geo-fence data matching is performed through a spatiotemporal feature fusion model;

[0066] When the image is judged to be human, multi-frame fusion super-resolution is used to evaluate facial quality and extract cross-camera ReID features.

[0067] Through the multimodal classification decision tree, the type of moving objects is identified:

[0068] Classify detected moving objects as non-living or living, and perform specific recognition tasks for different categories. Use a multimodal classification decision tree to identify the type of moving objects, combining visual features, motion patterns, and contextual information to form a hierarchical classification logic.

[0069] Extract the color histogram, texture features, and shape outline of the object based on MobileViT-XXS; extract speed, acceleration, and direction of movement from DeepSORT tracking results; combine scene semantics, such as whether it is in a parking lot or a vegetation area; input the detection results of moving objects, bounding boxes, motion trajectories, and appearance features; determine the category through multi-feature fusion, such as color texture, motion speed, and shape complexity; and output the final classification, judging whether it is living or non-living.

[0070] Decision rules for living or non-living things:

[0071] Non-living object discrimination: If the object's movement speed is lower than the threshold, such as <5 pixels / frame and its shape is regular, such as a rectangle or a circle, it is preferentially classified as a non-living object.

[0072] Organism discrimination: If an object's motion pattern is abrupt, such as sudden acceleration, or its shape is irregular, such as a human silhouette, it is classified as an organism;

[0073] When it is determined to be a non-living object:

[0074] Image salient ROI extraction:

[0075] Through visual saliency detection, the most informative area in the moving object image is located, such as the center of the moving object; the most informative area is the center of the moving object;

[0076] ROI cropping: Crop the minimum bounding box of the moving object based on the saliency map to reduce the amount of subsequent processing calculations;

[0077] Super-resolution reconstruction:

[0078] By inputting a low-resolution ROI image, such as 320×320, it outputs a high-resolution image, such as 1024×1024; image details are enhanced through perceptual loss and adversarial training; the detected object image region (ROI) is input, with a size of 320×320; the MobileViT-XXS lightweight visual model is used, combining CNN local modeling and Transformer global modeling capabilities, to extract high-dimensional features through multi-layer residual connections; the feature output is a global feature vector, such as with a dimension of 512 or 1024, which implies the color, texture, and shape information of the object; and specific features are further extracted from the feature vector of MobileViT-XXS, the input image is converted to color space, the histogram of each channel is calculated, such as the H / S / V channel of HSV, and normalized to 8-bin or 16-bin vectors; texture features are extracted through convolution pooling of the intermediate feature map of MobileViT-XXS; edge detection is used to calculate geometric properties such as aspect ratio, area, and perimeter;

[0079] EXIF metadata generation, including data content and storage format; data content includes the time when the target appears, the position of the target on the two-dimensional map, and the object attributes based on visual feature analysis; the storage format is embedded in the mobile object image file header in JSON format, while being compatible with traditional EXIF ​​standards, using EXIF ​​tools to parse the original EXIF ​​data, creating a new tag in the Exif sub-IFD, encoding the JSON string into ASCII or Unicode text, and writing it to the corresponding tag field to facilitate subsequent retrieval and analysis; recording the timestamp when the target is first detected, mapping the pixel coordinates of the target in the image to real-world map coordinates, combining the timestamp and world map coordinates to quickly locate the point of occurrence of the event, and generating descriptive information based on object classification and attribute analysis;

[0080] When a biological object is determined:

[0081] Based on EfficientNet-B4, we extract animal appearance features, such as fur color and body shape. We use LSTM to encode motion patterns between adjacent frames, capturing the temporal characteristics of moving objects, such as jumping and running. We use deep learning to classify and determine the biological category, while using unsupervised pre-training to enhance the model's adaptability to unknown species.

[0082] When an animal is identified, geofence data is matched against the local biological database to obtain common animal species in the monitored area. If the target is located in a known animal habitat, the species list in that area is prioritized. The species probability is adjusted based on seasonal changes, such as increasing the weight of migratory bird identification during the migration season. The system uses the Transformer architecture to stitch multimodal features together, and through the self-attention mechanism, learns the correlation between features and outputs a species probability distribution. When a new species is detected, the system automatically downloads the species' feature data from the cloud and compresses it through a knowledge distillation model.

[0083] Human recognition: Multi-frame fusion super-resolution is used to repair videos. Consecutive frames of blurred facial images are input, and a VSR model is used to generate clear faces. Optical flow is used to align inter-frame motion and eliminate the effects of jitter. Facial quality is assessed based on clarity, occlusion, and illumination uniformity. A quality score (0-1) is output, with images below a threshold (e.g., 0.7) not recognized. ArcFace and PCB models are used simultaneously for cross-camera ReID feature extraction.

[0084] ArcFace: Improves the discriminability of face recognition by adding angle constraint loss;

[0085] PCB: Divides the face into local areas, such as eyes and nose, and extracts features from each area to enhance robustness to partial occlusion.

[0086] And through the feature vector, 512-dimensional global features and local features are output for cross-camera matching;

[0087] The spatial modeling and trajectory prediction module optimizes camera calibration parameters and aligns with the map through multi-view SFM point cloud reconstruction; and predicts the trajectory of moving objects through the spatiotemporal graph neural network;

[0088] Reconstruction of multi-view SFM point cloud:

[0089] Through multi-view SFM point cloud reconstruction, SIFT / SURF or deep learning feature extractors are used to extract key points in the scene image. The camera pose is calculated by matching key points from different viewpoints. The 3D point cloud coordinates are calculated based on the disparity of the matched point pairs. By inputting images of moving objects in the same scene captured by multiple cameras, a sparse point cloud containing the object's position and shape information is output.

[0090] Key points: Stable and distinctive feature points in moving object images that can be repeatedly detected under different viewing angles, scales, rotations, or lighting conditions. They can be effectively distinguished from other areas through feature descriptions and are usually located at edges, corners, or areas with significant texture changes.

[0091] Camera calibration parameter optimization:

[0092] The Levenberg-Marquardt algorithm is used to minimize reprojection errors, optimize camera intrinsic and extrinsic parameters, improve the accuracy of point cloud reconstruction, and reduce deviations caused by lens distortion or installation errors. RANSAC is introduced to eliminate abnormal matching points and prevent the calibration results from being affected by noise.

[0093] Map registration:

[0094] Align the reconstructed point cloud with the known map to unify the spatial coordinates; iteratively calculate the nearest neighbor matching between the point cloud and the map, and optimize the transformation matrix by minimizing the Euclidean distance; introduce semantic information based on ICP and accelerate the registration process through semantic constraints;

[0095] Moving object trajectory prediction:

[0096] The scene is modeled as a graph structure, where nodes represent objects or scene elements and edges represent interactions between objects. Spatial dependencies are captured through graph neural networks, and temporal dynamics are modeled in combination with time series models. Chebyshev polynomial expansion or graph attention mechanism is used to aggregate neighbor node features. The current state of the object and scene semantic information are input. Transformer or LSTM is used to encode historical trajectories and extract long-term dependencies. Spatial features are concatenated with temporal features and input into a fully connected layer to generate trajectory prediction results, such as a coordinate sequence for the next 6 seconds. Historical trajectories are clustered and analyzed to extract typical motion patterns. When predicting, trajectory branches that are consistent with historical patterns are prioritized, such as when a pedestrian moves within the capture range of another camera within a period of time. Multiple possible trajectory branches are output simultaneously and assigned confidence levels.

[0097] Multiple trajectory seeds are randomly initialized, i.e., automatically selected starting points. The features of each seed are updated through a cross-attention mechanism. Each seed independently predicts a trajectory branch, and the optimal solution is selected through a scenario scoring module. Uncertainty modeling is also performed, using a Gaussian mixture model or Monte Carlo Dropout to quantify prediction uncertainty and output a probability distribution of the trajectory. For example, in a warehouse cross-route scenario, the STGCN model combines the pedestrian's historical trajectory (going straight), the current speed rapidly approaching the warehouse route, and the scene semantics (when the warehouse corner enters the capture range of the next camera) to predict the pedestrian's future trajectory as "turn left" or "go straight", and provides a confidence level (e.g., 70% left turn, 30% straight).

[0098] The intelligent scheduling module intelligently distributes bitstreams based on Voronoi diagram division of viewshed analysis and PID control of target motion vectors, and predictively adjusts camera positions based on moving object trajectory prediction.

[0099] Voronoi diagram: A spatial segmentation algorithm that divides the scene into multiple regions, where the points in each region are closest to their corresponding camera.

[0100] Based on the camera's installation position and viewing angle, a three-dimensional field of view coverage is constructed, dividing the coverage area into multiple sub-areas. Each area is monitored by a camera. When the target moves across areas, the optimal capture position of the camera is calculated through Monte Carlo simulation to maximize the coverage area and reduce blind spots. When the target crosses areas, the Kalman filter is used to predict the target trajectory and the tracking mode of adjacent cameras is activated in advance. The scheduling engine triggers the coordinated relay monitoring and tracking of adjacent cameras.

[0101] PID control: It is a closed-loop control algorithm used to adjust the camera's motion parameters, such as angle and focal length;

[0102] Intelligent bitrate distribution:

[0103] Encoding parameters are adjusted based on the type of moving objects and scene complexity. Adaptive frame rate and dynamic quantization parameters are enabled in scenes requiring high resolution. Fixed bandwidth is allocated to high-priority tasks, and key frames are cached at edge nodes to reduce cloud storage pressure. Low-resolution encoding is used for non-critical areas, and redundant frames may even be discarded. For example, during nighttime surveillance, the system automatically switches to low-bitrate mode to save bandwidth. When an illegal intrusion is detected, the bitrate of the relevant camera is immediately increased and high-definition video is uploaded to the cloud.

[0104] Make predictive adjustments to the camera position:

[0105] The target's speed and direction are obtained from the optical flow method or DeepSORT tracking results, the deviation between the target and the camera center point is calculated, and the camera's steering speed and focal length are dynamically adjusted based on the deviation to ensure that the target is always in the center of the picture; the PID gain is dynamically adjusted according to the target motion pattern, and the camera motion is smoothed through a low-pass filter to avoid frequent jitter caused by noise; the future position of the moving object is predicted based on the STGCN, and according to the predicted trajectory, the camera angle is adjusted in advance to cover the area where the target may appear, allocating additional computing resources to the high-probability target path.

[0106] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0107] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment as needed.

[0108] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. Multi-scenario IoT off-site dynamic inspection adaptive monitoring system, characterized by: The system includes: The mobile object detection module uses mixed Gaussian background modeling and optical flow method to detect moving objects. At the same time, it uses the lightweight MobileViT-XXS model to track moving objects. The moving object classification and recognition module uses a multimodal classification decision tree to identify the type of moving objects: When it is judged as a non-living object, ROI extraction is performed on the saliency of the moving object image, the optimal frame is captured, and EXIF ​​metadata is generated through super-resolution reconstruction; when it is judged as a living object, the biological category is determined: When it is determined to be an animal, geo-fence data matching is performed through a spatiotemporal feature fusion model; When it is judged to be human, multi-frame fusion super-resolution is used to evaluate facial quality and perform cross-camera ReID feature extraction; The spatial modeling and trajectory prediction module optimizes camera calibration parameters and aligns with the map through multi-view SFM point cloud reconstruction; and predicts the trajectory of moving objects through the spatiotemporal graph neural network; The intelligent scheduling module intelligently distributes bitstreams based on Voronoi diagram division of viewshed analysis and PID control of target motion vectors, and predictively adjusts camera positions based on moving object trajectory prediction.

2. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 1 is characterized in that: The process of detecting moving objects is as follows: S101: Dual-engine motion detection for moving objects is performed using mixed Gaussian background modeling and optical flow. A scene background model is established using mixed Gaussian background modeling to model and synthesize the static background in the scene of moving objects, used to distinguish between moving objects and the surrounding background. For moving objects, a weighted fusion strategy is used to integrate multi-source data and assign dynamic weights. And the part of the moving object where the pixels change rapidly is cut as the moving area; S102: Based on the improved YOLOv8 detection network, it accurately locates moving objects. It introduces GSConv gradient-stopped convolution, divides the inference process into normal mode and high-speed mode, and switches the resolution according to the moving speed of the moving object. S103: Continuous tracking of the target; In the target continuous tracking stage, a 512-dimensional feature vector is extracted from the target area through the appearance feature extractor, and a cascade matching strategy is adopted in combination with spatiotemporal consistency constraint tracking.

3. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 1 is characterized in that: The process of distinguishing the type of moving objects is as follows: Classify detected moving objects as non-living or living, and perform specific recognition tasks for different categories. Use a multimodal classification decision tree to identify the type of moving objects, combining visual features, motion patterns, and contextual information to form a hierarchical classification logic. Extract the color histogram, texture features and shape outline of the object based on MobileViT-XXS; Extract speed, acceleration and direction of movement based on DeepSORT tracking results; input the detection results of moving objects, bounding boxes, motion trajectories and appearance features, use multi-feature fusion to judge categories, and output biological and non-biological judgments.

4. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 3 is characterized in that: The process of generating EXIF ​​metadata through super-resolution reconstruction is as follows: Based on the saliency map, the minimum bounding box of the moving object is cropped to obtain a low-resolution ROI image. Through perceptual loss and adversarial training, a high-resolution image is output. The center part of the moving object image is located through visual saliency detection. Embed the moving object image file header in JSON format while being compatible with the traditional EXIF ​​standard. Use EXIF ​​tools to parse the raw EXIF ​​data, create a new tag in the Exif sub-IFD, encode the JSON string into ASCII, write the corresponding tag field, record the timestamp of the target's first detection, map the target's pixel coordinates in the moving object image to real-world map coordinates, and quickly locate the event occurrence point by combining the timestamp and world map coordinates.

5. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 4 is characterized in that: The process of biological category determination is as follows: Based on EfficientNet-B4, animal appearance features are extracted, LSTM is used to encode the motion patterns of adjacent frames, the temporal characteristics of moving objects are captured, and biological categories are classified and determined based on deep learning.

6. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 5 is characterized in that: When an animal is identified, geo-fence data matching is performed, and the process of the spatiotemporal feature fusion model is as follows: When an animal is identified, geo-fence data matching is performed, connecting to the local biological database to obtain common animal species in the monitoring area. If the target is located in a known animal habitat, the species list of the area is matched first. At the same time, the species probability is adjusted according to seasonal changes. The Transformer architecture is used to splice multimodal features, and the self-attention mechanism is used to learn the correlation between features and output the species probability distribution. When a new species is detected, the system automatically downloads the characteristic data of the species from the cloud.

7. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 6 is characterized in that: When the face is judged to be human, the process of performing facial quality assessment and cross-camera ReID feature extraction through multi-frame fusion super-resolution is as follows: Multi-frame fusion super-resolution is used to repair the video. Multiple consecutive frames of blurred face images are input, and a clear face is generated through the VSR model. The inter-frame motion is aligned using the optical flow method. The clarity, occlusion level, and lighting uniformity of the face image are evaluated through facial quality assessment. A quality score is output, and images below the threshold are not recognized. At the same time, ArcFace and PCB models are used to extract cross-camera ReID features.

8. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 1 is characterized in that: The process of map registration is as follows: Through multi-view SFM point cloud reconstruction, SIFT / SURF is used to extract key points in the scene image. The camera pose is calculated by matching key points from different viewpoints. The 3D point cloud coordinates are calculated based on the disparity of the matched point pairs. By inputting images of moving objects in the same scene captured by multiple cameras, the point cloud of the objects in the scene is output. The reconstructed point cloud is aligned with the known map to achieve the unification of spatial coordinates; the nearest neighbor matching of the point cloud and the map is iteratively calculated, and the transformation matrix is ​​optimized by minimizing the Euclidean distance; semantic information is introduced based on ICP, and the alignment process is accelerated through semantic constraints.

9. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 1 is characterized in that: The process of moving object trajectory prediction is as follows: Spatial dependencies are captured through graph neural networks, and neighbor node features are aggregated based on temporal dynamics modeled by time series models and graph attention mechanisms. The current state of the object and scene semantic information are input; LSTM is used to encode historical trajectories and extract long-term dependencies; spatial features are spliced ​​with temporal features, input into the fully connected layer to generate trajectory prediction results, and output motion patterns.

10. The multi-scenario Internet of Things off-site dynamic inspection adaptive monitoring system according to claim 1 is characterized in that: The process of predictively adjusting the camera position based on the moving object trajectory prediction is as follows: Adjust encoding parameters according to the type of moving object and scene complexity, enable adaptive frame rate and dynamic quantization parameters in scenes requiring high resolution, use low-resolution encoding for non-critical areas, obtain the target's speed and direction from the optical flow method or DeepSORT tracking results, calculate the deviation between the target and the camera center point, dynamically adjust the camera's steering speed and focal length based on the deviation, dynamically adjust the PID gain according to the target motion pattern, and smooth the camera motion through a low-pass filter. Predict the future position of the moving object based on STGCN, adjust the camera angle in advance according to the predicted trajectory to cover the area where the target appears, and allocate additional computing resources to high-probability target paths.

Images