A mobile control monitoring system and method based on edge computing
By combining edge computing and wireless solar-powered cameras, the mobile surveillance system can be rapidly deployed and efficiently processed, solving the flexibility and latency issues of traditional surveillance systems and improving emergency command and remote decision-making capabilities.
Patent Information
- Application Number
- CN202410482628.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-22
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-04-22
AI Technical Summary
Traditional monitoring systems rely on fixed monitoring points and centralized data processing centers, resulting in poor flexibility, large data transmission delays, and difficulty in responding to emergencies and temporary security needs.
A mobile surveillance system based on edge computing is adopted, which uses wireless solar-powered cameras to collect video data and combines edge computing terminals for real-time analysis, including adaptive lighting processing, multi-camera collaboration and dynamic resource scheduling, and transmits data through wireless CPE devices.
It improves the flexibility and response speed of the monitoring system, reduces data processing latency, enhances data security, and meets the high efficiency and high security standards required by modern cities.
Smart Images

Figure CN118540438B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of mobile monitoring technology, and in particular to a mobile control and monitoring system based on edge computing and a method thereof. Background Art
[0002] Traditional monitoring systems rely on fixed monitoring points, which are limited by the location and number of monitoring points and cannot flexibly respond to emergencies and temporary security needs. In addition, current security monitoring systems usually rely on centralized data processing centers, and a large amount of monitoring data needs to be transmitted to the central server for processing, which greatly increases the delay of data transmission and response.
[0003] Mobile rapid deployment can achieve fast and flexible deployment of surveillance cameras, so that monitoring equipment can be quickly put into use at critical moments and obtain on-site information in a timely manner. This deployment method gets rid of the limitations of traditional fixed monitoring and can adapt to various complex and changing environments, thereby improving the flexibility and efficiency of monitoring. And with the widespread deployment of Internet of Things (IoT) devices, the amount of data generated has increased dramatically, and the demand for real-time data processing and efficient security protection has also increased. In order to better ensure the orderly development of daily work, a mobile deployment monitoring system and method based on edge computing are provided in scenarios such as responding to emergencies, emergency rescue, and investigation and solving cases. As a temporary deployment in specific occasions and a supplementary mobile monitoring outside of fixed monitoring points, it can collect on-site audio and video, cooperate with edge computing terminals to perform face and vehicle comparison and recognition, make real-time judgments and rapid decisions, and thus improve video investigation, emergency command capabilities and remote decision-making capabilities. Summary of the invention
[0004] In view of the impact caused by the fact that traditional monitoring systems mostly rely on fixed monitoring points and centralized data processing centers, the present invention proposes a mobile control monitoring system and method based on edge computing.
[0005] In order to achieve the above object, the present invention is implemented by the following technical solutions:
[0006] A mobile control and monitoring system based on edge computing, the system comprising:
[0007] A mobile control module is used to perform mobile and rapid control according to control requirements, including a wireless solar camera that can freely adsorb poles or walls and achieve intelligent climbing through remote control, and uses the wireless solar camera to collect monitoring video data in real time;
[0008] An edge computing module, including an edge computing terminal, for receiving video data from a wireless solar camera for real-time analysis;
[0009] The user interface module is used to provide a visual interface so that operators can view video data, data analysis results, system status and alarm information in real time, play back historical videos, or manually intervene in the monitoring process;
[0010] The data transmission module includes a wireless CPE device. The wireless solar camera is connected to the wireless CPE device through a wireless network for data transmission. The edge computing module and the user interface module are connected to the wireless CPE device via a network cable for data transmission.
[0011] As a preferred solution of the present invention, the wireless solar camera has a built-in solar power supply module and a backup power supply module. When fully charged, it can support working in a light-free environment for at least 24 hours. The vertical angle can be adjusted to -50° to 0°. It has its own microphone and speaker, and stores video data through a TF card.
[0012] As a preferred solution of the present invention, the edge computing module includes an adaptive monitoring module, which annotates and collects video data under different lighting conditions, and uses the deep learning framework of TensorFlow for training to identify and adapt to different lighting conditions. According to the analysis results of the ambient light sensor or video data, the CSN network is used to automatically select the video processing model corresponding to different lighting conditions. Each branch in the CSN network is optimized for the video processing model under one lighting condition. The network selection logic is implemented by the softmax layer, and the formula is:
[0013] P(b|a)=softmax(W h ·h(a));
[0014] Where h(a) represents the feature parameter extracted from input a; W h is a weight matrix; P(b|a) represents the probability of selecting branch b under the condition of given input a.
[0015] As a preferred solution of the present invention, the edge computing module includes a video structuring module and a security incident response module;
[0016] The video structuring module is used to perform pre-processing operations such as decoding, frame extraction, and scaling of received video data through a selected video processing model, and to perform structured analysis on a human body or a vehicle through a target recognition algorithm, and to extract and store target attributes into a target database; the target recognition algorithm is specifically a YOLOv7-tiny algorithm with SENet added;
[0017] The security event response module is used to classify security events through the ResNet network, and automatically adjust the camera parameters according to the event type to optimize event capture or enable the alarm function, including a target comparison unit, a perimeter prevention unit, and a high-altitude throwing unit;
[0018] The target comparison unit is used to compare the target in the current video frame with a preset target database to identify whether there is a matching target attribute;
[0019] The perimeter prevention unit is used to identify objects or people crossing the perimeter in the perimeter prevention scene, and if an object or person is detected to have crossed the set perimeter, it will take a snapshot and trigger an alarm;
[0020] The high-altitude throwing unit is used to analyze the motion trajectory and object features in the video frame to identify the characteristic patterns in the throwing process. If a high-altitude throwing event is detected, the camera angle is adjusted to capture the throwing process and trigger an alarm.
[0021] As a preferred solution of the present invention, the classification of security events by using the ResNet network specifically includes:
[0022] The input layer receives the normalized input image and adds padding to the border of the input feature map, with padding = 1.
[0023] Feature extraction is performed through a 7×7 convolutional layer with a step size of 2 and 64 output channels;
[0024] Downsample through a 3×3 max pooling layer with a stride of 2;
[0025] Multiple residual blocks are stacked. Each residual block consists of two or three convolutional layers. The convolutional layers are nonlinearly transformed and normalized through ReLU activation functions and batch normalization layers. The output of each residual block is added to the input of the previous residual block through an identity connection to form residual learning.
[0026] After all residual blocks are processed, the feature map is converted into a feature vector through a global average pooling layer;
[0027] The security events are classified through the fully connected layer.
[0028] As a preferred solution of the present invention, the edge computing module also includes a multi-camera collaboration module and a dynamic resource scheduling module;
[0029] The multi-camera collaboration module is used to determine the position and viewing angle of each camera, and synchronously process the video data collected by multiple cameras on the edge computing terminal through a data fusion model;
[0030] The data fusion model searches for image corresponding points between different views based on the MVS technology, and reconstructs the three-dimensional scene according to the image corresponding points to integrate the image data from different cameras. The formula is:
[0031]
[0032] Where D(x,y) represents the depth value of the image coordinate (x,y); argmin d It means to find the d value that makes the following sum expression reach the minimum value, where d value refers to the disparity value; n is the number of cameras; SSD means sum of squared differences; I ref (x,y) represents the reference image I ref The pixel value at coordinate (x,y); Represents the i-th camera view I i After parallax correction, the coordinates The pixel value at denote the disparity from the reference image to the i-th camera view in the horizontal and vertical directions, respectively;
[0033] The dynamic resource scheduling module is used to monitor the computing resources and power status of the edge computing module in real time, and optimize and adjust task allocation according to the optimization algorithm;
[0034] The formula of the optimization algorithm is:
[0035]
[0036] Where Z is the total cost of the task; c m is the cost of task m; f m is a decision variable, indicating whether task m is selected for execution. m =1, then task m is selected for execution. If f m =0, task m is not executed; r m is the amount of resources required for task m; R is the total amount of available resources; M is the total number of tasks.
[0037] As a preferred solution of the present invention, the edge computing module also includes an extended API interface for accessing drone monitoring to perform joint monitoring of the air and the ground.
[0038] A mobile control and monitoring method based on edge computing, the method comprising:
[0039] Configure a wireless solar camera and adjust the vertical angle as needed. Use the wireless solar camera to collect video data of the specified area in real time and access the wireless CPE device through the wireless network for data transmission.
[0040] The wireless CPE device receives video data from the wireless solar camera and transmits the video data to the edge computing module and the user interface module via a network cable;
[0041] The edge computing terminal receives video data from the wireless CPE device and uses the CSN network to automatically select the video processing model corresponding to different lighting conditions;
[0042] The received video data is decoded, frame extracted, and scaled by the video processing model, and the video data is structurally analyzed by the target recognition algorithm to extract and store target attributes;
[0043] Use the ResNet network to classify security events and automatically adjust camera parameters or trigger alarm functions based on the event type;
[0044] Operators can view video data, data analysis results, system status and alarm information in real time through the visual interface of PC or mobile terminal, or manually intervene in the monitoring process.
[0045] As a preferred solution of the present invention, the edge computing terminal synchronously processes the video data collected by multiple cameras through a data fusion model, and the data fusion model specifically includes:
[0046] Based on the MVS technology, the image corresponding points between different views are found, and the three-dimensional scene is reconstructed according to the image corresponding points to integrate the image data from different cameras. The formula is:
[0047]
[0048] Where D(x,y) represents the depth value of the image coordinate (x,y); argmin d It means to find the d value that makes the following sum expression reach the minimum value, where d value refers to the disparity value; n is the number of cameras; SSD means sum of squared differences; I ref (x,y) represents the reference image I ref The pixel value at coordinate (x,y); Represents the i-th camera view I i After parallax correction, the coordinates The pixel value at Z represents the disparity from the reference image to the i-th camera view in the horizontal and vertical directions respectively.
[0049] The beneficial effects of the present invention are: using solar cameras to reduce dependence on external power supplies, enhancing the applicability and environmental adaptability of the system; it can be quickly deployed in any occasion that requires temporary monitoring, and is suitable for the security management of various temporary or special events, with strong flexibility and mobility; taking advantage of edge computing, by processing data at local nodes, the time for data processing and response is significantly reduced, latency is reduced, bandwidth pressure is alleviated, faster and safer data processing capabilities can be provided, and the need for large amounts of data to be transmitted to central servers is reduced, meeting the needs of modern cities for high efficiency and high safety standards; through data fusion, video data collected by multiple cameras can be processed synchronously, greatly improving response speed and decision-making efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0051] in:
[0052] Figure 1 A diagram showing a modular structure of a system in an embodiment of the present invention;
[0053] Figure 2 A schematic diagram of a visualization interface for configuring target attributes in an embodiment of the present invention;
[0054] Figure 3 4 is a flow chart of a method in an embodiment of the present invention. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.
[0056] like Figure 1 As shown, an embodiment of the present invention provides a mobile control and monitoring system based on edge computing, which can realize on-site audio and video acquisition, transmit through wireless networks, and cooperate with edge computing terminals for real-time analysis and decision-making, thereby improving video reconnaissance, emergency command and remote decision-making capabilities.
[0057] 1. Mobile control module
[0058] It is used for mobile and rapid deployment according to deployment requirements, including wireless solar cameras that can freely adsorb poles or walls and realize intelligent climbing through remote control, and use wireless solar cameras to collect monitoring video data in real time; the front-end device is powered by solar energy to ensure continuous operation in an environment without power supply. By combining solar power supply and wireless communication technology, the problem of difficult deployment of traditional monitoring systems in environments without power supply and difficult wiring is solved;
[0059] The wireless solar camera has a built-in solar power module and a backup power module. The battery capacity is 14.5Ah. A full charge can support working in a lightless environment for at least 24 hours. The protection level is IP66, supports voice intercom, comes with a microphone and speaker, supports standard 256GB MicroSD / MicroSDHC / MicroSDXC card storage, the highest resolution is 1920×1080@12.5fps, and the adjustable vertical angle is -50°~0°.
[0060] Video data is stored through a TF card with a nominal capacity of 128GB, Class 10 (read 95MB / s, write 27MB / s), an operating temperature of 0℃~70℃, and a storage temperature of -25℃~+85℃.
[0061] 2. Edge computing module
[0062] Includes edge computing terminals for receiving video data from wireless solar cameras for real-time analysis;
[0063] The edge computing terminal is equipped with a high-performance AI engine, which supports independent configuration of target recognition, perimeter defense, video structuring, high-altitude throwing and other engine mode switching, supports viewing the target extracted by the video by querying the target attributes, and supports viewing and downloading all pictures; Access capability: 4-channel H.264, H.265 format high-definition stream access; the list library capacity supports 100,000; Network interface: 2×RJ45 10 / 100 / 1000Mbps adaptive Ethernet port;
[0064] Edge nodes with computing capabilities are deployed at various key locations. Each node can perform preliminary data analysis, such as face recognition, abnormal behavior detection, etc., and develop special AI models to run on edge nodes to achieve real-time data analysis and instant decision-making. Edge nodes can detect abnormal situations and respond in the first time, such as automatic alarms. Abnormal event information can also be synchronized to the central server in real time for further analysis and recording. The central server regularly sends updated AI models to edge nodes to ensure continuous optimization of the algorithm.
[0065] By processing video streams instantly at edge nodes, reliance on central servers is reduced, and data processing speed and response time are improved. It supports multiple intelligent analysis modes such as face recognition, vehicle recognition, and perimeter protection, and can be switched according to the needs of different scenarios, achieving high flexibility and wide application.
[0066] In one embodiment, the edge computing module includes an adaptive monitoring module to annotate and collect video data under different lighting conditions, such as strong light conditions (direct or reflected sunlight may cause the video captured by the camera to be overexposed and details to be lost), weak light conditions (such as dusk, dawn or cloudy days, the light is weak, which may cause the video image to be blurred and have low contrast), backlight conditions (when the light source is opposite to the camera, such as the sun behind the camera, it may cause the target object to appear as a dark area in the image, and the background to be overexposed), night conditions (at night or in dim environments, artificial light sources are usually relied on, such as street lights, searchlights, etc., which may cause uneven lighting and shadows), flickering light conditions (caused by flashing lights). Flashing lights or light generated by electronic devices may cause flickering or interference in the video), dynamic lighting conditions (such as rapid changes in light caused by clouds blocking the sun, or local light changes caused by passing vehicles), high-contrast conditions (such as the sun and shadows appearing in the picture at the same time, resulting in extremely high contrast between bright and dark areas in the image), special light source conditions (such as shooting under non-visible light conditions such as infrared and ultraviolet rays), etc.; use TensorFlow or PyTorch deep learning frameworks for training to identify and adapt to different lighting conditions; use CSN networks (Conditional Switching Networks) to automatically select video processing models corresponding to different lighting conditions based on the analysis results of ambient light sensors or video data;
[0067] The CSN network contains multiple branches, each of which optimizes the video processing model under a certain lighting condition. The network selection logic is implemented through the softmax layer, and the formula is:
[0068] P(b|a)=softmax(W h ·h(a));
[0069] Where h(a) represents the feature parameter extracted from input a; W h is a weight matrix; P(b|a) represents the probability of selecting branch b under the condition of given input a.
[0070] In a specific embodiment, the edge computing module includes a video structuring module and a security incident response module;
[0071] The video structuring module is used to perform pre-processing operations such as decoding, frame extraction, and scaling of the received video data through the selected video processing model, and to perform structural analysis on targets such as people and vehicles through the target recognition algorithm, and to extract and store target attributes into the target database to facilitate subsequent retrieval by target attributes;
[0072] Target recognition algorithms include YOLOv7-tiny algorithm with SENet (Squeeze-and-Excitation Networks), SSD (Single Shot MultiBox Detector) algorithm, Faster R-CNN (Faster Region-based Convolutional Network), etc.
[0073] The security incident response module is used to classify security incidents through the ResNet network, and automatically adjust the camera parameters according to the event type to optimize event capture or activate the alarm function, including the target comparison unit, perimeter prevention unit and high-altitude throwing unit; the security incident response module can automatically adjust the camera focus and angle according to the specific event type identified (such as illegal intrusion, high-altitude throwing, etc.) to better track and record the progress of the event;
[0074] The target comparison unit is used to compare the target in the current video frame with a preset target database to identify whether there is a matching target attribute;
[0075] The perimeter protection unit is used to identify objects or people crossing the perimeter in the perimeter protection scenario. If an object or person is detected crossing the set perimeter, it will be captured and an alarm will be triggered;
[0076] The high-altitude object throwing unit is used to analyze the motion trajectory and object features in the video frame to identify the characteristic patterns in the object throwing process. If a high-altitude object throwing event is detected, the camera angle is adjusted to capture the object throwing process and trigger an alarm.
[0077] Furthermore, the security events are classified through the ResNet network, including:
[0078] The input layer receives the normalized input image and adds padding to the border of the input feature map, with padding = 1.
[0079] Feature extraction is performed through a 7×7 convolutional layer with a step size of 2 and 64 output channels;
[0080] Downsample through a 3×3 max pooling layer with a stride of 2;
[0081] Multiple residual blocks are stacked. Each residual block consists of two or three convolutional layers. The convolutional layers are nonlinearly transformed and normalized through ReLU activation functions and batch normalization layers. The output of each residual block is added to the input of the previous residual block through an identity connection to form residual learning.
[0082] After all residual blocks are processed, the feature map is converted into a feature vector through a global average pooling layer;
[0083] The security events are classified through the fully connected layer.
[0084] In another specific embodiment, the edge computing module may further include a multi-camera collaboration module and a dynamic resource scheduling module;
[0085] The multi-camera collaboration module is used to determine the position and viewing angle of each camera, and synchronously process the video data collected by multiple cameras on the edge computing terminal through the data fusion model; data fusion can improve the accuracy of target recognition and monitoring coverage;
[0086] The data fusion model is based on MVS (Multi-View Stereo) technology to find the image corresponding points between different views, and reconstruct the three-dimensional scene based on the image corresponding points to integrate the image data from different cameras. The formula is:
[0087]
[0088] Where D(x,y) represents the depth value of the image coordinate (x,y); argmin d It means to find the d value that makes the following sum expression reach the minimum value, where d value refers to the disparity value; n is the number of cameras; SSD means sum of squared differences; I ref (x,y) represents the reference image I ref The pixel value at coordinate (x,y); Represents the i-th camera view I i After parallax correction, the coordinates The pixel value at denote the disparity from the reference image to the i-th camera view in the horizontal and vertical directions, respectively;
[0089] The dynamic resource scheduling module is used to monitor the computing resources and power status of the edge computing module in real time, and optimize and adjust task allocation according to the optimization algorithm; optimize resource allocation (such as computing power and electricity) according to the current workload and power status to ensure that the system can maintain optimal performance even when energy is limited;
[0090] The formula of the optimization algorithm is:
[0091]
[0092] Where Z is the total cost of the task; c m is the cost of task m; f m is a decision variable, indicating whether task m is selected for execution. m =1, then task m is selected for execution. If f m =0, task m is not executed; r m is the amount of resources required for task m; R is the total amount of available resources; M is the total number of tasks.
[0093] Preferably, the edge computing module may also include an extended API interface for accessing drone monitoring to conduct joint air and ground monitoring, such as intercepting objects thrown from high altitudes through drones;
[0094] Provide an open API interface to allow third-party developers to develop new monitoring modules according to specific needs, such as drone automatic scheduling modules, so that the monitoring system can be seamlessly connected with drones for joint air and ground monitoring.
[0095] 3. User interface module
[0096] It is used to provide a visual interface for PC and mobile terminals, so that operators can configure target attributes, view video data, data analysis results, system status and alarm information in real time, play back historical videos, or manually intervene in the monitoring process;
[0097] Specifically, the manual intervention monitoring process includes configuring target attributes, such as Figure 2 As shown. The structured analysis in the video structuring module can be divided into human body analysis and vehicle analysis. In the target attribute configuration, the target attributes corresponding to human body analysis are divided into face attributes and body attributes. Face attributes include gender, age group, whether wearing glasses, whether wearing masks, expressions, etc. Body attributes include top color, gender, whether wearing glasses, age group, whether backpacking, whether riding a bicycle, etc.; The target attributes corresponding to vehicle analysis are divided into motor vehicle attributes and non-motor vehicle attributes. Motor vehicle attributes include license plate number, vehicle type, body color, vehicle brand, etc. Non-motor vehicle attributes include the rider's top color, top type, gender, whether wearing glasses or masks, whether backpacking, whether wearing hats, whether carrying passengers, non-motor vehicle type, etc.
[0098] 4. Data transmission module
[0099] It includes wireless CPE equipment, supports 4G / 5G transmission, wireless solar cameras access wireless CPE equipment through wireless network for data transmission, edge computing module and user interface module connect wireless CPE equipment through network cable for data transmission, which can reduce the physical wiring required for data transmission and improve deployment flexibility;
[0100] Optionally, the system can further implement advanced encryption technologies and access control strategies, such as Dual Layer Dynamic Key Encryption, which combines traditional symmetric encryption technology with a dynamic key generation mechanism to enhance data confidentiality and anti-attack capabilities, ensure the security of all transmitted and stored video data, prevent data leakage and unauthorized access, and enhance data security.
[0101] like Figure 3 As shown in FIG. 1 , another embodiment of the present invention provides a mobile control monitoring method based on edge computing, comprising the following steps:
[0102] Configure a wireless solar camera and adjust the vertical angle as needed. Use the wireless solar camera to collect video data of the specified area in real time and access the wireless CPE device through the wireless network for data transmission.
[0103] The wireless CPE device receives the video data from the wireless solar camera and transmits the video data to the edge computing module and the user interface module via the network cable;
[0104] The edge computing terminal receives video data from the wireless CPE device and uses the CSN network to automatically select the video processing model corresponding to different lighting conditions;
[0105] The received video data is decoded, frame extracted, and scaled by the video processing model, and the video data is structurally analyzed by the target recognition algorithm to extract and store target attributes.
[0106] Use the ResNet network to classify security events and automatically adjust camera parameters or trigger alarm functions based on the event type;
[0107] Operators can view video data, data analysis results, system status and alarm information in real time through the visual interface of PC or mobile terminal, or manually intervene in the monitoring process.
[0108] Furthermore, the edge computing terminal can synchronously process the video data collected by multiple cameras through the data fusion model. The data fusion model specifically includes:
[0109] Based on the MVS technology, we find the image corresponding points between different views and reconstruct the 3D scene based on the image corresponding points to integrate the image data from different cameras. The formula is:
[0110]
[0111] Where D(x,y) represents the depth value of the image coordinate (x,y); argmin d It means to find the d value that makes the following sum expression reach the minimum value, where d value refers to the disparity value; n is the number of cameras; SSD means sum of squared differences; I ref (x,y) represents the reference image I ref The pixel value at coordinate (x,y); Represents the i-th camera view I i After parallax correction, the coordinates The pixel value at denote the disparity from the reference image to the i-th camera view in the horizontal and vertical directions, respectively.
[0112] In summary, the present invention uses solar cameras to reduce dependence on external power supplies and enhance the applicability and environmental adaptability of the system; it can be quickly deployed in any occasion that requires temporary monitoring, is suitable for the security management of various temporary or special events, and has strong flexibility and mobility; it takes advantage of edge computing and processes data at local nodes, significantly reducing the time for data processing and response, reducing latency, and alleviating bandwidth pressure, and can provide faster and safer data processing capabilities, reducing the need for large amounts of data to be transmitted to central servers, and meeting the needs of modern cities for high efficiency and high safety standards; through data fusion, it can synchronously process video data collected by multiple cameras, greatly improving response speed and decision-making efficiency.
[0113] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.
[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A mobile control and monitoring system based on edge computing, characterized in that: The system comprises: A mobile control module is used to perform mobile and rapid control according to control requirements, including a wireless solar camera that can freely adsorb poles or walls and achieve intelligent climbing through remote control, and uses the wireless solar camera to collect monitoring video data in real time; An edge computing module, including an edge computing terminal, for receiving video data from a wireless solar camera for real-time analysis; The user interface module is used to provide a visual interface so that operators can view video data, data analysis results, system status and alarm information in real time, play back historical videos, or manually intervene in the monitoring process; The data transmission module includes a wireless CPE device, wherein the wireless solar camera is connected to the wireless CPE device through a wireless network for data transmission, and the edge computing module and the user interface module are connected to the wireless CPE device via a network cable for data transmission; The edge computing module also includes a multi-camera collaboration module and a dynamic resource scheduling module; The multi-camera collaboration module is used to determine the position and viewing angle of each camera, and synchronously process the video data collected by multiple cameras on the edge computing terminal through a data fusion model; The data fusion model searches for image corresponding points between different views based on the MVS technology, and reconstructs the three-dimensional scene according to the image corresponding points to integrate the image data from different cameras. The formula is: Where D(x,y) represents the depth value of the image coordinate (x,y); argmin d It means to find the d value that makes the following sum expression reach the minimum value, where d value refers to the disparity value; n is the number of cameras; SSD means sum of squared differences; I ref (x,y) represents the reference image I ref The pixel value at coordinate (x,y); Represents the i-th camera view I i After parallax correction, the coordinates The pixel value at denote the disparity from the reference image to the i-th camera view in the horizontal and vertical directions, respectively; The dynamic resource scheduling module is used to monitor the computing resources and power status of the edge computing module in real time, and optimize and adjust task allocation according to the optimization algorithm; The formula of the optimization algorithm is: Where Z is the total cost of the task; c m is the cost of task m; f m is a decision variable, indicating whether task m is selected for execution. m =1, then task m is selected for execution. If f m =0, task m is not executed; r m is the amount of resources required for task m; R is the total amount of available resources; M is the total number of tasks.
2. According to the mobile monitoring system based on edge computing of claim 1, it is characterized in that: The wireless solar camera has a built-in solar power supply module and a backup power supply module. A full charge can support working in a lightless environment for at least 24 hours. The vertical angle can be adjusted from -50° to 0°. It has its own microphone and speaker, and stores video data through a TF card.
3. According to the mobile monitoring system based on edge computing of claim 1, it is characterized in that: The edge computing module includes an adaptive monitoring module, which annotates and collects video data under different lighting conditions, and uses the deep learning framework of TensorFlow for training to identify and adapt to different lighting conditions. According to the analysis results of the ambient light sensor or video data, the CSN network is used to automatically select the video processing model corresponding to different lighting conditions. Each branch in the CSN network optimizes the video processing model corresponding to one lighting condition. The network selection logic is implemented through the softmax layer, and the formula is: P(b|a)=softmax(W h ×(h(a))); Where h(a) represents the feature parameter extracted from input a; W h is a weight matrix; P(b|a) represents the probability of selecting branch b under the condition of given input a.
4. The mobile control and monitoring system based on edge computing according to claim 3 is characterized in that: The edge computing module includes a video structuring module and a security incident response module; The video structuring module is used to perform pre-processing operations such as decoding, frame extraction, and scaling of received video data through a selected video processing model, and to perform structured analysis on a human body or a vehicle through a target recognition algorithm, and to extract and store target attributes into a target database; the target recognition algorithm is specifically a YOLOv7-tiny algorithm with SENet added; The security event response module is used to classify security events through the ResNet network, and automatically adjust the camera parameters according to the event type to optimize event capture or enable the alarm function, including a target comparison unit, a perimeter prevention unit, and a high-altitude throwing unit; The target comparison unit is used to compare the target in the current video frame with a preset target database to identify whether there is a matching target attribute; The perimeter prevention unit is used to identify objects or people crossing the perimeter in the perimeter prevention scene, and if an object or person is detected to have crossed the set perimeter, it will take a snapshot and trigger an alarm; The high-altitude throwing unit is used to analyze the motion trajectory and object features in the video frame to identify the characteristic patterns in the throwing process. If a high-altitude throwing event is detected, the camera angle is adjusted to capture the throwing process and trigger an alarm.
5. The mobile control and monitoring system based on edge computing according to claim 4 is characterized in that: The security incidents are classified by the ResNet network, specifically including: The input layer receives the normalized input image and adds padding to the border of the input feature map, with padding = 1. Feature extraction is performed through a 7×7 convolutional layer with a step size of 2 and 64 output channels; Downsample through a 3×3 max pooling layer with a stride of 2; Multiple residual blocks are stacked. Each residual block consists of two or three convolutional layers. The convolutional layers are nonlinearly transformed and normalized through ReLU activation functions and batch normalization layers. The output of each residual block is added to the input of the previous residual block through an identity connection to form residual learning. After all residual blocks are processed, the feature map is converted into a feature vector through a global average pooling layer; The security events are classified through the fully connected layer.
6. The mobile control and monitoring system based on edge computing according to claim 1 is characterized in that: The edge computing module also includes an extended API interface for accessing drone monitoring to conduct joint monitoring of the air and the ground.
7. A mobile control and monitoring method of a mobile control and monitoring system based on edge computing according to any one of claims 1 to 6, characterized in that: The method comprises: Configure a wireless solar camera and adjust the vertical angle as needed. Use the wireless solar camera to collect video data of the specified area in real time and access the wireless CPE device through the wireless network for data transmission. The wireless CPE device receives video data from the wireless solar camera and transmits the video data to the edge computing module and the user interface module via a network cable; The edge computing terminal receives video data from the wireless CPE device and uses the CSN network to automatically select the video processing model corresponding to different lighting conditions; The edge computing terminal synchronously processes the video data collected by multiple cameras through a data fusion model, and the data fusion model specifically includes: Based on the MVS technology, the image corresponding points between different views are found, and the three-dimensional scene is reconstructed according to the image corresponding points to integrate the image data from different cameras. The formula is: Where D(x,y) represents the depth value of the image coordinate (x,y); argmin d It means to find the d value that makes the following sum expression reach the minimum value, where d value refers to the disparity value; n is the number of cameras; SSD means sum of squared differences; I ref (x,y) represents the reference image I ref The pixel value at coordinate (x,y); Represents the i-th camera view I i After parallax correction, the coordinates The pixel value at denote the disparity from the reference image to the i-th camera view in the horizontal and vertical directions, respectively; The received video data is decoded, frame extracted, and scaled by the video processing model, and the video data is structurally analyzed by the target recognition algorithm to extract and store target attributes; Use the ResNet network to classify security events and automatically adjust camera parameters or trigger alarm functions based on the event type; Operators can view video data, data analysis results, system status and alarm information in real time through the visual interface of PC or mobile terminal, or manually intervene in the monitoring process.
