Intelligent edge preprocessing system and method based on equipment computing power

Abstract

The invention relates to an edge preprocessing technology, and discloses an intelligent edge preprocessing system and method based on equipment computing power, which are used for reducing dependence on cloud computing resources, optimizing storage and transmission efficiency and meeting the requirements of real-time monitoring and quick response. The method comprises the following steps: S1, collecting a video stream of a monitoring area in real time by a camera; s2, the edge device receives a video stream collected by a camera; s3, a monitoring application program on the edge device calls the target detection model to analyze the video stream in real time, extracts video key frames, marks event information and stores the event information to a storage unit; and S4, the monitoring application program on the edge device judges whether the event is an important event according to a preset rule, and uploads the key frame of the important event and the event marking information to the cloud. The method is especially suitable for scenes sensitive to real-time performance and resource consumption, such as security monitoring and traffic monitoring.

Description

Technical Field

[0001] The present invention relates to edge preprocessing technology, and in particular to an edge intelligent preprocessing system and method based on device computing power. Background Art

[0002] In the current monitoring technology landscape, traditional monitoring systems primarily rely on cloud servers for data storage and processing. This model presents a series of challenges in practical applications:

[0003] (1) Data relies on cloud storage:

[0004] Because traditional surveillance systems store large amounts of image data in the cloud, cloud storage costs remain high. With the increasing number of surveillance devices and the continued extension of monitoring time, the amount of stored data is growing exponentially, forcing businesses or organizations to pay substantial fees for cloud storage. Furthermore, the process of transmitting data from devices to the cloud requires high network bandwidth, further increasing operating costs. This is especially true when network conditions are poor, severely impacting the stability and efficiency of data transmission.

[0005] (2) Waste of storage resources:

[0006] In actual surveillance, many video clips may simply record scenes where no unusual events have occurred, such as corridors or empty parking lots where no one has passed through for extended periods. These clips have no substantial value for the surveillance target, but they consume a significant amount of storage space. Statistics show that in some common surveillance scenarios, invalid video data can account for over 70% of total storage capacity, resulting in a significant waste of storage resources.

[0007] (3) Poor real-time performance:

[0008] Because data must first be transmitted to the cloud for processing and then returned to the device, delays in data transmission and analysis are inevitable. In monitoring scenarios requiring extremely high real-time performance, such as bank surveillance and capturing traffic violations, even a delay of just a few seconds can result in the omission of important information, making it impossible to detect and address anomalies in a timely manner, making it difficult to meet the demands of real-time monitoring and rapid response.

[0009] (4) High computing resource consumption:

[0010] Traditional monitoring systems rely on high-performance servers for data processing. These servers need to have powerful computing capabilities to cope with the analysis tasks of large amounts of monitoring data, resulting in a large consumption of computing resources. Once the cloud server fails or is overloaded, the performance of the entire monitoring system will be seriously affected. In addition, this cloud-dependent model also limits the application of monitoring systems in some environments with limited network conditions or high privacy requirements. Summary of the Invention

[0011] The technical problem to be solved by the present invention is to propose an edge intelligent preprocessing system and method based on device computing power, reduce dependence on cloud computing resources, optimize storage and transmission efficiency, and meet the needs of real-time monitoring and rapid response.

[0012] The technical solution adopted by the present invention to solve the above technical problems is:

[0013] In one aspect, the present invention provides an edge intelligent preprocessing system based on device computing power, comprising: an edge device, a camera, and a monitoring application;

[0014] The edge device includes: a processing unit for running the target detection model; a display screen for displaying monitoring images and event alarms; a wifi module for establishing a connection with the camera; and a storage unit for storing video key frames and event marker information.

[0015] The camera is used to collect video streams of the monitored area in real time;

[0016] The monitoring application runs on the edge device and is used to receive the video stream collected by the camera in real time, call the target detection model to perform real-time analysis on the video stream, extract the video key frames and mark the event information, and store it in the storage unit, and determine whether the event is an important event according to preset rules, and upload the key frames and event marking information of the important event to the cloud.

[0017] Furthermore, the edge intelligent preprocessing system supports multiple edge devices to work together, and each edge device supports establishing connections with multiple cameras.

[0018] Furthermore, the important events include but are not limited to the following types: personnel intrusion events in security monitoring, traffic accident events in traffic monitoring, and equipment failure events in industrial monitoring.

[0019] On the other hand, the present invention also provides an edge intelligent preprocessing method based on device computing power, which is applied to the above system and includes the following steps:

[0020] S1. The camera collects the video stream of the monitoring area in real time;

[0021] S2. The edge device receives the video stream captured by the camera;

[0022] S3. The monitoring application on the edge device uses the target detection model to perform real-time analysis on the video stream, extract key frames from the video, mark event information, and store it in the storage unit.

[0023] S4. The monitoring application on the edge device determines whether the event is an important event based on preset rules and uploads the key frames and event marker information of the important event to the cloud.

[0024] Furthermore, in step S3, the monitoring application on the edge device preprocesses the image frames in the video stream before calling the target detection model to perform real-time analysis on the video stream. The preprocessing includes but is not limited to: color space conversion, noise removal, and image enhancement.

[0025] Furthermore, in step S3, the target detection model adopts an optimized YOLO Tiny model, where the optimized YOLO Tiny model is optimized based on the YOLO Tiny model, and the optimization includes model structure optimization, inference box optimization, and loss function optimization;

[0026] Among them, the model structure optimization is to reduce model parameters and calculation amount through pruning and quantization perception technology; the inference box optimization is to dynamically adjust the inference box size to adapt to different target sizes; the loss function optimization is to customize the optimization loss function according to the detection targets in different application scenarios.

[0027] Furthermore, in step S3, the video key frame is an image frame related to the detection target, and the marked event information includes event type, target category and timestamp, etc.

[0028] Furthermore, in step S3, after extracting the video key frames and marking the event information, the key frames and the corresponding event marking information are compressed using a compression algorithm and then stored in a storage unit.

[0029] Furthermore, in step S3, when storing the key frames and corresponding event marking information in the storage unit, the data is stored in storage media of different levels according to the event priority, including: storing the key frames and corresponding event marking information of emergency events in a cache area, and storing the key frames and corresponding event marking information of non-emergency events in a low-frequency access storage area.

[0030] Furthermore, in step S4, before uploading the key frames and event marker information of important events to the cloud, the data is encrypted, and the upload speed is automatically adjusted according to the network conditions during the upload process.

[0031] The beneficial effects of the present invention are:

[0032] (1) High efficiency: The solution of the present invention can analyze the collected video stream in real time and extract key frames through the YOLO Tiny model on the device side. There is no need to transmit the data to the cloud for processing. Therefore, it can quickly respond to changes in the monitoring system, quickly identify abnormal events, and issue alarms in a timely manner, ensuring a short reaction time and high real-time performance, greatly improving the response speed and processing efficiency of the monitoring system.

[0033] (2) Low resource consumption: The solution of the present invention can complete image processing and data storage on the device side, significantly reducing dependence on cloud computing resources and reducing the overall energy consumption and computing resource usage of the system.

[0034] (3) Storage and transmission optimization: The solution of the present invention only stores key frames and event marker data related to the detection target, and only selects key frames and marker information of important events to upload to the cloud, thereby greatly reducing the storage space requirements and the amount of data transmitted from the device to the cloud, thereby effectively reducing operating costs.

[0035] (4) Intelligence and Adaptability: The solution of the present invention is based on edge computing technology and can make intelligent decisions directly on the device side, thereby automatically adjusting the detection strategy according to different monitoring scenarios and improving the intelligence level of the system. For example, in traffic monitoring scenarios, the detection frequency and accuracy can be automatically adjusted according to the traffic volume at different time periods; in security monitoring scenarios, the image preprocessing parameters can be automatically optimized according to changes in ambient light to better adapt to various complex monitoring environments.

[0036] (5) Strong scalability: The solution of the present invention supports the collaborative work of multiple devices and can achieve efficient data processing and storage management among heterogeneous devices. For example, in a large-scale monitoring project, multiple Android devices with screens can be connected together to complete the monitoring task together. Each device is responsible for a part of the monitoring area, thereby expanding the monitoring range and improving the monitoring capabilities, and has strong system scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a structural block diagram of the edge intelligent preprocessing system based on device computing power in the present invention;

[0038] Figure 2 This is a flow chart of the edge intelligent preprocessing method based on device computing power in the present invention. DETAILED DESCRIPTION

[0039] The present invention aims to provide an edge intelligent preprocessing system and method based on device computing power, reducing reliance on cloud computing resources, optimizing storage and transmission efficiency, and meeting the needs of real-time monitoring and rapid response. Its core concept is to achieve real-time intelligent analysis and processing of surveillance video streams by integrating a lightweight YOLO Tiny model on edge devices. At the same time, the system only extracts and stores key frames and event marker data, and through data compression and low-bandwidth transmission strategies, significantly reduces storage space and network bandwidth consumption, thereby improving the real-time performance, intelligence level, and cost-effectiveness of the monitoring system. It is particularly suitable for scenarios sensitive to real-time performance and resource consumption, such as security monitoring and traffic monitoring.

[0040] In specific implementation, the edge intelligent preprocessing system structure based on device computing power provided by the present invention can be found in Figure 1 , which includes edge devices, cameras, and monitoring applications.

[0041] The edge device includes: a processing unit for running the target detection model; a display screen for displaying monitoring images and event alarms; a wifi module for establishing a connection with the camera; and a storage unit for storing video key frames and event marker information.

[0042] The camera is used to collect video streams of the monitored area in real time;

[0043] The monitoring application runs on the edge device and is used to receive the video stream collected by the camera in real time, call the target detection model to perform real-time analysis on the video stream, extract the video key frames and mark the event information, and store it in the storage unit, and determine whether the event is an important event according to preset rules, and upload the key frames and event marking information of the important event to the cloud.

[0044] Based on this system, the edge intelligent preprocessing method based on device computing power provided by the present invention can be found in Figure 2 , which includes the following implementation process:

[0045] S1. Real-time acquisition of video streams of the monitored area;

[0046] In this step, the camera collects real-time data of the monitoring area to obtain the monitoring video stream.

[0047] S2. The edge device receives the video stream;

[0048] In this step, the Wi-Fi module on the edge device establishes a connection with the camera to receive the video stream captured by the camera.

[0049] S3. The edge device analyzes the video stream in real time and extracts key frames and marked events for storage;

[0050] In this step, the monitoring application on the edge device calls the target detection model to perform real-time analysis on the video stream, extract the video key frames, mark the event information, and store it in the storage unit.

[0051] The target detection model can use the YOLO Tiny model. YOLO Tiny is an efficient target detection model based on a convolutional neural network architecture. Due to its high speed, high precision, and lightweight characteristics, it is widely used in real-time object detection tasks. In the present invention, in order to adapt to the limited computing power of the device end, the YOLO Tiny model can be further optimized to make it more lightweight. While ensuring detection accuracy, the model parameters and computational complexity are minimized, thus laying a good foundation for edge processing and computing.

[0052] Optimization content includes but is not limited to the following aspects:

[0053] Structural optimization: Reduce model parameters and computational complexity through pruning and quantization-aware technologies. For example, a pruning criterion based on the BatchNormalization layer scaling factor is used. By adding a sparse regularization term to the loss function, the scaling coefficient of some BN layers is made close to zero, thereby achieving channel pruning. Quantization-aware training (QAT) is used to convert model weights from floating-point format to low-precision format. Through structural optimization, the model size is made smaller and can be run on low-computing edge devices.

[0054] Inference box optimization: To further reduce the computational burden and improve real-time performance, the inference box size is dynamically adjusted to accommodate different target sizes. For example, a smaller inference box (e.g., 16x16) is used for small targets, and a larger inference box (e.g., 32x32) is used for large targets. This ensures that the inference box properly matches the target being detected, reducing the computational effort during inference. This is like narrowing the search range, thereby increasing detection speed. This optimization enables the model to deliver detection results in a shorter time when processing real-time video streams, ensuring the real-time responsiveness of the monitoring system.

[0055] Loss Function Optimization: This paper customizes the YOLOTiny model's training and optimizes its loss function for different application scenarios, such as traffic monitoring and security surveillance. For example, in a traffic monitoring scenario, where the focus is on vehicle and pedestrian detection, adjusting the loss function significantly improves the model's detection accuracy for both vehicles and pedestrians. This customized training allows the model to better adapt to the needs of specific scenarios and improves the accuracy of detecting specific targets.

[0056] Before the image frames in the video stream are fed into the optimized YOLO Tiny model, the present invention performs various preprocessing operations on the images, including color space conversion, noise removal, and image enhancement. Color space conversion converts the image into a format more suitable for model processing, noise removal reduces interference in the image, and image enhancement improves image clarity and contrast. These operations work together to enhance image quality and provide a more accurate data foundation for subsequent detection.

[0057] The preprocessed image is then used as input to the optimized YOLO Tiny model. When the model detects a specific target, such as a pedestrian, vehicle, or other object of interest, the system automatically extracts the keyframe containing the target and marks the timestamp and event type of the event, such as human intrusion, traffic accident, etc.

[0058] To conserve storage resources, the present invention only stores keyframes and event marker data relevant to the monitored target, eliminating redundant video streams and invalid data. Each keyframe is tagged with detailed information, such as event type, target category, and time, making subsequent storage and retrieval more efficient. When performing event retrieval, users can quickly locate the relevant keyframe based on information such as time and event type, significantly improving data processing efficiency.

[0059] To further save storage space, the present invention compresses the key frames and event marker data before storage, and can use efficient compression algorithms such as H.264 or HEVC. These algorithms can store data in a local SD card without losing too much image quality. During storage, through reasonable storage classification, different storage strategies are adopted for event data of different priorities. For example, the key frames and corresponding event marker information of emergency events are stored in a high-speed cache area, and the key frames and corresponding event marker information of non-emergency events are stored in a low-frequency access storage area. This method not only improves storage efficiency, but also ensures that important data can be quickly obtained when needed.

[0060] S4. The edge device uploads key frames and event marker information of important events to the cloud.

[0061] In this step, in order to further reduce the amount of data transmission, the monitoring application on the edge device determines whether the event is an important event based on preset rules, and uploads the key frames and event marker information of the important event to the cloud.

[0062] Among them, the preset rules can be set according to the actual application scenario, and whether it is an important event is determined based on the type of event, scope of impact, etc. For example: in security monitoring, the event of detecting a person entering a restricted area is marked as an important event and triggers an alarm; in industrial monitoring, equipment failure events are marked as important events; in traffic monitoring, congestion or traffic accidents are marked as important events, etc.

[0063] Example

[0064] In this embodiment, every link, from the selection of device hardware architecture to the development of system software and applications, and then to the formulation of data storage and optimization strategies, works closely together to ensure that the system runs efficiently and stably in actual applications.

[0065] 1. Device hardware architecture

[0066] In this embodiment, the edge device is an Android device with a screen. The device has basic hardware configurations such as a processing unit, a display, a Wi-Fi module, an SD card storage, and a battery. The device can connect to up to four cameras via the Wi-Fi hotspot function and support simultaneous reception of monitoring images from multiple cameras.

[0067] Device hardware description:

[0068] Android devices with screens: The configured screen not only displays the monitoring image but also intuitively displays information such as device operating status and event warnings, providing a convenient interactive interface for users. During the monitoring process, users can view the monitoring scene in real time through the screen. If an abnormal event occurs, a warning message will pop up on the screen to remind the user to take action.

[0069] Wi-Fi Module: The device supports Wi-Fi hotspot functionality, connecting up to four external cameras. The cameras connect wirelessly to the device via Wi-Fi, a convenient and fast connection method that reduces wiring hassles and increases deployment flexibility. In complex surveillance environments, such as large shopping malls and factories, users can flexibly adjust camera positions to achieve comprehensive surveillance coverage.

[0070] SD Card Storage: The device uses a built-in SD card as local storage for processing keyframe and event marker data. SD cards offer large storage capacity and fast read / write speeds, meeting all data storage needs. Furthermore, the removable SD card facilitates data backup and transfer when needed.

[0071] Processing Unit: Equipped with sufficient CPUs and GPUs, it provides powerful computing support for running lightweight AI models such as YOLO Tiny. The CPU handles basic system tasks and data scheduling, while the GPU focuses on the model's compute-intensive tasks, such as convolution operations. The two work together to ensure that the model can run efficiently on the device.

[0072] 2. System software and applications

[0073] A dedicated monitoring application is installed on the device, which is responsible for acquiring real-time video streams from the camera, performing image processing, and marking and storing events. The specific process is as follows:

[0074] Camera connection and video capture: Each camera connects to the device's hotspot via Wi-Fi. The device receives the video stream through the app and processes it in real time. During the connection process, the app automatically identifies the camera device and establishes a stable connection channel, ensuring real-time and stable transmission of the video stream.

[0075] AI reasoning and analysis: The application uses the YOLO Tiny model to analyze the camera video stream in real time, detecting target objects and generating keyframes. During this process, the application inputs the video stream frame by frame into the YOLO Tiny model, which rapidly analyzes each frame, identifies the target object, and generates keyframes based on predefined rules.

[0076] Keyframe Storage and Event Marking: Based on AI analysis results, important keyframes are extracted and event marker information, such as timestamp, target type, and event type, is recorded and saved to the SD card. During storage, the application will store data of different priorities in the appropriate location according to the storage classification strategy, ensuring efficient storage and fast retrieval.

[0077] Data synchronization with the cloud: For important event data, the device can choose to upload critical data to the cloud for long-term storage and further analysis. During the upload process, the application encrypts the data to ensure security. Furthermore, to avoid network congestion, the application automatically adjusts the upload speed based on network conditions to ensure stable data transmission.

[0078] 3. Data compression and optimized storage

[0079] In terms of data storage, the device uses an efficient data compression algorithm to store only necessary events and keyframe information. Storage optimization is achieved through the following methods:

[0080] Data compression: Use video compression algorithms such as HEVC or H.264 to compress stored key frames. These algorithms can compress the size of key frames to a fraction or even tens of times the original size while ensuring image quality, ensuring minimal storage usage.

[0081] Storage tiering: Event data of different priorities uses different storage strategies. Keyframe data from emergency events is prioritized for storage in local fast storage to ensure rapid access when needed. Other data is stored as less frequently accessed data in larger but slower read / write storage areas. This approach improves storage efficiency and reduces storage costs.

[0082] Although the embodiments of the present invention have been described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, all without departing from the scope of protection of the present invention.

Claims

1. Edge intelligent pre-processing system based on device computing power, It is characterized by: include: edge devices, cameras, and surveillance applications; The edge device includes: a processing unit for running the target detection model; a display screen for displaying monitoring images and event alarms; a wifi module for establishing a connection with the camera; and a storage unit for storing video key frames and event marker information. The camera is used to collect video streams of the monitored area in real time; The monitoring application runs on the edge device and is used to receive the video stream collected by the camera in real time, call the target detection model to perform real-time analysis on the video stream, extract the video key frames and mark the event information, and store it in the storage unit, and determine whether the event is an important event according to preset rules, and upload the key frames and event marking information of the important event to the cloud.

2. The edge intelligent preprocessing system based on device computing power according to claim 1, characterized in that: The edge intelligent preprocessing system supports multiple edge devices to work together, and each edge device supports establishing connections with multiple cameras.

3. The edge intelligent preprocessing system based on device computing power according to claim 1, characterized in that: The important events include: personnel intrusion events in security monitoring, traffic accident events in traffic monitoring, and equipment failure events in industrial monitoring.

4. The edge intelligent preprocessing method based on device computing power is applied to the edge intelligent preprocessing system based on device computing power as described in any one of claims 1 to 3, characterized in that: The method comprises the following steps: S1. The camera collects the video stream of the monitoring area in real time; S2. The edge device receives the video stream captured by the camera; S3. The monitoring application on the edge device uses the object detection model to perform real-time analysis on the video stream, extract key frames from the video, mark event information, and store it in the storage unit. S4. The monitoring application on the edge device determines whether the event is an important event based on preset rules and uploads the key frames and event marker information of the important event to the cloud.

5. The edge intelligent preprocessing method based on device computing power according to claim 4, characterized in that: In step S3, before the monitoring application on the edge device calls the target detection model to perform real-time analysis on the video stream, it also preprocesses the image frames in the video stream. The preprocessing includes: color space conversion, noise removal and image enhancement.

6. The edge intelligent preprocessing method based on device computing power according to claim 4, characterized in that: In step S3, the target detection model adopts the optimized YOLO Tiny model, which is optimized based on the YOLO Tiny model. The optimization content includes model structure optimization, inference box optimization and loss function optimization; Among them, the model structure optimization is to reduce model parameters and calculation amount through pruning and quantization perception technology; the inference box optimization is to dynamically adjust the inference box size to adapt to different target sizes; the loss function optimization is to customize the optimization loss function according to the detection targets in different application scenarios.

7. The edge intelligent preprocessing method based on device computing power according to claim 4, characterized in that: In step S3, the video key frame is an image frame related to the detection target, and the marked event information includes event type, target category and timestamp.

8. The edge intelligent preprocessing method based on device computing power according to claim 4, characterized in that: In step S3, after extracting the video key frames and marking the event information, the key frames and the corresponding event marking information are compressed using a compression algorithm and then stored in a storage unit.

9. The edge intelligent preprocessing method based on device computing power according to claim 8, characterized in that: In step S3, when storing the key frames and corresponding event marking information in the storage unit, the data is stored in storage media of different levels according to the event priority, including: storing the key frames and corresponding event marking information of emergency events in a cache area, and storing the key frames and corresponding event marking information of non-emergency events in a low-frequency access storage area.

10. The edge intelligent preprocessing method based on device computing power according to claim 9, characterized in that: In step S4, before uploading the key frames and event marker information of important events to the cloud, the data is encrypted, and the upload speed is automatically adjusted according to the network conditions during the upload process.

Images