Deposit detection system based on edge calculation

Abstract

The invention discloses a deposit detection system based on edge calculation, and relates to the technical field of public safety management, and the system comprises a server-side model training module which is used for training a YOLOv5 model through employing a corridor deposit image data set, and obtaining a deposit detection model; the model conversion module is used for performing format conversion on the deposit detection model to generate a format model adaptive to the edge computing equipment; the edge computing terminal is configured with a TogeetherROS operating system and a Hobt DNN reasoning framework, and is used for loading and running the format model and detecting the acquired corridor image in real time through the format model; and the communication module comprises a 4G DTU module connected through a serial port and is used for sending the detection result to a mobile terminal through an MQTT protocol. According to the invention, the edge computing technology is adopted to realize the detection of the corridor deposits.

Description

Technical Field

[0001] This invention relates to the field of public safety management technology, and in particular to an edge computing-based debris detection system. Background Technology

[0002] Residential building corridors are not only passageways for daily walking, but also fire safety passages and vital lifelines in emergencies. Keeping them unobstructed is of great significance. However, in reality, individuals or organizations often occupy residential building corridors, and cardboard boxes and plastic bottles are frequently seen in corridors. These items not only easily accelerate the spread of fire, but also seriously affect residents' evacuation and escape in emergency situations.

[0003] Currently, detecting clutter in stairwells mainly relies on two methods: installing cameras at stairwells or conducting manual patrols. When installing cameras, fixed-angle cameras create blind spots; while rotating cameras can expand the monitoring range, they easily record the entrance areas of residents' homes, infringing on residents' privacy. Therefore, the practical implementation of camera installation faces difficulties. Manual patrols of residential stairwells consume a significant amount of manpower and greatly increase property management costs. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides an edge computing-based accumulation detection system, which solves the problems of blind spots in monitoring, easy infringement of residents' privacy and high labor costs of the existing detection methods.

[0005] The present invention adopts the following technical solution: In a first aspect, the present invention provides an accumulation detection system based on edge computing, comprising: The server-side model training module is used to train the YOLOv5 model using a dataset of images of debris in stairwells to obtain a debris detection model. The model conversion module is used to convert the stacking detection model into a format that is compatible with edge computing devices. An edge computing terminal is equipped with the TogetherROS operating system and the Hobot DNN inference framework, which is used to load and run the format model and perform real-time detection on the collected corridor images through the format model. The communication module includes a 4G DTU module connected via a serial port, which is used to send the detection results to the mobile terminal via the MQTT protocol.

[0006] Preferably, training the YOLOv5 model using a dataset of images of debris piled up in stairwells specifically includes the following steps: Images of debris piled up in stairwells were collected, and multiple images of debris piled up in stairwells were cleaned, labeled, and data augmented to obtain a dataset of debris piled up images; The YOLOv5 model was trained using a dataset of piled-up images to obtain a piled-up detection model.

[0007] Preferably, the format conversion of the accumulation detection model specifically includes the following steps: Convert the PyTorch format model of the accumulation detection model to ONNX format; The toolset is used to quantize and convert the ONNX format stacking detection model to generate a .bin format model adapted for edge computing devices.

[0008] Preferably, before sending the detection results to the mobile terminal via the MQTT protocol, the detected human body is coded.

[0009] Preferably, the cloud forwarding module, based on the rule engine in the Alibaba Cloud IoT platform, forwards the detection results to the mobile terminal through the message service.

[0010] Compared with the prior art, the above-mentioned at least one technical solution adopted by the present invention can achieve the following beneficial effects: First, this invention employs edge computing technology to detect accumulated objects in stairwells, offering a lower level of intrusion compared to traditional camera systems based on continuous video surveillance. Camera-based solutions often involve long-term video capture and storage, raising privacy concerns; however, this invention localizes image information, performing human detection on the accumulated objects before uploading the recognition results, and blurring the faces of detected individuals. This clear approach effectively protects personal privacy and meets the practical needs of intelligent monitoring in public areas.

[0011] Secondly, regarding hardware deployment and maintenance costs, the edge devices used in this invention are small in size and low in power consumption, eliminating the need for continuous video transmission and large storage devices. They can complete the uploading of recognition data simply by working with a 4G DTU module, significantly reducing the overall system deployment and communication costs. Simultaneously, the selected model has been quantized and optimized, enabling it to run on resource-constrained devices, avoiding reliance on high-performance GPU servers and further reducing hardware investment and maintenance pressure.

[0012] Finally, the system has a simple overall structure, combining real-time performance and scalability, making it easy to deploy quickly in various corridors and passageways, achieving low-cost and highly reliable intelligent detection, and helping to improve the automation level of corridor safety management. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of an edge computing-based pile detection system according to the present invention; a block diagram of the edge computing-based pile detection system. Figure 2 This is a flowchart of the edge device image recognition process of the present invention; Figure 3 This is a diagram of the data flow method of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] This invention, based on deep learning and edge computing technologies, proposes a method for detecting and visualizing debris in building corridors, addressing the problem of debris identification and processing in the field of public safety management. The system can quickly identify, process edges, remotely upload, and display abnormal debris within a target area on mobile devices, significantly improving the intelligence level of corridor safety management.

[0017] This invention integrates multiple technologies, including edge computing, deep learning, wireless communication, and cloud platform services. It covers key aspects such as image dataset construction, model deployment, and IoT communication, and features flexible deployment, rapid response, and intuitive information delivery. It is applicable to various scenarios such as building security and community governance. This invention aims to achieve efficient debris detection in smart building scenarios.

[0018] First, a dataset of stacked objects on stairs is constructed by collecting data from multiple scenarios, laying the foundation for model training. Second, a convolutional neural network is used to perform deep training on the dataset, enabling the model to recognize stacked objects. Then, the trained model is deployed to an edge computing platform, allowing edge devices to identify the stacked objects. Before uploading the identified information to the cloud, the system performs human detection and blurs the faces of identified individuals to minimize the risk of privacy leaks. Finally, using a 4G DTU device, the recognition results are uploaded to a mobile app, allowing managers to monitor the situation on-site in a timely manner. This technical solution effectively integrates data collection, model training, edge computing, and data transmission, enabling efficient detection of stacked objects, significantly improving the level of intelligent building management, and powerfully promoting the construction and development of smart buildings.

[0019] This system includes a server-side model training module, a model conversion module, an edge computing terminal, a communication module, and a cloud forwarding module.

[0020] The server-side model training module is used to train the YOLOv5 model using a dataset of images of debris in stairwells, resulting in a debris detection model.

[0021] The model conversion module is used to convert the stacking detection model into a format that is compatible with edge computing devices.

[0022] The edge computing terminal is equipped with the TogetherROS operating system and the Hobot DNN inference framework, which are used to load and run the format model and perform real-time detection on the collected corridor images.

[0023] The communication module includes a 4G DTU module connected via a serial port, which is used to send the detection results to the mobile terminal via the MQTT protocol.

[0024] The cloud-based forwarding module, based on the rules engine of the Alibaba Cloud IoT platform, forwards the detection results to the mobile terminal via message service. Reference Figures 1-3 This system also includes training a debris recognition model, deploying the recognition model on an edge computing platform, and displaying the recognition results via an app.

[0025] Step 1: Train the pile recognition model.

[0026] Step 1.1: Create the stack dataset.

[0027] In the process of constructing the dataset, this invention combines publicly available online resources with actual on-site images to build an initial image library of accumulated objects. Target annotation processing is performed on the self-collected data, and online images are used to supplement sample diversity. Through steps such as cleaning, annotation, and data augmentation, a high-quality image library for object recognition is finally constructed. Images and their labels irrelevant to the target scene are removed, and most of the collected images are single-category images. To address the insufficient number of accumulated object samples, additional data is collected from stacked objects in stairwell scenes to achieve a balanced distribution of data across different categories.

[0028] Step 1.2: Data augmentation.

[0029] In this step, to improve the model's generalization ability, the image data was enhanced using the Albumentations library. Specific operations included random rotation, adding Gaussian noise, adjusting image brightness, and adding the new data to the dataset of this invention.

[0030] Step 1.3: Train the recognition model.

[0031] In this step, the YOLOv5 algorithm is used. YOLOv5 boasts excellent real-time detection performance, ensuring high recognition accuracy while rapidly processing image information. This is crucial for scenarios requiring timely detection of debris on stairs to ensure pedestrian safety. Common activation functions include Sigmoid, SiLU, and Leaky ReLU. The X3 platform offers good support for Leaky ReLU, enabling hardware acceleration. Due to hardware platform operator limitations, some calculations with Sigmoid and SiLU run on the CPU, significantly reducing inference speed. YOLOv5 version 2.0 uses Leaky ReLU as the activation function; from version 3.0 onwards, SiLU is used. Therefore, this study selects YOLOv5 version 2.0 for model training. The training environment is a PC terminal equipped with an NVIDIA GeForce RTX 4070 Laptop graphics card, running Ubuntu 20.04.1 LTS, with Python 3.10, CUDA version 12.0, and PyTorch as the deep learning framework. When training the detection model based on YOLOv5 2.0, the batch size was set to 16 and the training epochs were 200. By selecting YOLOv5 2.0 and its hardware-friendly activation functions, the model's running efficiency on edge platforms was improved, achieving a good balance between recognition accuracy and speed.

[0032] Step 2: Deploy the recognition model on the edge computing platform.

[0033] Step 2.1: Format conversion of the model.

[0034] In this step, the edge computing platform used is RDK X3. The recognition model needs to be converted to a format usable by RDK X3 for stacking object recognition. The model trained in step 1.3 is converted from PyTorch format (.pt) to a device-compatible binary format (.bin). First, the PyTorch model is exported to ONNX format. Second, using the toolset in the Horizon Robotics OpenExplorer development platform, the hb_mapper checker is used to verify the runnability of the ONNX model on the BPU (edge ​​AI acceleration hardware). After successful verification, the hb_mapper makertbin tool is used to quantize the model, converting the model weights from floating-point to 8-bit integers to reduce computational complexity and storage resource consumption while maintaining model accuracy as much as possible. Finally, a deployment-ready .bin format model is generated.

[0035] Step 2.2: Deploy the model on an edge computing platform.

[0036] After model training and quantization optimization, this invention deploys the model on the TogetherROS operating system based on the Horizon Robotics platform. TogetherROS is an embedded robot operating system independently developed by Horizon Robotics, which integrates the Hobot DNN inference framework.

[0037] Using the Hobot DNN inference engine, the quantized model can be quickly loaded onto the RDK X3 edge computing device, ensuring both inference speed and accuracy even with limited computing resources. The process of inferring an image is as follows: Figure 2 As shown, the system utilizes the hardware and software collaboration interface provided by TogetherROS, combined with a real-time image acquisition module from a camera, to achieve a complete process from image input and model inference to target detection result output. Finally, through deployment and debugging in the TogetherROS operating system, edge detection of the stacked object was achieved, ensuring the stable operation of the edge device in complex environments and providing a reliable data foundation and real-time support for subsequent data uploading and visualization management.

[0038] Step 2.3: Privacy Protection.

[0039] Before uploading images of accumulated debris to the Alibaba Cloud IoT platform, the system performs local preprocessing on the collected images. To effectively prevent the risk of personal privacy leaks, the system automatically identifies human targets in the images. Once a human body is detected in the image, the system will blur that area to conceal human features, effectively reducing the privacy risks that travelers may face during image collection. This processing not only improves the security of the system deployed in public environments but also enhances users' acceptance and trust in the intelligent monitoring system, laying the foundation for its future application in more complex scenarios.

[0040] Step 3: The APP displays the recognition results.

[0041] Step 3.1 The edge device data is sent to the Alibaba Cloud IoT platform.

[0042] In this step, when the edge computing device detects accumulated debris in the corridor, it uploads the data to the Alibaba Cloud IoT platform. To achieve remote data transmission, the system connects the edge device to a 4G DTU module via a serial interface. The 4G DTU module is a smart communication device that converts serial data into data for cellular network transmission. It offers advantages such as wide-area coverage, stable connection, and easy deployment, making it suitable for IoT systems in non-Wi-Fi, weak signal, or complex network environments. In practical applications, the edge device acts as an MQTT client, publishing the detection data to a specified topic. The Alibaba Cloud IoT platform receives this data and parses, forwards, or stores it according to a pre-configured rule engine. This provides the foundation for the subsequent display of images of the accumulated debris in a mobile app.

[0043] Step 3.2 Configure Alibaba Cloud data forwarding.

[0044] In this step, the edge device uploads the identified pile images and related information to the Alibaba Cloud IoT platform via the network. The platform, through a configured rules engine, automatically encapsulates the received image information into messages and forwards them to a designated topic via the MQTT protocol. After subscribing to this topic, the mobile app can receive image pushes in real time, thus achieving rapid synchronization and display of the recognition results. This data transmission process can be found in [reference needed]. Figure 3 The system structure diagram is shown below.

[0045] Step 3.3 The APP displays the recognition results.

[0046] The mobile app maintains a continuous connection with the Alibaba Cloud IoT platform via the MQTT protocol, enabling real-time reception of recognition results. To address the limitation of single MQTT message size for image data, the system employs a packet-splitting and receiver-end reassembly mechanism, splitting the complete image data into multiple smaller data packets for sequential transmission. This packet-splitting and reassembly mechanism improves the stability and integrity of large data transmissions in weak network environments.

[0047] During the mobile terminal's data reception process, the app numbers and caches each data packet, and automatically reassembles and decodes the images after all data has been received. This mechanism effectively ensures the integrity and reliability of image data during transmission, enabling administrators to accurately and promptly view and identify scene images on their mobile terminals, thereby improving event response efficiency and the accuracy of handling.

[0048] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0049] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A stacking detection system based on edge computing, characterized in that, Includes the following steps: The server-side model training module is used to train the YOLOv5 model using a dataset of images of debris in stairwells to obtain a debris detection model. The model conversion module is used to convert the stacking detection model into a format that is compatible with edge computing devices. An edge computing terminal is equipped with the TogetherROS operating system and the Hobot DNN inference framework, which is used to load and run the format model and perform real-time detection on the collected corridor images through the format model. The communication module includes a 4G DTU module connected via a serial port, which is used to send the detection results to the mobile terminal via the MQTT protocol.

2. The edge computing-based material detection system as described in claim 1, characterized in that, The training of the YOLOv5 model using a dataset of images of debris piled up in stairwells specifically includes the following steps: Images of debris piled up in stairwells were collected, and multiple images of debris piled up in stairwells were cleaned, labeled, and data augmented to obtain a dataset of debris piled up images; The YOLOv5 model was trained using a dataset of piled-up images to obtain a piled-up detection model.

3. The edge computing-based material detection system as described in claim 1, characterized in that, The format conversion of the accumulation detection model specifically includes the following steps: Convert the PyTorch format model of the accumulation detection model to ONNX format; The toolset is used to quantize and convert the ONNX format stacking detection model to generate a .bin format model adapted for edge computing devices.

4. The edge computing-based material detection system as described in claim 1, characterized in that, Before sending the detection results to the mobile device via the MQTT protocol, the detected human body is censored.

5. The edge computing-based material detection system as described in claim 1, characterized in that, Also includes: The cloud forwarding module, based on the rule engine in the Alibaba Cloud IoT platform, forwards the detection results to the mobile terminal through the message service.

Images