Edge type antiriot detection method and system based on video and sound

Abstract

The invention relates to the technical field of edge computing, in particular to an edge type anti-riot detection method and system based on videos and voices.The method comprises the following steps that a sensing camera is adopted as a detection tool, and 24-hour 360-degree dead-corner-free monitoring is achieved in a network-free environment; the sensing camera has the function of sensing and monitoring behaviors and dialogues of sounds and videos, the monitoring content is analyzed through a built-in system, and when abnormal behaviors are found, quick response is made in the system behaviors; the beneficial effect is that the anti-riot detection of campus, society and crowd designated places is realized. The frame-by-frame detection of the video stream is realized through artificial intelligence, and the characteristics of quick response, quick storage and the like are realized in the artificial intelligence deployment edge equipment. And information does not need to be uploaded, so that privacy security is achieved. In addition, the terminal equipment is provided with microphone sound detection to improve the detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of edge computing technology, and in particular to an edge-type riot control detection method and system based on video and sound. Background Art

[0002] It is necessary to propose a video and sound-based edge riot detection method and system. Summary of the Invention

[0003] The object of the present invention is to provide a video and sound-based edge-type riot control detection method and system to solve the problems raised in the above background technology.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a video and sound-based edge-type riot control detection method, comprising the following steps:

[0005] Using perception cameras as detection tools, 24-hour 360-degree surveillance without blind spots is achieved in a network-free environment. The perception camera has the function of behavioral and conversation perception monitoring of sound and video, analyzes the monitoring content through the built-in system, and quickly responds in system behavior when abnormal behavior is found. At the same time, a customized GUI interface is provided, through which users can change event levels, set contact methods, set alarm phone numbers, perform performance tuning, enhance control, set behavior expectations, and perform permission encryption operations in the system.

[0006] Preferably, the following steps are also included: the user interaction GUI interface of the perception camera follows the principles of simplicity, ease of operation, and ease of use. The system comes with default rules and can be used directly after deployment; users use this interface to adjust the camera's distance, switch night vision mode, control sound, and perform zoom operations; at the same time, the system implements authority management for personnel, and encrypts, backs up, and locally stores monitoring data.

[0007] Preferably, it also includes: in terms of hardware architecture, adopting a new heterogeneous computing architecture, integrating a graphics processing unit with powerful parallel computing capabilities and a low-power, high-performance central processing unit into the edge device; at the same time, it is equipped with a high-speed cache and a large-capacity random access memory for fast storage and reading of image data and intermediate calculation results, reducing data reading and writing delays to meet the needs of real-time monitoring and fast processing.

[0008] Preferably, it also includes: in algorithm application, optimizing and streamlining the traditional deep learning image recognition algorithm; removing redundant connections and neurons in the network through pruning technology to reduce model complexity; using quantization technology to convert model parameters from high-precision data types to low-precision data types, while ensuring that the recognition accuracy is not significantly reduced, reducing the model storage capacity and computational complexity, so that it can run quickly on the limited computing resources of edge devices, and realize rapid identification of abnormal behavior in surveillance videos and sounds.

[0009] Preferably, it also includes an adaptive task scheduling mechanism, designs an adaptive task scheduling algorithm, and dynamically allocates computing resources according to the current load of the edge device and the priority of the image task; when the device load is low, high-priority tasks are given priority; when the load is too high, low-priority tasks are appropriately delayed or downgraded to ensure the overall real-time and stability of the system; at the same time, an image preprocessing module is integrated on the edge device to perform denoising, grayscale, and contrast enhancement operations on the collected original image, and adopts an artificial intelligence-based image enhancement algorithm to automatically adjust the enhancement parameters according to the image content, highlight the key features in the image, and improve the accuracy of subsequent recognition algorithms; in addition, the system feeds back the image recognition results to the user or other related devices in real time. When the recognition result is wrong or the new image sample features do not match the existing model, the edge device automatically collects new data and uploads the data to the cloud for model update training through secure communication with the cloud server. The updated model is then downloaded to the edge device to achieve dynamic optimization of the model and continuously improve the recognition accuracy.

[0010] A system for edge-type riot detection methods based on video and sound. The system uses perception cameras as the core detection tool to achieve 24-hour 360-degree monitoring without blind spots in a network-free environment. The perception cameras have the function of behavioral and conversational perception monitoring of sound and video, and perform real-time analysis of the monitored content through the built-in system. When abnormal behavior is detected, the system behavior will respond quickly. At the same time, the system provides a customized GUI interface, through which users can change event levels, set contact methods, set alarm phone numbers, perform performance tuning, enhance control, set behavior expectations, and perform permission encryption operations in the system to meet the riot detection needs in different scenarios.

[0011] Preferably, the user interaction GUI interface design of the perception camera follows the principles of simplicity, ease of operation and ease of use. The system comes with default rules and can be put into use directly after deployment. Users can use this interface to adjust the camera's distance, switch night vision mode, control sound and zoom to adapt to different monitoring environments. In addition, the system implements strict authority management on personnel to ensure the security of monitoring data, and encrypts, backs up and locally stores monitoring data to prevent data loss and leakage.

[0012] Preferably, the system adopts a hardware architecture of a new heterogeneous computing architecture, combining a graphics processing unit with powerful parallel computing capabilities with a low-power, high-performance central processing unit, and integrating them into edge devices; at the same time, it is equipped with a high-speed cache and a large-capacity random access memory for fast storage and reading of image data and intermediate calculation results, effectively reducing data reading and writing delays, and ensuring the efficiency and stability of the system during real-time monitoring and rapid processing.

[0013] Preferably, in terms of algorithm application, the system optimizes and streamlines the traditional deep learning image recognition algorithm; removes redundant connections and neurons in the network through pruning technology to reduce model complexity; uses quantization technology to convert model parameters from high-precision data types to low-precision data types, and greatly reduces the model storage capacity and computational complexity while ensuring that the recognition accuracy is not significantly reduced, so that it can run quickly on the limited computing resources of edge devices, thereby achieving rapid and accurate recognition of abnormal behaviors in surveillance videos and sounds.

[0014] Preferably, the system has an adaptive task scheduling mechanism and designs an adaptive task scheduling algorithm to dynamically allocate computing resources according to the current load of the edge device and the priority of the image task; when the device load is low, high-priority tasks are prioritized to ensure that emergencies are responded to in a timely manner; when the load is too high, low-priority tasks are appropriately delayed or downgraded to ensure the overall real-time and stability of the system; at the same time, an image preprocessing module is integrated on the edge device to perform denoising, grayscale, and contrast enhancement operations on the collected original image, and an artificial intelligence-based image enhancement algorithm is used to automatically adjust the enhancement parameters according to the image content, highlight the key features in the image, and improve the accuracy of subsequent recognition algorithms; in addition, the system can feed back the image recognition results to the user or other related devices in real time. When the recognition results are erroneous or the new image sample features do not match the existing model, the edge device automatically collects these new data and uploads the data to the cloud for model update training through secure communication with the cloud server. The updated model is then downloaded to the edge device to achieve dynamic optimization of the model and continuously improve the recognition accuracy to adapt to the ever-changing needs of riot detection.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] The video- and sound-based edge-based riot detection method and system proposed in this paper enable riot detection in designated locations, such as on campuses, in public settings, and among crowds. Using artificial intelligence (AI), frame-by-frame detection of video streams is achieved. Deploying AI in edge devices offers fast response and storage. Privacy is ensured by eliminating the need to upload information. Furthermore, microphone-based sound detection in terminal devices enhances detection accuracy.

[0017] The system has built-in facial recognition. When the device detects a violation, the video stream is converted into multiple images based on the time of day. The system then identifies the individuals in the images. After identification, the images are annotated in the video and images. The annotated information is then sent to the administrator and saved locally.

[0018] The present invention also provides a user-friendly interactive interface, and the system realizes local deployment. Information is stored locally using encryption, verification, anti-tampering and other security measures to ensure data security, consistency and permanence. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Flow chart of the method of the present invention;

[0020] Figure 2 This is the face entry flow chart of the present invention. DETAILED DESCRIPTION

[0021] In order to clearly and completely describe the objectives and technical solutions of the present invention and make the advantages more clearly understood, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, not all of them, and are only used to explain the embodiments of the present invention, not to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0022] For example 1, please refer to Figures 1 to 2 The present invention provides a technical solution: a video and sound-based edge-type riot detection method, comprising the following steps:

[0023] 1. General Introduction

[0024] The final version of this tool will be a perception camera, enabling 24 / 7 360-degree surveillance without a network connection. It monitors audio and video for behavior and conversations, enabling rapid system response if abnormal behavior is detected. This tool offers a customizable GUI interface. It allows users to modify event levels, contact information, alarm phone numbers, and perform performance tuning, control enhancements, behavior predictions, and permission encryption.

[0025] 2. User Interaction GUI

[0026] This interface is designed for simplicity, ease of use, and ease of use. The system comes with pre-installed default rules and is ready for immediate use. The system provides controls for camera range, night vision, sound, and zoom. It also manages personnel permissions and provides data encryption, backup, and local storage.

[0027] 3. Hardware architecture optimization

[0028] This technology utilizes a novel heterogeneous computing architecture, combining a powerful parallel computing graphics processing unit (GPU) with a low-power, high-performance central processing unit (CPU) and integrating them into edge devices. It also features high-speed cache and large-capacity random access memory (RAM) for rapid storage and access of image data and intermediate computational results, reducing data read and write latency.

[0029] 4. Lightweight AI algorithm

[0030] Optimize and streamline traditional deep learning image recognition algorithms (such as convolutional neural networks (CNNs)). Pruning techniques are used to remove redundant connections and neurons in the network, reducing model complexity. Quantization techniques are used to convert model parameters from high-precision data types to low-precision data types. This reduces model storage capacity and computational complexity without significantly reducing recognition accuracy, enabling it to run quickly on the limited computing resources of edge devices.

[0031] 5. Adaptive task scheduling

[0032] Design an adaptive task scheduling algorithm that dynamically allocates computing resources based on the edge device's current load (including CPU usage, GPU usage, and memory usage) and the priority of image tasks (for example, image recognition tasks for emergency security events take precedence over general environmental monitoring tasks). When the device load is low, high-priority tasks are prioritized; when the load is high, low-priority tasks are appropriately delayed or downgraded to ensure the real-time and stability of the overall system.

[0033] 6. Image preprocessing and enhancement

[0034] An image preprocessing module is integrated into the edge device to perform operations such as denoising, grayscale conversion, and contrast enhancement on the collected raw images. An AI-based image enhancement algorithm is used to automatically adjust enhancement parameters based on image content, highlighting key features in the image and improving the accuracy of subsequent recognition algorithms.

[0035] 7. Real-time feedback and updates

[0036] The system can provide real-time feedback on image recognition results to users or other related devices. Furthermore, when recognition errors occur or new image sample features don't match the existing model, the edge device automatically collects this new data and, through secure communication with the cloud server, uploads it to the cloud for model update training. The updated model is then downloaded to the edge device, enabling dynamic model optimization and continuously improving recognition accuracy.

[0037] 8. Face Recognition Library

[0038] Data collection can be performed in a variety of ways, such as using a camera to capture facial images in various environments, including indoors and outdoors, and under varying lighting conditions. Alternatively, facial frames can be extracted from video surveillance footage, or user facial data can be collected using a mobile device's camera. During the collection process, consideration should be given to factors such as the device's resolution, shooting angle, and lighting conditions to ensure sufficient diversity and quality of the collected data.

[0039] To train face recognition models, collected facial images need to be annotated. This annotation typically includes facial identifiers, facial position and pose information, and expression tags. Identifiers are used to distinguish individuals and are the core annotation information for face recognition tasks. Facial position and pose information can help the model better locate and align faces, improving recognition accuracy.

[0040] To ensure the quality and performance of the face recognition library, data must be regularly cleaned and optimized. Cleaning operations include removing duplicate data, erroneous data, and data of poor quality. Optimization operations include data compression and index optimization to improve data storage efficiency and access speed.

[0041] Facial recognition databases contain a large amount of personal facial data, which involves user privacy. Therefore, data security and privacy protection are key issues in facial recognition database technology. Encryption technology is required to protect data during storage and transmission to prevent theft or tampering. Furthermore, data usage must comply with relevant laws, regulations, and privacy policies to ensure user privacy is protected.

[0042] Example 2, based on Example 1, proposes a system for edge-type riot detection methods based on video and sound. The system uses perception cameras as the core detection tool to achieve 24-hour 360-degree no-blind-angle monitoring in a network-free environment; the perception camera has the function of behavioral and conversation perception monitoring of sound and video, and performs real-time analysis of the monitored content through the built-in system. When abnormal behavior is detected, it quickly responds in the system behavior; at the same time, the system provides a customized GUI interface, through which users can change event levels, set contact methods, set alarm phones, perform performance tuning, enhance control, set behavior expectations, and perform permission encryption operations in the system to meet the riot detection needs in different scenarios.

[0043] The user interaction GUI interface of the perception camera is designed to be simple, easy to operate, and user-friendly. The system comes with default rules and can be put into use directly after deployment. Users can use this interface to adjust the camera's perspective, switch night vision mode, control sound, and perform zoom operations to adapt to different monitoring environments. In addition, the system implements strict personnel authority management to ensure the security of monitoring data, and encrypts, backs up, and locally stores monitoring data to prevent data loss and leakage.

[0044] The system adopts a new hardware architecture of heterogeneous computing architecture, combining a graphics processing unit with powerful parallel computing capabilities with a low-power, high-performance central processing unit, and integrating them into edge devices; at the same time, it is equipped with high-speed cache and large-capacity random access memory for fast storage and reading of image data and intermediate calculation results, effectively reducing data reading and writing delays, and ensuring the efficiency and stability of the system during real-time monitoring and rapid processing.

[0045] In terms of algorithm application, the system optimizes and streamlines traditional deep learning image recognition algorithms; uses pruning technology to remove redundant connections and neurons in the network to reduce model complexity; and uses quantization technology to convert model parameters from high-precision data types to low-precision data types. While ensuring that the recognition accuracy is not significantly reduced, the model storage capacity and computational complexity are greatly reduced, allowing it to run quickly on the limited computing resources of edge devices, thereby achieving rapid and accurate identification of abnormal behaviors in surveillance videos and sounds.

[0046] The system has an adaptive task scheduling mechanism and an adaptive task scheduling algorithm designed to dynamically allocate computing resources based on the current load of the edge device and the priority of the image task. When the device load is low, high-priority tasks are prioritized to ensure timely response to emergencies. When the load is too high, low-priority tasks are appropriately delayed or downgraded to ensure the overall real-time and stability of the system. At the same time, an image preprocessing module is integrated on the edge device to perform denoising, grayscale, and contrast enhancement operations on the collected original images, and an artificial intelligence-based image enhancement algorithm is used to automatically adjust the enhancement parameters according to the image content, highlight the key features in the image, and improve the accuracy of subsequent recognition algorithms. In addition, the system can feedback the image recognition results to users or other related devices in real time. When the recognition results are incorrect or the new image sample features do not match the existing model, the edge device automatically collects the new data and uploads the data to the cloud for model update training through secure communication with the cloud server. The updated model is then downloaded to the edge device to achieve dynamic optimization of the model and continuously improve the recognition accuracy to adapt to the ever-changing needs of riot detection.

[0047] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A video and sound-based edge riot detection method, characterized by: The following steps are involved: Using perception cameras as detection tools, 24-hour 360-degree monitoring without blind spots is possible in an offline environment. This perception camera has the function of behavioral and conversation perception monitoring of sound and video, analyzes the monitoring content through the built-in system, and quickly responds in system behavior when abnormal behavior is detected; at the same time, it provides a customized GUI interface, through which users can change event levels, set contact methods, set alarm phone numbers, perform performance tuning, enhance control, set behavior expectations, and perform permission encryption operations in the system.

2. The edge-type riot control detection method based on video and sound according to claim 1, characterized in that: The following steps are also included: The user interaction GUI interface of the perception camera follows the principles of simplicity, ease of operation, and ease of use. The system comes with default rules and can be used directly after deployment. Users can use this interface to adjust the camera's perspective, switch night vision mode, control sound, and operate zoom. At the same time, the system implements personnel authority management and encrypts, backs up, and locally stores monitoring data.

3. The edge-type riot control detection method based on video and sound according to claim 2, characterized in that: Also includes: In terms of hardware architecture, a new heterogeneous computing architecture is adopted to integrate a graphics processing unit with powerful parallel computing capabilities and a low-power, high-performance central processing unit into the edge device; at the same time, it is equipped with a high-speed cache and a large-capacity random access memory for fast storage and reading of image data and intermediate calculation results, reducing data reading and writing delays to meet the needs of real-time monitoring and fast processing.

4. The edge-type riot control detection method based on video and sound according to claim 3, characterized in that: Also includes: In terms of algorithm application, the traditional deep learning image recognition algorithm is optimized and streamlined; Pruning technology is used to remove redundant connections and neurons in the network to reduce model complexity; quantization technology is used to convert model parameters from high-precision data types to low-precision data types. While ensuring that the recognition accuracy is not significantly reduced, the model storage capacity and computational complexity are reduced, allowing it to run quickly on the limited computing resources of edge devices, thereby achieving rapid identification of abnormal behavior in surveillance videos and sounds.

5. The edge-type riot control detection method based on video and sound according to claim 4, characterized in that: It also includes an adaptive task scheduling mechanism, designs an adaptive task scheduling algorithm, and dynamically allocates computing resources according to the current load of the edge device and the priority of the image task; when the device load is low, high-priority tasks are prioritized; when the load is too high, low-priority tasks are appropriately delayed or downgraded to ensure the overall real-time and stability of the system; at the same time, an image preprocessing module is integrated on the edge device to perform denoising, grayscale, and contrast enhancement operations on the collected original image, and adopts an artificial intelligence-based image enhancement algorithm to automatically adjust the enhancement parameters according to the image content, highlight the key features in the image, and improve the accuracy of subsequent recognition algorithms; in addition, the system feeds back the image recognition results to the user or other related devices in real time. When the recognition result is wrong or the new image sample features do not match the existing model, the edge device automatically collects new data and uploads the data to the cloud for model update training through secure communication with the cloud server. The updated model is then downloaded to the edge device to achieve dynamic optimization of the model and continuously improve the recognition accuracy.

6. A system for the edge-type riot control detection method based on video and sound according to claim 5, characterized in that: The system uses perception cameras as its core detection tool, achieving 24-hour 360-degree surveillance without blind spots in a network-free environment. The perception cameras have the function of behavioral and conversational perception monitoring of sound and video, and conduct real-time analysis of monitoring content through the built-in system. When abnormal behavior is detected, the system will quickly respond in terms of system behavior. At the same time, the system provides a customized GUI interface, through which users can change event levels, set contact methods, set alarm phone numbers, perform performance tuning, enhance control, set behavior expectations, and perform permission encryption operations in the system to meet the riot detection needs in different scenarios.

7. A system according to claim 6, characterized in that: The user interaction GUI interface of the perception camera is designed to be simple, easy to operate, and user-friendly. The system comes with default rules and can be put into use directly after deployment. Users can use this interface to adjust the camera's perspective, switch night vision mode, control sound, and perform zoom operations to adapt to different monitoring environments. In addition, the system implements strict personnel authority management to ensure the security of monitoring data, and encrypts, backs up, and locally stores monitoring data to prevent data loss and leakage.

8. A system according to claim 7, characterized in that: The system adopts a new hardware architecture of heterogeneous computing architecture, combining a graphics processing unit with powerful parallel computing capabilities with a low-power, high-performance central processing unit, and integrating them into edge devices; at the same time, it is equipped with high-speed cache and large-capacity random access memory for fast storage and reading of image data and intermediate calculation results, effectively reducing data reading and writing delays, and ensuring the efficiency and stability of the system during real-time monitoring and rapid processing.

9. A system according to claim 8, characterized in that: In terms of algorithm application, the system optimizes and streamlines traditional deep learning image recognition algorithms; uses pruning technology to remove redundant connections and neurons in the network to reduce model complexity; and uses quantization technology to convert model parameters from high-precision data types to low-precision data types. While ensuring that the recognition accuracy is not significantly reduced, the model storage capacity and computational complexity are greatly reduced, allowing it to run quickly on the limited computing resources of edge devices, thereby achieving rapid and accurate identification of abnormal behaviors in surveillance videos and sounds.

10. A system according to claim 9, characterized in that: The system has an adaptive task scheduling mechanism and an adaptive task scheduling algorithm designed to dynamically allocate computing resources based on the current load of the edge device and the priority of the image task. When the device load is low, high-priority tasks are prioritized to ensure timely response to emergencies. When the load is too high, low-priority tasks are appropriately delayed or downgraded to ensure the overall real-time and stability of the system. At the same time, an image preprocessing module is integrated on the edge device to perform denoising, grayscale, and contrast enhancement operations on the collected original images, and an artificial intelligence-based image enhancement algorithm is used to automatically adjust the enhancement parameters according to the image content, highlight the key features in the image, and improve the accuracy of subsequent recognition algorithms. In addition, the system can feedback the image recognition results to users or other related devices in real time. When the recognition results are incorrect or the new image sample features do not match the existing model, the edge device automatically collects the new data and uploads the data to the cloud for model update training through secure communication with the cloud server. The updated model is then downloaded to the edge device to achieve dynamic optimization of the model and continuously improve the recognition accuracy to adapt to the ever-changing needs of riot detection.

Images