A wildlife inspection system based on YOLO v5 and edge computing

Abstract

The present invention provides a wildlife inspection system based on YOLO v5 and edge computing, which belongs to the field of target detection technology. A multi-category wildlife database is used to train a target detection model. The wildlife target detection algorithm model is embedded in the edge end through a Docker container. The front-end camera inspects 24 hours a day at a rate of 50fps. Multi-channel cameras transmit video streams to the edge computing box through an ad hoc network. After the edge end detects the designated target, the first frame image, the target video file, and the recognition result are transmitted back to the server end in the form of WIFI transmission. The system visualizes the transmitted result to the middle end. A ferocious beast database is preset. When the edge end recognizes a ferocious beast, the village smart loudspeaker is triggered to broadcast warning information and the location of the ferocious beast in real time to avoid the occurrence of beast-injuring incidents.

Description

Technical Field

[0001] The present invention relates to technical fields such as edge computing, computer vision, target detection, deep convolutional neural network algorithms, and in particular to a wildlife inspection system based on YOLO v5 and edge computing. Background Art

[0002] (1) Thanks to a series of measures in recent years, such as in situ and ex situ conservation, implementation of major ecological projects, and improvement of policies and regulations, my country has achieved remarkable results in biodiversity conservation. The living environment of wild animals has been greatly improved, and some rare and precious animals have been gradually discovered;

[0003] (2) Due to historical reasons, many villages in my country are located near natural forests and suitable forest land. Ferocious wild animals often sneak into villages and cause casualties. Human-animal conflicts continue to escalate. At present, the safety of villages is mainly ensured by mobile patrols, but safety and timeliness are difficult to guarantee.

[0004] (3) In recent years, edge computing and computer vision technologies have gradually matured. They have the advantages of low cost, high efficiency, strong linkage, and contactless monitoring. However, the recognition effect of incomplete images of wild animals and blurred images at night is not ideal, and the processing method is mostly a front-end device-server processing mode, which has high requirements for network transmission in mountainous and remote areas and is difficult to achieve. The feedback timeliness of the recognition results is poor. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a wildlife inspection system based on YOLO v5 and edge computing, which broadcasts warning information and the location of ferocious beasts in real time to avoid incidents of beasts injuring people.

[0006] The technical solution of the present invention is:

[0007] A wildlife inspection system based on YOLO v5 and edge computing, including a fixed camera, a track, a servo motor, an inspection camera, an edge computing box, a server, an intelligent speaker, and a visualization platform.

[0008] The fixed position camera is arranged at the entrance and exit of the village, continuously collects real-time images, and transmits them back to the edge computing box through the transmission mode of the self-organizing network; the track is laid to cover the edge of the village, and the inspection position camera moves at a uniform speed based on the power of the servo motor. When an obstacle is detected in front, a stop command is issued to the servo motor, and a fault log is sent to the server. When the obstacle in front is cleared, an output power command is issued. The inspection position camera collects real-time images at a rate of 50fps and transmits them back to the edge computing box through the transmission mode of the self-organizing network; the edge computing box comprehensively analyzes and identifies the video streams returned by the fixed position camera and the inspection position camera. After detecting the specified target, the first frame image, the target video file, and the recognition result are transmitted back to the server in the form of WIFI transmission; at the same time, when the edge computing box detects a ferocious beast, the warning information is transmitted back to the smart speaker through the transmission mode of the self-organizing network to remind nearby villagers to be vigilant against the beast; the visualization platform is responsible for displaying the image data and warning information returned by the edge end, so that managers can comprehensively analyze the distribution of biodiversity and timely understand the distribution of nearby beasts for subsequent management;

[0009] Furthermore,

[0010] The edge computing box processes video stream data including the following steps:

[0011] S1. Regularize the ImageNet dataset and early camera data collection, and design the model basic dataset in a one-to-one format of jpg-xml;

[0012] S2. Use the random() function in Python to randomly divide the basic data set into a training set and a validation set in a ratio of 8:2. The training set is used for wildlife feature extraction, and the validation set is used for optimal judgment.

[0013] S3, YOLO v5 deep convolutional neural network training: Comprehensive target detection accuracy and edge computing box computing power, based on YOLO v5 designed deep convolutional neural network to train and verify the basic data set, monitor algorithm loss and fitting degree, when tending to the fitting state, the model training is terminated, and the optimal weight file is output for subsequent algorithm reasoning;

[0014] S4. Video stream inference algorithm design: Use multi-threading to implement video stream target detection inference, deploy data push thread Thread-LoadDataSet, which is responsible for accepting and caching video frames, deploy algorithm inference thread Thread-Detect, perform target detection based on the video frames pushed by Thread-LoadDataSet thread, deploy data storage thread Thread-Storage, store the images, videos, and recognition results recognized by Thread-Detect, and transmit the recognition results back to the server;

[0015] S5. Edge computing box interface deployment: Design the Fastapi interface based on the optimal weight and video stream inference algorithm, package it into the container through docker-compose, infer and analyze the video streams sent back by fixed-position cameras and inspection-position cameras in self-organizing network transmission, communicate with the remote server in WIFI transmission, and communicate with the smart broadcast in self-organizing network transmission.

[0016] Preferably, the early camera acquisition data in step S1 includes infrared images and incomplete images of specific wild animals taken during the day and at night, which are used to improve the robustness of subsequent models.

[0017] Preferably, the deep convolutional neural network data preprocessing part in step S3 specifically includes Mosaic data enhancement, adaptive anchor frame calculation, and adaptive image scaling.

[0018] Preferably, the position loss function CIOU_Loss in the deep convolutional neural network model based on YOLO v5 in step S3 is specifically defined as:

[0019]

[0020] Where IOU represents the intersection-over-union ratio between the predicted box and the true box, Distance_2 represents the Euclidean distance between the predicted box and the center point of the true box, Distance_C represents the diagonal distance between the predicted box and the minimum circumscribed rectangle of the true box, and v represents the parameter for measuring the consistency of the aspect ratio, which is:

[0021] Preferably, the confidence loss function BCE_Loss in the deep convolutional neural network model based on YOLO v5 in step S3 is specifically defined as:

[0022]

[0023] In the formula, y is the true value and x is the model output value;

[0024] Preferably, in order to avoid the serious imbalance problem of positive and negative samples, the deep convolutional neural network model based on YOLO v5 in step S3 designs a Focal_Loss loss function to reduce the influence of easy-to-classify samples on the loss function, which is specifically defined as:

[0025] Focal_Loss(p t )=-(1-p t ) τ log(p t )

[0026] Preferably, the inference algorithm design described in step S4 can infer the video stream in a multi-process manner if the core computing power allows, and the inference speed will be improved by 50%-100%.

[0027] The beneficial effects of the present invention are

[0028] (1) Fixed-position cameras and patrol-position cameras are installed to fully cover the monitored villages, and low-cost, small-error, high-efficiency, and all-weather monitoring is achieved based on computer vision technology and edge computing boxes;

[0029] (2) A wildlife target detection model based on a deep neural network model uses the ImageNet dataset and field-collected datasets to make the model more generalizable. Undefined wildlife targets will be transmitted to professionals at the edge for identification and feedback of identification results, providing real-time feedback for wildlife protection and biodiversity monitoring.

[0030] (3) A new method for automatically identifying multiple types of wild animals was proposed, and the identification process was less affected by environmental factors;

[0031] (4) The combination of self-organizing network and edge computing box effectively reduces the data transmission cost and greatly improves the monitoring timeliness;

[0032] (5) Design a database of ferocious wild animals. When the system detects a ferocious wild animal, it will issue instructions to the intelligent broadcast through the self-organizing network, which can achieve real-time early warning and avoid economic and human losses caused by wild animals entering the village.

[0033] Computer vision technology can avoid contact with Asian elephants and reduce stress responses. Compared with the manual method of identifying Asian elephants, the method involved in this patent has the characteristics of low cost, high efficiency and small error, and can achieve 24-hour early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a working schematic diagram of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0036] like Figure 1 As shown, the present invention provides a wildlife inspection system based on YOLO v5 and edge computing, which includes a fixed-position camera, a track, a servo motor, an inspection-position camera, an edge computing box, a server, a smart speaker, and a visualization platform.

[0037] in,

[0038] The fixed-position camera is arranged at the entrance and exit of the village, continuously collects real-time images, and transmits them back to the edge computing box through the transmission mode of the self-organizing network; the track is laid to cover the edge of the village, and the inspection position camera moves at a uniform speed based on the power of the servo motor. When an obstacle is detected in front, a stop command is issued to the servo motor, and a fault log is sent to the server. When the obstacle in front is cleared, an output power command is issued. The inspection position camera collects real-time images at a rate of 50fps and transmits them back to the edge computing box through the transmission mode of the self-organizing network; the edge computing box comprehensively analyzes and identifies the video streams returned by the fixed-position camera and the inspection position camera, and after detecting the specified target, the first frame image, the target video file, and the recognition result are transmitted back to the server in the form of WIFI transmission; at the same time, when the edge computing box detects a ferocious beast, the warning information is transmitted back to the smart speaker through the transmission mode of the self-organizing network to remind nearby villagers to be vigilant against the beast; the visualization platform is responsible for displaying the image data and warning information returned by the edge end, so that managers can comprehensively analyze the distribution of biodiversity and timely understand the distribution of nearby beasts for subsequent management.

[0039] The edge computing box processes video stream data including the following steps:

[0040] S1. Regularize the ImageNet dataset and early camera data collection, and design the model basic dataset in a one-to-one format of jpg-xml;

[0041] S2. Use the random() function in Python to randomly divide the basic data set into a training set and a validation set in a ratio of 8:2. The training set is used for wildlife feature extraction, and the validation set is used for optimal judgment.

[0042] S3, YOLO v5 deep convolutional neural network training: Comprehensive target detection accuracy and edge computing box computing power, based on YOLO v5 designed deep convolutional neural network to train and verify the basic data set, monitor algorithm loss and fitting degree, when tending to the fitting state, the model training is terminated, and the optimal weight file is output for subsequent algorithm reasoning;

[0043] S4. Video stream inference algorithm design: Use multi-threading to implement video stream target detection inference, deploy data push thread Thread-LoadDataSet, which is responsible for accepting and caching video frames, deploy algorithm inference thread Thread-Detect, perform target detection based on the video frames pushed by Thread-LoadDataSet thread, deploy data storage thread Thread-Storage, store the images, videos, and recognition results recognized by Thread-Detect, and transmit the recognition results back to the server;

[0044] S5. Edge computing box interface deployment: Design the Fastapi interface based on the optimal weight and video stream inference algorithm, package it into the container through docker-compose, infer and analyze the video streams sent back by fixed-position cameras and inspection-position cameras in self-organizing network transmission, communicate with the remote server in WIFI transmission, and communicate with the smart broadcast in self-organizing network transmission.

[0045] The above description is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A wildlife inspection system based on YOLO v5 and edge computing, characterized in that: Including fixed position cameras, tracks, servo motors, inspection position cameras, edge computing boxes, servers, smart speakers, and visualization platforms. The fixed position camera is arranged at the entrance and exit of the card port, continuously collects real-time images, and transmits them back to the edge computing box through the transmission mode of the self-organizing network; The track is laid to cover the edge of the village. The inspection camera moves at a constant speed based on the power of the servo motor. When an obstacle is detected in front, a stop command is issued to the servo motor, and a fault log is sent to the server. When the obstacle in front is cleared, an output power command is issued. The inspection camera collects real-time images at a rate of 50fps and transmits them back to the edge computing box via a self-organizing network transmission method; The edge computing box comprehensively analyzes and identifies the video streams sent back by the fixed position camera and the inspection position camera, and after detecting the designated target, sends the first frame image, the target video file, and the recognition result back to the server in the form of WIFI transmission; At the same time, when the edge computing box detects a ferocious beast, it transmits the warning information back to the smart speaker in a self-organizing network transmission mode to warn the beast; The visualization platform is responsible for displaying the image data and warning information sent back by the edge end, so that managers can comprehensively analyze the distribution of biodiversity and timely understand the distribution of nearby wild animals for subsequent management; The edge computing box processes video stream data including the following steps: S1. Regularize the ImageNet dataset and early camera data collection, and design the model basic dataset in a one-to-one format of jpg-xml; S2. Use the random() function in Python to randomly divide the basic data set into a training set and a validation set in a ratio of 8:

2. The training set is used for wildlife feature extraction, and the validation set is used for optimal judgment. S3, YOLO v5 deep convolutional neural network training: Comprehensive target detection accuracy and edge computing box computing power, based on YOLO v5 designed deep convolutional neural network to train and verify the basic data set, monitor algorithm loss and fitting degree, when tending to the fitting state, the model training is terminated, and the optimal weight file is output for subsequent algorithm reasoning; S4. Video stream inference algorithm design: Use multi-threading to implement video stream target detection inference, deploy data push thread Thread-LoadDataSet, which is responsible for accepting and caching video frames, deploy algorithm inference thread Thread-Detect, perform target detection based on the video frames pushed by Thread-LoadDataSet thread, deploy data storage thread Thread-Storage, store the images, videos, and recognition results recognized by Thread-Detect, and transmit the recognition results back to the server; S5. Edge computing box interface deployment: Design the Fastapi interface based on the optimal weight and video stream inference algorithm, package it into the container through docker-compose, infer and analyze the video streams sent back by fixed-position cameras and inspection-position cameras in self-organizing network transmission, communicate with the remote server in WIFI transmission, and communicate with the smart broadcast in self-organizing network transmission.

2. The system according to claim 1, characterized in that A multi-category wildlife database is used to train the target detection model. The wildlife target detection algorithm model is embedded in the edge through a Docker container. The front-end camera conducts 24-hour uninterrupted inspections at a rate of 50fps. Multiple cameras transmit video streams to the edge computing box through an ad hoc network. After the edge detects the specified target, the first frame image, the target video file, and the recognition result are transmitted back to the server in the form of WIFI transmission. The system visualizes the returned results to the middle platform. A database of ferocious beasts is preset. When the edge recognizes a ferocious beast, the smart speaker is triggered to broadcast warning information and the location of the ferocious beast in real time to avoid incidents of beasts injuring people.

3. The system according to claim 1, characterized in that The early camera data collected in step S1 includes infrared images and incomplete images of wild animals taken during the day and at night, which are used to improve the robustness of subsequent models.

4. The system according to claim 1, characterized in that The deep convolutional neural network data preprocessing part described in step S3 specifically includes Mosaic data enhancement, adaptive anchor frame calculation, and adaptive image scaling.

5. The system according to claim 1, wherein: The position loss function CIOU_Loss in the deep convolutional neural network model based on YOLO v5 described in step S3 is specifically defined as: Where IOU represents the intersection-over-union ratio between the predicted box and the true box, Distance_2 represents the Euclidean distance between the predicted box and the center point of the true box, Distance_C represents the diagonal distance between the predicted box and the minimum circumscribed rectangle of the true box, and v represents the parameter for measuring the consistency of the aspect ratio, which is:

6. The system according to claim 1, wherein: The confidence loss function BCE_Loss in the deep convolutional neural network model based on YOLO v5 described in step S3 is specifically defined as: In the formula, y is the true value and x is the model output value.

7. The system according to claim 1, characterized in that In order to avoid the serious imbalance problem of positive and negative samples, the deep convolutional neural network model based on YOLO v5 in step S3 designs the Focal_Loss loss function to reduce the influence of easy-to-classify samples on the loss function, which is specifically defined as:

8. The system of claim 1, wherein: The inference algorithm described in step S4 is designed to infer the video stream in a multi-process manner if the core computing power allows.

Images