An electronic fence early warning method based on target detection and target tracking

Abstract

The application discloses an electronic fence early warning method based on target detection and target tracking, which uses a YOLOX target detection network to detect video stream data returned by a field camera frame by frame in real time, detects the existence and state of a target object, assigns an id to a target behavior track by a DeepSORT tracking algorithm, and issues an alarm once an id owner approaches the fence. The application has the beneficial effects that the application acquires images in a monitoring area through a camera or other image acquisition equipment, processes and analyzes the images by using computer vision technology, identifies the motion track of an object or a person, and thus realizes safety monitoring and preventive measures in the monitoring area, and can be applied to many fields with extremely low cost and extremely high deployment flexibility.

Description

Technical Field

[0001] This invention relates to the field of electronic fences, and in particular to an electronic fence early warning method based on target detection and target tracking. Background Technology

[0002] The history of electronic fence technology can be traced back to the early 1950s, when a US military company used radar technology to establish the first "virtual fence" to protect a US Air Force base. This "virtual fence" protected the base by placing radar equipment around it to monitor moving objects in real time. With the development of electronic and computer technologies, electronic fence technology has been continuously improved and perfected.

[0003] In the 1970s, with the advancement and application of computer technology, electronic fence technology began to enter the commercial field and was applied in areas such as property security and prison monitoring. At this time, electronic fence technology mainly used sensors such as infrared and ultrasonic sensors to detect the movement of objects.

[0004] In the 1990s, with the rise of video surveillance technology, electronic fence technology began to integrate with it, forming video surveillance-based electronic fence technology, also known as image-based electronic fence technology. This technology primarily utilizes video surveillance cameras to capture images of the surrounding environment and uses image analysis and motion detection algorithms to detect and trigger alarms for abnormal behavior. This technology not only improves the efficiency and accuracy of security monitoring but also enables remote monitoring and data storage.

[0005] Traditional electronic fence technology is mostly implemented in hardware, which is difficult to deploy and has high purchase and maintenance costs. Summary of the Invention

[0006] The purpose of this invention is to provide an electronic fence early warning method based on target detection and target tracking. It achieves efficient and accurate early warning of the monitored area with the simplest hardware conditions by using the YOLOX target detection network and the DeepSort target tracking network.

[0007] This invention is achieved through the following technical solution: an electronic fence early warning method based on target detection and target tracking, characterized in that it includes the following steps:

[0008] Step 1: Based on the video shooting scene corresponding to the video capture module, the user draws several straight lines to form a closed area, and the straight lines serve as an electronic fence.

[0009] Step 2: Read the video and send the video frames into the computing module in sequence. Perform object detection through the YOLOX object detection network in the computing module to obtain the detection results Bounding Boxes, and send the detection results Bounding Boxes into the buffer queue.

[0010] Step 3: Determine whether a new target appears in the video frame. If a new target appears, the video frame is returned to Step 2 as the first video frame of the new target and sent to the YOLOX target detection network for target detection.

[0011] Otherwise, the detection result of the current video frame is sent to the DeepSort target tracking network in the calculation module for matching, to find the same target in adjacent frames and assign the same ID to the same target;

[0012] Step 4: Confirm the coordinates of the target's location point from the Bounding Boxes detection results for different targets;

[0013] Step 5: Calculate the axial distance between each target location point and each straight line; take the minimum value as the distance between the target and the fence in the real scene; for the axial distance between the location point coordinates of the i-th target and line j... The formula is as follows;

[0014]

[0015] x i Let y be the horizontal coordinate of target i. i Let k be the vertical coordinate of target i; j Let b be the slope of line j. j This is the offset of line j.

[0016] Step 6: If the distance between the target location point coordinates and the minimum axis of any straight line is greater than or equal to the threshold, check if the target ID is in the buffer. If the target ID is in the buffer, remove the target ID from the buffer.

[0017] Step 7: If the distance between the target location point coordinates and the minimum axis of a certain straight line is less than the threshold, determine whether the target ID is in the buffer. If it is in the buffer, no alarm will be triggered this time.

[0018] If no alarm has been triggered before the given time interval and the target ID is not in the buffer, then the target ID is sent to the buffer and an alarm is issued;

[0019] The buffer is a memory area in which the processing module stores the IDs of the testing personnel;

[0020] Step 8: Determine whether the target's location point coordinates exist in the video frame. If not, end the determination; if so, return to step 5.

[0021] Compared with previous technologies, the beneficial effects of the present invention are as follows:

[0022] This invention acquires images of a monitored area using cameras or other image acquisition devices, and processes and analyzes these images using computer vision technology (YOLOX object detection network and DeepSort object tracking network) to identify the movement trajectories of objects or people. This enables security monitoring and preventative measures within the monitored area, and can be applied to many fields with extremely low cost and high deployment flexibility. Examples include security monitoring and intrusion prevention in public places such as factories, warehouses, parking lots, schools, and parks, as well as cargo tracking and theft prevention in industries such as logistics and warehousing. This technology can improve the efficiency and accuracy of security monitoring, reduce manpower and equipment costs, and better protect the safety of people and property.

[0023] 2. High efficiency and safety: The electronic fence uses advanced image processing technology to accurately locate the area and can issue alarms in a timely manner, ensuring safety and reliability.

[0024] 3. Flexible and controllable: The electronic fence supports flexible settings and modifications. Users can adjust the range, shape, time and other parameters of the electronic fence at any time according to their needs to meet the needs of different scenarios.

[0025] 4. Intelligent Management: Electronic fences can be connected to the Internet, the Internet of Things and other technologies to achieve intelligent management and remote control, which facilitates management and monitoring.

[0026] 5. Low cost and easy installation: Compared with traditional physical fences, electronic fences require no construction, are easy to install, have low cost, and are easy to maintain. This product only requires one camera.

[0027] 6. Applicable to multiple scenarios: Electronic fences can be used in various scenarios such as homes, businesses, schools, and communities, for example, to monitor room entrances and exits, vehicle parking areas, and item storage areas.

[0028] 7. Data visualization: Electronic fences can enable real-time monitoring of people, vehicles, and items within the fence, and provide more intelligent and efficient management methods through data analysis and visualization. Attached Figure Description

[0029] Figure 1 This is the overall flowchart of the present invention;

[0030] Figure 2 Flowchart for single video frame determination;

[0031] Figure 3 Schematic diagrams of different types of fences;

[0032] Figure 4 This is a flowchart for determining a single video frame. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings:

[0034] like Figure 1-4 As shown: An electronic fence early warning method based on target detection and target tracking includes the following steps:

[0035] Step 1: Based on the video shooting scene corresponding to the video capture module, the user draws several straight lines to form a closed area, and the straight lines serve as an electronic fence.

[0036] To accommodate applications in non-closed areas, electronic fences are constructed using straight lines rather than fixed closed areas, making them more adaptable to various terrains and conditions. We utilize OpenCV's built-in mouse event handlers to connect the coordinates of adjacent points clicked by the user on the map, forming a straight line – this line constitutes the electronic fence.

[0037] Generally, the enclosed area here is a square area, bounded by horizontal and vertical sides. Considering video distortion and the arbitrariness of line angles, straight lines will be defined here, specifically horizontal and vertical lines.

[0038] Specifically, the slope and intercept of a line determine whether it is a vertical or horizontal line; where, if the slope k of the line belongs to the set... If the slope k of the line belongs to the set, then it is considered a vertical line. If it is, then it is considered a horizontal line.

[0039] In actual use, the straight lines used to form the electronic fence in step 1 are divided into inner straight lines and outer straight lines; the inner straight line is the departure alarm line used to determine the departure of the target; the outer straight line is the entry alarm line used to determine the entry of the target.

[0040] Step 1 also includes a proximity alarm line, which is a virtual line with two lines, inner and outer. The inner proximity alarm line is located inside the departure alarm line and the distance between the two lines is equal to a threshold. The outer proximity alarm line is located outside the entry alarm line and the distance between the two lines is equal to a threshold.

[0041] We offer electronic fence alarm lines with different functions to meet users' diverse needs. See the actual results for details. Figure 3The green line represents the approaching alarm line, the red line represents the leaving alarm line, and the blue line represents the approaching alarm line. When different types of alarm lines are close together, only the alarm line that is first touched will trigger an alarm.

[0042] The video acquisition module is a camera, and the computing module is a computer with embedded YOLOX target detection network and DeepSort target tracking network. The video acquisition module transmits video to the computing module via wired or wireless means.

[0043] Step 2: Read the video and send the video frames into the computing module in sequence. Perform object detection through the YOLOX object detection network in the computing module to obtain the detection results Bounding Boxes, and send the detection results Bounding Boxes into the buffer queue.

[0044] Step 3: Determine whether a new target appears in the video frame. If a new target appears, the video frame is returned to Step 2 as the first video frame of the new target and sent to the YOLOX target detection network for target detection.

[0045] Otherwise, the detection result of the current video frame is sent to the DeepSort target tracking network in the calculation module for matching, to find the same target in adjacent frames and assign the same ID to the same target;

[0046] Step 4: Confirm the target's location coordinates from the Bounding Boxes of different target detection results; To ensure the accuracy of the location, the position of the location coordinates is limited here. The determination of the target's location coordinates in Step 4 is based on the following: Generate detection boxes from the Bounding Boxes of different target detection results, obtain the length and width information and center point coordinates of the detection boxes, calculate the bottom midpoint coordinates of each target detection box, and use the bottom midpoint as the location coordinates;

[0047] The detection box is a rectangle, and the four sides of the rectangle pass through the limit pixels of the target in the four directions of up, down, left, and right.

[0048] Step 5: Calculate the axial distance between each target location point and each straight line; take the minimum value as the distance between the target and the fence in the real scene; for the axial distance between the location point coordinates of the i-th target and line j... The formula is as follows;

[0049]

[0050] x i Let y be the horizontal coordinate of target i. i Let k be the vertical coordinate of target i; j Let b be the slope of line j.j The offset of line j;

[0051] Step 6: If the distance between the target location point coordinates and the minimum axis of any straight line is greater than or equal to the threshold, check if the target ID is in the buffer. If the target ID is in the buffer, remove the target ID from the buffer.

[0052] In step 6, the straight line that needs to be measured from the target is taken as the target straight line. First, determine the type of the target straight line. If the target straight line is a vertical line, calculate the horizontal axis distance; if it is a horizontal line, calculate the vertical axis distance; if it is neither, calculate both the horizontal axis distance and the vertical axis distance. Take the minimum axis distance as the distance between the target and the fence straight line in the real scene.

[0053] Step 7: If the distance between the target location point coordinates and the minimum axis of a certain straight line is less than the threshold, determine whether the target ID is in the buffer. If it is in the buffer, no alarm will be triggered this time.

[0054] If no alarm has been triggered before the given time interval and the target ID is not in the buffer, then the target ID is sent to the buffer and an alarm is issued;

[0055] The buffer is a memory area in which the processing module stores the IDs of the testing personnel;

[0056] This determination reduces excessive alarms and prevents the entry alarm line, which is wrapped around the exit alarm line, from being triggered when the target triggers the exit alarm line, enabling the system to accurately determine entry and exit behaviors.

[0057] The threshold in steps 6 and 7 is one-fifth of the width of the target detection box. Selecting one-fifth of the detection box width as the warning distance can effectively avoid the negative impact of the same distance in the actual scene appearing as unequal distances in the pixel coordinate system under different viewing distances, because the detection box also shrinks proportionally in the pixel coordinate system as the viewing distance increases.

[0058] Step 8: Determine whether the target's location point coordinates exist in the video frame. If not, end the determination; if so, return to step 5.

[0059] It should be noted that because there are multiple targets in a video frame, we calculate the distances of all targets in the video frame to each line before determining whether there are any targets whose distances exceed the threshold. In practice, the threshold is indeed determined after the distance determination; however, an additional step of checking if it is the last target point in the frame is added. This step is only used to indicate the existence of a loop and does not affect the implementation of this invention.

[0060] The essence of this invention lies in the following: using video stream data transmitted from on-site cameras, the YOLOX target detection network is used to detect the existence and status of target objects frame by frame in real time. The DeepSORT tracking algorithm assigns IDs to the target's behavioral trajectory, and an alarm is triggered once the ID owner is detected approaching the fence.

[0061] The core of the computation lies in two main components: the YoloX object detection network, which is the latest object detection network in the Yolo (You Only Look Once) series released by Megvii Technology in August 2021.

[0062] In terms of detection, compared to the anchor-based approach used in previous YOLOv3, YOLOv4, and YOLOv5 networks, it uses an anchor-free method similar to that used in networks like CenterNet and Nanodet as the detection head. This method reduces computation by eliminating prior prediction boxes, speeds up forward propagation, alleviates the problem of uneven distribution of positive and negative samples, and avoids parameter tuning of prior anchors. Regarding accuracy, since the prior boxes obtained by the clustering methods used in previous YOLO series are not generally representative and do not significantly improve accuracy in practical engineering applications, the anchor-free approach does not negatively impact accuracy compared to the anchor-based approach. In fact, because YOLOX's anchor-free approach uses a decoupled head detector, compared to the previous YOLO series networks that used the same set of parameters for both regression and prediction, accuracy is improved to some extent.

[0063] For training, YoloX uses the SimOTA algorithm, an improved version of the OTA optimal transport allocation algorithm, for label assignment. Previous YoloX models relied too heavily on a grid and IOU (Intersection over Union) threshold for assignment, which inevitably resulted in some blurry boxes being filtered out when objects were occluded, leading to poor generalization. The OTA algorithm treats the IOU loss between the predicted and ground truth boxes and the loss between the predicted class of the current predicted box and the actual class of the ground truth box as costs, establishing a mathematical model for optimal transport to solve for these costs. This improves the model's generalization ability by enhancing the rationality and accuracy of label assignment. SimOTA reduces the steps of globally searching for the optimal solution compared to OTA, using the Dynamic_k algorithm to dynamically extract the k minimum-cost boxes from the ground truth from the cost matrix as positive samples, significantly reducing computation time and resources. Compared to the OTA algorithm, SimOTA effectively reduces training time by 25%.

[0064] Another is the DeepSort object tracking network. The DeepSort (Simple Online and Realtime Tracking with a Deep Association Metric) algorithm integrates motion and appearance information by combining two metrics: Mahalanobis distance and feature cosine distance. It matches the identified target against consecutive frames and assigns an ID to achieve tracking. Appearance information refers to the use of a simple CNN network, Extractor, to extract the appearance features of the object detected by the object detection network. Motion information refers to the results of Kalman filter predictions.

[0065] Because the Sort algorithm, the predecessor of DeepSort, was a relatively coarse tracking algorithm, it was particularly prone to losing its ID when an object was occluded. The DeepSort algorithm, based on the Sort (Simple Online and Realtime Tracking) algorithm, adds cascading matching and confirmation of new tracks. Tracks are divided into confirmed and unconfirmed states. Newly generated tracks are in the unconfirmed state; unconfirmed tracks must match detections consecutively a certain number of times (default is 3) before they can be converted into confirmed tracks. Confirmed tracks must mismatch detections consecutively a certain number of times (default is 30) before they are deleted.

[0066] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An electronic fence early warning method based on target detection and target tracking, characterized in that: It includes the following steps: Step 1: Based on the video shooting scene corresponding to the video capture module, the user draws several straight lines to form a closed area, and the straight lines serve as an electronic fence. Step 2: Read the video and send the video frames into the computing module in sequence. Perform object detection through the YOLOX object detection network in the computing module to obtain the detection results Bounding Boxes, and send the detection results Bounding Boxes into the buffer queue. Step 3: Determine whether a new target appears in the video frame. If a new target appears, the video frame is returned to Step 2 as the first video frame of the new target and sent to the YOLOX target detection network for target detection. Otherwise, the detection result of the current video frame is sent to the DeepSort target tracking network in the calculation module for matching, to find the same target in adjacent frames and assign the same ID to the same target; Step 4: Confirm the coordinates of the target's location point from the Bounding Boxes detection results for different targets; Step 5: Calculate the axial distance between each target location point and each straight line; take the minimum value as the distance between the target and the fence in the real scene; for the axial distance between the location point coordinates of the i-th target and line j... The formula is as follows; ； x i Let y be the horizontal coordinate of target i. i Let k be the vertical coordinate of target i; j Let b be the slope of line j. j The offset of line j; Step 6: If the distance between the target location point coordinates and the minimum axis of any straight line is greater than or equal to the threshold, check if the target ID is in the buffer. If the target ID is in the buffer, remove the target ID from the buffer. Step 7: If the distance between the target location point coordinates and the minimum axis of a certain straight line is less than the threshold, determine whether the target ID is in the buffer. If it is in the buffer, no alarm will be triggered this time. If no alarm has been triggered before the given time interval and the target ID is not in the buffer, then the target ID is sent to the buffer and an alarm is issued; The buffer is a memory area in the computer where the IDs of the inspectors are stored. Step 8: Determine whether the target's location point coordinates exist in the video frame. If not, end the determination; if so, return to step 5.

2. The electronic fence early warning method based on target detection and target tracking according to claim 1, characterized in that: In step 1, the slope and intercept of the line determine whether it is a vertical or horizontal line; where, if the slope k of the line belongs to the set... If the slope k of the line belongs to the set, then it is considered a vertical line. If it is, then it is considered a horizontal line.

3. The electronic fence early warning method based on target detection and target tracking according to claim 1, characterized in that: The determination of the target's location point coordinates in step 4 is based on the following: generating detection boxes from the detection results of different targets in Bounding Boxes, obtaining the length and width information and center point coordinates of the detection boxes, calculating the bottom midpoint coordinates of each target detection box, and using the bottom midpoint as the location point coordinates; The detection box is a rectangle, and the four sides of the rectangle pass through the limit pixels of the target in the four directions of up, down, left, and right.

4. The electronic fence early warning method based on target detection and target tracking according to claim 3, characterized in that: The threshold in steps 6 and 7 is one-fifth of the width of the target detection box.

5. The electronic fence early warning method based on target detection and target tracking according to claim 1, characterized in that: In step 6, the straight line that needs to be measured from the target is taken as the target straight line. First, determine the type of the target straight line. If the target straight line is a vertical line, calculate the horizontal axis distance; if it is a horizontal line, calculate the vertical axis distance; if it is neither, calculate both the horizontal axis distance and the vertical axis distance. Take the minimum axis distance as the distance between the target and the fence straight line in the real scene.

6. The electronic fence early warning method based on target detection and target tracking according to claim 3, characterized in that: The straight lines used to form the electronic fence in step 1 are divided into inner straight lines and outer straight lines; the inner straight line is the departure alarm line used to determine the departure of the target; the outer straight line is the entry alarm line used to determine the entry of the target.

7. The electronic fence early warning method based on target detection and target tracking according to claim 6, characterized in that: Step 1 also includes proximity alarm lines, which are virtual lines with inner and outer ones. The inner proximity alarm line is located inside the exit alarm line and the distance between the two lines is equal to a threshold. The outer proximity alarm line is located outside the entry alarm line and the distance between the two lines is equal to a threshold. The area between the inner and outer proximity alarm lines forms a buffer zone.

8. The electronic fence early warning method based on target detection and target tracking according to claim 1, characterized in that: The video acquisition module is a camera, and the computing module is a computer with embedded YOLOX target detection network and DeepSort target tracking network. The video acquisition module transmits video to the computing module via wired or wireless means.

Images