A seat occupancy detection method for campus libraries based on YOLO v5

By using YOLO V5 and K-means algorithm in the library combined with DeepSort's seat occupation detection method, we can identify whether there are people in the library seats in real time, and identify malicious seat occupation by timekeeping, solving the problem of seat shortage and seat occupation disputes, and improving the Seat utilization and security of the library environment.

CN114170573BActive Publication Date: 2024-11-26FUZHOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111545932.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-17
Publication Date
2024-11-26
Estimated Expiration
2041-12-17

AI Technical Summary

Technical Problem

The shortage of seats in school libraries, and the seat-occupying behavior leads to low position utilization, and even causes disputes. It is difficult for the existing technology to effectively detect and solve this problem.

Method used

Using the seat-occupying detection method based on YOLO V5, combined with the K-means clustering algorithm and the DeepSort algorithm, the surveillance camera detects tables, chairs, people and schoolbags in the library in real time, calculates the IOU ratio and determines whether there is a person in the seat, and recognizes malicious intentions through timing. Take the seat and light the traffic light.

Benefits of technology

It improves the utilization rate of seats, reduces disputes, realizes real-time monitoring and identification of seat occupation behavior, prevents vicious seat occupation, and improves the utilization efficiency of library seats.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114170573B_ABST
    Figure CN114170573B_ABST
Patent Text Reader

Abstract

The present invention relates to a seat occupancy detection method for a campus library based on YOLO v5. A camera and a YOLO V5 target detection algorithm monitor the vacancy of seats in the library in real time. For example, if someone is on the seat, it will be determined that there is someone on the seat. If no one is detected on the seat, it is necessary to determine whether it is a malicious seat occupancy behavior at this time. The light on the table will light up in red. If it is detected that there is a book but no one on the seat, it is determined to be a seat occupancy behavior. At this time, the timing starts. If the seat occupancy time is greater than 30 minutes, it is considered to be a malicious seat occupancy at this time. The light on the table will turn green, indicating that the students without seats can sit down. The last person on this seat has left for too long or other students occupy multiple positions by themselves. What needs to be done is to first detect whether there is someone on the seat. If no one is on the seat, then detect whether there are larger seat occupancy items such as books or bags on the table, and then calculate the placement time of the seat occupancy items. If it exceeds 30 minutes, it is determined to be a malicious seat occupancy behavior, and other students can sit down with confidence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a seat occupancy detection method for a campus library based on YOLO v5. Background Art

[0002] In recent years, with the surge in the number of students taking postgraduate entrance examinations and civil service examinations, the number of seats in the school library is limited, and it is impossible to satisfy all students. In addition, many students or friends help to occupy seats, and place a few books or schoolbags on the seats to occupy seats. There may be no one in the seat for a whole morning, but because of the books, students without seats will think that someone will occupy the seat in a while, resulting in not being able to find a seat to study, but the seat may be idle for a long time or even several hours, which makes the seats more scarce and greatly reduces the utilization rate of the seats, which is not conducive to students' preparation for the exam. There are even students who have disputes because of occupying seats. Therefore, a device is needed to detect seat occupation behavior. The judgment of the machine can also reduce disputes and greatly improve the utilization rate of seats, and prevent malicious seat occupation. Summary of the invention

[0003] The purpose of the present invention is to provide a seat occupancy detection method for a campus library based on YOLO v5, which uses a YOLO V5 detection model and a K-means clustering algorithm that can obtain a more reasonable anchor frame to quickly retrieve key points and identify a monitoring system for tables, chairs, people, books, bags and other objects.

[0004] To achieve the above purpose, the technical solution of the present invention is: a seat occupancy detection method for campus libraries based on YOLO v5, which adopts the YOLO 5 detection method fused with K-means to obtain a more reasonable anchor frame algorithm and improve the accuracy of classified target detection. The specific implementation steps of the method are as follows:

[0005] Step S1, obtaining a scene image from the surveillance video of the surveillance area from the camera in the library, and detecting the table and chair in the image according to the deep learning target detection algorithm framework YOLO 5;

[0006] Step S2: Use K-means clustering algorithm to obtain more reasonable 9 anchor boxes required by YOLO V5;

[0007] Step S3: After the chair and the person are detected, the IOU ratio of the overlap between the images of the chair and the person is calculated to determine whether there is a person on the chair at this time;

[0008] Step S4, judging whether the distance is less than a threshold value, if so, it is determined that there is someone on the seat and a red light is turned on, if not, the book and bag detection continues;

[0009] Step S5: If no books or bags are detected, it is determined that no one is sitting at this position, and the green light is turned on;

[0010] Step S6: If books and bags are detected, start timing. If the seat is unoccupied for more than 30 minutes, it is determined to be malicious seat occupation and the green light is turned on.

[0011] In one embodiment of the present invention, in step S1, the training of YOLO V5 includes the following steps:

[0012] Step A1: First, construct a data set including tables, chairs and students, and then expand the data set by methods including affine transformation and rotation to generate a sufficiently large data set with high image quality;

[0013] Step A2: Build the neural network model required by the YOLO V5 framework, using the pytorch network framework and the Relu activation function to solve the problem of neuron death when the input value is negative;

[0014] Step A3: The data is divided into training set: validation set: test set = 8:1:1. Auto Learning Bounding Box Anchors is added. The custom data set is analyzed using k-means and genetic learning algorithms to obtain a preset anchor box suitable for object bounding box prediction in the custom data set. This box is automatically learned based on the training data.

[0015] In one embodiment of the present invention, Darknet-53 is used as the backbone network, and three feature layers of different scales are used, namely 13×13, 26×26, and 52×52. YOLO V5 first sets three prior frames for each downsampling scale to cluster and obtain nine sizes of prior frames; in the entire YOLO V5 structure, there is no pooling layer and a fully connected layer, and the network downsampling is achieved by setting the stride of the convolution to 3; on the basis of the current YOLO V5 algorithm for target recognition, the DeepSort algorithm is integrated, and the prediction heads are integrated and applied to drone photography, and finally a technology that can accurately locate targets in high-density scenes and continuously track them is achieved.

[0016] Compared with the prior art, the present invention has the following beneficial effects: the present invention equips the library with a surveillance camera and uses the YOLO V5 model to monitor whether there are occupied seats on the tables in the library, and sends the video frames acquired by the camera to the YOLO V5 model. On the basis of identifying targets based on the current YOLO V5 algorithm, the DeepSort algorithm is integrated, and prediction heads are integrated for application in drone photography. Finally, a technology that can accurately locate targets in high-density scenes and conduct continuous tracking is achieved to achieve the purpose of real-time monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of the network structure of YOLO V5;

[0018] Figure 2 It is the loss function formula of the YOLO V5 model;

[0019] Figure 3 It is a schematic diagram of the working process of an embodiment of the present invention;

[0020] Figure 4 It is the recognition result of whether the chair in the hall is vacant by the present invention;

[0021] Figure 5 It is the recognition result of whether there is an object occupying the seat when detecting an empty seat. DETAILED DESCRIPTION

[0022] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0023] The present invention discloses a seat occupancy detection method for a campus library based on YOLO v5, which adopts the YOLO5 detection method integrated with K-means to obtain a more reasonable anchor frame algorithm and improve the accuracy of classified target detection. The specific implementation steps of the method are as follows:

[0024] Step S1, obtaining a scene image from the surveillance video of the surveillance area from the camera in the library, and detecting the table and chair in the image according to the deep learning target detection algorithm framework YOLO 5;

[0025] Step S2: Use K-means clustering algorithm to obtain more reasonable 9 anchor boxes required by YOLO V5;

[0026] Step S3: After the chair and the person are detected, the IOU ratio of the overlap between the images of the chair and the person is calculated to determine whether there is a person on the chair at this time;

[0027] Step S4, judging whether the distance is less than a threshold value, if so, it is determined that there is someone on the seat and a red light is turned on, if not, the book and bag detection continues;

[0028] Step S5: If no books or bags are detected, it is determined that no one is sitting at this position, and the green light is turned on;

[0029] Step S6: If books and bags are detected, start timing. If the seat is unoccupied for more than 30 minutes, it is determined to be malicious seat occupation and the green light is turned on.

[0030] The following are specific implementation examples of the present invention.

[0031] This embodiment provides a campus library seat occupancy detection method based on YOLO V5. The detection of students, tables, chairs, books and bags in the present invention uses the end-to-end deep learning target detection algorithm YOLO V5, adopts the K-means algorithm, optimizes the model, and improves the detection performance. The YOLO V5 network model is shown in the attached figure. Figure 1 shown.

[0032] Specifically, the YOLO V5 model uses ReLU as the activation function and adopts an end-to-end method for training. The loss function used by the YOLO V5 model when performing the gradient descent method is as follows: Figure 2 As shown:

[0033] The first and second parts are responsible for predicting the bbox (bounding box) of the object. The first part represents the error value between the center point coordinates obtained by the forward propagation of the image along the neural network and the ground truth center point. The second part measures the error value between the width and height of the border obtained by the forward propagation of the image along the neural network and the width and height of the ground truth. The third part represents the error value of the confidence of the predicted border containing the target object. The confidence of the predicted border containing the target object should reach 1 after training. The fourth part represents the error value of the confidence of the predicted border that does not contain the target object. The confidence of the predicted widening that does not contain the target object should reach 0 after training. The fifth part is the grid prediction classification error term that contains the target object.

[0034] The following will be combined with the attached Figure 3 The workflow diagram illustrates the specific implementation steps:

[0035] Step 1: Obtain a scene image from a surveillance video of a surveillance area captured by a surveillance camera;

[0036] Step 2: Use the K-means algorithm to generate the initial anchor boxes required by the 9 YOLO V5 models;

[0037] Step 3: Detect students and tables and chairs in the image using the deep learning target detection algorithm framework YOLO 5, and mark and output them with rectangular borders;

[0038] Step 4: Determine whether a person is sitting on the chair based on the IOU ratio of the chair and the person image prior frame;

[0039] Step 5: If it is determined that there is someone on the chair, the red light will be turned on; if it is determined that there is no one, the book and bag will be checked;

[0040] Step 6: If someone is detected, a 30-minute countdown starts. If no one is present for more than 30 minutes, the behavior is considered as occupying a seat.

[0041] Figure 4 Is it to detect whether there is someone sitting on the seat: detect the IOU ratio of people, chairs and their prior frames through different detection frames, and judge whether the person is sitting on the chair based on this ratio value;

[0042] Figure 5 This is a rendering of the effect of detecting whether there is any object occupying a seat on the table when no one is there.

[0043] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A seat occupancy detection method for a campus library based on YOLO v5, characterized in that: The implementation steps are as follows: Step S1, obtaining a scene image from the surveillance video of the surveillance area from the camera in the library, and detecting the table and chair in the image according to the deep learning target detection algorithm framework YOLO v5; Step S2: Use K-means clustering algorithm to obtain the 9 anchor boxes required by YOLO v5; Step S3: After the chair and the person are detected, the IOU ratio of the overlap between the images of the chair and the person is calculated to determine whether there is a person on the chair at this time; Step S4: determine whether the IOU ratio of the overlap between the images of the chair and the person is less than a threshold value. If so, it is determined that there is someone on the seat and a red light is turned on. If no one is on the seat, the book and bag detection continues. Step S5: If no books or bags are detected, it is determined that no one is sitting at this position, and the green light is turned on; Step S6: If a book or bag is detected, start timing. If the seat is unoccupied for more than 30 minutes, it is determined to be malicious seat occupation and the green light is turned on. In step S1, the training of YOLO v5 includes the following steps: Step A1: First, construct a data set including tables, chairs and students, and then expand the data set by methods including affine transformation and rotation to generate a sufficiently large data set with high image quality; Step A2: Build the neural network model required by the YOLO v5 framework, using the pytorch network framework and the Relu activation function to solve the problem of neuron death when the input value is negative; Step A3: The data is divided into training set: validation set: test set = 8:1:1, and Auto Learning BoundingBox Anchors-adaptive anchor boxes are added. The custom data set is analyzed using k-means and genetic learning algorithms to obtain preset anchor boxes suitable for object bounding box prediction in the custom data set, and this box is automatically learned based on the training data; Taking Darknet-53 as the backbone network, three feature layers of different scales are used, namely 13×13, 26×26, and 52×52. YOLO v5 first sets three prior boxes for each downsampling scale and clusters them to obtain nine prior boxes of different sizes. In the entire YOLO v5 structure, there are no pooling layers and fully connected layers, and the network downsampling is achieved by setting the stride of the convolution to 3.