Invisible area people counting method based on people behavior trend space vector analysis

Abstract

The invention discloses an invisible area people counting method based on people behavior trend spatial vector analysis, which comprises the following steps: arranging a camera device in a visible area, detecting a target in a field of view of the camera device through a YOLO detection algorithm, and marking and tracking the target; and analyzing the behavior trend of the detected target, judging whether the detected target enters a specific invisible area, and calculating the number of people in the invisible area based on a judgment result. According to the invention, accurate statistics of the number of people in a plurality of non-public areas is realized at the same time.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a method for counting people in an invisible area based on spatial vector analysis of personnel behavior trends. Background Art

[0002] With the development of social economy, accurately grasping the personnel situation in various regions plays an increasingly important role in production and life. For example, when disasters such as coal mine safety accidents and office building fires occur, rescue departments can optimize resource allocation and formulate more efficient rescue plans through accurate population data, thereby minimizing the losses caused by disasters and increasing the success rate of rescue operations. Therefore, whether it is a visible area or an invisible area, the statistics of the number of people are very important.

[0003] The current methods of counting people mainly include infrared detection, wireless communication, and machine vision technology. Infrared detection technology uses infrared radiation released by the human body to detect the presence and movement of the human body, but infrared detection can only detect the number of people in a certain section. When multiple people walk side by side, if they block each other, infrared detection may not be able to accurately identify the number of people. Most wireless communication technologies use Ultra-Wideband (UWB) and Radio Frequency Identification (RFID). Among them, UWB technology uses the characteristics of large bandwidth and short pulses to transmit a large amount of data in a very short time. However, because it is used for short-distance high-speed data transmission, it is necessary to deploy communication base stations, and the engineering cost is expensive. RFID technology is a technology that identifies specific targets through radio signals. The RFID system consists of a reader and a tag. The reader sends a radio signal to activate the nearby RFID tag, and the tag receives the signal and responds to the data containing the identification information. However, the personnel who need to go down the mine need to wear an electronic tag, so the loss or damage of the electronic tag may cause positioning errors and other problems.

[0004] With the continuous development of deep learning target detection technology, the use of machine vision technology to perform real-time statistical analysis of personnel has become a current research hotspot. However, most current methods for counting the number of people still rely on cameras to directly monitor the detected area, which means that effective personnel counting cannot be performed in areas that the camera cannot directly illuminate.

[0005] Therefore, there is currently no mature solution for counting the number of people in invisible areas in the prior art. Visible areas specifically refer to places where installation of facilities such as cameras is allowed, such as corridors and halls, etc.; invisible areas specifically refer to places where installation of facilities such as cameras is not allowed or is not convenient, such as offices and warehouses. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a method for counting people in invisible areas based on spatial vector analysis of people behavior trends. By analyzing the people behavior trends collected by cameras in public areas, accurate counting of people in multiple non-public areas can be achieved at the same time, providing important data support for rescue of people in emergency situations.

[0007] To achieve the above object, the present invention provides a method for counting people in an invisible area based on spatial vector analysis of people behavior trends, comprising:

[0008] Deploy a camera device in the visible area, detect the target in the field of view of the camera device by using the YOLO detection algorithm, and mark and track the target;

[0009] Analyze the behavior trend of the detected target, determine whether the detected target enters a specific invisible area, and calculate the number of people in the invisible area based on the determination result.

[0010] Preferably, detecting the target in the field of view of the camera device by using the YOLO detection algorithm includes:

[0011] A detection area is drawn in the visible area. When a target appears in the detection area, a preset operation is triggered, and continuous video frames of the same target in the detection area are analyzed by the YOLO detection algorithm; wherein the detection area is set at the boundary between the visible area and the invisible area.

[0012] Preferably, if several targets appear in the detection area at the same time, it is necessary to ensure that the blocked target is higher than the extension line between the camera device and the highest point of other targets in front, otherwise the target detection is invalid.

[0013] Preferably, detecting the target in the field of view of the camera device by using the YOLO detection algorithm also includes:

[0014] The detection line is drawn through OpenCV and the coordinates of the detection line. The position of the detection line is determined based on the position of the camera device and the invisible area, and the center of the camera field of view is taken as the origin (0, 0). The position coordinates of the two end points of the detection line are obtained, and the detection line is drawn through the position coordinates.

[0015] Preferably, analyzing the behavior trend of the detected target includes:

[0016] When the detected target is a single object, the positional relationship and the angle between the direction vector of the detected target and the direction vector of the detection line are calculated to determine whether the detected target enters the specific invisible area, specifically:

[0017] Firstly, the detection line direction vector and the real-time direction vector of the detected target are obtained by vector calculation, and it is determined whether the detected target crosses the line;

[0018] Then, the cross product of the two vectors is calculated, and the positional relationship of the two vectors is determined by the positive or negative result. If the cross product is greater than zero and the angle between the two vectors is between 0 and 180 degrees, it is determined that the target has entered a specific invisible area. When the cross product is less than zero and the angle between the two vectors is between 0 and 180 degrees, it is determined that the target has walked out of the specific invisible area.

[0019] Preferably, analyzing the behavior trend of the detected target further includes:

[0020] When there are multiple detected targets, the Byte Track tracking algorithm is combined to mark the detected targets in each frame with IDs and distinguish and record them, and the detected targets are judged separately.

[0021] Preferably, when judging whether the detected object enters a specific invisible area, the bottom center point of the detection object boundary box is used as a reference to judge whether the detection object crosses the detection line.

[0022] Compared with the prior art, the present invention has the following advantages and technical effects:

[0023] This paper uses the advanced YOLOv8 recognition algorithm combined with ByteTrack target tracking technology, and introduces regional intrusion and vector crossing detection technology to achieve accurate statistics of the number of people in non-public areas. By arranging cameras in the corridors of public areas, the cost is greatly reduced, providing an effective solution for counting the number of people in non-public areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 This is a flow chart of a method for counting people in an invisible area based on spatial vector analysis of people behavior trends according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram comparing the performance of various versions of YOLO models in an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of a person that can be detected by a pre-trained model according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of an obstruction situation that occurs during area intrusion detection according to an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of camera configuration parameters according to an embodiment of the present invention;

[0030] Figure 6 An example diagram of drawing a detection line according to an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of a detected target entering a room according to an embodiment of the present invention;

[0032] Figure 8 A schematic diagram of a detected target entering a room according to an embodiment of the present invention;

[0033] Fig. 9 A schematic diagram of a detected target exiting a room according to an embodiment of the present invention;

[0034] Fig.10 It is a schematic diagram of a case where there are multiple detected targets in the detection field of view of an embodiment of the present invention;

[0035] Fig.11 This is a schematic diagram of ByteTrack performance indicators of an embodiment of the present invention;

[0036] Fig.12 A schematic diagram of a detection area drawn according to an embodiment of the present invention;

[0037] Fig.13 A schematic diagram of a detected person according to an embodiment of the present invention;

[0038] Fig.14 This is an example diagram of personnel tracking according to an embodiment of the present invention;

[0039] Fig.15 This is a display interface diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] The present invention proposes a method for counting people in invisible areas based on spatial vector analysis of people behavior trends, such as Figure 1 ,include:

[0043] Deploy a camera device in the visible area, detect the target in the field of view of the camera device by using the YOLO detection algorithm, and mark and track the target;

[0044] Analyze the behavior trend of the detected target, determine whether the detected target enters a specific invisible area, and calculate the number of people in the invisible area based on the determination result.

[0045] This paper uses the advanced YOLOv8 recognition algorithm combined with ByteTrack target tracking technology, and introduces regional intrusion and vector crossing detection technology to achieve accurate statistics of the number of people in non-public areas. By arranging cameras in the corridors of public areas, the cost is greatly reduced, providing an effective solution for counting the number of people in non-public areas.

[0046] Specifically, the target in the field of view of the camera device is detected by the YOLO detection algorithm, including:

[0047] A detection area is drawn in the visible area. When a target appears in the detection area, a preset operation is triggered, and continuous video frames of the same target in the detection area are analyzed by the YOLO detection algorithm; wherein the detection area is set at the boundary between the visible area and the invisible area.

[0048] Furthermore, in order to count the number of people in non-public areas, this embodiment adopts the method of arranging cameras in the corridor and ensuring that more than two doorways can be observed. First, an area needs to be demarcated at each doorway for subsequent regional intrusion detection. These demarcated areas will help determine which room a person has entered. Next, a detection line is drawn in the middle of each area, which is used to identify whether a person is entering or leaving a room.

[0049] When someone appears in the camera's field of view, the target detection algorithm is first used to identify and mark the people in the field, and their location information is saved. These marked targets are tracked, and once a person enters the previously marked area and crosses the detection line in the direction of the door, it is recorded as a person entering the room, and the number of people in the room increases accordingly. If they move out of the room and cross the detection line, the number of people in the room decreases accordingly. In this way, the number of people in each room can be accurately counted.

[0050] This embodiment needs to detect specific targets (ie, people) in the field of view before subsequent analysis can be performed, so it is necessary to introduce a target detection algorithm.

[0051] In this embodiment, the target detection algorithm is required to be trainable, fast, and accurate. The YOLO series (You Only Look Once) detection algorithm is used. This series of algorithms learns the features in the image through a deep neural network to achieve efficient and accurate target detection. By comparing the performance of various versions of the YOLO model, Figure 2 As shown, it can be seen that when the number of model parameters in each version is almost the same, YOLO8 has the highest accuracy. Table 1 shows the pre-trained model version number and corresponding indicator parameters of YOLO8 trained with coco128 as training data. Figure 3 The pre-trained model can detect people. Therefore, this embodiment uses the pre-trained model of yolov8, and this time uses the yolov8s version.

[0052] Table 1

[0053]

[0054] After detecting people in the public area, it is necessary to determine the room number that the detected target has entered, which is achieved by using the regional intrusion detection algorithm.

[0055] Draw a detection area near the door and use it as a double judgment with subsequent cross-line detection. This way, you can determine the room number that the detected target enters and exits.

[0056] The OpenCV library is used to capture video feeds and define specific areas to be monitored, such as doorways or windows, by drawing rectangles or polygons. Successive video frames within these defined areas are then analyzed.

[0057] Use OpenCV to determine whether the target position coordinates obtained by the detection algorithm are within the detection area. When a detected target enters the detection area, it will trigger pre-set operations such as alarms.

[0058] Furthermore, if several targets appear in the detection area at the same time, it is necessary to ensure that the blocked target is higher than the extension line of the highest point of the camera device and other targets in front, otherwise the target detection will be invalid.

[0059] When performing area intrusion detection, occlusion is difficult to avoid. Placing the camera at a higher position can effectively reduce some occlusion, but when two people completely overlap in front of the camera, the person behind must be higher than the extension line of the line connecting the highest point of the camera and the person in front to be effectively detected. Figure 4 .

[0060] Furthermore, detecting the target in the field of view of the camera device by using the YOLO detection algorithm also includes:

[0061] The detection line is drawn through OpenCV and the coordinates of the detection line. The position of the detection line is determined based on the position of the camera device and the invisible area, and the center of the camera field of view is taken as the origin (0, 0). The position coordinates of the two end points of the detection line are obtained, and the detection line is drawn through the position coordinates.

[0062] Specifically, the detection line is drawn using OpenCV and the coordinates of the detection line. The distance between the detection line and the door should be moderate, ensuring that the distance is far enough to detect the person entering the door in advance, but not too far to avoid misdetecting the person passing by the door as entering. Figure 5 Under the camera configuration shown, the limit distance of accurate detection is 23m. In this embodiment, it is assumed that the center of the field of view is the origin (0, 0), and the position coordinates of the two end points of the detection line are obtained based on this. Figure 6 shown.

[0063] Further, analyzing the behavior trend of the detected target includes:

[0064] When the detected target is a single object, the positional relationship and the angle between the direction vector of the detected target and the direction vector of the detection line are calculated to determine whether the detected target enters the specific invisible area, specifically:

[0065] Firstly, the detection line direction vector and the real-time direction vector of the detected target are obtained by vector calculation, and it is determined whether the detected target crosses the line;

[0066] Then, the cross product of the two vectors is calculated, and the positional relationship of the two vectors is determined by the positive or negative result. If the cross product is greater than zero and the angle between the two vectors is between 0 and 180 degrees, it is determined that the target has entered a specific invisible area. When the cross product is less than zero and the angle between the two vectors is between 0 and 180 degrees, it is determined that the target has walked out of the specific invisible area.

[0067] Specifically, by calculating the positional relationship between the direction vector of the detected target and the direction vector of the detection line and the angle between them, it is determined whether the detected target is entering or leaving the door. Figure 7 This is the schematic diagram of the detected target entering the room.

[0068] First, the detection line direction vector (assuming that the north direction is the direction of the detection line direction vector) and the real-time direction vector of the detected target (obtained by calculating the position of the current frame and the position of the previous frame) are obtained through vector calculation, and it is determined whether the target crosses the line.

[0069] The direction vector calculation formula is: Let the coordinates of point A be (x1, y1), the coordinates of point B be (x2, y2), and the vector

[0070] Then calculate the cross product of the two vectors, and the positive or negative result can be used to determine the positional relationship of the two vectors.

[0071] Cross product of vectors in two-dimensional space: Given two two-dimensional vectors a = (a x ,a y ),b=(b x ,b y ), their cross product is defined as:

[0072] Geometric meaning: The absolute value of this scalar is equal to the area of ​​the parallelogram formed by vectors a and b.

[0073] Direction determination: The sign of the cross product can be used to determine the relative direction between two vectors.

[0074] If a×b>0, then b is counterclockwise of a.

[0075] If a×b<0, then b is clockwise from a.

[0076] If a×b=0, a and b are collinear (parallel or antiparallel).

[0077] Then, the two vector angles can be obtained through the vector angle calculation formula.

[0078] The formula for calculating the vector angle in two-dimensional space is:

[0079] In the above formula, the direction vector of the detection line is vector a, and the direction vector of the detected target is vector b. When the cross product of vector a and vector b is greater than zero and the angle between the two vectors is between 0 and 180 degrees, it is entering the door. When the cross product of vector a and vector b is less than zero and the angle between the two vectors is between 0 and 180 degrees, it is leaving the door. Figure 8 To enter the door, Fig. 9 To go out.

[0080] Further, analyzing the behavior trend of the detected target also includes:

[0081] When there are multiple detected targets, the Byte Track algorithm is used to mark the detected targets in each frame with IDs and distinguish and record them, and the detected targets are judged separately.

[0082] Specifically, when there are multiple detected targets in the field of view, the positions of all detected targets in each frame are placed in a tuple without being distinguished or recorded, which results in a poor one-to-one correspondence when comparing the positions of detected targets in the current frame with those in the previous frame, leading to false detections. Fig.10 shown.

[0083] Therefore, based on the single-person crossing-the-line detection scheme and combined with the tracking algorithm, the detected targets in each frame are marked with IDs and recorded separately to solve the problem of false detection.

[0084] This embodiment uses the ByteTrack tracking algorithm. ByteTrack is an efficient and accurate multi-target tracking algorithm that improves tracking performance by rationally utilizing low-confidence detection boxes. The algorithm first uses traditional methods to initially associate high-confidence detection boxes, then predicts the position of the trajectory through a Kalman filter, and matches unassociated low-confidence detection boxes with these predicted positions, thereby enhancing tracking accuracy and robustness. Experimental results of ByteTrack on multiple public datasets show that it performs well in dealing with situations such as target loss and occlusion, and is suitable for scenarios that require real-time and high accuracy, such as video surveillance, unmanned driving, and sports event analysis. ByteTrack performance indicators such as Fig.11 shown.

[0085] In the case of multi-person detection, it is necessary to combine the tracking algorithm with the single-person detection solution. By combining the two systems into one, the tracking algorithm is also introduced in the single-person detection, so that the cross-line detection can accurately detect the target's cross-line situation regardless of the single-person or multi-person detection scenario.

[0086] Further, when determining whether the detected object enters a specific invisible area, the bottom center point of the detection object boundary box is used as a reference to determine whether the detection object crosses the detection line.

[0087] When detecting crossing the line, the bottom center point of the bounding box of the detected person is used to determine whether the detection line has been crossed. The reason is that if the center point of the detection box is used as the basis for judgment, then when the detected target just passes by the door and continues to walk forward, the center point of the detection box will also cross the line.

[0088] In order to more clearly express the technical solution of the present invention, the following specific embodiments are provided to introduce the solution:

[0089] Draw the detection area:

[0090] Use OpenCV to draw the detection area in the field of view. The detection area should be set near the door. The specific location can be adjusted according to the location of the door. Fig.12 shown.

[0091] Draw the test line

[0092] Use OpenCV to draw a detection line in the field of view. The detection line should be set near the door, and the specific position can be adjusted according to the location of the door.

[0093] Person Detection

[0094] Detects people in the field of view and marks detected targets. Fig.13 is the person detected in this embodiment.

[0095] People tracking

[0096] Track the detected target, and the white line is the tracking line. Fig.14 shown.

[0097] Door entry and exit judgment

[0098] In this embodiment, when the detected target enters the left room, "jin" will be displayed on its body; when the detected target comes out of the left room, "chu" will be displayed on its body.

[0099] Final effect display

[0100] After backend calculation, the room number and the number of people corresponding to it are finally displayed on the UI interface. Fig.15 shown.

[0101] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for counting people in invisible areas based on spatial vector analysis of people behavior trends, characterized in that: include: Deploy a camera device in the visible area, detect the target in the field of view of the camera device by using the YOLO detection algorithm, and mark and track the target; Analyze the behavior trend of the detected target, determine whether the detected target enters a specific invisible area, and calculate the number of people in the invisible area based on the determination result.

2. The method for counting people in invisible areas based on spatial vector analysis of personnel behavior trends according to claim 1 is characterized in that: The YOLO detection algorithm is used to detect the target in the field of view of the camera device, including: A detection area is drawn in the visible area. When a target appears in the detection area, a preset operation is triggered, and continuous video frames of the same target in the detection area are analyzed by the YOLO detection algorithm; wherein the detection area is set at the boundary between the visible area and the invisible area.

3. The method for counting people in invisible areas based on spatial vector analysis of personnel behavior trends according to claim 2 is characterized in that: If several targets appear in the detection area at the same time, it is necessary to ensure that the blocked target is higher than the extension line of the camera device and the highest point of other targets in front, otherwise the target detection will be invalid.

4. The method for counting people in invisible areas based on spatial vector analysis of personnel behavior trends according to claim 2 is characterized in that: Detecting the target in the field of view of the camera device through the YOLO detection algorithm also includes: The detection line is drawn through OpenCV and the coordinates of the detection line. The position of the detection line is determined based on the position of the camera device and the invisible area, and the center of the camera field of view is taken as the origin (0, 0). The position coordinates of the two end points of the detection line are obtained, and the detection line is drawn through the position coordinates.

5. The method for counting people in invisible areas based on spatial vector analysis of personnel behavior trends according to claim 1, characterized in that: Analyze the behavior trend of the detected target, including: When the detected target is a single object, the positional relationship and the angle between the direction vector of the detected target and the direction vector of the detection line are calculated to determine whether the detected target enters the specific invisible area, specifically: Firstly, the detection line direction vector and the real-time direction vector of the detected target are obtained by vector calculation, and it is determined whether the detected target crosses the line; Then, the cross product of the two vectors is calculated, and the positional relationship of the two vectors is determined by the positive or negative result. If the cross product is greater than zero and the angle between the two vectors is between 0 and 180 degrees, it is determined that the target has entered a specific invisible area. When the cross product is less than zero and the angle between the two vectors is between 0 and 180 degrees, it is determined that the target has walked out of the specific invisible area.

6. The method for counting people in invisible areas based on spatial vector analysis of people behavior trends according to claim 5 is characterized in that: Analyzing the behavior trend of the detected target also includes: When there are multiple detected targets, the Byte Track tracking algorithm is combined to mark the detected targets in each frame with IDs and distinguish and record them, and the detected targets are judged separately.

7. The method for counting people in invisible areas based on spatial vector analysis of people behavior trends according to claim 1, characterized in that: When determining whether the detected object enters a specific invisible area, the bottom center point of the detection object boundary box is used as a reference to determine whether the detection object crosses the detection line.

Images