A method and system for monitoring wandering behavior of personnel

Abstract

The present invention discloses a method and system for monitoring the wandering behavior of a person. The method includes: collecting video and analyzing the video frame by frame to identify pedestrians in each frame; judging whether the identified pedestrians are real pedestrians; generating the trajectory of the same target pedestrian in the video collection area; gridding the video collection area together with the pedestrian trajectory, calculating the distance the pedestrian trajectory moves in each direction in each grid and comparing the maximum value, and counting the direction of the maximum pedestrian trajectory movement distance in all grids; judging whether the pedestrian has wandering behavior by comparing the number of times the direction of the maximum pedestrian trajectory movement distance in all grids appears with a threshold. The present invention has low requirements for video resolution and clarity. The method of judging the number of gridded main movement directions has fewer calculation steps, a simple process, accurate results, and is suitable for practical use.

Description

Technical Field

[0001] The invention relates to a method and a system for monitoring the wandering behavior of people. Background Art

[0002] Before committing a crime, criminals often scout the target location, move around the location, and linger. Therefore, their activity trajectory has obvious wandering characteristics. By detecting and alarming wandering people, the police or dispatched personnel can provide timely warnings and identify targets that need to be monitored in a timely manner.

[0003] The existing technology for monitoring pedestrian wandering behavior is difficult to recall and is extremely demanding on the trajectory of wandering pedestrians, making it difficult to implement in actual projects. Summary of the Invention

[0004] In view of this, the present invention provides a method and system for monitoring pedestrian wandering behavior, which improves the accuracy and feasibility of identifying pedestrian wandering behavior in surveillance videos.

[0005] To solve the above technical problems, the technical solution of the present invention is to adopt a method for monitoring the wandering behavior of people, comprising:

[0006] Collect videos and analyze them frame by frame to identify pedestrians in each frame.

[0007] Generate the trajectory of the same target pedestrian in the video acquisition area;

[0008] Grid the video capture area and pedestrian trajectories, calculate the distance the pedestrian trajectory moves in each direction in each grid, compare the maximum value, and count the directions of the maximum pedestrian trajectory distance in all grids;

[0009] The number of times the direction of the maximum pedestrian trajectory distance in all grids appears is compared with the threshold to determine whether the pedestrian has wandering behavior.

[0010] As an improvement, before generating pedestrian trajectories, it is determined whether the identified pedestrian is a real pedestrian.

[0011] As a further improvement, the determination of whether the identified pedestrian is a real pedestrian includes:

[0012] Searching the misidentified pedestrian feature database, if the same or similar misidentified pedestrian features as the identified pedestrian are found in the misidentified pedestrian feature database, it is determined that the identified pedestrian is not a real pedestrian;

[0013] If no identical or similar features of the identified pedestrian are found in the feature library of the identified pedestrian, the geometric features of the identified pedestrian are judged; if the geometric features do not match those of the pedestrian, the pedestrian is judged as not being a real pedestrian.

[0014] As another further improvement, the establishment of the misidentified pedestrian feature library includes:

[0015] Build a neural network model;

[0016] Manually select objects that are easily misidentified as pedestrians;

[0017] Use a neural network model to extract features from objects that are easily misidentified as pedestrians;

[0018] The extracted features are stored in the misidentified pedestrian feature library.

[0019] As an improvement, the geometric feature judgment condition is that the aspect ratio of the pedestrian in the image is between 1 / 2 and 1 / 5.

[0020] As an improvement, a target tracking algorithm is used to find the same pedestrian from the identified pedestrians and generate the trajectory of the pedestrian in the video acquisition area.

[0021] As an improvement, the video acquisition area and pedestrian trajectories are gridded, the distance of the pedestrian trajectory in each grid is calculated in each direction and the maximum value is compared, and the number of times the direction with the maximum pedestrian trajectory distance appears in all grids is counted.

[0022] Grid the video acquisition area and pedestrian trajectories into several square grids, and place the gridded video acquisition area and pedestrian trajectories in the plane coordinate system of the XY axis;

[0023] Obtain the coordinates of the starting point and end point of the pedestrian trajectory in the grid, and set the value obtained by subtracting the X-axis coordinate of the end point coordinate from the X-axis coordinate of the starting point coordinate as the X-axis motion value. If the X-axis motion value is positive, it is determined that the pedestrian trajectory moves along the positive direction of the X-axis; if it is negative, it is determined that the pedestrian trajectory moves along the negative direction of the X-axis; if it is 0, it is determined that the pedestrian trajectory does not move in the X-axis direction; set the value obtained by subtracting the Y-axis coordinate of the end point coordinate from the Y-axis coordinate of the starting point coordinate as the Y-axis motion value. If the Y-axis motion value is positive, it is determined that the pedestrian trajectory moves along the positive direction of the Y-axis; if it is negative, it is determined that the pedestrian trajectory moves along the negative direction of the Y-axis; if it is 0, it is determined that the pedestrian trajectory does not move in the Y-axis direction;

[0024] The direction of the larger one of the absolute value of the X-axis motion value and the absolute value of the Y-axis motion value is recorded as the main movement direction;

[0025] Count the main movement directions in all grids.

[0026] As an improvement, the method of comparing the number of times the direction of the maximum pedestrian trajectory distance in all grids appears with a threshold value to determine whether the pedestrian has wandering behavior includes:

[0027] The sum of the number of times the main movement direction is in the positive direction of X and the number of times the main movement direction is in the negative direction of X in all grids is compared with the number threshold. If it is greater than the number threshold, the ratio of the number of times the main movement direction is in the positive direction of X to the number of times the main movement direction is in the negative direction of X is compared with the ratio threshold. If it is greater than the ratio threshold or less than the inverse of the ratio threshold, the pedestrian is judged to have wandering behavior.

[0028] Alternatively, / or in combination, the sum of the number of times the main moving direction is positive Y and the number of times the main moving direction is negative Y in all grids is compared with the number threshold. If it is greater than the number threshold, the ratio of the number of times the main moving direction is positive Y to the number of times the main moving direction is negative Y is compared with the proportion threshold. If it is greater than the proportion threshold or less than the inverse of the proportion threshold, it is determined that the pedestrian has wandering behavior.

[0029] The present invention also provides a system for monitoring personnel wandering behavior, comprising:

[0030] Video acquisition module, used to collect pedestrian video images;

[0031] Pedestrian recognition module, used to analyze the collected video frame by frame and identify pedestrians in each frame;

[0032] Pedestrian trajectory generation module, used to generate the motion trajectory of the same pedestrian within the video acquisition range;

[0033] The main motion direction statistics module is used to grid the video acquisition area together with the pedestrian trajectory, calculate the distance the pedestrian trajectory moves in each direction in each grid, compare the maximum value, and count the direction of the maximum pedestrian trajectory distance in all grids;

[0034] The wandering judgment module is used to judge whether the pedestrian has wandering behavior by comparing the number of times the direction of the maximum pedestrian trajectory distance in all grids appears with the threshold.

[0035] As an improvement, it also includes:

[0036] The pedestrian identification module is used to determine whether the identified pedestrian is a real pedestrian.

[0037] The present invention is beneficial in that it grids pedestrian trajectories and counts the primary direction of movement within each grid. The number of occurrences of these primary directions across all grids is compared with a threshold to determine pedestrian wandering behavior. This method requires low video resolution and clarity. This grid-based method for determining the number of primary directions has few computational steps, a simple process, and accurate results, making it suitable for practical use. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flowchart of the present invention.

[0039] Figure 2 A grid diagram.

[0040] Figure 3 It is a structural principle diagram of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods.

[0042] like Figure 1 As shown, the present invention provides a method for monitoring the wandering behavior of a person, comprising:

[0043] S1 collects video and analyzes it frame by frame to identify pedestrians in each frame.

[0044] S2 determines whether the identified pedestrian is a real pedestrian;

[0045] S3 generates the trajectory of the same target pedestrian in the video acquisition area;

[0046] S4 grids the video acquisition area together with the pedestrian trajectory, calculates the distance the pedestrian trajectory moves in each direction in each grid, compares the maximum value, and counts the direction of the maximum pedestrian trajectory distance in all grids;

[0047] S5 determines whether the pedestrian has wandering behavior by comparing the number of times the direction of the maximum pedestrian trajectory distance in all grids appears with the threshold.

[0048] Specifically, in step S1, video information is collected in the monitoring area, and pedestrian target detection is performed using the neural network model YOLOv5 to identify pedestrian targets in each frame image of the video information.

[0049] In step S2, the accuracy of pedestrian detection by the neural network model is affected by the video resolution and clarity, and errors may occur, such as misidentifying other objects as pedestrians. Therefore, the pedestrians identified by the neural network model need to be judged again. The specific steps include:

[0050] S21 searches the misidentified pedestrian feature library. If the same or similar misidentified pedestrian features as the identified pedestrian are found in the misidentified pedestrian feature library, the identified pedestrian is determined not to be a real pedestrian. The misidentified pedestrian feature library is a library consisting of features of objects that are easily misidentified as pedestrians (such as cones, leaf shadows, etc.). If the pedestrian identified by the neural network model contains these features, it means that the pedestrian has been misidentified. The misidentified pedestrian feature library can be established in advance, and the establishment steps include:

[0051] S211 builds a neural network model;

[0052] S212 manually selects objects that are easily misidentified as pedestrians;

[0053] S213 uses a neural network model to extract features of objects that are easily misidentified as pedestrians;

[0054] S214 stores the extracted features in a misidentified pedestrian feature library.

[0055] If no identical or similar features of the identified pedestrian are found in the falsely identified pedestrian feature library, the identified pedestrian undergoes geometric feature verification. If the pedestrian's geometric features do not match, the pedestrian is determined to be not a true pedestrian. In this embodiment, the geometric feature verification condition is that the aspect ratio of the pedestrian in the image is between 1 / 2 and 1 / 5. After passing the falsely identified pedestrian feature library, geometric feature verification is required. Only if the geometric feature verification passes can the identified pedestrian be confirmed to be a true pedestrian and not some other object.

[0056] In step S3, this embodiment uses a target tracking algorithm to find the same pedestrian from the identified pedestrians and generates a trajectory of the pedestrian in the video acquisition area.

[0057] Step S4 specifically includes the following steps:

[0058] S41 Figure 2 As shown, the video capture area and pedestrian trajectories are gridded into several square grids, and the gridded video capture area and pedestrian trajectories are placed in the XY coordinate system. Of course, the finer the grid division, the higher the judgment accuracy, but also the greater the system overhead. Therefore, for existing video images, a 9*7 grid is generally sufficient.

[0059] S42 obtains the coordinates of the starting point and the end point of the pedestrian trajectory in the grid, and sets the value obtained by subtracting the X-axis coordinate of the end point coordinate from the X-axis coordinate of the starting point coordinate as the X-axis motion value. If the X-axis motion value is positive, it is determined that the pedestrian trajectory moves in the positive direction of the X-axis; if it is negative, it is determined that the pedestrian trajectory moves in the negative direction of the X-axis; if it is 0, it is determined that the pedestrian trajectory does not move in the X-axis direction; set the value obtained by subtracting the Y-axis coordinate of the end point coordinate from the Y-axis coordinate of the starting point coordinate as the Y-axis motion value. If the Y-axis motion value is positive, it is determined that the pedestrian trajectory moves in the positive direction of the Y-axis; if it is negative, it is determined that the pedestrian trajectory moves in the negative direction of the Y-axis; if it is 0, it is determined that the pedestrian trajectory does not move in the Y-axis direction; for example, in a certain grid, the starting point coordinates of the pedestrian trajectory are (1.1, 1.8) and the end point coordinates are (1.7, 1.3), then the X-axis motion value is 0.6 and the Y-axis motion value is -0.5, that is, the pedestrian trajectory moves in the positive direction of the X-axis and in the negative direction of the Y-axis.

[0060] S43 records the direction of the larger absolute value of the X-axis motion value and the Y-axis motion value as the main movement direction. If the absolute value of the X-axis motion value is 0.6 and the absolute value of the Y-axis motion value is 0.5, then the positive direction of the X-axis is the main movement direction in this grid. If the absolute value of the X-axis motion value is equal, it can be determined that the grid has no main movement direction.

[0061] S44 counts the main movement directions in all grids. If a direction in a grid is the main movement direction, then +1 is added to the statistics.

[0062] The rules for judging in step S5 are as follows:

[0063] The sum of the number of times the main movement direction is in the positive X direction and the number of times the main movement direction is in the negative X direction in all grids is compared with the number threshold. If it is greater than the number threshold, the ratio of the number of times the main movement direction is in the positive X direction to the number of times the main movement direction is in the negative X direction is compared with the ratio threshold. If it is greater than the ratio threshold or less than the inverse of the ratio threshold, the pedestrian is judged to have wandering behavior. For example, after statistics, the X-axis has moved in the positive direction 5 times and in the negative direction 3 times. The set number threshold is 9. Then 5 + 3 < 9, indicating that there is no wandering behavior in the X-axis direction.

[0064] Alternatively, the sum of the number of times the primary Y-axis movement direction is positive plus the number of times the primary Y-axis movement direction is negative across all grids is compared with a number threshold. If the sum is greater than the number threshold, the ratio of the number of times the primary Y-axis movement direction is positive to the number of times the primary Y-axis movement direction is negative is compared with a ratio threshold. If the sum is greater than the ratio threshold or less than the reciprocal of the ratio threshold, the pedestrian is considered to have wandered. For example, if the number of positive Y-axis movements is 8 and the number of negative Y-axis movements is 6, and the number threshold is set to 9, then 8 + 6 > 9, and the next step is to determine the pedestrian's behavior. If the ratio threshold is set to 1 / 2, then 6 / 8 > 1 / 2, and the pedestrian is considered to have wandered along the Y-axis. Alternatively, the ratio threshold can be set to 2, the reciprocal of 1 / 2, so 8 / 6 < 2 is also considered wandering. The threshold and the size of the judgment can be adjusted based on the denominator of the ratio (larger than smaller, or smaller than larger).

[0065] When judging, as long as there is wandering behavior in a certain direction, the person is considered to have wandering behavior. Therefore, the order of judgment can be X axis first and then Y axis, or Y axis first and then X axis, or both can be judged at the same time, which is not limited in the present invention.

[0066] like Figure 3 As shown, the present invention also provides a monitoring system for personnel wandering behavior, which is characterized by comprising:

[0067] Video acquisition module, used to collect pedestrian video images;

[0068] Pedestrian recognition module, used to analyze the collected video frame by frame and identify pedestrians in each frame;

[0069] Pedestrian identification module, used to determine whether the identified pedestrian is a real pedestrian;

[0070] Pedestrian trajectory generation module, used to generate the motion trajectory of the same pedestrian within the video acquisition range;

[0071] The main motion direction statistics module is used to grid the video acquisition area together with the pedestrian trajectory, calculate the distance the pedestrian trajectory moves in each direction in each grid, compare the maximum value, and count the direction of the maximum pedestrian trajectory distance in all grids;

[0072] The wandering judgment module is used to judge whether the pedestrian has wandering behavior by comparing the number of times the direction of the maximum pedestrian trajectory distance in all grids appears with the threshold.

[0073] The above are merely preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be construed as limiting the present invention, and the scope of protection of the present invention should be determined by the scope defined in the claims. Persons skilled in the art will appreciate that improvements and modifications may be made without departing from the spirit and scope of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for monitoring the wandering behavior of a person, characterized in that include: Collect videos and analyze them frame by frame to identify pedestrians in each frame. Generate the trajectory of the same target pedestrian in the video acquisition area; Grid the video capture area and pedestrian trajectories. Calculate the distances traveled by pedestrian trajectories in each direction within each grid and compare the maximum values. Count the directions of the maximum distances of pedestrian trajectories in all grids, specifically including: The video acquisition area and the pedestrian trajectory are gridded, the distance of the pedestrian trajectory moving in each direction in each grid is calculated and the maximum value is compared, the video acquisition area and the pedestrian trajectory are gridded into a plurality of square grids, and the gridded video acquisition area and the pedestrian trajectory are placed in the plane coordinate system of the XY axis; Obtain the coordinates of the starting point and end point of the pedestrian trajectory in the grid, and set the value obtained by subtracting the X-axis coordinate of the end point coordinate from the X-axis coordinate of the starting point coordinate as the X-axis motion value. If the X-axis motion value is positive, it is determined that the pedestrian trajectory moves along the positive direction of the X-axis; if it is negative, it is determined that the pedestrian trajectory moves along the negative direction of the X-axis; if it is 0, it is determined that the pedestrian trajectory does not move in the X-axis direction; set the value obtained by subtracting the Y-axis coordinate of the end point coordinate from the Y-axis coordinate of the starting point coordinate as the Y-axis motion value. If the Y-axis motion value is positive, it is determined that the pedestrian trajectory moves along the positive direction of the Y-axis; if it is negative, it is determined that the pedestrian trajectory moves along the negative direction of the Y-axis; if it is 0, it is determined that the pedestrian trajectory does not move in the Y-axis direction; The direction of the larger one of the absolute value of the X-axis motion value and the absolute value of the Y-axis motion value is recorded as the main movement direction; Count the main movement directions in all grids; The sum of the number of times the main movement direction is in the positive direction of X and the number of times the main movement direction is in the negative direction of X in all grids is compared with the number threshold. If it is greater than the number threshold, the ratio of the number of times the main movement direction is in the positive direction of X to the number of times the main movement direction is in the negative direction of X is compared with the ratio threshold. If it is greater than the ratio threshold or less than the inverse of the ratio threshold, the pedestrian is judged to have wandering behavior. Alternatively, / or in combination, the sum of the number of times the main moving direction is positive Y and the number of times the main moving direction is negative Y in all grids is compared with the number threshold. If it is greater than the number threshold, the ratio of the number of times the main moving direction is positive Y to the number of times the main moving direction is negative Y is compared with the proportion threshold. If it is greater than the proportion threshold or less than the inverse of the proportion threshold, it is determined that the pedestrian has wandering behavior.

2. The method for monitoring loitering behavior of a person according to claim 1, characterized in that: Before generating pedestrian trajectories, it is determined whether the identified pedestrian is a real pedestrian.

3. A method for monitoring wandering behavior of a person according to claim 2, characterized in that The determining whether the identified pedestrian is a real pedestrian includes: searching in a misidentified pedestrian feature library, and if a misidentified pedestrian feature identical or similar to that of the identified pedestrian is found in the misidentified pedestrian feature library, determining that the identified pedestrian is not a real pedestrian; If no identical or similar features of the identified pedestrian are found in the feature library of the identified pedestrian, the geometric features of the identified pedestrian are judged; if the geometric features do not match those of the pedestrian, the pedestrian is judged as not being a real pedestrian.

4. A method for monitoring wandering behavior of a person according to claim 3, characterized in that The establishment of the misidentified pedestrian feature library includes: establishing a neural network model; Manually select objects that are easily misidentified as pedestrians; Use a neural network model to extract features from objects that are easily misidentified as pedestrians; The extracted features are stored in the misidentified pedestrian feature library.

5. The method for monitoring loitering behavior of a person according to claim 3, characterized in that: The geometric feature judgment condition is that the aspect ratio of the pedestrian in the image is between 1 / 2 and 1 / 5.

6. The method for monitoring loitering behavior of a person according to claim 1, characterized in that: The target tracking algorithm is used to find the same pedestrian from the identified pedestrians and generate the trajectory of the pedestrian in the video acquisition area.

7. A system for monitoring the wandering behavior of a person, for deploying the method for monitoring the wandering behavior of a person as claimed in any one of claims 1 to 6, characterized in that include: Video acquisition module, used to collect pedestrian video images; Pedestrian recognition module, used to analyze the collected video frame by frame and identify pedestrians in each frame; Pedestrian trajectory generation module, used to generate the motion trajectory of the same pedestrian within the video acquisition range; The main motion direction statistics module is used to grid the video acquisition area together with the pedestrian trajectory, calculate the distance the pedestrian trajectory moves in each direction in each grid, compare the maximum value, and count the direction of the maximum pedestrian trajectory distance in all grids; The wandering judgment module is used to judge whether the pedestrian has wandering behavior by comparing the number of times the direction of the maximum pedestrian trajectory distance in all grids appears with the threshold.

8. A system for monitoring wandering behavior of a person according to claim 7, characterized in that Also includes: The pedestrian identification module is used to determine whether the identified pedestrian is a real pedestrian.

Images