Wandering detection method, wandering detection device, and computer-readable storage medium

Abstract

The application provides a loitering detection method, a loitering detection device and a computer readable storage medium. The loitering detection method comprises the following steps: acquiring a first detection image of a first camera and a second detection image of a second camera collected at the same time; transforming the second detection image to the coordinate system of the first detection image based on the transformation relationship between the first camera and the second camera, to obtain a fusion image of the first detection image and the second detection image; detecting a trajectory path of a pedestrian in the fusion image; judging whether the number of rounds of the pedestrian is greater than a preset number based on the trajectory path; and if yes, confirming that the pedestrian has a loitering behavior. In the above manner, the loitering detection device realizes loitering detection in a large range through multi-camera splicing, and improves the personnel loitering recognition rate in key places through multi-camera linkage image capture.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent security and protection, in particular to a loitering detection method, a loitering detection device and a computer readable storage medium. BACKGROUND

[0002] With the development of computer vision technology, image processing and video analysis and other related technologies are applied to the traditional security and protection industry, and the traditional security and protection system is gradually moving towards intelligentization from digitization and networking. More and more attention is paid to target detection, classification, identification, tracking and feature point extraction of monitoring video in intelligent security and protection systems. Loitering detection is to judge whether the human target in the video appears loitering behavior, to judge abnormal trajectory, so as to discover and handle potential dangers such as theft, terrorist attacks and pickpocketing as soon as possible.

[0003] The current loitering detection method mainly includes four parts, namely video picture processing, pedestrian detection, trajectory tracking and trajectory analysis. The commonly used loitering detection mainly uses a single high-definition camera to collect pictures, and then performs noise reduction and other processing. Then, the processed pictures are sent to the detector for identification, the pedestrian features are matched by the pedestrian tracking algorithm to obtain the trajectory information of the pedestrian, and finally the loitering situation and reverse movement of the personnel are judged according to the turning back of the trajectory points.

[0004] The traditional loitering detection algorithm uses a single camera and is suitable for loitering detection in small scenes. In actual use scenarios, high-risk personnel will conduct site visits around key places and loiter in a large range, so it is not easy to identify them using a single camera. SUMMARY

[0005] The present application provides a loitering detection method, a loitering detection device and a computer readable storage medium.

[0006] The present application provides a loitering detection method, which comprises:

[0007] obtaining a first detection image of a first camera and a second detection image of a second camera collected at the same time;

[0008] transforming the second detection image to the coordinate system of the first detection image based on the transformation relationship between the first camera and the second camera, to obtain a fusion image of the first detection image and the second detection image;

[0009] detecting a trajectory path of a pedestrian in the fusion image;

[0010] judging whether the number of round trips of the pedestrian is greater than a preset number based on the trajectory path;

[0011] If yes, it is confirmed that the pedestrian appears loitering behavior.

[0012] wherein the detecting a trajectory path of the pedestrian in the fusion image comprises:

[0013] extracting motion information and appearance features of the pedestrian in the plurality of fusion images;

[0014] linearly fusing the motion information and the appearance features of the pedestrian in each fusion image to obtain first measurement information;

[0015] associating the first measurement information of the plurality of fusion images to obtain a first association result;

[0016] obtaining an intersection over union of a bounding box of the motion information and a bounding box of the appearance features that fail to match to obtain second measurement information;

[0017] associating the second measurement information that fails to match to obtain a second association result;

[0018] generating the trajectory path of the pedestrian according to the first association result and the second association result.

[0019] wherein the extracting motion information and appearance features of the pedestrian in the plurality of fusion images comprises:

[0020] in the fusion image, if the pedestrian appears in a first detection image and a second detection image after transformation at the same time, obtaining a first image quality of the first detection image and a second image quality of the second detection image;

[0021] when the first image quality is higher than the second image quality, extracting the motion information and the appearance features of the pedestrian from the first detection image;

[0022] when the first image quality is lower than the second image quality, extracting the motion information and the appearance features of the pedestrian from the second detection image.

[0023] wherein the loitering detection method further comprises:

[0024] based on the trajectory path, obtaining a motion direction of the pedestrian at each trajectory point;

[0025] based on a number of times of transformation of the motion direction of the pedestrian per unit time, obtaining a number of times of back-and-forth movement of the pedestrian.

[0026] wherein the obtaining the motion direction of the pedestrian at each trajectory point based on the trajectory path comprises:

[0027] mapping the trajectory path to a first quadrant of a coordinate axis of the fusion image;

[0028] Obtain the slope of each trajectory point relative to the origin;

[0029] The direction of motion of each trajectory point is determined based on the slope difference between adjacent trajectory points.

[0030] The step of determining the motion direction of each trajectory point based on the slope difference between adjacent trajectory points includes:

[0031] Get the slope difference between the current trajectory point and its adjacent trajectory points;

[0032] When the slope difference is greater than 0, the motion direction of the current trajectory point is set to a first preset value;

[0033] When the slope difference is less than 0, the motion direction of the current trajectory point is set to a second preset value;

[0034] The step of obtaining the number of round trips of the pedestrian based on the number of times the pedestrian changes direction of movement per unit time includes:

[0035] Obtain the motion direction values ​​of all trajectory points along the pedestrian's trajectory path;

[0036] The number of times the combination of movement direction values ​​appears in all the trajectory points is determined as the number of round trips for the pedestrian.

[0037] The combination of motion direction values ​​consists of adjacent motion direction values ​​being a first preset value and a second preset value.

[0038] Wherein, after detecting the pedestrian's trajectory path in the fused image, the loitering detection method further includes:

[0039] Based on the trajectory path, determine whether the number of times the pedestrian makes round trips is greater than half of a preset number;

[0040] If so, then it is confirmed that the pedestrian exhibited suspected loitering behavior;

[0041] Control the first camera and / or the second camera to acquire images at a fast capture speed, wherein the fast capture speed is higher than the normal capture speed.

[0042] The control of the first camera and / or the second camera to acquire images at a fast capture speed includes:

[0043] When the suspected loitering behavior appears in the first detection image, the first camera is controlled to capture images at a fast capture speed, and the second camera is controlled to capture images at a normal capture speed.

[0044] when the suspected loitering behavior appears in the second detection image, controlling the first camera to capture images at a normal snapshot speed and the second camera to capture images at a fast snapshot speed;

[0045] when the suspected loitering behavior appears in the junction area of the first detection image and the transformed second detection image in the fusion image, controlling the first camera to capture images at a fast snapshot speed and the second camera to capture images at a fast snapshot speed.

[0046] wherein, after the controlling the first camera to capture images at a fast snapshot speed and the second camera to capture images at a normal snapshot speed, the loitering detection method further comprises:

[0047] transforming the boundary of the second camera to the one-way tripwire intrusion rule line of the first camera according to the transformation relationship;

[0048] when the pedestrian moving with suspected loitering behavior moves to the one-way tripwire intrusion rule line, controlling the second camera to capture images at a fast snapshot speed.

[0049] The application also provides a loitering detection device, which comprises a processor and a memory, the memory stores program data, and the processor is used to execute the program data to realize the loitering detection method as described above.

[0050] The application also provides a computer readable storage medium for storing program data, which is used to realize the loitering detection method as described above when executed by a processor.

[0051] The application has the beneficial effect that the loitering detection device acquires a first detection image of a first camera and a second detection image of a second camera captured at the same time; based on the transformation relationship between the first camera and the second camera, the second detection image is transformed to the coordinate system of the first detection image to obtain a fusion image of the first detection image and the second detection image; the trajectory path of the pedestrian is detected in the fusion image; based on the trajectory path, it is judged whether the number of round trips of the pedestrian is greater than a preset number; if yes, it is confirmed that the pedestrian has loitering behavior. Through the above-mentioned manner, the loitering detection device realizes loitering detection in a larger range through multi-camera splicing, and improves the personnel loitering recognition rate in key places through multi-camera linkage photographing. BRIEF DESCRIPTION OF DRAWINGS

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0053] Figure 1 This is a flowchart illustrating an embodiment of the loitering detection method provided in this application;

[0054] Figure 2 This is a schematic flowchart of an embodiment of the loitering detection method provided in this application;

[0055] Figure 3 yes Figure 1 The flowchart of step S14 of the loitering detection method is shown below.

[0056] Figure 4 This is a schematic diagram showing the slope changes of each trajectory point provided in this application;

[0057] Figure 5 This is a flowchart illustrating another embodiment of the loitering detection method provided in this application;

[0058] Figure 6 This is a schematic diagram of the structure of an embodiment of the loitering detection device provided in this application;

[0059] Figure 7 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0061] The technical means by which this application solves the problems of the prior art is to intelligently match the preset position that the user wants to set by comparing several common and universal feature items that are pre-stored, thereby reducing the steps of the user manually adjusting the camera direction and automatically adjusting the camera direction.

[0062] Please refer to details. Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating an embodiment of the loitering detection method provided in this application. Figure 2This is a schematic diagram of the overall process of an embodiment of the loitering detection method provided in this application.

[0063] The loitering detection method of this application is applied to a loitering detection device, which can be a server or a system consisting of a server and a terminal device working together. Accordingly, the various parts of the loitering detection device, such as units, subunits, modules, and submodules, can be all located in the server, or they can be located separately in the server and the terminal device.

[0064] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed servers, or as a single software program or software module; no specific limitation is made here. In some possible implementations, the loitering detection method of this application embodiment can be implemented by the processor calling computer-readable instructions stored in memory.

[0065] It should be noted that the loitering detection device in this application embodiment can be mounted on a camera or webcam to directly control the camera or webcam, or it can be used as a camera or webcam.

[0066] like Figure 2 As shown, the loitering detection method based on multi-camera linkage in this application mainly includes six parts: camera linkage configuration, video and image acquisition, image quality assessment, pedestrian detection, trajectory analysis, and loitering judgment analysis.

[0067] Specifically, the first part is camera linkage configuration. This involves setting up the main and auxiliary cameras and using homography rigid body transformation spatial sampling technology to obtain the homography transformation matrix of the auxiliary camera stitched to the main camera. The second part is image acquisition. This is achieved by capturing multiple real-time video recordings using multiple front-end high-definition cameras, and then extracting images from the real-time video recordings second by second and storing them in a register. The third part is image quality assessment. Since the images obtained in the second part may have issues such as occlusion or blurring, they are filtered in this part. The fourth part is pedestrian detection, using YOLOv5, which has high detection accuracy and fast detection speed, for real-time analysis. The fifth part is trajectory analysis. This part uses the homography transformation matrix obtained in the first part to stitch the images from the auxiliary camera and the main camera, and uses a YOLOv5-based multi-pedestrian tracking algorithm to track the pedestrians detected in the video, thereby obtaining the pedestrian trajectories. The sixth part is loitering judgment analysis, using a pedestrian trajectory-based detection method, using the number of changes in the slope direction of trajectory points as the basis for determining whether loitering behavior exists.

[0068] The following section will continue to introduce the implementation process of the specific loitering detection method. Figure 2 The specific contents of the six parts shown are as follows:

[0069] Specifically, such as Figure 1 As shown, the loitering detection method in this application specifically includes the following steps:

[0070] Step S11: Acquire the first detection image from the first camera and the second detection image from the second camera, which are captured at the same time.

[0071] In the embodiments of this application, such as Figure 2 The camera linkage configuration shown requires attention to the camera installation method when using multi-camera linkage for loitering detection.

[0072] Multiple cameras need to be distinguished as main cameras and auxiliary cameras. The main camera is typically installed in a frontal position at key locations and is usually a close-up camera. Auxiliary cameras are generally installed near key locations to extend the main camera's field of view, and their fields of view partially overlap with the main camera's. In subsequent identification and calculation processes, the location information is standardized based on the main camera's position. In this embodiment, a first camera is used as the main camera, and a second camera as the auxiliary camera. In other embodiments, the same setup can be used, with the first camera as the auxiliary camera and the second camera as the main camera.

[0073] Because pedestrians don't walk very fast, both the main and auxiliary cameras use the normal capture speed (v) when capturing images. low Image capture is performed, and images are captured and stored in a register at corresponding time intervals.

[0074] Furthermore, embodiments of this application can also achieve relay data acquisition between the main camera and the auxiliary camera through tripwire intrusion detection, enabling the camera with suspected loitering personnel to capture images at a high speed v. high Data collection was conducted, and cameras were used to collect images of individuals suspected of loitering. low The data is collected quickly. This ensures accurate capture of suspected loitering individuals while reducing the resource consumption of the camera.

[0075] Step S12: Based on the transformation relationship between the first camera and the second camera, transform the second detection image to the coordinate system of the first detection image to obtain a fused image of the first detection image and the second detection image.

[0076] In this embodiment of the application, before processing the detection image, the loitering detection device can also preprocess and screen the detection image.

[0077] Please refer to details. Figure 2In the image quality assessment section shown, some of the cropped images do not perform well. There may be instances where pedestrians are blurry and their individual features are indistinguishable, or they may be occluded, preventing further identification. Such images are therefore excluded. Therefore, a quality assessment model is used to evaluate the images, determining the completeness and blurriness of pedestrians within the images. Thresholds are set to filter out images.

[0078] After image quality assessment, the loitering detection device uses the remaining detection images for fusion: specifically, the loitering detection device uses a homography transformation matrix to transform the detection image from the auxiliary camera onto the detection image from the main camera, and completes the fusion and stitching of the detection images to obtain a fused image.

[0079] Therefore, the fused image can simultaneously include the acquisition range of both the main camera and the auxiliary camera, which can effectively achieve loitering detection over a large area.

[0080] Step S13: Detect the pedestrian trajectory path in the fused image.

[0081] In the embodiments of this application, such as Figure 2 The diagram shows two parts: pedestrian detection and trajectory analysis. The loitering detection device detects the position of pedestrians by analyzing the position of pedestrians in each frame of the fused image, and then generates the pedestrian's trajectory path by analyzing the pedestrian positions in multiple consecutive frames of the fused image.

[0082] Specifically, in the pedestrian detection module, the YOLOv5 model is used for detection. CIoU Loss is used as the loss function for target regression. Mixed precision training is also employed to reduce the model size.

[0083] In the trajectory analysis module, the loitering detection device first needs to extract pedestrian motion information and appearance features from the pedestrian detection images. During motion information extraction, due to the use of multiple cameras working in tandem, the pedestrian detection data from the auxiliary cameras needs to be processed to convert their location information to the corresponding location on the main camera. If a pedestrian appears in both the main and auxiliary cameras at the same time, the image with the better image evaluation quality is selected for appearance feature extraction.

[0084] Then, the loitering detection device linearly fuses the motion information and appearance feature measurements as the first measurement, and uses this result as input to analyze the data using the Hungarian algorithm to obtain the association results and update the trajectory. Next, for pedestrians and trajectories that fail to match, the Intersection over Union (IoU) is measured, and the Hungarian algorithm is used again for a second data matching to increase the number of successful matches and improve tracking accuracy. It should be noted that the second data matching focuses on the Intersection over Union of motion information and the detection bounding boxes of the appearance features; the matching accuracy is slightly lower than that of the first data matching, but it effectively increases the matching success rate.

[0085] Step S14: Based on the trajectory path, determine whether the number of times the pedestrian travels back and forth is greater than the preset number.

[0086] In this embodiment, the loitering detection device determines the pedestrian's direction of travel based on the trajectory path. If the direction of travel changes, such as turning back, the number of round trips is counted. Within a preset time period, the number of round trips is counted to determine whether the number of round trips is greater than the preset number. If it is greater, it is confirmed that the pedestrian is loitering and the process proceeds to step S15; if it is less, it is confirmed that the pedestrian is not loitering.

[0087] Specifically, to improve the accuracy of statistical data on pedestrian back-and-forth trips, this application provides a specific calculation method for determining loitering. Please refer to [link / reference] for details. Figure 3 , Figure 3 yes Figure 1 The diagram shows the specific flow chart of step S14 in the loitering detection method.

[0088] The loitering detection device maps the trajectory information of the same pedestrian to the first quadrant. It calculates the difference in slope between adjacent trajectory points and performs a binary mapping of the difference to 1 and -1. It iterates through and counts the number of consecutive 1s or consecutive -1s within a trajectory segment, then merges the statistical results. If the number of slope changes within a uniform time period exceeds a threshold, it is determined to be loitering; if it is less than or equal to the threshold, it is determined that no loitering behavior has occurred.

[0089] Specifically, such as Figure 3 As shown, the loitering detection method in this application specifically includes the following steps:

[0090] Step S141: Based on the trajectory path, obtain the direction of movement of the pedestrian at each trajectory point.

[0091] In this embodiment of the application, the loitering detection device maps the pedestrian trajectory to the first quadrant of the coordinate axis, wherein the origin of the coordinate axis is determined by the coordinate point of the lower left corner of the fused image.

[0092] When pedestrians move in the same direction, any point p on their trajectory curve i =(x i ,y i The slope of the line connecting the origin C = (x0, y0) to the origin follows the same direction, where i ranges from 0 to N. The change in slope reflects the change in the angle between the trajectory point and the x-axis. When a pedestrian moves in the same direction, the slope changes at each trajectory point, as shown below. Figure 4 As shown.

[0093] The slope calculation formula of this application is as follows:

[0094] Since the minimum bounding rectangle is used to label the position of pedestrians in pedestrian recognition, in the embodiments of this application, the center point of the rectangle is used as the generation point of the trajectory curve, reducing the problem of abnormal trajectory points caused by the deformation of the rectangle.

[0095] The wandering detection device calculates the slope for each trajectory point in the trajectory, calculates the difference between the slopes of adjacent trajectory points, and performs binary mapping on the difference. When the slope difference is k i > 0, its value at the corresponding position is 1, indicating that at this trajectory point, the pedestrian walks in the same moving direction. When k i < 0, its value at the corresponding position is -1, indicating that at this trajectory point, the pedestrian walks in the opposite moving direction.

[0096] Step S142: Obtain the number of round trips of the pedestrian based on the number of times the moving direction of the pedestrian changes per unit time.

[0097] In the embodiments of this application, the wandering detection device combines consecutive 1s and -1s, records the number of discontinuities per unit time, that is, the value combinations of "1, -1" and "-1, 1", as the number of round trips of the pedestrian.

[0098] Step S15: Confirm that the pedestrian exhibits a wandering behavior.

[0099] In the embodiments of this application, when the number of discontinuities Ir in the trajectory path of the pedestrian is greater than the threshold p, it is determined that the pedestrian exhibits a wandering behavior.

[0100] In the embodiments of this application, the wandering detection device acquires the first detection image of the first camera and the second detection image of the second camera collected at the same moment; based on the transformation relationship between the first camera and the second camera, transforms the second detection image into the coordinate system of the first detection image to obtain a fused image of the first detection image and the second detection image; detects the trajectory path of the pedestrian in the fused image; based on the trajectory path, determines whether the number of round trips of the pedestrian is greater than a preset number; if so, confirms that the pedestrian exhibits a wandering behavior. By the above method, the wandering detection device realizes wide-range wandering detection through multi-camera stitching and improves the recognition rate of personnel wandering in key places through multi-camera linked image capture.

[0101] Please continue to refer to Figure 2 the camera image acquisition part shown in Figure 5 , Figure 5 which is a schematic flowchart of another embodiment of the wandering detection method provided by this application.

[0102] Specifically, as Figure 5 shown, the wandering detection method of the embodiments of this application specifically includes the following steps:

[0103] Step S21: Based on the trajectory path, determine whether the number of times the pedestrian makes round trips is greater than half of the preset number.

[0104] In this embodiment of the application, in order to reduce the amount of data processed, tripwire intrusion is used to capture images of multiple cameras in a coordinated manner.

[0105] Step S22: Confirm that the pedestrian is exhibiting suspected loitering behavior.

[0106] Step S23: Control the first camera and / or the second camera to acquire images at a fast capture speed, wherein the fast capture speed is higher than the normal capture speed.

[0107] In this embodiment of the application, when no suspected loitering person is detected, the main and auxiliary cameras operate at a slower rate v. low When capturing images, if the main / secondary camera detects suspected loitering behavior, the main / secondary camera will use a faster shutter speed (v). high Take a screenshot.

[0108] It should be noted that if a pedestrian stays in the designated area for more than t1 or the number of discontinuous pedestrians Ir is greater than t1, then the pedestrian is considered to have stayed in the designated area for more than t1. At that time, it was determined to be suspected loitering behavior.

[0109] Furthermore, regarding the specific location of a pedestrian when they trigger suspected loitering behavior, this application also provides the following camera acquisition speed design scheme:

[0110] If a suspected loitering person appears in the main camera's field of view (excluding the boundary area between the main and auxiliary cameras), the main camera will move at a speed of v. high Capture images, with the auxiliary camera at speed v low Capture the image. Use homography transformation on the boundary of the auxiliary camera to obtain the one-way tripwire intrusion rule line rule1 in the main camera (from the main camera to the auxiliary camera).

[0111] When a suspected loitering individual triggers a tripwire intrusion in the main camera, the secondary camera begins using speed v. high Capture an image. Apply homography transformation to the boundary of the main camera to obtain the one-way tripwire intrusion rule line rulw2 (from the main camera to the auxiliary camera) in the auxiliary camera. When a suspected loitering person triggers a tripwire intrusion in the auxiliary camera, the main camera resumes its operating speed v. low Take a screenshot.

[0112] If a suspected loitering person appears in the secondary camera's field of view (excluding the boundary area between the primary and secondary cameras), the secondary camera will move at a speed of v. high To capture an image, the main camera moves at speed v. low Capture the image. Transform the boundary of the main camera to the single tripwire intrusion rule line rule3 in the auxiliary camera (from the auxiliary camera towards the main camera).

[0113] When a suspected loitering individual triggers a tripwire intrusion in the secondary camera, the main camera uses speed v. high Capture an image. Use homography transformation on the boundary of the auxiliary camera to obtain the one-way tripwire intrusion rule line rule4 (from the auxiliary camera to the main camera) in the main camera. When a suspected loitering person triggers a tripwire intrusion in the main camera, the auxiliary camera resumes capturing the image at its normal speed. low .

[0114] If a suspected loitering individual appears in the area where the main and auxiliary cameras meet, both the main and auxiliary cameras should use speed v. high Take a screenshot.

[0115] In this embodiment, the loitering detection device uses homography rigid body transformation spatial sampling technology for image matching, thereby determining loitering through multi-camera linkage. This enables identification even in situations with a large loitering range, achieving loitering detection over a wider area. The loitering detection device judges loitering by collecting pedestrian trajectory data and utilizing changes in the slope of the data. The device also achieves multi-camera linkage image capture through tripwire intrusion, reducing the amount of data processed.

[0116] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0117] To implement the loitering detection method of the above embodiments, this application also proposes a loitering detection device, please refer to the details below. Figure 6 , Figure 6 This is a schematic diagram of an embodiment of the loitering detection device provided in this application.

[0118] The loitering detection device 300 of this application embodiment includes a memory 31 and a processor 32, wherein the memory 31 and the processor 32 are coupled together.

[0119] The memory 31 is used to store program data, and the processor 32 is used to execute the program data to implement the loitering detection method described in the above embodiments.

[0120] In this embodiment, processor 32 can also be referred to as a CPU (Central Processing Unit). Processor 32 may be an integrated circuit chip with signal processing capabilities. Processor 32 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 32 can be any conventional processor.

[0121] To implement the loitering detection method of the above embodiments, this application also provides a computer-readable storage medium, such as... Figure 7 As shown, the computer-readable storage medium 400 is used to store program data 41, which, when executed by a processor, is used to implement the loitering detection method as described in the above embodiments.

[0122] This application also provides a computer program product, wherein the computer program product includes a computer program operable to cause a computer to perform the loitering detection method as described in the embodiments of this application. The computer program product may be a software installation package.

[0123] The loitering detection method described in the above embodiments of this application, when implemented as a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0124] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for detecting loitering, characterized in that, The loitering detection method includes: Acquire the first detection image from the first camera and the second detection image from the second camera, both captured at the same time. Based on the transformation relationship between the first camera and the second camera, the second detected image is transformed to the coordinate system of the first detected image to obtain a fused image of the first detected image and the second detected image. The trajectory path of pedestrians is detected in the fused image; Based on the trajectory path, determine whether the number of times the pedestrian travels back and forth is greater than a preset number; If so, then it is confirmed that the pedestrian is exhibiting loitering behavior; The process of detecting the pedestrian's trajectory path in the fused image includes: Extract the pedestrian's motion information and appearance features from the multi-frame fused image; The motion information and appearance features of pedestrians in each frame of the fused image are linearly fused, and the linear fusion result is used as the first measurement information. The first metric information of the multi-frame fused image is correlated to obtain the first correlation result; The intersection-union ratio (CUIR) of the detection boxes for motion information that failed to match and the detection boxes for appearance features is obtained, and the CUIR is used as a second metric. The second set of metrics that failed to match is correlated to obtain the second correlation result; Based on the first association result and the second association result, the pedestrian's trajectory path is generated; The step of associating the second set of metrics that failed to match, and obtaining the second association result, includes: After the first measurement association, for cases where the match fails, the intersection-union ratio is used for a second measurement, and the Hungarian algorithm is used again for data association to obtain the second association result.

2. The loitering detection method according to claim 1, characterized in that, Extracting the pedestrian's motion information and appearance features from the multi-frame fused image includes: In the fused image, if the pedestrian appears in both the first detection image and the transformed second detection image, the first image quality of the first detection image and the second image quality of the second detection image are obtained. When the quality of the first image is higher than that of the second image, the motion information and appearance features of the pedestrian are extracted from the first detected image; When the quality of the first image is lower than that of the second image, the motion information and appearance features of the pedestrian are extracted from the second detected image.

3. The loitering detection method according to claim 1, characterized in that, The loitering detection method further includes: Based on the trajectory path, the direction of movement of the pedestrian at each trajectory point is obtained; The number of times the pedestrian changes direction per unit time is used to obtain the number of round trips.

4. The loitering detection method according to claim 3, characterized in that, The step of obtaining the pedestrian's movement direction at each trajectory point based on the trajectory path includes: Map the trajectory path to the first quadrant of the fused image coordinate axes; Obtain the slope of each trajectory point relative to the origin; The direction of motion of each trajectory point is determined based on the slope difference between adjacent trajectory points.

5. The loitering detection method according to claim 4, characterized in that, Determining the direction of motion for each trajectory point based on the slope difference between adjacent trajectory points includes: Get the slope difference between the current trajectory point and its adjacent trajectory points; When the slope difference is greater than 0, the motion direction of the current trajectory point is set to a first preset value; When the slope difference is less than 0, the motion direction of the current trajectory point is set to a second preset value; The step of obtaining the number of round trips of the pedestrian based on the number of times the pedestrian changes direction of movement per unit time includes: Obtain the motion direction values ​​of all trajectory points along the pedestrian's trajectory path; The number of times the combination of movement direction values ​​appears in all the trajectory points is determined as the number of round trips for the pedestrian. The combination of motion direction values ​​consists of adjacent motion direction values ​​being a first preset value and a second preset value.

6. The loitering detection method according to claim 1, characterized in that, After detecting the pedestrian's trajectory path in the fused image, the loitering detection method further includes: Based on the trajectory path, determine whether the number of times the pedestrian makes round trips is greater than half of a preset number; If so, then it is confirmed that the pedestrian exhibited suspected loitering behavior; Control the first camera and / or the second camera to acquire images at a fast capture speed, wherein the fast capture speed is higher than the normal capture speed.

7. The loitering detection method according to claim 6, characterized in that, The control of the first camera and / or the second camera to acquire images at a fast capture speed includes: When the suspected loitering behavior appears in the first detection image, the first camera is controlled to capture images at a fast capture speed, and the second camera is controlled to capture images at a normal capture speed. When the suspected loitering behavior appears in the second detection image, the first camera is controlled to capture images at a normal capture speed, and the second camera is controlled to capture images at a fast capture speed. When the suspected loitering behavior appears in the boundary area between the first detection image and the transformed second detection image in the fused image, the first camera is controlled to acquire images at a fast capture speed, and the second camera is controlled to acquire images at a fast capture speed.

8. The loitering detection method according to claim 7, characterized in that, After controlling the first camera to acquire images at a fast capture speed and the second camera to acquire images at a normal capture speed, the loitering detection method further includes: The boundary of the second camera is transformed into the one-way tripwire intrusion rule line of the first camera according to the transformation relationship; When a pedestrian suspected of loitering moves to the one-way tripwire intrusion rule line, the second camera is controlled to capture images at a fast capture speed.

9. A loitering detection device, characterized in that, The loitering detection device includes a processor and a memory, the memory storing program data, and the processor executing the program data to implement the loitering detection method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program data, which, when executed by a processor, is used to implement the loitering detection method according to any one of claims 1-8.

Images