Multi-target object wandering detection method, system, medium and electronic device

Abstract

The application discloses a loitering detection method, system, medium and electronic equipment for multiple target objects, and the method comprises the following steps: receiving an image to be processed from a camera; when the image contains at least one target object, extracting all face frames and body frames in the image to obtain a face cut image sequence and a body cut image sequence; determining whether each target object meets a loitering condition according to the face cut image sequence and the body cut image sequence; and warning the target object meeting the loitering condition. When a large number of target objects are detected, all face frames and body frames in the image can be detected and extracted, the face cut image sequence and the body cut image sequence are obtained, and loitering detection is performed through the sequence, so that the tracking failure caused by incomplete body shielding can be prevented, the re-identification false detection rate of the target object entering and exiting the monitoring picture multiple times is low, the deduplication effect is good, the loitering behavior of multiple target objects in the image can be accurately identified, and the accuracy of tracking detection is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, system, medium, and electronic device for detecting loitering of multiple targets. Background Technology

[0002] In related technologies, to promptly detect loitering individuals, the common approach is to use cameras in key areas to capture real-time images of the individuals' activities and input them into a model for analysis. This method can achieve tracking and detection when there is only one target in the image. However, when there are multiple targets in the image, occlusion between them can occur, leading to misidentification by the model if a complete human body cannot be detected, thus reducing the accuracy of tracking and detection. Furthermore, the false detection rate is high for individuals who repeatedly enter and exit the monitoring screen. Summary of the Invention

[0003] This application provides a method, system, medium, and electronic device for detecting loitering of multiple targets. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general description, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0004] In a first aspect, embodiments of this application provide a method for detecting loitering of multiple target objects, the method comprising:

[0005] It receives images of the target monitoring area from the camera in real time and is ready for processing.

[0006] When the image to be processed is detected to contain at least one target object, the regions of all face bounding boxes and body bounding boxes in the image to be processed are extracted to obtain face cropping sequences and body cropping sequences.

[0007] Based on the face image sequence and the human body image sequence, determine whether each target object meets the hovering condition;

[0008] Issue a warning to target objects that meet the loitering criteria.

[0009] Optionally, based on the face image sequence and the body image sequence, determine whether each target object meets the loitering condition, including:

[0010] Pair and associate the face slice sequence and the human body slice sequence to obtain the target pairing relationship of each target object;

[0011] Based on the target pairing relationship of each target object, calculate the travel distance and dwell time of each target object in the target monitoring area;

[0012] Based on the distance traveled and the time spent in the area, determine whether each target object meets the loitering condition.

[0013] Optionally, the face image sequence and the body image sequence are paired and associated to obtain the target pairing relationship for each target object, including:

[0014] Calculate the target distance between the center point coordinates of each face slice in the face slice sequence and the center point coordinates of each human body slice in the human body slice sequence;

[0015] Determine the target human body slice with the minimum target distance corresponding to each face slice;

[0016] Determine the scores for each face slice and the target human body slice, and multiply the scores of each face slice and the target human body slice with the preset weights to obtain the total score;

[0017] When the total score is greater than the preset threshold, the sequence number of each face slice and the target human body slice is associated to obtain the target pairing relationship of each target object.

[0018] Optionally, based on the target pairing relationship of each target object, calculate the travel distance and dwell time of each target object in the target monitoring area, including:

[0019] Set a pairing ID for the target pairing relationship of each target object;

[0020] AI target tracking technology is used to track and identify each target object and determine the tracking ID of each target object;

[0021] The tracking IDs of each target object are corrected by using AI pedestrian re-identification technology to obtain the corrected tracking IDs of each target object;

[0022] The correction tracking ID of each target object is associated with the paired ID of the target object to obtain the ID mapping relationship of each target object;

[0023] Based on the ID mapping relationship, calculate the travel distance and dwell time of each target object in the target monitoring area.

[0024] Optionally, based on the ID mapping relationship, calculate the travel distance and dwell time of each target object in the target monitoring area, including:

[0025] When the number of images to be processed received from the camera exceeds a preset number, the actual tracking ID of each target object in each image to be processed is determined.

[0026] In the ID mapping relationship, determine the actual paired ID corresponding to the actual tracking ID;

[0027] Based on the pairing relationship to which the actual pairing ID belongs, the movement distance and displacement of each target object are calculated to obtain the travel distance of each target object in the target monitoring area;

[0028] The length of each target object's stay on the monitoring screen is taken as the dwell time of each target object.

[0029] Optionally, based on the distance traveled and the time spent in the area, determine whether each target object meets the loitering criteria, including:

[0030] The total appearance rate of each target object is obtained by dividing the total dwell time of each target object by the number of dwell times.

[0031] Calculate the absolute value of the difference between the travel distance and displacement of each target object;

[0032] If the absolute value is greater than the preset absolute value, the total occurrence ratio of each target object is greater than the preset ratio, and the dwell time of each target object is greater than the preset duration, determine whether each target object meets the wandering condition.

[0033] Optionally, determine the scores for each face slice and the target human body slice, including:

[0034] Obtain the first image feature parameters corresponding to each face slice; the first image feature parameters are the image feature parameters of all face slices in the existing database for each face slice;

[0035] Obtain the second image feature parameters corresponding to the target human body slice image; the second image feature parameters are the image feature parameters of all human body slice images in the existing database of each target human body slice image.

[0036] Based on the first image feature parameters and the second image feature parameters, calculate the similarity between each face slice and the first image feature parameters, and the similarity between the target human body slice and the second image feature parameters;

[0037] Based on the similarity, a score is determined for each face slice and the target human body slice.

[0038] Secondly, embodiments of this application provide a loitering detection system for multiple targets, the system comprising:

[0039] The image receiving module is used to receive images to be processed from the camera capturing the target monitoring area in real time.

[0040] The region extraction module is used to extract the regions of all face bounding boxes and body bounding boxes in the image to be processed when it is detected that the image to be processed contains at least one target object, so as to obtain face cropping sequence and body cropping sequence.

[0041] The loitering judgment module is used to determine whether each target object meets the loitering condition based on the face slice sequence and the human body slice sequence;

[0042] The warning module is used to issue warnings to target objects that meet the lingering conditions.

[0043] Thirdly, embodiments of this application provide a computer storage medium storing multiple instructions adapted for loading and execution of the above-described method steps by a processor.

[0044] Fourthly, embodiments of this application provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed by the above-described method steps.

[0045] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0046] In this embodiment, the multi-target object loitering detection system first receives images of the target monitoring area captured by a camera in real time. Then, when the image contains at least one target object, it extracts the regions of all face and body bounding boxes in the image to obtain face and body image sequences. Next, based on the face and body image sequences, it determines whether each target object meets the loitering condition. Finally, it issues a warning to target objects that meet the loitering condition. This application detects and extracts the regions of all face and body bounding boxes in the image separately when there are many target objects, obtaining face and body image sequences. Using these sequences for loitering detection prevents tracking failures due to incomplete human occlusion. Furthermore, it has a low false positive rate for re-identifying target objects that repeatedly enter and exit the monitoring screen, resulting in good deduplication. It can intelligently and accurately identify the loitering behavior of multiple target objects in the image, thereby improving the accuracy of tracking and detection.

[0047] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] Figure 1 This is a flowchart illustrating a method for detecting loitering of multiple targets provided in an embodiment of this application;

[0050] Figure 2 This is a flowchart illustrating a target object loitering behavior recognition process provided in an embodiment of this application;

[0051] Figure 3 This is a schematic block diagram of a multi-target object loitering detection process provided in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the structure of a loitering detection system for multiple targets provided in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them.

[0055] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0056] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with some aspects of this application as detailed in the appended claims.

[0057] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0058] This application provides a method, system, medium, and electronic device for detecting loitering of multiple targets, in order to solve the problems existing in the aforementioned related technologies. In the technical solution provided by this application, when a large number of target objects are detected, the application separately detects and extracts the regions of all face bounding boxes and body bounding boxes present in the image, obtaining face and body cropping sequences. Using these sequences for loitering detection can prevent tracking failures due to incomplete occlusion of the human body. Furthermore, it has a low false detection rate for re-identification of target objects that repeatedly enter and exit the monitoring screen, and has good deduplication effects. It can intelligently and accurately identify the loitering behavior of multiple target objects in the image, thereby improving the accuracy of tracking and detection. The following is a detailed description using exemplary embodiments.

[0059] The following will be combined with the appendix Figure 1 -Appendix Figure 3 This application provides a detailed description of the loitering detection method for multiple targets provided in its embodiments. This method can be implemented using a computer program and can run on a multi-target loitering detection system based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application.

[0060] Please see Figure 1 This is a flowchart illustrating a method for detecting loitering of multiple targets, as provided in an embodiment of this application. Figure 1 As shown, the method in this application embodiment may include the following steps:

[0061] S101 receives images of the target monitoring area collected by the camera in real time;

[0062] The target monitoring area is the region where multiple target objects need to be detected loitering. The images to be processed are image frames captured in real time by the camera within the target monitoring area.

[0063] In this embodiment of the application, when performing loitering detection of multiple target objects in the target monitoring area, the camera first captures images of the target monitoring area in real time and sends them to the server via wired or wireless means. Then, the server receives the image frames captured in real time from the camera in the target monitoring area and obtains the images to be processed.

[0064] S102, when it is detected that the image to be processed contains at least one target object, extract the regions of all face frames and body frames in the image to be processed to obtain face image sequence and body image sequence;

[0065] The target object can be a pedestrian, or it can be a target that meets the preset characteristics. The image to be processed may not contain a target object, or it may contain one target object, or it may contain multiple target objects.

[0066] In this embodiment of the application, it is necessary to determine whether the image to be processed contains a target object. If no target object is found, the step of receiving the image to be processed from the camera in real time is continued until the image to be processed is detected to contain a target object.

[0067] Furthermore, in the presence of a target object, existing technologies can be used to directly track and detect the target object, or the regions of all face and body bounding boxes in the image to be processed can be extracted using this application to obtain face and body image sequences.

[0068] Furthermore, in the case of multiple target objects, it is necessary to extract the regions of all face bounding boxes and body bounding boxes in the image to be processed to obtain face cutout sequences and body cutout sequences.

[0069] Specifically, the AI ​​face recognition technology (Insightface) detects and extracts the faces of target objects in the image to be processed, obtaining face slices. These face slices can be saved to a dynamic pedestrian database, which currently stores up to 12 face slices from different angles for each ID. Specifically, the AI ​​face recognition technology (Insightface) using the ArcFace loss function detects and extracts the faces of target objects in the image to be processed; for example, the face detection function can obtain the face bounding box coordinates (face_bbox), face pose (face_pose), and face features (face_embedding) for each face in the image to be processed.

[0070] Specifically, AI human body recognition technology (YOLOv5s) is used to detect and extract human bodies from the target objects in the image to be processed, resulting in human body slices. These slices can be saved to a dynamic pedestrian database, which currently stores up to 12 human body slices from different angles for each ID. The AI ​​human body recognition technology (YOLOv5s) is used to detect human bodies in the image to be processed; for example, the human body detection function can obtain human body attributes such as the bounding box coordinates (person_bbox) for each person in the image.

[0071] S103, Based on the face image sequence and the human body image sequence, determine whether each target object meets the hovering condition;

[0072] In this embodiment of the application, when determining whether each target object meets the loitering condition based on the face image sequence and the body image sequence, the face image sequence and the body image sequence are first paired and associated to obtain the target pairing relationship of each target object; then, based on the target pairing relationship of each target object, the travel distance and dwell time of each target object in the target monitoring area are calculated; finally, based on the travel distance and dwell time, it is determined whether each target object meets the loitering condition.

[0073] In this embodiment, when pairing and associating face image sequences and body image sequences to obtain target pairing relationships for each target object, the target distance between the center point coordinates of each face image in the face image sequence and the center point coordinates of each body image in the body image sequence is first calculated; then, the target body image with the minimum target distance corresponding to each face image is determined; then, the scores of each face image and the target body image are determined, and the scores of each face image and the target body image are multiplied by a preset weight to obtain a total score; finally, when the total score is greater than a preset threshold, the sequence numbers of each face image and the target body image are associated to obtain the target pairing relationship for each target object.

[0074] Specifically, during the pairing process, the center coordinates of the face image corresponding to the detected face bounding box (face_bbox) are first calculated; then, the center coordinates of the human image corresponding to the detected human bounding box (person_bbox) are calculated; next, the center coordinates of the human image closest to the center coordinates of the face image are found, and the two are paired; then, the scores for the face and the human are given separately and multiplied by their respective weights (the sum of the face weight and the human weight is 1; the weights can be flexibly adjusted online to adapt to the size of the face or human being captured by the camera; for example, if the captured face is small, the face weight is reduced, and the corresponding human weight is increased to increase the overall recognition rate of the person, thus obtaining the total score of the target object; finally, based on this score, it can be determined whether the two are the best match.

[0075] For example, the face weight is 0.3 and the body weight is 0.7. The total score of the target person is obtained by multiplying the face score by the face weight and adding the body score by the body weight. If the score is greater than the given maximum score value, the face and body are considered to be the best match and match a certain ID in the existing dynamic pedestrian database. The corresponding data is then stored in the dynamic pedestrian database.

[0076] In this embodiment, when calculating the travel distance and dwell time of each target object in the target monitoring area based on the target pairing relationship of each target object, firstly, a pairing ID is set for the target pairing relationship of each target object; then, each target object is tracked and identified using AI target tracking technology, and the tracking ID of each target object is determined; next, the tracking ID of each target object is corrected using AI pedestrian re-identification technology to obtain the corrected tracking ID of each target object; then, the corrected tracking ID of each target object is associated with the pairing ID of the target object to obtain the ID mapping relationship of each target object; finally, the travel distance and dwell time of each target object in the target monitoring area are calculated according to the ID mapping relationship.

[0077] Specifically, the target object is tracked and identified using AI target tracking technology (DeepSORT) to determine its tracking ID. The target object's tracking ID is then corrected using AI pedestrian re-identification technology (FastReID). The corrected tracking ID is then associated with its paired ID and stored in a dynamic pedestrian database.

[0078] In this embodiment of the application, when calculating the travel distance and dwell time of each target object in the target monitoring area according to the ID mapping relationship, firstly, when the number of images to be processed received from the camera is greater than a preset number, the actual tracking ID of each target object in each image to be processed is determined; then, in the ID mapping relationship, the actual pairing ID corresponding to the actual tracking ID is determined; secondly, based on the pairing relationship to which the actual pairing ID belongs, the movement distance and displacement of each target object are calculated to obtain the travel distance of each target object in the target monitoring area; finally, the dwell time of each target object in the monitoring screen is taken as the dwell time of each target object.

[0079] Specifically, when determining whether each target object meets the loitering condition based on the travel distance and dwell time, the total occurrence ratio of each target object is first obtained by dividing the total dwell time of each target object by the number of dwell times; then, the absolute value of the difference between the travel distance and displacement of each target object is calculated; finally, if the absolute value is greater than the preset absolute value, the total occurrence ratio of each target object is greater than the preset ratio, and the dwell time of each target object is greater than the preset duration, it is determined whether each target object meets the loitering condition.

[0080] For example Figure 2 As shown, Figure 2 This is a flowchart illustrating a target object loitering behavior recognition process provided in this application. The processing flow is as follows:

[0081] Step 1: Perform trajectory determination every 30 frames;

[0082] Step 2: Calculate the distance and displacement of the target person.

[0083] Step 3: Calculate the cumulative dwell time (appear_time) and total appearance ratio (appear_ratio) of the target personnel;

[0084] Step 4: If the absolute value of the difference between distance and displacement exceeds threshold1, and the total appearance ratio of the target personnel is greater than threshold2, and the cumulative stay time of the target personnel is greater than threshold3, then a warning is issued.

[0085] Step 5: If five consecutive warnings occur, start saving and drawing the motion trajectory, and start correcting the motion trajectory every 30 frames.

[0086] Furthermore, when determining the scores for each face slice and the target human body slice, the first image feature parameter corresponding to each face slice is first obtained; the first image feature parameter is the image feature parameter of all face slices in the existing database of the face slices; then the second image feature parameter corresponding to the target human body slice is obtained; the second image feature parameter is the image feature parameter of all human body slices in the existing database of the target human body slices; next, based on the first image feature parameter and the second image feature parameter, the similarity between each face slice and the first image feature parameter and the similarity between the target human body slice and the second image feature parameter are calculated; finally, based on the similarity, the scores for each face slice and the target human body slice are determined.

[0087] S104, issue a warning to the target object that meets the lingering condition.

[0088] In this embodiment of the application, a warning can be issued once to a target object that meets the loitering condition. When the number of warnings exceeds the preset number of warnings, a severe warning is issued to the target object, ordering it to leave the target monitoring area as soon as possible. Considering that relevant personnel will be arranged to handle the warning immediately after it is issued, and if there is no response, it may be accepted by default, the warning will last for 90 frames and will disappear automatically after 90 frames.

[0089] For example Figure 3 As shown, Figure 3 This is a schematic block diagram of a multi-target object loitering detection process provided in this application. The process includes the following steps:

[0090] S1, real-time acquisition of images of target personnel (pedestrians);

[0091] S2 uses AI facial recognition technology (Insightface) to identify the face of the target person in the detected image;

[0092] S3 uses AI human body recognition technology (YOLOv5s) to identify the human body of the target person in the image;

[0093] S4, calculate the center point coordinates of the detected face bounding boxes and the center point coordinates of the detected human bounding boxes, and find the closest pair of face and human body bounding boxes;

[0094] S5 uses AI target tracking technology (DeepSORT) to track and identify target personnel;

[0095] S6 uses AI pedestrian re-identification technology (FastReID) to perform target tracking and correction;

[0096] S7, through pairing relationships, tracks and calculates the target person's travel distance and dwell time in the current frame to determine the target person's loitering characteristics;

[0097] S8 issues an alert or warning to individuals loitering near the target location.

[0098] In this embodiment, the multi-target object loitering detection system first receives images of the target monitoring area captured by a camera in real time. Then, when the image contains at least one target object, it extracts the regions of all face and body bounding boxes in the image to obtain face and body image sequences. Next, based on the face and body image sequences, it determines whether each target object meets the loitering condition. Finally, it issues a warning to target objects that meet the loitering condition. This application detects and extracts the regions of all face and body bounding boxes in the image separately when there are many target objects, obtaining face and body image sequences. Using these sequences for loitering detection prevents tracking failures due to incomplete human occlusion. Furthermore, it has a low false positive rate for re-identifying target objects that repeatedly enter and exit the monitoring screen, resulting in good deduplication. It can intelligently and accurately identify the loitering behavior of multiple target objects in the image, thereby improving the accuracy of tracking and detection.

[0099] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0100] Please see Figure 4 This illustration shows a schematic diagram of a loitering detection system for multiple targets provided in an exemplary embodiment of this application. This loitering detection system can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The system 1 includes an image receiving module 10, a region extraction module 20, a loitering judgment module 30, and a warning module 40.

[0101] Image receiving module 10 is used to receive images to be processed from the camera capturing the target monitoring area in real time;

[0102] The region extraction module 20 is used to extract the regions of all face frames and body frames in the image to be processed when it is detected that the image to be processed contains at least one target object, so as to obtain face image sequence and body image sequence.

[0103] The loitering judgment module 30 is used to determine whether each target object meets the loitering condition based on the face image sequence and the human body image sequence.

[0104] Warning module 40 is used to issue warnings to target objects that meet the lingering conditions.

[0105] It should be noted that the above embodiments of the multi-target object loitering detection system are only illustrated by the division of the functional modules described above when executing the multi-target object loitering detection method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the multi-target object loitering detection system and the multi-target object loitering detection method embodiments provided above belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0106] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0107] In this embodiment, the multi-target object loitering detection system first receives images of the target monitoring area captured by a camera in real time. Then, when the image contains at least one target object, it extracts the regions of all face and body bounding boxes in the image to obtain face and body image sequences. Next, based on the face and body image sequences, it determines whether each target object meets the loitering condition. Finally, it issues a warning to target objects that meet the loitering condition. This application detects and extracts the regions of all face and body bounding boxes in the image separately when there are many target objects, obtaining face and body image sequences. Using these sequences for loitering detection prevents tracking failures due to incomplete human occlusion. Furthermore, it has a low false positive rate for re-identifying target objects that repeatedly enter and exit the monitoring screen, resulting in good deduplication. It can intelligently and accurately identify the loitering behavior of multiple target objects in the image, thereby improving the accuracy of tracking and detection.

[0108] This application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the multi-target object loitering detection method provided in the above-described method embodiments.

[0109] This application also provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the multi-target object loitering detection method of the various method embodiments described above.

[0110] Please see Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0111] The communication bus 1002 is used to realize the connection and communication between these components.

[0112] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0113] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0114] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the electronic device 1000 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip, without being integrated into the processor 1001.

[0115] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage system located remotely from the aforementioned processor 1001. Figure 5 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a loitering detection application for multiple targets.

[0116] exist Figure 5 In the illustrated electronic device 1000, the user interface 1003 is mainly used to provide an input interface for the user and to acquire user input data; while the processor 1001 can be used to call the multi-target object loitering detection application stored in the memory 1005 and specifically perform the following operations:

[0117] It receives images of the target monitoring area from the camera in real time and is ready for processing.

[0118] When the image to be processed is detected to contain at least one target object, the regions of all face bounding boxes and body bounding boxes in the image to be processed are extracted to obtain face cropping sequences and body cropping sequences.

[0119] Based on the face image sequence and the human body image sequence, determine whether each target object meets the hovering condition;

[0120] Issue a warning to target objects that meet the loitering criteria.

[0121] In one embodiment, when the processor 1001 determines whether each target object meets the hovering condition based on the face image sequence and the human body image sequence, it specifically performs the following operations:

[0122] Pair and associate the face slice sequence and the human body slice sequence to obtain the target pairing relationship of each target object;

[0123] Based on the target pairing relationship of each target object, calculate the travel distance and dwell time of each target object in the target monitoring area;

[0124] Based on the distance traveled and the time spent in the area, determine whether each target object meets the loitering condition.

[0125] In one embodiment, when processor 1001 performs pairing and association of face slice sequences and body slice sequences to obtain target pairing relationships for each target object, it specifically performs the following operations:

[0126] Calculate the target distance between the center point coordinates of each face slice in the face slice sequence and the center point coordinates of each human body slice in the human body slice sequence;

[0127] Determine the target human body slice with the minimum target distance corresponding to each face slice;

[0128] Determine the scores for each face slice and the target human body slice, and multiply the scores of each face slice and the target human body slice with the preset weights to obtain the total score;

[0129] When the total score is greater than the preset threshold, the sequence number of each face slice and the target human body slice is associated to obtain the target pairing relationship of each target object.

[0130] In one embodiment, when the processor 1001 calculates the travel distance and dwell time of each target object in the target monitoring area based on the target pairing relationship of each target object, it specifically performs the following operations:

[0131] Set a pairing ID for the target pairing relationship of each target object;

[0132] AI target tracking technology is used to track and identify each target object and determine the tracking ID of each target object;

[0133] The tracking IDs of each target object are corrected by using AI pedestrian re-identification technology to obtain the corrected tracking IDs of each target object;

[0134] The correction tracking ID of each target object is associated with the paired ID of the target object to obtain the ID mapping relationship of each target object;

[0135] Based on the ID mapping relationship, calculate the travel distance and dwell time of each target object in the target monitoring area.

[0136] In one embodiment, when the processor 1001 calculates the travel distance and dwell time of each target object in the target monitoring area based on the ID mapping relationship, it specifically performs the following operations:

[0137] When the number of images to be processed received from the camera exceeds a preset number, the actual tracking ID of each target object in each image to be processed is determined.

[0138] In the ID mapping relationship, determine the actual paired ID corresponding to the actual tracking ID;

[0139] Based on the pairing relationship to which the actual pairing ID belongs, the movement distance and displacement of each target object are calculated to obtain the travel distance of each target object in the target monitoring area;

[0140] The length of each target object's stay on the monitoring screen is taken as the dwell time of each target object.

[0141] In one embodiment, when the processor 1001 determines whether each target object meets the loitering condition based on the travel distance and dwell time, it specifically performs the following operations:

[0142] The total appearance rate of each target object is obtained by dividing the total dwell time of each target object by the number of dwell times.

[0143] Calculate the absolute value of the difference between the travel distance and displacement of each target object;

[0144] If the absolute value is greater than the preset absolute value, the total occurrence ratio of each target object is greater than the preset ratio, and the dwell time of each target object is greater than the preset duration, determine whether each target object meets the wandering condition.

[0145] In one embodiment, when the processor 1001 performs the operation of determining the scores of each face slice and the target human body slice, it specifically performs the following operations:

[0146] Obtain the first image feature parameters corresponding to each face slice; the first image feature parameters are the image feature parameters of all face slices in the existing database for each face slice;

[0147] Obtain the second image feature parameters corresponding to the target human body slice image; the second image feature parameters are the image feature parameters of all human body slice images in the existing database of each target human body slice image.

[0148] Based on the first image feature parameters and the second image feature parameters, calculate the similarity between each face slice and the first image feature parameters, and the similarity between the target human body slice and the second image feature parameters;

[0149] Based on the similarity, a score is determined for each face slice and the target human body slice.

[0150] In this embodiment, the multi-target object loitering detection system first receives images of the target monitoring area captured by a camera in real time. Then, when the image contains at least one target object, it extracts the regions of all face and body bounding boxes in the image to obtain face and body image sequences. Next, based on the face and body image sequences, it determines whether each target object meets the loitering condition. Finally, it issues a warning to target objects that meet the loitering condition. This application detects and extracts the regions of all face and body bounding boxes in the image separately when there are many target objects, obtaining face and body image sequences. Using these sequences for loitering detection prevents tracking failures due to incomplete human occlusion. Furthermore, it has a low false positive rate for re-identifying target objects that repeatedly enter and exit the monitoring screen, resulting in good deduplication. It can intelligently and accurately identify the loitering behavior of multiple target objects in the image, thereby improving the accuracy of tracking and detection.

[0151] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for detecting loitering of multiple targets can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program for detecting loitering of multiple targets can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0152] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for detecting loitering of multiple targets, characterized in that, The method includes: It receives images of the target monitoring area from the camera in real time and is ready for processing. When the image to be processed is detected to contain at least one target object, the regions of all face bounding boxes and body bounding boxes in the image to be processed are extracted to obtain face cropping sequences and body cropping sequences; the face croppings are saved to a dynamic pedestrian database, which can store multiple face croppings from different angles for each ID; the body croppings are also saved to a dynamic pedestrian database, which can store multiple body croppings from different angles for each ID. The face image sequence and the human body image sequence are paired and associated to obtain the target pairing relationship of each target object; based on the target pairing relationship of each target object, the travel distance and dwell time of each target object in the target monitoring area are calculated; based on the travel distance and dwell time, it is determined whether each target object meets the loitering condition. Based on the target pairing relationships of each target object, the travel distance and dwell time of each target object in the target monitoring area are calculated, including: setting a pairing ID for the target pairing relationships of each target object; tracking and identifying each target object using AI target tracking technology and determining the tracking ID of each target object; correcting the tracking ID of each target object using AI pedestrian re-identification technology to obtain the corrected tracking ID of each target object; associating the corrected tracking ID of each target object with the pairing ID of the target object to obtain the ID mapping relationship of each target object; and calculating the travel distance and dwell time of each target object in the target monitoring area according to the ID mapping relationship. A warning is issued to target objects that meet the aforementioned lingering conditions.

2. The method according to claim 1, characterized in that, The process of pairing and associating the face image sequence and the human body image sequence to obtain the target pairing relationship for each target object includes: Calculate the target distance between the center point coordinates of each face slice in the face slice sequence and the center point coordinates of each human body slice in the human body slice sequence; Determine the target human body slice with the minimum target distance corresponding to each face slice; Determine the scores for each face slice and the target human body slice, and multiply the scores of each face slice and the target human body slice with a preset weight to obtain the total score; When the total score is greater than a preset threshold, the sequence number of each face slice is associated with the sequence number of the target human body slice to obtain the target pairing relationship of each target object.

3. The method according to claim 1, characterized in that, The step of calculating the travel distance and dwell time of each target object in the target monitoring area based on the ID mapping relationship includes: When the number of images to be processed received from the camera exceeds a preset number, the actual tracking ID of each target object in each image to be processed is determined. In the ID mapping relationship, determine the actual paired ID corresponding to the actual tracking ID; Based on the pairing relationship to which the actual pairing ID belongs, the movement distance and displacement of each target object are calculated to obtain the travel distance of each target object in the target monitoring area; The length of each target object's stay on the monitoring screen is taken as the dwell time of each target object.

4. The method according to claim 1, characterized in that, The step of determining whether each target object meets the loitering conditions based on the travel distance and dwell time includes: The total occurrence ratio of each target object is obtained by dividing the total dwell time of each target object by the number of dwell times. Calculate the absolute value of the difference between the travel distance and displacement of each target object; If the absolute value is greater than a preset absolute value, the total occurrence ratio of each target object is greater than a preset ratio, and the dwell time of each target object is greater than a preset duration, then it is determined whether each target object meets the loitering condition.

5. The method according to claim 2, characterized in that, The determination of the scores for each face slice and the target human body slice includes: Obtain the first image feature parameters corresponding to each face slice; the first image feature parameters are the image feature parameters of all face slices in the existing database for each face slice; Obtain the second image feature parameters corresponding to the target human body slice image; the second image feature parameters are the image feature parameters of all human body slice images in the existing database of each target human body slice image. Based on the first image feature parameters and the second image feature parameters, calculate the similarity between each face slice and the first image feature parameters, and the similarity between the target human body slice and the second image feature parameters; Based on the similarity, a score is determined for each face slice and the target human body slice.

6. A loitering detection system for multiple targets implemented using the method described in any one of claims 1-5, characterized in that, The system includes: The image receiving module is used to receive images to be processed from the camera capturing the target monitoring area in real time. The region extraction module is used to extract the regions of all face frames and body frames in the image to be processed when it is detected that the image to be processed contains at least one target object, so as to obtain face image sequence and body image sequence. The loitering judgment module is used to determine whether each target object meets the loitering condition based on the face image sequence and the human body image sequence. The warning module is used to issue warnings to target objects that meet the lingering conditions.

7. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the method as described in any one of claims 1-5.

8. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed as described in any one of claims 1-5.

Images