Littering Behavior Recognition Method Based on Key Point Detection and Trajectory Fitting

Abstract

The present invention discloses a method for identifying littering behavior based on key point detection and trajectory fitting, which includes the steps of: obtaining surveillance images at consecutive intervals of N frames for human target recognition, performing key point detection on each recognized human target, and saving the latest M frames of detected images; performing garbage target recognition on the detected images, when a garbage target is detected, continuously obtaining subsequent multiple frames of surveillance images for garbage target recognition, and determining whether the garbage is discarded artificially; if so, performing pose analysis on the human key points in the frame where the garbage appears and several previous frames of detected images, and if the movement trajectory of the wrist key point conforms to the littering characteristics of the garbage target, it is determined that there is a current littering behavior. The present invention is based on the analysis of human key points and garbage movement trajectories, can overcome problems such as complex background and random behavior, and effectively improve the recognition accuracy and real-time performance.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method for identifying littering behavior based on key point detection and trajectory fitting. Background Art

[0002] With the acceleration of the urbanization process, urban waste management has become an increasingly severe problem. Littering behavior not only seriously affects the appearance of the city, but may also have a negative impact on the environment and public health. Therefore, improving the efficiency and accuracy of urban waste management, especially timely detecting and stopping littering behavior, has become an urgent problem that urban managers need to solve.

[0003] Currently, the supervision of littering behavior mainly relies on two methods: manual inspection and video monitoring.

[0004] 1. Manual inspection: Manual inspection is that urban management departments regularly inspect specific areas through urban management personnel to discover and stop littering behavior. However, this method has problems such as limited coverage, low efficiency, and high costs. Especially in complex urban areas or when there is a large-area monitoring requirement, the timeliness and accuracy of manual inspection are insufficient.

[0005] 2. Video monitoring: With the development of computer vision technology, video monitoring has become another common means. Existing video monitoring systems collect video stream data through cameras installed in areas such as streets and public places, and analyze these videos through computer vision technology to detect and identify littering behavior. However, although video monitoring technology has certain advantages, the application of existing technologies in complex environments still faces many challenges.

[0006] Most current video monitoring systems are based on traditional object detection methods, and judge whether it is littering by analyzing the relative position relationship between the garbage object and people in the image. This method usually has the following deficiencies:

[0007] 1. Complex background of multiple people and multiple objects: In the urban environment, there are often multiple people and objects in the monitoring screen, and the behavior patterns are complex and changeable, which easily causes target overlap and misjudgment. In such a complex scenario, relying solely on the relative position relationship between the garbage and people to judge whether it is littering has low accuracy.

[0008] 2. Randomness and instantaneity of behavior: Littering behavior is usually fast and random, and the garbage object is often close to other objects or people, and may be instantly misjudged as normal object movement or picking behavior. This poses a great challenge to existing detection algorithms.

[0009] 3. Lack of detailed behavior analysis: Existing garbage detection systems mainly rely on image analysis and simple target detection, often ignoring human motion analysis and the dynamic trajectory of garbage behavior. This makes it difficult for the system to accurately determine whether a certain action is littering.

[0010] In addition, the existing system lacks a comprehensive analysis of human behavior, especially the tracking and analysis of human limb movements (such as wrists, hands and other key parts). Human movements, especially wrist movements, are closely related to the trajectory of garbage throwing. Therefore, how to combine the dynamic characteristics of human key points and garbage targets to conduct comprehensive behavior analysis is a key problem that needs to be solved in the existing technology. Summary of the invention

[0011] In order to solve the deficiencies existing in the above-mentioned prior art, the present invention aims to provide a littering behavior recognition method based on the analysis of human key points and garbage movement trajectories, which can overcome the problems of complex background and random behavior, and effectively improve the recognition accuracy and real-time performance.

[0012] In order to achieve the above-mentioned invention object, the technical solution provided by the present invention includes:

[0013] The littering behavior recognition method based on key point detection and trajectory fitting includes the following steps:

[0014] Obtain surveillance images at intervals of N frames continuously to identify human targets, perform key point detection on each identified human target, and save the latest M frames of detection images;

[0015] The detection image is used to identify garbage targets. When a garbage target is detected, multiple subsequent monitoring images are continuously acquired to identify the garbage target and determine whether the garbage is discarded by humans.

[0016] If yes, then perform posture analysis on the key points of the human body in the garbage appearing frame and the previous frames. If the movement trajectory of the wrist key point meets the littering characteristics of the garbage target, it is determined that there is currently littering behavior;

[0017] The littering features of the garbage target include: the wrist key point approaches the position where the garbage first appears; the garbage first appears when the wrist key point stops moving; and the movement trajectory of the wrist key point matches the movement trajectory of the garbage.

[0018] Preferably, the method for determining whether the garbage is discarded by humans includes:

[0019] Obtain the garbage detection frame in the garbage target identification and calculate the coordinates of its center point. If the vertical movement distance of the center point coordinate on the time flow exceeds the preset movement limit, it is determined that the garbage is discarded by humans.

[0020] Preferably, the method for determining whether the garbage is discarded by humans includes:

[0021] Obtain the garbage detection box in garbage target recognition and calculate its center point coordinates, fit the center point movement line. If the angle between the center point movement line and the vertical line is less than the first preset angle, it is determined that the garbage is discarded by humans.

[0022] Preferably, the human key points for pose analysis are the human key points of the human target near the position where the garbage first appears in the detection image.

[0023] Preferably, the method for determining whether the movement trajectory of the wrist key point matches the movement trajectory of the garbage includes:

[0024] Obtain the garbage detection box in garbage target recognition and calculate its center point coordinates, fit the center point movement line;

[0025] Obtain the wrist key point coordinates in the garbage appearance frame and several previous frames of detection images, fit the wrist key point movement line;

[0026] If the angle between the center point movement line and the wrist key point movement line is less than the second preset angle, it is determined that the movement trajectory of the wrist key point matches the movement trajectory of the garbage.

[0027] Preferably, the preset movement limit value is 0.

[0028] Preferably, the first preset angle is any angle between 40° and 60°.

[0029] Preferably, the second preset angle is any angle between 20° and 40°.

[0030] Beneficial effects

[0031] 1. Improve recognition accuracy: The present invention adopts a method combining human key point detection and garbage target detection. By matching the movement trajectory of the wrist with the movement trajectory of the garbage target, it can accurately identify the behavior of littering, effectively distinguish normal behavior from littering behavior, and avoid misrecognition in traditional methods.

[0032] 2. Reduce the amount of calculation: The present invention significantly reduces the amount of calculation through strategies such as frame skipping detection and only performing pose analysis when garbage is detected, improves the real-time response ability of the system, and reduces the calculation burden per frame.

[0033] 3. Dynamic trajectory analysis, accurately identify littering behavior: The present invention can effectively solve the problem of identifying the instantaneous and random nature of littering behavior in the prior art by fitting and comparing the movement trajectory of the garbage target and the movement trajectory of the wrist key point.

[0034] 4. Adapt to complex scenarios: By combining the detection of key points of the human body and the detection of litter targets, it is possible to accurately identify littering behaviors in an environment with multiple people and objects, avoiding the recognition errors caused by target overlap or behavior confusion in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic flow diagram of a littering behavior recognition method based on key point detection and trajectory fitting in a preferred embodiment of the present invention;

[0036] Figure 2 Schematic diagram of the process of determining whether the litter is discarded by humans in a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings. In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0038] Embodiment

[0039] As Figure 1 shown, this embodiment provides a littering behavior recognition method based on key point detection and trajectory fitting, including the steps of:

[0040] S1. Continuously obtain surveillance images at intervals of N frames for human target recognition, perform key point detection on each recognized human target, and save the latest M frames of detection images.

[0041] In order to improve the detection efficiency and reduce the computational complexity, the present invention adopts a method of continuously obtaining surveillance images at intervals of N frames for human target recognition. Human actions are usually continuous. Therefore, in a video surveillance scenario, it is not necessary to perform independent detection on each frame of the image. Instead, the method of skipping frames can be used to perform human target detection every several frames. This method can effectively reduce the frequency of image processing, thereby reducing the consumption of computing resources and improving the processing speed. For example, when obtaining surveillance images every 5 frames, the system can reduce the number of human detections per second while ensuring a high accuracy rate, significantly improving the operating efficiency of the entire system.

[0042] In addition, the purpose of saving the M-frame detection images is to directly apply the detection results of the human target key points in the subsequent processing steps, thereby avoiding the repeated calculation of key point recognition and detection, and further optimizing the system performance. By saving a certain number of historical detection images, the system can retrieve the human key point data in these images at any time for the analysis and judgment of subsequent frames, reducing the overhead of real-time calculation.

[0043] For example, if it is set to obtain a monitoring image every 5 frames and save 6 frames (i.e., 6 times) of detection images, assuming the frame rate of the monitoring video is 30 frames per second, then 6 detection images will be saved per second. These 6 frames of images cover the action information within one second, enabling the system to use this historical data for continuous action analysis or behavior recognition without reprocessing each frame of the image. In this way, the system can achieve efficient image detection and data saving, avoid redundant calculations, and ensure high real-time performance and accuracy in complex scenarios.

[0044] It should be understood that the implementation methods of target recognition and key point detection described in the present invention all adopt mature technologies widely used in the art, such as deep learning models, convolutional neural networks (CNNs), etc., and are not limited to any specific implementation manner. Since this part of the content is not the core innovation of the present invention, the present invention does not make further technical limitations on this.

[0045] S2. Identify garbage targets in the detection images. When a garbage target is detected, continuously obtain multiple subsequent frames of monitoring images for garbage target identification, and determine whether the garbage is discarded manually.

[0046] As Figure 2 shown, on the basis of the foregoing step S1, first obtain images by frame skipping and perform human target recognition and key point detection on them, and then identify garbage targets in these detection images. This method can effectively avoid independent garbage target recognition for each frame of the image, thereby significantly reducing the calculation amount and improving the processing efficiency of the system. During the process of obtaining images by frame skipping, the system only processes every several frames of images, selectively performs human and garbage target recognition, and avoids repeated calculations for each frame, thereby improving the processing speed. It should be understood that the garbage recognition is performed on the frame-skipped images at this time, and it is possible that this frame is not the actual first appearance frame of the garbage, resulting in a missing time for the first appearance of the garbage, but the time deviation is only 1 / 6 second, so it has little impact on the final result.

[0047] Garbage target recognition uses existing garbage object detection algorithms to classify the objects in each frame of the surveillance image. The detection results usually include the category of the object (such as paper towels, beverage bottles, plastic bags, etc.) and the position of the object in the image (represented by a bounding box or other means). Once a garbage target is detected in a certain frame of the image, the system will further obtain multiple subsequent frames of surveillance images for continuous garbage target recognition, and at this time, frame skipping will no longer be performed. This is because littering behavior usually has continuity within a short period of time, and the movement of the garbage target is an important basis for evaluating whether it is discarded by humans. Therefore, through continuous multi-frame garbage target recognition, the dynamic changes of the garbage object can be tracked more accurately, and the behavioral characteristics of the garbage object can be further verified.

[0048] For example, when it is detected that the garbage object suddenly falls from the air and there is an obvious change in the movement trajectory (such as a large vertical movement), it can be inferred that the garbage may be discarded by humans.

[0049] In some preferred embodiments, a method for judging whether the garbage is discarded by humans considering the movement in the vertical direction is given, specifically including:

[0050] Obtain the garbage detection box in the garbage target recognition and calculate its center point coordinates. If the vertical movement distance of the center point coordinates in the time stream exceeds the preset movement limit value, it is determined that the garbage is discarded by humans. This judgment method effectively excludes the stationary garbage targets on the ground, reduces the interference of non-discarded-by-humans garbage, and ensures the accurate recognition of the littering behavior by the system. By setting an appropriate preset movement limit value, the system can flexibly adjust the detection sensitivity according to the actual situation to adapt to the garbage behavior characteristics in different surveillance environments. In addition, by combining the movement information in the vertical direction, not only can it be more accurate to judge whether the garbage is a discarded object, but also false judgments caused by wind or other natural factors can be avoided. In some other preferred embodiments, the preset movement limit value is 0, that is, it is determined that the garbage with movement in the vertical direction is discarded by humans.

[0051] Furthermore, in some other preferred embodiments, a method for judging whether the garbage is discarded by humans based on the angle between the movement trajectory of the garbage target and the vertical direction is also proposed, specifically including:

[0052] Obtain the garbage detection box in garbage target recognition and calculate the coordinates of its center point. Fit the straight line of the center point movement. If the angle between the straight line of the center point movement and the vertical line is less than the first preset angle, it is determined that the garbage is discarded manually. Among them, the vertical line is a virtual construction line perpendicular to the ground in the vertical direction. The fitting of the straight line of the center point movement refers to describing the movement direction of the garbage in the monitoring screen by fitting the straight line of the center point movement of the garbage target, which can be realized by common trajectory fitting methods in the field such as the least squares method. The present invention does not make further limitations on this. By analyzing the angle between the movement direction of the garbage target and the vertical direction, it is possible to flexibly determine whether the garbage target conforms to normal physical behavior, such as the falling trajectory. If the angle is less than the set preset angle (for example, any angle between 40° and 60°), it indicates that the falling trajectory of the garbage is relatively close to the vertical direction, which usually means that the garbage is generated by manual throwing rather than the dropping of an object in a natural state. Compared with the foregoing method for judging manual discard based on the movement situation in the vertical direction, this embodiment avoids simply relying on the judgment of the object's falling distance and provides a more robust and detailed way to identify the behavior of manually discarding garbage, which is particularly suitable for environments with complex backgrounds and multi-object interference. By setting a reasonable preset angle, the system can flexibly adjust its recognition criteria in different application scenarios, improving the accuracy and adaptability of recognition.

[0053] In some other preferred embodiments, the two methods of considering the movement situation in the vertical direction and considering the angle between the movement trajectory and the vertical direction to judge whether the garbage is discarded manually can be used together, that is, first consider the movement situation in the vertical direction to confirm that this is the garbage just thrown to exclude the garbage that was originally on the ground. Then consider the angle between the movement trajectory and the vertical direction to confirm which person specifically threw the garbage.

[0054] S3. If so, perform pose analysis on the human key points in the garbage appearance frame and several previous frame detection images. If the movement trajectory of the wrist key point conforms to the littering characteristics of the garbage target, it is determined that there is a current littering behavior; the littering characteristics of the garbage target include: the wrist key point moves closer to the position where the garbage first appears; the garbage first appears when the wrist key point stops moving; the movement trajectory of the wrist key point matches the movement trajectory of the garbage.

[0055] The littering characteristics of the litter target refer to specific patterns or behavioral features in surveillance videos or images that can help identify and distinguish littering behaviors. These features are mainly reflected in the changes in the movement trajectory, speed, direction, etc. of the litter target, as well as the correlation with human actions (especially hand actions). Specifically, these features provide the key basis for the system to distinguish between normal object drops and human littering. The core of the littering characteristics of the litter target is closely related to the synchronization with human actions, especially the synchronization of hand actions. This is because when humans litter, it is usually accompanied by specific actions of the wrist or hand, such as throwing, tossing, or swinging. By real-time detecting the movement trajectory of human key points, especially the wrist key points, it can be determined whether the hand action of a person matches the movement trajectory of the litter target. Specifically, the moving direction of the wrist usually remains consistent with the movement direction of the litter target. Specifically, when a person throws litter, the wrist usually faces the position where the litter first appears, and during the action, the relative position and movement direction between the wrist and the litter target will remain synchronized. This synchronization is an important feature for judging whether there is a littering behavior. Further, another significant littering characteristic is that the litter object first appears in the surveillance screen when the wrist action stops. This feature indicates that the litter is generated by a human throwing action, rather than natural dropping or the movement of an object under other external forces. This temporal synchronization is a typical sign for identifying littering behaviors. The system can further confirm whether the behavior belongs to littering by accurately detecting the end time of the wrist action and the appearance time of the litter target. In the present invention, when the above three littering characteristics simultaneously meet the following conditions, it can be accurately determined that there is a littering behavior currently.

[0056] It should be noted that the human key points for pose analysis are preferably the human key points of the human target near the position where the litter first appears in the detected image. This is because the scenes in surveillance videos are usually very complex, including multiple objects and people. Therefore, performing full-image human pose analysis may bring a huge computational burden, especially in the case of multiple people or long-term surveillance. By selecting the key points of the human target near the position where the litter first appears, the calculation can be focused on the most relevant area, avoiding redundant calculation for irrelevant areas. This not only improves the processing efficiency of the system but also reduces unnecessary computational overhead.

[0057] In some preferred embodiments, a method for determining whether the movement trajectory of the wrist key point matches the movement trajectory of the litter is given, specifically including:

[0058] S31. Obtain the garbage detection box in the garbage target recognition and calculate its center point coordinates, and fit the center point movement line.

[0059] S32. Obtain the wrist key point coordinates in the garbage appearance frame and several previous detection images, and fit the moving straight line of the wrist key points.

[0060] S33. If the included angle between the moving straight line of the center point and the moving straight line of the wrist key points is less than the second preset angle, it is determined that the moving trajectory of the wrist key points matches the moving trajectory of the garbage.

[0061] It should be noted that the movement trajectories of the wrist key points and the garbage target need to be tracked simultaneously to ensure that their trajectories can be compared within the same time window. This is because only when the wrist movement is synchronized with the movement trajectory of the garbage target can it be accurately determined as an act of human discard. Fitting the trajectories of the wrist and the garbage target and calculating the included angle is an effective way to determine whether they are consistent. If the included angles of the movement trajectories of the garbage target and the wrist with the vertical direction are large, it may be due to external factors (such as wind) or the natural fall of the object. By setting an appropriate preset angle, the behavior of human discard and natural fall can be effectively distinguished. The garbage target of natural fall usually has a large included angle (for example, the angle is greater than the preset threshold), so this technical solution can effectively avoid misjudgment and ensure the accurate identification of littering behavior. Specifically, a straight line fitting method (such as the least squares method) can be used to determine the movement trajectories of the wrist and the garbage target. This is because straight line fitting can simplify complex trajectory calculations, enabling the system to analyze quickly and efficiently. Straight line fitting can obtain the movement direction of an object by processing the center point coordinates of the object in a series of consecutive image frames. Further, the setting of the first preset angle needs to be reasonably selected according to the actual scenario, ensuring that it can tolerate some small errors and not be too broad to cause misjudgment, and can be optimized according to the specific scenario and test data. In some preferred embodiments, the second preset angle is any angle between 20° and 40°.

[0062] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A littering behavior recognition method based on key point detection and trajectory fitting, characterized in that: Includes steps: Obtain surveillance images at intervals of N frames continuously for human target recognition, perform key point detection on each recognized human target, and save the latest M frames of detection images; The detection image is subjected to garbage target identification. When a garbage target is detected, subsequent multiple frames of monitoring images are continuously acquired to identify the garbage target and determine whether the garbage is discarded by humans; If yes, then perform posture analysis on the key points of the human body in the garbage appearing frame and the previous frames. If the movement trajectory of the wrist key point meets the littering characteristics of the garbage target, it is determined that there is currently littering behavior; The littering features of the garbage target include: the wrist key point approaches the position where the garbage first appears; the garbage first appears when the wrist key point stops moving; and the movement trajectory of the wrist key point matches the movement trajectory of the garbage.

2. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 1, characterized in that: The method for determining whether the garbage is discarded by humans includes: Obtain the garbage detection frame in the garbage target identification and calculate the coordinates of its center point. If the vertical movement distance of the center point coordinate on the time flow exceeds the preset movement limit, it is determined that the garbage is discarded by humans.

3. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 1, characterized in that: The method for determining whether the garbage is discarded by humans includes: Obtain the garbage detection frame in the garbage target recognition and calculate the coordinates of its center point, fit the center point moving straight line, and if the angle between the center point moving straight line and the vertical line is less than a first preset angle, it is determined that the garbage is discarded by humans.

4. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 1, characterized in that: The human key points for posture analysis are the human key points of the human target near the first appearance position of the garbage in the detection image.

5. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 1, characterized in that: The method for determining whether the wrist key point movement trajectory matches the garbage movement trajectory includes: Get the garbage detection frame in garbage target recognition and calculate its center point coordinates, and fit the center point moving straight line; Obtain the coordinates of the wrist key points in the garbage appearing frame and the detection images of several previous frames, and fit the wrist key point moving straight line; If the angle between the center point moving straight line and the wrist key point moving straight line is smaller than the second preset angle, it is determined that the wrist key point moving trajectory matches the garbage moving trajectory.

6. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 2, characterized in that: The preset movement limit value is 0.

7. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 3, characterized in that: The first preset angle is any angle between 40° and 60°.

8. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 5, characterized in that: The second preset angle is any angle between 20° and 40°.

Images