Trash throwing behavior identification method based on key point detection and trajectory fitting

Abstract

The invention discloses a littering behavior identification method based on key point detection and trajectory fitting, and the method comprises the steps: continuously obtaining a monitoring image at an interval of N frames, carrying out the recognition of a human body target, carrying out the key point detection of each recognized human body target, and storing the latest M frames of detection images; performing junk target identification on the detection image, when a junk target is detected, continuously acquiring subsequent multiple frames of monitoring images to perform junk target identification, and judging whether the junk is artificially discarded; and if yes, performing posture analysis on the human body key points in the garbage appearing frame and the plurality of previous frames of detection images, and if the movement track of the wrist key points conforms to the littering feature of the garbage target, judging that the littering behavior exists at present. Based on human body key point and garbage motion trail analysis, the problems of complex background, random behaviors and the like can be solved, and the recognition accuracy and real-time performance are effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for identifying littering behavior based on key point detection and trajectory fitting. Background Art

[0002] With the acceleration of urbanization, urban waste management has become an increasingly serious problem. Littering not only seriously affects the appearance of the city, but may also have a negative impact on the environment and public health. Therefore, improving the efficiency and accuracy of urban waste management, especially timely detecting and stopping littering, has become a problem that urban managers urgently need to solve.

[0003] At present, the supervision of littering mainly relies on two methods: manual inspection and video surveillance: 1. Manual inspection: Manual inspection is a process where urban management departments have urban management personnel regularly inspect specific areas to discover and stop littering. However, this method has problems such as limited coverage, low efficiency, and high cost. Especially when there is a need for monitoring in complex urban areas or large areas, the timeliness and accuracy of manual inspections are insufficient.

[0004] 2. Video surveillance: With the development of computer vision technology, video surveillance has become another common means. Existing video surveillance systems collect video stream data through cameras installed in streets, public places and other areas, and analyze these videos through computer vision technology to detect and identify littering. However, despite the advantages of video surveillance technology, the application of existing technology in complex environments still faces many challenges.

[0005] Most current video surveillance systems are based on traditional target detection methods, which determine whether littering occurs by analyzing the relative position of garbage objects and people in the image. This method usually has the following shortcomings: 1. Complex background with multiple people and objects: In urban environments, there are often multiple people and objects in the surveillance images, and their behavior patterns are complex and changeable, which can easily lead to target overlap and misjudgment. In such complex scenes, the accuracy of judging whether it is littering is low only by the relative position relationship between garbage and people.

[0006] 2. Randomness and instantaneity of behavior: Littering behavior is usually fast and random, and garbage objects are often close to other objects or people, which may be misjudged as normal object movement or picking behavior. This poses a great challenge to existing detection algorithms.

[0007] 3. Lack of detailed behavior analysis: Existing garbage detection systems mainly rely on image analysis and simple target detection, often ignoring human motion analysis and the dynamic trajectory of garbage behavior. This makes it difficult for the system to accurately determine whether a certain action is littering.

[0008] In addition, the existing system lacks a comprehensive analysis of human behavior, especially the tracking and analysis of human limb movements (such as wrists, hands and other key parts). Human movements, especially wrist movements, are closely related to the trajectory of garbage throwing. Therefore, how to combine the dynamic characteristics of human key points and garbage targets to conduct comprehensive behavior analysis is a key problem that needs to be solved in the existing technology. Summary of the invention

[0009] In order to solve the deficiencies existing in the above-mentioned prior art, the present invention aims to provide a littering behavior recognition method based on the analysis of human key points and garbage movement trajectories, which can overcome the problems of complex background and random behavior, and effectively improve the recognition accuracy and real-time performance.

[0010] In order to achieve the above-mentioned invention object, the technical solution provided by the present invention includes: The littering behavior recognition method based on key point detection and trajectory fitting includes the following steps: Obtain surveillance images at intervals of N frames continuously to identify human targets, perform key point detection on each identified human target, and save the latest M frames of detection images; The detection image is subjected to garbage target identification. When a garbage target is detected, multiple subsequent monitoring images are continuously acquired to identify the garbage target and determine whether the garbage is discarded by humans; If yes, then perform posture analysis on the key points of the human body in the garbage appearing frame and the previous frames. If the movement trajectory of the wrist key point meets the littering characteristics of the garbage target, it is determined that there is currently littering behavior; The littering features of the garbage target include: the wrist key point approaches the position where the garbage first appears; the garbage first appears when the wrist key point stops moving; and the movement trajectory of the wrist key point matches the movement trajectory of the garbage.

[0011] Preferably, the method for determining whether the garbage is discarded by humans includes: Obtain the garbage detection frame in the garbage target identification and calculate the coordinates of its center point. If the vertical movement distance of the center point coordinate on the time flow exceeds the preset movement limit, it is determined that the garbage is discarded by humans.

[0012] Preferably, the method for determining whether the garbage is discarded by humans includes: Obtain the garbage detection frame in the garbage target recognition and calculate the coordinates of its center point, fit the center point moving straight line, and if the angle between the center point moving straight line and the vertical line is less than a first preset angle, it is determined that the garbage is discarded by humans.

[0013] Preferably, the key points of the human body for posture analysis are key points of the human body of the human target near the position where the garbage first appears in the detection image.

[0014] Preferably, the method for determining whether the wrist key point movement trajectory matches the garbage movement trajectory includes: Get the garbage detection frame in garbage target recognition and calculate its center point coordinates, and fit the center point moving straight line; Obtain the coordinates of the wrist key points in the garbage appearing frame and the detection images of several previous frames, and fit the wrist key point moving straight line; If the angle between the center point moving straight line and the wrist key point moving straight line is smaller than the second preset angle, it is determined that the wrist key point moving trajectory matches the garbage moving trajectory.

[0015] Preferably, the preset movement limit is 0.

[0016] Preferably, the first preset angle is any angle between 40° and 60°.

[0017] Preferably, the second preset angle is any angle between 20° and 40°.

[0018] Beneficial Effects 1. Improve recognition accuracy: The present invention adopts a combination of human key point detection and garbage target detection. The movement trajectory of the wrist is matched with the movement trajectory of the garbage target, so as to accurately identify littering behavior, effectively distinguish normal behavior from littering behavior, and avoid misidentification in traditional methods.

[0019] 2. Reduce the amount of calculation: The present invention significantly reduces the amount of calculation through strategies such as frame skipping detection and posture analysis only when garbage is detected, improves the real-time response capability of the system, and reduces the calculation burden of each frame.

[0020] 3. Dynamic trajectory analysis to accurately identify littering behavior: The present invention can effectively solve the problem of identifying the instantaneous and random nature of littering behavior in the prior art by fitting and comparing the motion trajectory of the garbage target and the motion trajectory of the key points of the wrist.

[0021] 4. Adapt to complex scenarios: By combining key point detection of the human body with garbage target detection, it is able to accurately identify littering behavior in an environment with multiple people and multiple objects, avoiding recognition errors caused by target overlap or behavior confusion in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of a method for identifying littering behavior based on key point detection and trajectory fitting in a preferred embodiment of the present invention; Figure 2 This is a schematic diagram of the process of determining whether garbage is discarded by humans in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described below in conjunction with the accompanying drawings. In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention.

[0024] Example like Figure 1 As shown, this embodiment provides a method for identifying littering behavior based on key point detection and trajectory fitting, including the steps of: S1. Obtain surveillance images at intervals of N frames continuously for human target recognition, perform key point detection on each recognized human target, and save the latest M frames of detection images.

[0025] In order to improve detection efficiency and reduce the amount of calculation, the present invention adopts the method of acquiring monitoring images at continuous intervals of N frames to perform human target recognition. Human body movements are usually continuous, so in video surveillance scenarios, it is not necessary to independently detect each frame of the image, but it is possible to detect human targets every few frames by skipping frames. This method can effectively reduce the frequency of image processing, thereby reducing the consumption of computing resources and increasing the processing speed. For example, when acquiring a monitoring image every 5 frames, the system can reduce the number of human detections per second while ensuring a high accuracy rate, significantly improving the operating efficiency of the entire system.

[0026] In addition, the purpose of saving M frames of detection images is to directly apply the detection results of human target key points in subsequent processing steps, thereby avoiding repeated calculations of key point recognition and detection, and further optimizing system performance. By saving a certain number of historical detection images, the system can retrieve the human key point data in these images at any time for analysis and judgment of subsequent frames, reducing the overhead of real-time calculations.

[0027] For example, if the monitoring image is acquired every 5 frames and 6 frames (i.e. 6 times) of detection images are saved, assuming that the frame rate of the monitoring video is 30 frames per second, 6 detection images will be saved every second. These 6 frames cover the action information within one second, allowing the system to use these historical data for continuous action analysis or behavior recognition without reprocessing each frame of the image. In this way, the system can achieve efficient image detection and data preservation, avoid redundant calculations, and ensure high real-time performance and accuracy in complex scenarios.

[0028] It should be understood that the target recognition and key point detection methods described in the present invention all adopt mature technologies that have been widely used in the field, such as deep learning models, convolutional neural networks (CNNs), etc., and are not limited to any specific implementation method. Since this part of the content is not the core of the innovation of the present invention, the present invention does not make further technical limitations on this.

[0029] S2. Perform garbage target recognition on the detection image. When a garbage target is detected, continuously obtain subsequent multiple frames of monitoring images to perform garbage target recognition and determine whether the garbage is discarded by humans.

[0030] like Figure 2 As shown, based on the aforementioned step S1, firstly, images are acquired by skipping frames and human target recognition and key point detection are performed on them, and then junk target recognition is performed on these detected images. This method can effectively avoid independent junk target recognition for each frame of the image, thereby significantly reducing the amount of calculation and improving the processing efficiency of the system. In the process of skipping frames to acquire images, the system only processes images every few frames, selectively recognizes human bodies and junk targets, avoids repeated calculations for each frame, and thus improves the processing speed. It should be understood that the junk recognition at this time is performed on the skipped frame image, and the frame may not be the actual first appearance frame of the junk, resulting in the missing time of the first appearance of the junk, but the time deviation is only 1 / 6 second, so it has little impact on the final result.

[0031] Garbage object recognition uses existing garbage object detection algorithms to classify objects in each frame of surveillance images. The detection results usually include the category of the object (such as paper towels, beverage bottles, plastic bags, etc.) and the location of the object in the image (represented by a bounding box or other means). Once a garbage object is detected in a certain frame of an image, the system will further obtain subsequent multiple frames of surveillance images for continuous garbage object recognition, and frame skipping will no longer be performed. This is because littering behavior usually has a short-term continuity, and the movement of garbage objects is an important basis for evaluating whether it is artificially discarded. Therefore, through continuous multiple-frame garbage object recognition, the dynamic changes of garbage objects can be tracked more accurately, and the behavioral characteristics of the garbage objects can be further verified.

[0032] For example, when a garbage object is detected to suddenly fall from the air and there is an obvious change in its motion trajectory (such as a large vertical movement), it can be inferred that the garbage may have been discarded by humans.

[0033] In some preferred embodiments, a method for determining whether garbage is discarded manually considering vertical movement is provided, which specifically includes: Obtain the garbage detection frame in the garbage target identification and calculate the coordinates of its center point. If the vertical movement distance of the center point coordinates on the time flow exceeds the preset movement limit, the garbage is judged to be discarded by humans. This judgment method effectively eliminates stationary garbage targets on the ground, reduces the interference of non-human discarded garbage, and ensures the system's accurate identification of littering behavior. By setting appropriate preset movement limits, the system can flexibly adjust the detection sensitivity according to actual conditions to adapt to the garbage behavior characteristics in different monitoring environments. In addition, combined with the vertical movement information, it can not only more accurately judge whether the garbage is discarded, but also avoid misjudgment caused by wind or other natural factors. In some other preferred embodiments, the preset movement limit is 0, that is, it is determined that garbage that moves in the vertical direction is discarded by humans.

[0034] Furthermore, in some other preferred embodiments, a method for judging whether the garbage is discarded manually based on the angle between the moving trajectory of the garbage target and the vertical direction is proposed, which specifically includes: Obtain the garbage detection frame in the garbage target identification and calculate its center point coordinates, fit the center point moving straight line, and if the angle between the center point moving straight line and the vertical line is less than the first preset angle, the garbage is judged to be discarded manually. Among them, the vertical line is a virtual construction line perpendicular to the ground in the vertical direction. The fitting of the center point moving straight line refers to describing the movement direction of the garbage in the monitoring screen by fitting the center point moving straight line of the garbage target, which can be implemented by using the least squares method and other commonly used trajectory fitting methods in this field, and the present invention does not further limit this. By analyzing the angle between the movement direction of the garbage target and the vertical direction, it is possible to flexibly judge whether the garbage target conforms to normal physical behavior, such as a falling trajectory. If the angle is less than the preset angle (for example, any angle between 40°-60°), it indicates that the falling trajectory of the garbage is close to the vertical direction, which usually means that the garbage is generated by artificial throwing, rather than falling objects in a natural state. Compared with the above-mentioned method of judging whether garbage has been discarded manually based on the movement in the vertical direction, this embodiment avoids relying solely on the judgment of the falling distance of the object, and provides a more robust and detailed way to identify the behavior of discarding garbage manually, which is particularly suitable for environments with complex backgrounds and multiple objects. By setting a reasonable preset angle, the system can flexibly adjust its recognition criteria in different application scenarios to improve the accuracy and adaptability of recognition.

[0035] In some other preferred embodiments, the two methods of considering the vertical movement and the angle between the movement trajectory and the vertical direction to determine whether the garbage is discarded by humans can be used together, that is, first consider the vertical movement to confirm that it is the garbage that has just been thrown away, so as to exclude the garbage that has been on the ground. Then consider the angle between the movement trajectory and the vertical direction to confirm who threw the garbage.

[0036] S3. If so, perform posture analysis on the key points of the human body in the garbage appearing frame and the previous several frames of detection images. If the moving trajectory of the wrist key point meets the littering characteristics of the garbage target, it is determined that there is currently littering behavior; the littering characteristics of the garbage target include: the wrist key point approaches the position where the garbage first appears; the garbage first appears when the wrist key point stops moving; the moving trajectory of the wrist key point matches the moving trajectory of the garbage.

[0037] The littering characteristics of the garbage target refer to specific patterns or behavioral characteristics that can help identify and distinguish littering behavior in the surveillance video or image. These characteristics are mainly reflected in the changes in the movement trajectory, speed, direction, etc. of the garbage target, as well as the correlation with human movements (especially hand movements). Specifically, these characteristics provide the system with the key basis for distinguishing normal object drops from human garbage discarding. The core of the littering characteristics of the garbage target is closely related to the synchronization of human movements, especially the synchronization of hand movements. This is because when humans litter, they are usually accompanied by specific movements of the wrist or hand, such as throwing, throwing or swinging. By real-time detection of the movement trajectory of the key points of the human body, especially the key points of the wrist, it can be determined whether the hand movement of the person matches the movement trajectory of the garbage target. Specifically, the movement direction of the wrist is usually consistent with the movement direction of the garbage target. Specifically, when a person throws garbage, the wrist is usually facing the position where the garbage first appears, and during the movement, the relative position and movement direction between the wrist and the garbage target will remain synchronized. This synchronization is an important feature for determining whether there is littering behavior. Furthermore, another significant feature of littering is that the garbage object first appears in the monitoring screen when the wrist movement stops. This feature indicates that the garbage is produced by human throwing action, rather than natural falling or the movement of objects under the action of other external forces. This time synchronization is a typical sign of identifying littering behavior. The system can further confirm whether the behavior is littering by accurately detecting the end time of the wrist movement and the appearance time of the garbage target. In the present invention, when the above three littering features meet the following conditions at the same time, it can be accurately determined that there is currently littering behavior.

[0038] It should be noted that the key points of the human body for posture analysis are preferably the key points of the human body of the human body target near the first appearance position of the garbage in the detection image. This is because the scenes in the surveillance video are usually very complex and contain multiple objects and people. Therefore, performing human posture analysis across the entire image may bring a huge computational burden, especially in the case of multiple people or long-term monitoring. By selecting the key points of the human body target near the first appearance position of the garbage, the calculation can be focused on the most relevant area, avoiding redundant calculations on irrelevant areas. This not only improves the processing efficiency of the system, but also reduces unnecessary computational overhead.

[0039] In some preferred embodiments, a method for determining whether the wrist key point movement trajectory matches the garbage movement trajectory is provided, specifically including: S31. Obtain the garbage detection frame in the garbage target recognition and calculate the coordinates of its center point, and fit the center point moving straight line.

[0040] S32. Obtain the coordinates of the wrist key points in the garbage appearing frame and the detection images of several preceding frames, and fit the wrist key point moving straight line.

[0041] S33. If the angle between the center point moving straight line and the wrist key point moving straight line is smaller than the second preset angle, it is determined that the wrist key point moving trajectory matches the garbage moving trajectory.

[0042] It should be noted that the wrist key point and the motion trajectory of the garbage target need to be tracked at the same time to ensure that the trajectories of the two can be compared within the same time window. This is because only when the wrist movement is synchronized with the motion trajectory of the garbage target can it be accurately determined as an act of human discarding. Fitting the trajectories of the wrist and the garbage target and calculating the angle is an effective way to determine whether the two are consistent. If the angle between the motion trajectory of the garbage target and the wrist and the vertical direction is large, it may be due to external factors (such as wind) or the natural fall of the object. By setting a suitable preset angle, the behavior of human discarding and natural falling can be effectively distinguished. Naturally falling garbage targets usually have a large angle (for example, the angle is greater than the preset threshold), so this technical solution can effectively avoid misjudgment and ensure accurate identification of littering behavior. Specifically, a straight line fitting method (such as the least squares method) can be used to determine the motion trajectory of the wrist and the garbage target. This is because straight line fitting can simplify complex trajectory calculations, allowing the system to analyze quickly and efficiently. Straight line fitting can obtain the direction of motion of the object by processing the coordinates of the center point of the object in a series of continuous image frames. Furthermore, the setting of the first preset angle needs to be reasonably selected according to the actual scene, ensuring that it can tolerate some small errors and is not too wide to cause misjudgment, and can be optimized according to the specific scene and test data. In some preferred embodiments, the second preset angle is any angle between 20° and 40°.

[0043] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A littering behavior recognition method based on key point detection and trajectory fitting, characterized in that: Includes steps: Obtain surveillance images at intervals of N frames continuously for human target recognition, perform key point detection on each recognized human target, and save the latest M frames of detection images; The detection image is subjected to garbage target identification. When a garbage target is detected, subsequent multiple frames of monitoring images are continuously acquired to identify the garbage target and determine whether the garbage is discarded by humans; If yes, then perform posture analysis on the key points of the human body in the garbage appearing frame and the previous frames. If the movement trajectory of the wrist key point meets the littering characteristics of the garbage target, it is determined that there is currently littering behavior; The littering features of the garbage target include: the wrist key point approaches the position where the garbage first appears; the garbage first appears when the wrist key point stops moving; and the movement trajectory of the wrist key point matches the movement trajectory of the garbage.

2. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 1, characterized in that: The method for determining whether the garbage is discarded by humans includes: Obtain the garbage detection frame in the garbage target identification and calculate the coordinates of its center point. If the vertical movement distance of the center point coordinate on the time flow exceeds the preset movement limit, it is determined that the garbage is discarded by humans.

3. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 1, characterized in that: The method for determining whether the garbage is discarded by humans includes: Obtain the garbage detection frame in the garbage target recognition and calculate the coordinates of its center point, fit the center point moving straight line, and if the angle between the center point moving straight line and the vertical line is less than a first preset angle, it is determined that the garbage is discarded by humans.

4. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 1, characterized in that: The human key points for posture analysis are the human key points of the human target near the first appearance position of the garbage in the detection image.

5. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 1, characterized in that: The method for determining whether the wrist key point movement trajectory matches the garbage movement trajectory includes: Get the garbage detection frame in garbage target recognition and calculate its center point coordinates, and fit the center point moving straight line; Obtain the coordinates of the wrist key points in the garbage appearing frame and the detection images of several previous frames, and fit the wrist key point moving straight line; If the angle between the center point moving straight line and the wrist key point moving straight line is smaller than the second preset angle, it is determined that the wrist key point moving trajectory matches the garbage moving trajectory.

6. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 2, characterized in that: The preset movement limit value is 0.

7. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 3, characterized in that: The first preset angle is any angle between 40° and 60°.

8. The method for identifying littering behavior based on key point detection and trajectory fitting as claimed in claim 5, characterized in that: The second preset angle is any angle between 20° and 40°.

Images