Pedestrian bending behavior detection method and system for intelligent garbage booth

Abstract

The invention discloses a pedestrian bowing behavior detection method and system for an intelligent garbage booth, and relates to the technical field of intelligent Internet of Things, and the method comprises the steps: employing YOLOv8-Pose to detect key point coordinates and bounding boxes, generating a multi-scale optical flow field through an RAFT algorithm, constructing a sampling set, obtaining an affine transformation matrix and an optical flow residual error, marking an abnormal point, and carrying out the detection of a pedestrian bowing behavior. A replacement optical flow is generated by optimizing an objective function, a spatio-temporal feature tensor is constructed and input into a Transform encoder, smooth key points are output, and updated key point coordinates are generated by screening and complementing; according to the method, through YOLOv8-Pose attitude estimation and RAFT multi-scale optical flow fusion, the precision and robustness of bowing behavior detection are improved, and through Transform space-time coding and HMM state sequence optimization fusion, the real-time performance and reliability of intelligent garbage booth pedestrian behavior detection are improved.

Description

Technical Field

[0001] This invention relates to the field of smart Internet of Things (IoT) technology, and in particular to a method and system for detecting pedestrian bending behavior at smart garbage collection points. Background Technology

[0002] With the rapid development of IoT and AI technologies, intelligent waste management systems have gradually evolved from traditional mechanical waste collection into highly integrated intelligent platforms, achieving significant progress, especially in the field of urban environmental sanitation. Early intelligent waste kiosks mainly relied on infrared sensors and basic image recognition algorithms to detect user behavior. However, with the rise of deep learning models, detection methods combining multimodal sensors have become widespread, used to capture pedestrian posture and distance data in real time. Spatiotemporal features are extracted through optical flow algorithms and key point estimation models to improve the accuracy of behavior detection. The introduction of the Transformer architecture further enhances the processing capability of sequential data, enabling the system to better cope with noise interference and lighting changes in complex environments.

[0003] However, existing technologies still have shortcomings. Traditional detection methods are limited by single-scale optical flow analysis, which is not sensitive enough to subtle posture changes and is prone to misjudgment or omission. This results in insufficient accuracy in the reliability score of key points and the calculation of geometric consistency error, affecting the overall robustness of detection. Furthermore, the lack of effective outlier set replacement mechanisms and state sequence optimization of HMM models leads to low efficiency in identifying potential bending sequences and fails to meet the real-time response requirements in high-density crowd environments. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method and system for detecting pedestrian bending behavior in intelligent garbage kiosks. It solves the problems of traditional detection methods, which are limited by single-scale optical flow analysis and have insufficient sensitivity to subtle posture changes, making them prone to misjudgment or omission. This results in inaccurate reliability scoring and geometric consistency error calculation of key points, affecting the overall robustness of detection. Furthermore, the lack of an effective outlier replacement mechanism and state sequence optimization of the HMM model leads to low efficiency in identifying potential bending sequences and fails to meet the real-time response requirements in high-density crowd environments.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for detecting pedestrian bending behavior in an intelligent garbage station, comprising, Real-time data is collected and preprocessed. The YOLOv8-Pose model is used to detect human keypoints and pedestrian bounding boxes in each frame of the image. Based on the pedestrian bounding boxes, each frame of the image is cropped to generate initial ROI image patches. The initial ROI image patches are scaled using bilinear interpolation to obtain ROI image blocks. Based on the ROI image blocks, the RAFT algorithm is used to generate an optical flow vector field. The keypoint coordinates are set as the center, and a proportionally proportional pixel neighborhood is extracted. Its local curvature features and dynamic curvature threshold are calculated. Combined with the optical flow vector field, the reliability score and optical flow features of the keypoints are obtained. Based on optical flow features, the matching degree between key points is calculated. A sampling set is constructed by combining the reliability score. The affine transformation matrix of the key points in the sampling set is calculated to generate the optical flow residual of the key points. By filtering, the inlier set and the outlier set are obtained. An optimization objective function is constructed for the outlier set to generate a replacement optical flow feature vector. The fused optical flow is obtained by combining the optical flow feature vector of the inlier set and constructing a spatiotemporal feature tensor. The tensor is input into the Transformer encoder to generate smooth key point coordinates. The optical flow consistency error is calculated and updated key point coordinates are generated by filtering and completion. Based on the updated key point coordinates, the left included angle, right included angle, included angle, and the rate of change of distance between the waist and knee are calculated. Potential bending frames are obtained through screening, and a potential bending sequence is generated. The optimal state sequence is obtained through the HMM model, the proportion of bending state is calculated, the effective bending behavior is obtained, and an alarm is triggered.

[0007] As a preferred embodiment of the pedestrian bending behavior detection method for intelligent garbage bins described in this invention, wherein obtaining the reliability score and optical flow characteristics of key points includes: The YOLOv8-Pose model is used to detect human key points and pedestrian bounding boxes in each frame of the image. Based on the pedestrian bounding boxes, each frame of the image is cropped to obtain ROI image patches. Based on ROI image patches, the RAFT algorithm is used to generate multi-scale dense optical flow fields, including low, medium and high resolution optical flow vector fields. Based on the optical flow vector field, the key point coordinates are set as the center, and a proportional pixel neighborhood is extracted. Its local curvature features and dynamic curvature threshold are calculated. Combined with the optical flow vector field, the geometric consistency error is calculated to obtain the reliability score and optical flow features of the key point.

[0008] As a preferred embodiment of the pedestrian bending behavior detection method for the intelligent garbage pavilion described in this invention, the step of generating smooth key point coordinates, calculating the optical flow consistency error, and generating updated key point coordinates through filtering and completion includes: The matching degree between optical flow features is calculated, and a sampling set is constructed by combining the reliability score. The affine transformation matrix of the key points in the sampling set is calculated, and the optical flow residual of all key points is calculated. The inlier set and the outlier set are obtained by filtering. Based on the optical flow feature vectors corresponding to the outlier set and the interior set, an optimization objective function is constructed and solved using the gradient descent method. This generates a replacement optical flow feature vector for the outlier set, calculates the fused optical flow, constructs a spatiotemporal feature tensor, inputs it into a Transformer encoder, generates smooth keypoint coordinates, calculates the optical flow consistency error, and selects smooth keypoint coordinates with an optical flow consistency error greater than a proportional pixel value for completion. The completed keypoint coordinates are then replaced with their corresponding smooth keypoint coordinates to generate updated keypoint coordinates.

[0009] As a preferred embodiment of the pedestrian bending behavior detection method for intelligent garbage bins described in this invention, the step of obtaining potential bending frames through screening and generating potential bending sequences includes: Based on the updated key point coordinates, calculate the rate of change of the included angle and the distance to the waist and knee; If the included angle is greater than the included angle threshold and the rate of change of the distance between the waist and knee is greater than the rate of change threshold, then the corresponding video frame is marked as a potential bending frame. If k consecutive frames are potential bending frames, then they are arranged horizontally to generate a potential bending sequence.

[0010] As a preferred embodiment of the pedestrian bending behavior detection method for the intelligent garbage bin described in this invention, the step of obtaining the optimal state sequence through an HMM model includes: Based on the potential bending sequence, the optimal state sequence is calculated using the HMM model, and the proportion of bending states is calculated. Valid bending behaviors are obtained by filtering those that are greater than the bending state proportion threshold.

[0011] As a preferred embodiment of the pedestrian bending behavior detection method for the intelligent garbage pavilion described in this invention, the step of calculating the bending state ratio, obtaining valid bending behavior, and triggering an alarm includes: Based on the effective bending behavior, a local alarm is triggered, including activating the built-in buzzer and LEDs in the garbage bin. The corresponding optimal state sequence, updated key point coordinates, included angles, and the rate of change of waist and knee distance are stored locally with timestamps and pushed to the cloud server, and then transmitted to the management personnel via APP or web page.

[0012] As a preferred embodiment of the pedestrian bending behavior detection method for the intelligent garbage pavilion described in this invention, the step of collecting real-time data and performing preprocessing includes: High-definition cameras and infrared ranging sensors are deployed on the top of smart trash cans and the sides of trash kiosks to collect real-time data, including real-time video streams and distances to pedestrians' waists and knees, for preprocessing.

[0013] Secondly, this invention provides a pedestrian bending behavior detection system for intelligent garbage collection points, comprising, The data acquisition and processing module is used to deploy high-definition cameras and infrared ranging sensors to collect video streams and distance data in real time, and to preprocess the data by Gaussian filtering for noise reduction, adaptive histogram equalization for illumination correction, and median filtering for distance smoothing. The key point detection and optical flow feature extraction module is used to detect human key points and generate ROI image patches using the YOLOv8-Pose model. It extracts multi-scale optical flow fields based on the RAFT algorithm and calculates local curvature features, optical flow features and their matching degree to construct sampling sets and affine transformation matrices. The key point optimization and fusion module is used to generate a fused optical flow by calculating the optical flow residual, optimizing the optical flow of outliers, and fusing the internal points and replacement optical flow. It constructs a spatiotemporal feature tensor input to the Transformer encoder to smooth the key point coordinates, and completes the key point coordinates based on the consistency error. The bending behavior recognition module is used to filter potential bending sequences by calculating the left and right angles and the rate of change of waist-knee distance based on the updated key point coordinates, initialize the HMM model to calculate the optimal state sequence and evaluate the proportion of bending states to determine effective bending behavior. The alarm storage module is used to activate the local buzzer and LED alarm for valid bending behavior. It stores the relevant sequence, coordinates, angle and rate of change together with the timestamp locally and pushes it to the cloud server, which can then be transmitted to the administrator via APP or web page.

[0014] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the pedestrian bending behavior detection method for intelligent garbage kiosks as described in the first aspect of the present invention.

[0015] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the pedestrian bending behavior detection method for intelligent garbage kiosks as described in the first aspect of the present invention.

[0016] The beneficial effects of this invention are as follows: This invention uses YOLOv8-Pose to detect key point coordinates and bounding boxes, generates a multi-scale optical flow field through the RAFT algorithm, constructs a sampling set, obtains the affine transformation matrix and optical flow residual, marks outliers, generates replacement optical flow by optimizing the objective function, constructs a spatiotemporal feature tensor input to a Transformer encoder, outputs smooth key points, and generates updated key point coordinates by filtering and completion; it improves the accuracy and robustness of bending behavior detection, and enhances the real-time performance and reliability of pedestrian behavior detection in intelligent garbage kiosks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the pedestrian bending behavior detection method for the intelligent garbage station in Example 1.

[0019] Figure 2 This is a schematic diagram of the pedestrian bending behavior detection system for the intelligent garbage station in Example 1. Detailed Implementation

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a method for detecting pedestrian bending behavior in an intelligent garbage station, including the following steps: S1. Collect real-time data and preprocess it. Use the YOLOv8-Pose model to detect human key points and pedestrian bounding boxes in each frame of the image. Based on the pedestrian bounding boxes, crop each frame of the image to generate an initial ROI image patch. Scale the initial ROI image patch using bilinear interpolation to obtain the ROI image patch. Based on the ROI image patch, use the RAFT algorithm to generate an optical flow vector field. Set the key point coordinates as the center, extract the proportional pixel neighborhood, calculate its local curvature features and dynamic curvature threshold, and combine the optical flow vector field to obtain the reliability score and optical flow features of the key points. Specifically, real-time data is collected and preprocessed, including: High-definition cameras and infrared ranging sensors are deployed on the top of smart trash cans and the sides of trash kiosks to collect real-time data, including real-time video streams and the distance to pedestrians' waist and knees, for preprocessing. The preprocessing includes denoising each frame of the image using Gaussian filtering, performing illumination correction using adaptive histogram equalization, and smoothing the distance between the pedestrian's waist and knee using median filtering.

[0024] By deploying high-definition cameras and infrared ranging sensors on the top and sides of the smart waste collection station, the system can capture multi-dimensional real-time data, including video streams and distance information between a pedestrian's waist and knees. This multi-sensor collaborative data acquisition method significantly improves the comprehensiveness and accuracy of data collection. High-definition cameras provide high-resolution visual information, capturing details of pedestrian behavior, while infrared ranging sensors compensate for the limitations of cameras in complex lighting or occlusion environments, ensuring the reliability of depth information. The preprocessing stage effectively improves data quality through Gaussian filtering for noise reduction, adaptive histogram equalization for illumination correction, and median filtering for smoothing. Gaussian filtering effectively suppresses high-frequency noise in the image while preserving the edge and texture information required for behavior detection. Adaptive histogram equalization dynamically adjusts for different lighting conditions, enhancing image contrast and feature recognizability. Median filtering smooths the infrared ranging data, reducing errors caused by sensor jitter or environmental interference. The combination of these preprocessing techniques significantly improves the robustness and accuracy of subsequent key point detection, laying the foundation for accurate identification of pedestrian bending behavior in complex outdoor environments.

[0025] Furthermore, the reliability scores and optical flow characteristics of the key points are obtained, including: The YOLOv8-Pose model is used to detect human key points and pedestrian bounding boxes in each frame of the image. The key points of the human body include the coordinates of the left shoulder, right shoulder, left hip, right hip, left knee, and right knee, which represent the pixel positions in the plane of each frame image. It refers to a coordinate system with the upper left corner of each frame image as the origin, the X-axis to the right, and the Y-axis downward. Based on the pedestrian bounding box, each frame of the image is cropped to generate an initial ROI image patch, which refers to the image region contained in the pedestrian bounding box. The initial ROI image patch is scaled by bilinear interpolation to obtain the ROI image patch. Based on ROI image patches, the RAFT algorithm is used to generate multi-scale dense optical flow fields, including low, medium, and high-resolution optical flow vector fields, as shown in the formula: , , in, For scale The optical flow vector field, and For scale The pixel displacement along the X-axis and the pixel displacement along the Y-axis, , as well as For low, medium and high resolution, and The x and y axes are the horizontal and vertical coordinates. Based on the optical flow vector field, the keypoint coordinates are set as the center, and a proportionally scaled pixel neighborhood is extracted to calculate its local curvature features. The formula is as follows: , in, Key point In scale The local curvature characteristics, The area of ​​the pixel neighborhood is proportional. Key point Equal-proportion pixel areas For gradient operators; The dynamic curvature threshold is calculated using the following formula: , in, Local curvature features The dynamic curvature threshold, The pedestrian depth is the average distance between the pedestrian's waist and knee. This is the scaling factor; The formula for calculating the geometric consistency error of key points is as follows: , in, Key point In scale Geometric consistency error, Key point coordinates Key points coordinates Key point and key points The distance between them represents the standard bone length. Number of key points; The formula for calculating the reliability score of key points is: , in, Key point In scale Reliability rating Scale factor; The optical flow features of key points, including direction and amplitude (pixels), are calculated using the following formula: , in, Key point In scale Optical flow characteristics, and Key point In scale The pixel displacement along the X-axis and the pixel displacement along the Y-axis, It is the arctangent function. For scale The optical flow feature vector is directly extracted from the optical flow vector field.

[0026] The system utilizes the YOLOv8-Pose model for human keypoint detection and pedestrian bounding box extraction. It efficiently and accurately locates the coordinates of key points such as the waist and knees of pedestrians, providing a foundation for dynamic analysis of bending behavior. The efficiency and high accuracy of the YOLOv8-Pose model make it suitable for real-time scenarios, with a lower false detection rate in complex backgrounds, significantly improving detection robustness. ROI image patches are generated by cropping pedestrian bounding boxes and scaled using bilinear interpolation to further focus on the pedestrian behavior area, reducing interference from irrelevant background information and improving computational efficiency. The multi-scale dense optical flow field generated by the RAFT algorithm provides dynamic motion information for keypoints, combined with local curvature characteristics... By calculating the geometric consistency error, the system can comprehensively characterize the spatiotemporal characteristics of pedestrian behavior. The local curvature feature formula extracts the motion trajectory features of key points through the gradient operator. The dynamic curvature threshold is dynamically adjusted according to the average pedestrian depth and distance, enhancing the algorithm's adaptability to different heights, body types, and distances. The introduction of reliability scoring is the core innovation of this step. By combining local curvature features and geometric consistency error, the system can quantify the credibility of each key point, avoiding misjudgments caused by occlusion or changes in lighting. The calculation of optical flow features further captures the motion direction and amplitude of key points, providing high-dimensional feature support for subsequent behavior analysis and significantly improving the accuracy and stability of bending behavior detection.

[0027] S2. Based on optical flow features, calculate the matching degree between key points, construct a sampling set by combining reliability scores, calculate the affine transformation matrix of key points in the sampling set, generate optical flow residuals of key points, obtain inlier set and outlier set by filtering, construct an optimization objective function for outlier set, generate replacement optical flow feature vector, obtain fused optical flow by combining optical flow feature vector of inlier set, construct spatiotemporal feature tensor, input into Transformer encoder, generate smooth key point coordinates, calculate optical flow consistency error, and generate updated key point coordinates by filtering and completion. Specifically, smooth keypoint coordinates are generated, optical flow consistency error is calculated, and updated keypoint coordinates are generated through filtering and completion, including: The formula for calculating the matching degree between optical flow features is: , in, For time Key points and key points In scale The degree of matching, For time Key points In scale Optical flow characteristics, For time Key points In scale Optical flow characteristics, To match sensitivity; Construct a sampling set, including sampling probabilities obtained based on reliability scores and matching degrees, sorting them in descending order, selecting the top K key points, and ensuring that the key points satisfy both a reliability score greater than a reliability score threshold and a matching degree greater than a matching degree threshold. Arrange these key points vertically to obtain the sampling set, as shown in the formula: , in, Key point In scale The sampling probability; The formula for calculating the affine transformation matrix of the key points in the sampling set is as follows: , in, For scale The affine transformation matrix, For scale The sampling set, This is the initial affine transformation matrix; Based on the affine transformation matrix, the optical flow residuals at all key points are calculated using the following formula: , in, Key point In scale optical flow residual, and For curvature weights and geometric weights; Key points with optical flow residuals less than or equal to the optical flow residual threshold are selected and marked as inliers; otherwise, they are marked as outliers. The sampling set is reconstructed for the unmarked key points. The above operation is repeated until all key points are marked. The inlier set and the outlier set are obtained, both of which are horizontally arranged. Based on the optical flow feature vectors corresponding to the outlier set and the interior set, an optimization objective function is constructed. Gradient descent is used to minimize the optimization objective function, yielding the minimized optimization objective function, where the optical flow feature vector is the replacement optical flow feature vector. The formula is: , , , , in, To optimize the objective function, , as well as The objective function values ​​are optimized for optical flow, depth, and geometry. For depth-constrained weights, For time Key points depth, The depth function represents the depth value obtained by mapping two-dimensional coordinates to three-dimensional space using camera built-in parameters, and then smoothing it using a Kalman filter. These are geometric constraint weights; The formula for calculating the fused optical flow is: , in, To integrate optical flow, For scale The fusion weight, Replace the optical flow feature vectors of the outlier set and the interior set; Based on fused optical flow, a spatiotemporal feature tensor is constructed and normalized, as shown in the formula: , in, For the spatiotemporal feature tensor, For time Key point coordinates, For time The infrared distance data represents the distance between a pedestrian's waist and knees. The size of the time window. For time The fusion of optical flow, For time The fusion optical flow; Input the spatiotemporal feature tensor into the Transformer encoder to generate smooth keypoint coordinates; The formula for calculating optical flow consistency error is: , in, To smooth key points In scale Optical flow consistency error, For time The Coordinates of a smooth key point, For time The Coordinates of a smooth key point; Select smooth keypoint coordinates with optical flow consistency errors greater than a proportional pixel value for completion, obtain the completed keypoint coordinates, replace their corresponding smooth keypoint coordinates, and generate updated keypoint coordinates. The formula is as follows: , in, For time The Complete the coordinates of the key points. For frame index, For frames The supplementary weights, For time The Coordinates of a smooth key point, This is the time offset.

[0028] By filtering key points using optical flow feature matching degree and sampling probability, the system can effectively eliminate outliers and retain a set of highly reliable interior points. The introduction of the affine transformation matrix optimizes the key point position through geometric constraints, reducing coordinate deviations caused by sensor noise or occlusion. The calculation and filtering mechanism of optical flow residuals further improves the robustness of key point coordinates, ensuring the continuity and consistency of behavior detection. The optimized objective function combines depth constraints and geometric constraints, and uses gradient descent to generate replacement optical flow feature vectors for outlier sets, significantly improving the system's self-repair capability for outlier data. The construction of integrated optical flow and the normalization of spatiotemporal feature tensors integrate multi-scale and multi-modal information into a unified feature table. The Transformer encoder provides high-quality input data and leverages its powerful sequence modeling capabilities to generate smooth keypoint coordinates, effectively capturing the dynamic patterns of pedestrian bending behavior. The calculation and completion mechanism of optical flow consistency error further optimizes the continuity of keypoint coordinates, avoiding misjudgments of behavior caused by keypoint loss or jitter. The final updated keypoint coordinates, by fusing multi-frame information and time offset, significantly improve the spatiotemporal consistency of behavior detection, not only enhancing detection accuracy but also strengthening the system's adaptability in complex scenarios, providing reliable technical support for the interactive functions of intelligent waste collection stations.

[0029] S3. Based on the updated key point coordinates, calculate the left included angle, right included angle, included angle, and the rate of change of distance between the waist and knees. Through screening, obtain potential bending frames, generate potential bending sequence, obtain the optimal state sequence through HMM model, calculate the bending state ratio, obtain effective bending behavior, and trigger alarm. Specifically, potential bending frames are obtained through screening, and potential bending sequences are generated, including: Based on the updated keypoint coordinates, calculate the vectors from the left shoulder to the left hip and from the left hip to the left knee, defined as the left-side angle. Calculate the vectors from the right shoulder to the right hip and from the right hip to the right knee, defined as the right-side angle. The angle is calculated using the following formula: , , , in, Let be the vector from the left shoulder to the left hip and the vector from the right shoulder to the right hip. For the left and right hips, For the left and right shoulders, Angle The value of the cosine function, Let be the vector from the left hip to the left knee and the vector from the right hip to the right knee. Pi; The formula for calculating the rate of change of the distance between the waist and knees is: , in, The rate of change of the distance between the waist and knees. The distance measured is from waist level to the garbage bin. The distance measured from the knee to the garbage bin; If the included angle is greater than the included angle threshold and the rate of change of the distance between the waist and knee is greater than the rate of change threshold, then the corresponding video frame is marked as a potential bending frame. If k consecutive frames are all potential bending frames, then they are arranged horizontally to generate a potential bending sequence.

[0030] The technical solution that filters potential bending frames and generates sequences by calculating the left and right angles and the rate of change of distance between the waist and knees has significant accuracy and efficiency. By using the vector angle formula and the distance change rate formula, it accurately captures the geometric features and dynamic changes of pedestrian bending actions, avoiding the limitations of traditional single image feature detection. The joint filtering mechanism of angle threshold and change rate threshold significantly reduces the false judgment rate and enhances robustness to complex scenes, generating a continuous k-frame potential bending sequence. Furthermore, through continuity analysis in the time dimension, it improves the stability of behavior recognition and lays a reliable data foundation for subsequent state sequence optimization. In the scenario of intelligent garbage kiosks, it can efficiently distinguish bending over to dispose of garbage from other actions such as passing by, thus optimizing the level of intelligence in garbage classification monitoring.

[0031] Furthermore, the optimal state sequence is obtained through the HMM model, including: Initialize the HMM model, including defining the standing and bending states, calculating the initial probability states, and constructing the state transition matrix, as shown in the formula: , in, B is the state transition matrix. For state Transition to state The probability of; The formula for calculating the Gaussian distribution is: , in, and For state The mean and covariance matrices of the observed data, including the left angle, right angle, and angle, For time steps Observational data, For state The probability density, It follows a multivariate normal distribution; The formula for calculating the optimal state sequence is: , in, For time status The optimal state sequence is obtained by horizontally stacking the states with the highest probability. and For state index, For state and state The state transition probability between states.

[0032] By employing a state sequence optimization method based on Hidden Markov Models (HMMs), this study provides high-precision state modeling and prediction capabilities for bending-over behavior detection. By defining standing and bending states and combining initial probabilities, state transition matrices, and Gaussian distribution formulas, the HMM model can effectively capture the temporal sequence characteristics of pedestrian actions. Through dynamic probability calculation, it significantly improves the recognition accuracy of continuous actions, especially performing well in handling noise interference or atypical actions. The application of multivariate normal distribution, which comprehensively considers multidimensional features such as the left and right angles and the rate of change of waist-knee distance, further enhances the model's generalization ability, providing real-time and accurate behavioral state analysis capabilities for intelligent garbage kiosks, and is suitable for behavioral monitoring in dynamic environments.

[0033] Furthermore, the proportion of bending-over states is calculated to identify valid bending-over behaviors, and alarms are triggered, including: Calculate the proportion of bending over postures, filter those exceeding a bending over posture proportion threshold, and determine valid bending over behaviors. The formula is: , in, Proportion for a bent-over position. For frame index, The total number of frames in the sliding time window. The number of frames analyzed at the tail end is the number of frames within the sliding time window. Bit, This is the optimal state sequence; Based on the effective bending behavior, a local alarm is triggered, including activating the built-in buzzer and LEDs in the garbage bin. The corresponding optimal state sequence, updated key point coordinates, included angles, and the rate of change of waist and knee distance are stored locally with timestamps and pushed to the cloud server, and then transmitted to the management personnel via APP or web page.

[0034] By calculating the proportion of bending-over states and combining it with thresholds to filter valid bending-over behaviors, an efficient transformation from state sequences to actual behavior judgments is achieved. The sliding time window design takes into account both the continuity and real-time nature of the action, significantly improving the sensitivity and specificity of detection. The linkage mechanism that triggers local alarms and cloud data pushes not only provides immediate alerts for abnormal behaviors but also supports subsequent data analysis and behavior tracing through the storage of timestamps and key features. Pushing data to managers via APP or web page optimizes the operational efficiency of garbage collection points, reduces the cost of manual inspections, and provides data-driven decision support for urban waste classification management, resulting in significant social and economic benefits.

[0035] This embodiment also provides a pedestrian bending behavior detection system for intelligent garbage collection points, including: The data acquisition and processing module is used to deploy high-definition cameras and infrared ranging sensors to collect video streams and distance data in real time, and to preprocess the data by Gaussian filtering for noise reduction, adaptive histogram equalization for illumination correction, and median filtering for distance smoothing. The key point detection and optical flow feature extraction module is used to detect human key points and generate ROI image patches using the YOLOv8-Pose model. It extracts multi-scale optical flow fields based on the RAFT algorithm and calculates local curvature features, optical flow features and their matching degree to construct sampling sets and affine transformation matrices. The key point optimization and fusion module is used to generate a fused optical flow by calculating the optical flow residual, optimizing the optical flow of outliers, and fusing the internal points and replacement optical flow. It constructs a spatiotemporal feature tensor input to the Transformer encoder to smooth the key point coordinates, and completes the key point coordinates based on the consistency error. The bending behavior recognition module is used to filter potential bending sequences by calculating the left and right angles and the rate of change of waist-knee distance based on the updated key point coordinates, initialize the HMM model to calculate the optimal state sequence and evaluate the proportion of bending states to determine effective bending behavior. The alarm storage module is used to activate the local buzzer and LED alarm for valid bending behavior. It stores the relevant sequence, coordinates, angle and rate of change together with the timestamp locally and pushes it to the cloud server, which can then be transmitted to the administrator via APP or web page.

[0036] This embodiment also provides a computer device applicable to the method for detecting pedestrian bending behavior at intelligent garbage kiosks, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for detecting pedestrian bending behavior at intelligent garbage kiosks as proposed in the above embodiment.

[0037] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0038] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the pedestrian bending behavior detection method for intelligent garbage kiosks as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0039] In summary, this invention uses YOLOv8-Pose to detect keypoint coordinates and bounding boxes, generates a multi-scale optical flow field using the RAFT algorithm, constructs a sampling set, obtains the affine transformation matrix and optical flow residuals, marks outliers, generates replacement optical flow by optimizing the objective function, constructs a spatiotemporal feature tensor input to a Transformer encoder, outputs smooth keypoints, and generates updated keypoint coordinates through filtering and completion; thus improving the accuracy and robustness of bending behavior detection and enhancing the real-time performance and reliability of pedestrian behavior detection in intelligent garbage kiosks.

[0040] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting a pedestrian bending behavior for an intelligent garbage kiosk, characterized in that: Comprising, Collect real-time data and preprocess, detect each frame of image using YOLOv8-Pose model, get human key points and pedestrian bounding box, based on pedestrian bounding box, crop each frame of image to generate initial ROI image block, scale initial ROI image block through bilinear interpolation to get ROI image block, based on ROI image block, generate optical flow vector field using RAFT algorithm, set key point coordinates as center, extract equal proportion pixel neighborhood, calculate its local curvature feature and dynamic curvature threshold, combine optical flow vector field to get reliability score and optical flow feature of key points; Based on optical flow feature, calculate matching degree between key points, combine reliability score to construct sample set, calculate affine transformation matrix of key points in sample set, generate optical flow residual of key points, get inlier set and outlier set by screening, construct optimization objective function based on outlier set, generate replacement optical flow feature vector, combine optical flow feature vector of inlier set to get fused optical flow, construct space-time feature tensor, input into Transformer encoder to generate smoothed key point coordinates, calculate optical flow consistency error, complete to generate updated key point coordinates by screening; Based on updated key point coordinates, calculate left angle, right angle, angle and waist and knee distance change rate, get potential bending frame through screening, generate potential bending sequence, get optimal state sequence through HMM model, calculate bending state proportion to get effective bending behavior and trigger alarm.

2. The method of claim 1, wherein the method further comprises: detecting a bending behavior of the pedestrian using the camera. The reliability score and optical flow feature of the key points comprise: Detect each frame of image using YOLOv8-Pose model to get human key points and pedestrian bounding box, crop each frame of image based on pedestrian bounding box to get ROI image block; Based on ROI image block, generate multi-scale dense optical flow field using RAFT algorithm, including low, medium and high resolution optical flow vector field; Based on optical flow vector field, set key point coordinates as center, extract equal proportion pixel neighborhood, calculate its local curvature feature and dynamic curvature threshold, combine optical flow vector field to calculate geometric consistency error to get reliability score and optical flow feature of key points.

3. The method of claim 2, wherein the method further comprises: determining a bending angle of the person based on the bending angle data; and determining the bending behavior of the person based on the bending angle. The generation of smoothed key point coordinates, calculation of optical flow consistency error, and completion of updated key point coordinates by screening comprise: Calculate matching degree between optical flow features, combine reliability score to construct sample set, calculate affine transformation matrix of key points in sample set, and calculate optical flow residual of all key points, get inlier set and outlier set by screening; Based on optical flow feature vectors corresponding to outlier set and inlier set, construct optimization objective function, use gradient descent method to solve it to generate replacement optical flow feature vector of outlier set, calculate fused optical flow, construct space-time feature tensor, input into Transformer encoder to generate smoothed key point coordinates, calculate optical flow consistency error, screen smoothed key point coordinates with equal proportion pixels whose optical flow consistency error is greater than or equal to, complete to get complete key point coordinates, replace corresponding smoothed key point coordinates to generate updated key point coordinates.

4. The method of claim 3, wherein the method further comprises: detecting a bending behavior of the pedestrian. The potential bending sequence is generated by screening, including: Based on the updated key point coordinates, the included angle and the change rate of the distance between the waist and the knee are calculated; If the included angle is greater than the included angle threshold and the change rate of the distance between the waist and the knee is greater than the change rate threshold, the corresponding video frame is marked as a potential bending frame, and if k consecutive frames are potential bending frames, a potential bending sequence is generated by transverse arrangement.

5. The method of claim 4, wherein the method further comprises: determining a bending angle of the person based on the bending angle of the person and the height of the person. The optimal state sequence is obtained by the HMM model, including: Based on the potential bending sequence, the optimal state sequence is calculated by the HMM model, and the bending state proportion is calculated, and the effective bending behavior is obtained by screening the bending state proportion greater than the bending state proportion threshold.

6. The method of claim 5, wherein the method further comprises: determining a bending angle of the person based on the bending angle of the person and the height of the person. The bending state proportion is calculated, the effective bending behavior is obtained, and the alarm is triggered, including: Based on the effective bending behavior, the local alarm is triggered, including activating the buzzer and LED in the garbage pavilion, storing the corresponding optimal state sequence, updated key point coordinates, included angle and change rate of the distance between the waist and the knee in combination with the timestamp locally and pushing to the cloud server, and transmitting to the management personnel through APP or Web page.

7. The method of claim 6, wherein the method further comprises: determining a bending angle of the person based on the bending angle of the person and the height of the person. The real-time data is collected and preprocessed, including: High-definition cameras and infrared distance sensors are deployed on the top of the intelligent garbage can and the side of the garbage pavilion to collect real-time data, including real-time video stream and the distance between the waist and the knee of pedestrians, and the data is preprocessed.

8. A system for detecting a pedestrian bending behavior for an intelligent garbage kiosk, based on the method for detecting a pedestrian bending behavior for an intelligent garbage kiosk according to any one of claims 1 to 7, characterized in that: Including, The data acquisition and processing module is used to deploy high-definition cameras and infrared distance sensors to collect video stream and distance data in real time, and to preprocess the data through Gaussian filter denoising, adaptive histogram equalization light correction and median filter smoothing distance; The key point detection and optical flow feature extraction module is used to detect human key points and generate ROI image blocks using YOLOv8-Pose model, extract multi-scale optical flow field based on RAFT algorithm, calculate local curvature feature, optical flow feature and matching degree to construct sample set and affine transformation matrix; The key point optimization and fusion module is used to generate fused optical flow by calculating optical flow residual, optimizing abnormal point optical flow and fusing inlier and replacement optical flow, construct spatiotemporal feature tensor input Transformer encoder to smooth key point coordinates, and complete based on consistency error to obtain updated key point coordinates; The bending behavior recognition module is used to calculate the left and right included angles and the change rate of the distance between the waist and the knee based on the updated key point coordinates to screen the potential bending sequence, initialize the HMM model to calculate the optimal state sequence and evaluate the bending state proportion to determine the effective bending behavior; The alarm storage module is used to activate the local buzzer and LED alarm for the effective bending behavior, store the related sequence, coordinates, included angle and change rate in combination with the timestamp locally and push to the cloud server, and transmit to the management personnel through APP or Web page. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the pedestrian bending behavior detection method for the intelligent garbage pavilion in any one of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the pedestrian bending behavior detection method for the intelligent garbage pavilion in any one of claims 1-7.

Images