Intelligent climbing behavior identification method based on target detection and attitude estimation

Abstract

The invention discloses a climbing behavior intelligent identification method based on target detection and posture estimation, and the method comprises the steps: firstly obtaining a training image data set, carrying out the marking, and training a target detection model and a human body posture estimation model; synchronously inputting a video frame image to be recognized into the two trained models, and obtaining a human body target detection frame containing behavior category information and human body key point position information; performing cross-frame association on the same target in the video sequence based on the information, and generating a target track sequence with a time sequence identifier; and finally, extracting human body motion characteristics based on the track sequence, and judging whether a climbing behavior occurs or not through preset multi-condition judgment logic. According to the method, the climbing behavior can be accurately recognized by fusing target detection and attitude estimation technologies and combining time sequence feature analysis and a multi-condition judgment mechanism, and the method is suitable for an intelligent security and protection monitoring system.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring, and specifically to an intelligent recognition method for climbing behavior based on target detection and attitude estimation. Background Technology

[0002] With the continuous development of security monitoring technology, intelligent recognition of abnormal behavior has become an important research direction in the field of security protection. Climbing behavior, as a common security hazard, has wide-ranging application value in areas such as public safety, boundary protection, and the protection of critical facilities. Currently, climbing behavior recognition technology is mainly based on computer vision and deep learning methods, achieving this through the detection and behavioral analysis of human targets in video images.

[0003] However, existing climbing behavior recognition technologies still suffer from the following problems: First, they suffer from high false alarm rates and frequent indiscriminate alarms. Current technologies struggle to accurately distinguish genuine climbing behavior from normal activity, leading to numerous false alarms and severely reducing the reliability and practicality of security systems. Second, they exhibit poor environmental adaptability, with recognition performance significantly decreasing under conditions of changing lighting, occlusion, and complex backgrounds. Third, a balance between deployment cost and detection effectiveness is difficult to achieve. High-precision solutions require multimodal sensors and expensive high-performance computing equipment, resulting in high costs, while low-cost solutions cannot meet the detection accuracy requirements in complex environments. Fourth, traditional monitoring systems, based on simple motion detection or area intrusion algorithms, lack in-depth analysis of human posture and behavioral patterns, making it impossible to effectively identify complex climbing behaviors. Furthermore, existing technologies are insufficient in cross-frame correlation and long-term behavior determination, making it difficult to effectively track and identify continuous climbing behavior.

[0004] Therefore, there is an urgent need for an intelligent identification method that can accurately identify climbing behavior, reduce false alarm rates, adapt to complex environments, and has a reasonable deployment cost, so as to improve the practicality and reliability of security monitoring systems. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an intelligent climbing behavior recognition method based on target detection and attitude estimation, which solves the problems of high false alarm rate, poor environmental adaptability, difficulty in balancing deployment cost and detection effect in existing climbing behavior recognition technologies, and the large number of false alarms caused by traditional monitoring systems based on simple algorithms. The invention achieves the technical effects of high-precision recognition, low false alarm rate, cost-effectiveness balance, strong environmental adaptability and good real-time performance.

[0006] In a first aspect, this application provides an intelligent recognition method for climbing behavior based on target detection and posture estimation, comprising the following steps:

[0007] Obtain a training image dataset, and annotate the images in the dataset to obtain object detection annotation data and human pose estimation annotation data;

[0008] A target detection model is trained based on the target detection annotation data to locate human targets in an image and output corresponding behavior category information; a human pose estimation model is trained based on the human pose estimation annotation data to output the location information of human key points.

[0009] The video frame images to be identified are synchronously input into the trained target detection model and human pose estimation model to obtain human target detection boxes containing behavior category information and human key point location information, respectively.

[0010] Based on the human target detection box and human key point coordinate information, cross-frame association is performed on the same target in the video sequence to generate a target trajectory sequence with temporal identifier.

[0011] Human motion features are extracted based on the target trajectory sequence, and a pre-set multi-condition judgment logic is used to determine whether climbing behavior has occurred.

[0012] Furthermore, it also includes preprocessing the video frame images to be identified, specifically including: normalizing the image size, and performing adaptive contrast enhancement and sharpening of small target areas.

[0013] Furthermore, cross-frame association of the same target to generate a target trajectory sequence with temporal identifiers includes:

[0014] Matching calculations are performed based on the intersection-union ratio of human target detection boxes between the target in the current frame and each target in the previous frame, as well as the consistency of the human key point positions.

[0015] If the matching calculation result meets the preset conditions, the target in the current frame is determined to be the same target as the corresponding target in the previous frame, and the target in the current frame is assigned the same identity ID; otherwise, the target in the current frame is determined to be a new target, and a new identity ID is assigned to it.

[0016] Furthermore, for a target in the current frame that is determined to be the same target as the target in the previous frame, the system adds its human target detection box, behavior category information, and human key point location information to the target trajectory sequence corresponding to the associated existing identity ID; the target trajectory sequence is used to support subsequent human motion feature extraction and multi-condition determination.

[0017] Furthermore, the human motion characteristics include displacement change characteristics and posture change characteristics calculated based on the positions of key points on the human body. The displacement change characteristics include the displacement rate of the target in the vertical direction; the posture change characteristics include the swing amplitude of the upper or lower limbs and the duration of the legs being in a stationary state.

[0018] Furthermore, the judgment conditions upon which the multi-condition judgment logic is based include at least one of the following:

[0019] Based on the temporal statistical conditions of the behavior category labels in the target trajectory sequence, the conditions include: the duration of continuous occurrence of the climbing behavior category reaches a first threshold.

[0020] The abnormal state conditions based on the single-model output confidence in the target trajectory sequence include:

[0021] The duration of a high-confidence abnormal state output only by one of the target detection model or the human pose estimation model reaches a second threshold.

[0022] Furthermore, after determining that climbing behavior has occurred, the following steps are also included:

[0023] Generate and save an early warning log, which includes at least: the identity ID of the abnormal target, the specific condition type that triggered the judgment, the coordinates of the key points of the human body of the target in the current frame, and the motion feature parameter values ​​on which the judgment is based; at the same time, trigger an early warning signal for the abnormal target.

[0024] Furthermore, the target detection annotation data includes labels identifying climbing or normal walking behavior categories; the human pose estimation annotation data includes location information of key points on the human body.

[0025] Secondly, this application provides an intelligent recognition system for climbing behavior based on target detection and posture estimation, comprising:

[0026] The acquisition and annotation module acquires the training image dataset, annotates the images in the dataset, and obtains object detection annotation data and human pose estimation annotation data.

[0027] The data training module trains a target detection model based on the target detection annotation data, which is used to locate human targets in the image and output corresponding behavior category information; it also trains a human pose estimation model based on the human pose estimation annotation data, which is used to output the location information of human key points.

[0028] The data input module synchronously inputs the video frame images to be identified into the trained target detection model and human pose estimation model to obtain human target detection boxes containing behavior category information and human key point location information, respectively.

[0029] The target association module, based on the human target detection box and the coordinate information of human key points, performs cross-frame association on the same target in the video sequence to generate a target trajectory sequence with temporal identifiers;

[0030] The behavior determination module extracts human motion features based on the target trajectory sequence and determines whether climbing behavior has occurred through preset multi-condition determination logic.

[0031] The beneficial effects of the technical solution of the intelligent recognition method for climbing behavior based on target detection and posture estimation of the present invention include at least the following:

[0032] By employing a dual-model collaborative framework of object detection and pose estimation, combined with multi-dimensional conditional decision logic, high-precision climbing behavior recognition is achieved. The object detection and pose estimation models achieved average accuracies of 90.2% and 95.6%, respectively, on the validation set. The dual-model collaborative detection and multi-conditional decision logic effectively reduced the false alarm rate and improved system reliability. Under moderate hardware cost constraints, high-precision, low-false-alarm climbing behavior detection and judgment were achieved, effectively balancing recognition performance and hardware economy. Multi-scenario data training and data augmentation methods enhanced the model's generalization ability in different environments. The system can detect climbing behavior in real time and trigger alerts, save screenshots and video recordings, and annotate abnormal target IDs, trigger conditions, and human key point states, demonstrating excellent real-time performance. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic flowchart of an intelligent climbing behavior recognition method for target detection and attitude estimation provided in the embodiments of this specification.

[0035] Figure 2 This is a schematic diagram illustrating the behavior category labeling provided for embodiments of this specification.

[0036] Figure 3 This is a schematic diagram of key human body point annotations provided for embodiments of this specification.

[0037] Figure 4 This is a schematic diagram of the climbing behavior recognition logic provided in the embodiments of this specification.

[0038] Figure 5 This is a schematic diagram illustrating the target detection and recognition effect provided in the embodiments of this specification.

[0039] Figure 6 This is a schematic diagram illustrating the key point detection effect provided in the embodiments of this specification.

[0040] Figure 7 This diagram illustrates the evaluation of the training effect of the target detection model provided in the embodiments of this specification.

[0041] Figure 8 This diagram illustrates the evaluation of the training effect of the attitude estimation model provided in the embodiments of this specification.

[0042] Figure 9 This is a schematic diagram of the climbing behavior recognition interface provided in the embodiments of this specification.

[0043] Figure 10 This is a schematic diagram of a climbing behavior intelligent recognition method system for target detection and attitude estimation provided in the embodiments of this specification.

[0044] Figure 11 A schematic diagram of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0046] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to facilitate the description of the embodiments and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.

[0047] A method for intelligent recognition of climbing behavior based on target detection and pose estimation; please refer to [link / reference]. Figure 1 This includes the following steps:

[0048] Data acquisition and annotation: A training image dataset was obtained, and the images in the dataset were annotated. Based on visual features of climbing behavior, such as body posture and scene association, climbing behavior datasets for various scenes were collected, and the data were standardized in size and format. The X-Anylabeling tool was used to perform two types of annotation on the dataset: one for training the object detection model, annotating only climbing and normal pedestrians, resulting in object detection labeled data; the other for training the pose estimation model, performing detailed annotation on each human body keypoint, including but not limited to the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle, resulting in human pose estimation labeled data. Please refer to the appendix. Figure 2 and attached Figure 3 The two types of annotation results are stored in JSON format, including target location, category label, and keypoint coordinates. This ultimately yields target detection annotation data and human pose estimation annotation data.

[0049] In some embodiments, to improve the model's generalization ability, the JSON annotation file is converted to YOLO format, and the dataset is divided into training, validation, and test sets. Abnormal data is cleaned, removing annotation errors such as missing bounding boxes, keypoint coordinates exceeding the image range, and blurry images (data with a resolution below a threshold), which are then used for subsequent model training.

[0050] Model training involves training an object detection model based on the labeled object detection data to locate human targets in images and output corresponding behavior category information; and training a human pose estimation model based on the labeled human pose estimation data to output the location information of human key points. Using YOLOv11 as the basic framework, object detection models are constructed for locating and initially classifying human bodies, while the pose estimation model extracts human key point features. A joint training strategy is employed to optimize model performance. Based on the YOLOv11 feature extraction, feature fusion, and object prediction structure, the network is fine-tuned for human targets, and data augmentation methods such as geometric transformation and pixel adjustment are used during training to improve the model's generalization ability.

[0051] Appendix Figure 4 The diagram shown is a logic diagram for climbing behavior recognition, which includes data preprocessing: preprocessing the video frame image to be recognized, normalizing the image size, and performing contrast adaptive enhancement and small target area sharpening. Through image enhancement and dual-model inference, invalid targets with low confidence and missing key points are excluded, and only potential objects with climbing behavior characteristics are retained.

[0052] Model inference and feature extraction: The video frame images to be identified are synchronously input into the trained target detection model and human pose estimation model to obtain human target detection boxes containing behavior category information and human key point location information, respectively.

[0053] Cross-frame target association: Based on the human target detection box and human key point coordinate information, cross-frame association is performed on the same target in the video sequence to generate a target trajectory sequence with temporal identifier, and high-confidence abnormal targets and their human key points are extracted.

[0054] The specific steps include:

[0055] Matching calculations are performed based on the intersection-union ratio of human target detection boxes between the target in the current frame and each target in the previous frame, as well as the consistency of the human key point positions.

[0056] If the matching calculation result meets the preset conditions, it is determined that the target in the current frame is the same target as the corresponding target in the previous frame, and the target in the current frame is assigned the same identity ID.

[0057] Otherwise, the target in the current frame is identified as a new target, and a new identity ID is assigned to it.

[0058] For a target in the current frame that is determined to be the same target as the target in the previous frame, the system appends its human target detection bounding box, behavior category information, and human key point location information to the target trajectory sequence corresponding to the associated existing identity ID. The target trajectory sequence is used to support subsequent human motion feature extraction and multi-condition judgment.

[0059] In one embodiment, a tracking association strategy based on DeepSort is used to perform cross-frame association of the same target, specifically including:

[0060] The DeepSort tracker performs cross-frame association between the target detection box and the pose detection box, converts the bounding box output by the target detection into the center coordinates + width and height (xywh) format, and inputs the detection confidence and category information into the tracker.

[0061] The tracker assigns a unique tracking ID (PID) to each target based on ReID feature matching and motion model prediction.

[0062] The preset condition is: when the target matching confidence score output by the tracker is ≥0.5 and the target matches successfully in consecutive frames, it is determined to be the same target, thus achieving stable tracking across frames.

[0063] In another embodiment, the association employs a weighted scoring strategy, specifically including:

[0064] Normalize the intersection-union ratio to the range [0,1] to obtain N;

[0065] After normalizing the average Euclidean distance D of the human body key points, the similarity is calculated as: S = 1 - D;

[0066] Calculate the overall matching degree: M = 0.6 × N + 0.4 × S;

[0067] The preset condition is: when M>0.7 and the number of effective limb key points is greater than or equal to 3, they are judged as the same target.

[0068] Trajectory sequence filtering: Before extracting human motion features based on the target trajectory sequence, verify the number of visible human key points of the target corresponding to the target trajectory sequence in the video frame. If the number of visible points is lower than the preset threshold, skip the subsequent behavior judgment of the target trajectory sequence.

[0069] Human motion feature extraction: Human motion features are extracted based on the target trajectory sequence. These features include displacement change features and posture change features calculated based on the positions of key points on the human body. Displacement change features include the target's displacement rate in the vertical direction; posture change features include the swing amplitude of the upper limbs.

[0070] Multi-condition judgment logic: This logic uses pre-defined multi-condition judgment logic to determine whether climbing behavior has occurred. The judgment conditions used by this logic include:

[0071] Temporal statistical conditions based on behavioral category labels in the target trajectory sequence:

[0072] Continuous condition: The duration of continuous occurrence of the climbing behavior category reaches the first threshold of 1.5 seconds, which corresponds to the basic continuous warning and is applicable to continuous climbing scenarios along walls;

[0073] Abnormal state conditions based on the single-model output confidence in the target trajectory sequence:

[0074] The duration of a high-confidence abnormal state output by only one of the object detection model or human pose estimation model reaches the second threshold of 2 seconds.

[0075] Warning processing: After determining that climbing behavior has occurred, a warning log is generated and saved. The warning log includes at least: the identity ID of the abnormal target, the specific condition type that triggered the determination, the coordinate information of the human key points of the target in the current frame, and the motion feature parameter values ​​on which the determination is based; at the same time, a warning signal is triggered for the abnormal target.

[0076] In a preferred embodiment, the target detection annotation data includes labels identifying climbing or normal walking behavior categories; the human pose estimation annotation data includes location information of key human points. Through this architecture of dual-model collaborative reasoning, ID tracking, and multi-dimensional conditional judgment, climbing behavior can be accurately identified, effectively eliminating false positives such as taking photos or remaining stationary, thus achieving intelligent identification and early warning for different types of climbing behavior.

[0077] Furthermore, to verify the effectiveness of a climbing behavior intelligent recognition method based on target detection and pose estimation, this embodiment trained target detection and human pose estimation models based on the YOLOv11 framework and effectively applied them to the climbing behavior recognition task.

[0078] The results of training the object detection model using YOLOv11 are as follows. Training and validation used labeled climbing and normal pedestrian image datasets, with two models trained using different annotation methods. Each model was trained using approximately 5000 images, and the training cycle was 150 epochs. Figure 5 , Figure 6 The results show the class recognition performance and key point recognition performance of the two models on the validation set, respectively.

[0079] Furthermore, to comprehensively evaluate detection performance, multiple metrics such as average precision, precision, recall, and frame rate are used for evaluation. Average precision is the most commonly used evaluation metric in object detection, comprehensively reflecting the average precision value of the model under different recall rates; the closer its value is to 1, the better the model performance. The calculation methods are as follows:

[0080] ;

[0081] Where mAP is the average precision, n is the number of classes, and AP i Let be the average precision of the i-th category.

[0082] The final target detection and pose estimation models achieved average accuracies of 90.2% and 95.6%, respectively, on the validation set. Figure 7 and Figure 8 This is a diagram showing the effect during actual training.

[0083] Then, combining the characteristics of each of the two models, a method for recognizing climbing behavior in videos or surveillance footage was designed. Currently, the collected video data is used in conjunction with the two models for climbing behavior recognition. The data includes 20 videos containing climbing and 34 videos of normal pedestrians without climbing, each video being approximately 10 seconds long. The test results show a recall rate of 95% and an accuracy rate of 98.18%.

[0084] It should be noted that in this embodiment, the recall rate includes the percentage of videos in which climbing behavior was correctly identified. Furthermore, the precision rate is the number of correctly identified samples divided by the total number of samples. Finally, the climbing behavior recognition interface in the video is as follows: Figure 9 As shown.

[0085] This embodiment also provides an intelligent climbing behavior recognition system that includes target detection and attitude estimation. Please refer to [link to relevant documentation]. Figure 10 It includes a data acquisition and annotation module, a data training module, a data input module, a target association module, and a behavior determination module.

[0086] The acquisition and annotation module acquires the training image dataset and annotates the images within it. This module collects climbing behavior datasets from various scenarios, focusing on visual features of climbing behavior such as body posture and scene association, and standardizes the data size and format. Then, the X-Anylabeling tool is used to perform two types of annotations on the dataset: one for object detection model training, annotating only climbing and normal pedestrians; and the other for pose estimation model training, annotating the skeletal keypoints of each person. Both types of annotation results are stored in JSON format, containing object location, category label, and keypoint coordinates. To improve model generalization ability, the annotated data is preprocessed by converting the JSON annotation files to YOLO format and dividing them into training, validation, and test sets. Abnormal data is cleaned, removing annotation errors such as missing bounding boxes, keypoint coordinates outside the image area, and blurry images, before being used for subsequent model training.

[0087] The data training module trains an object detection model based on labeled object detection data to locate human targets in images and output corresponding behavior category information; it also trains a human pose estimation model based on labeled human pose estimation data to output the location information of human key points. This module uses YOLOv11 as its basic framework, constructing object detection models for human location and preliminary classification, and a pose estimation model for extracting human key point features. A joint training strategy is employed to optimize model performance. Based on YOLOv11's feature extraction, feature fusion, and object prediction structure, the network is fine-tuned for human targets, and data augmentation methods such as geometric transformations and pixel adjustments are used during training to improve the model's generalization ability.

[0088] The data input module synchronously inputs the video frame images to be identified into the trained target detection model and human pose estimation model, respectively obtaining human target detection boxes containing behavior category information and human keypoint location information. This module uses image enhancement and dual-model inference to eliminate invalid targets with low confidence or missing keypoints, retaining only potential objects with climbing behavior characteristics.

[0089] The target association module, based on human target detection bounding boxes and human keypoint coordinate information, performs cross-frame association on the same target in the video sequence, generating a target trajectory sequence with temporal identifiers. To avoid behavior breaks caused by inter-frame target displacement, a fixed ID is assigned to the same human body through an ID tracking mechanism, and historical behavior data is continuously recorded to provide temporal basis for subsequent multi-condition judgments.

[0090] The behavior determination module extracts human motion features based on the target trajectory sequence and uses pre-set multi-condition judgment logic to determine whether climbing behavior has occurred. Based on historical data, this module uses a two-tier judgment logic of pre-filtering and core triggering to exclude non-climbing scenarios and sets core judgment conditions: a continuous judgment condition is a continuous abnormal duration ≥ 1.5 seconds, corresponding to a basic continuous warning, applicable to continuous wall climbing scenarios; a single-model high-threshold warning judgment condition is that a single model detects abnormal behavior exceeding a set threshold for 2 seconds. Meeting either condition determines it as climbing behavior, achieving accurate identification of climbing actions.

[0091] In a preferred embodiment, the system employs a joint training strategy, combining the training processes of the target detection model and the human pose estimation model. By sharing a feature extraction layer, the system improves the synergistic effect between the models and reduces computational resource consumption.

[0092] In another preferred embodiment, the behavior determination module also includes a scene adaptability adjustment function, which can automatically adjust the determination threshold according to different application scenarios, such as schools, parks, borders, etc., to improve the accuracy of the system in different environments.

[0093] Figure 11 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this specification.

[0094] like Figure 11As shown, the electronic device 1100 may include: at least one processor 1101, at least one network interface 1104, a user interface 1103, a memory 1105, and at least one communication bus 1102. The communication bus 1102 can be used to connect and communicate with the aforementioned components. The user interface 1103 may include buttons, and optionally may include standard wired or wireless interfaces. The network interface 1104 may include, but is not limited to, a Bluetooth module, an NFC module, or a Wi-Fi module. The processor 1101 may include one or more processing cores. The processor 1101 connects to various parts within the electronic device 1100 using various interfaces and lines, and performs various functions of the routing device and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1105, and by calling data stored in the memory 1105. Optionally, the processor 1101 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 1101 may integrate one or more combinations of CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content that the display screen needs to show; and the modem is used for wireless communication.

[0095] It is understandable that the aforementioned modem may not be integrated into the processor 1101, but may be implemented using a separate chip.

[0096] The memory 1105 may include RAM or ROM. Optionally, the memory 1105 may include a non-transitory computer-readable medium. The memory 1105 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1105 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1105 may also be at least one storage device located remotely from the aforementioned processor 1101. As a computer storage medium, the memory 1105 may include an operating system, a network communication module, a user interface module, and application programs. The processor 1101 may be used to call the application programs stored in the memory 1105 and execute the methods in the above-described embodiments.

[0097] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform multiple steps as described in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0098] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the multiple steps described in the above embodiments.

[0099] Where there is no conflict, the technical features in this embodiment and implementation scheme can be combined arbitrarily.

[0100] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes multiple computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates multiple available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0101] When implemented through hardware or firmware, the aforementioned method flow is programmed into the hardware circuit to obtain the corresponding hardware circuit structure and achieve the corresponding function. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit, whose logic function is determined by the user programming the device. Designers can program a digital system onto a PLD themselves, eliminating the need for chip manufacturers to design and fabricate dedicated integrated circuit chips. Furthermore, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, similar to the software compiler used in program development. The original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There is not just one HDL, but many. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of the aforementioned hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logic method flow can be easily obtained.

[0102] The above description is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined in the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for intelligent recognition of climbing behavior based on target detection and posture estimation, characterized in that, Includes the following steps: Obtain a training image dataset, and annotate the images in the dataset to obtain object detection annotation data and human pose estimation annotation data; A target detection model is trained based on the target detection annotation data to locate human targets in an image and output corresponding behavior category information; a human pose estimation model is trained based on the human pose estimation annotation data to output the location information of human key points. The video frame images to be identified are synchronously input into the trained target detection model and human pose estimation model to obtain human target detection boxes containing behavior category information and human key point location information, respectively. Based on the human target detection box and human key point coordinate information, cross-frame association is performed on the same target in the video sequence to generate a target trajectory sequence with temporal identifier. Human motion features are extracted based on the target trajectory sequence, and a pre-set multi-condition judgment logic is used to determine whether climbing behavior has occurred.

2. The intelligent climbing behavior recognition method based on target detection and attitude estimation according to claim 1, characterized in that, It also includes preprocessing the video frame images to be identified, specifically including: normalizing the image size, and performing adaptive contrast enhancement and sharpening of small target areas.

3. The intelligent recognition method for climbing behavior based on target detection and attitude estimation according to claim 1, characterized in that, Cross-frame correlation of the same target generates a target trajectory sequence with temporal identifiers, including: Matching calculations are performed based on the intersection-union ratio of human target detection boxes between the target in the current frame and each target in the previous frame, as well as the consistency of the human key point positions. If the matching calculation result meets the preset conditions, the target in the current frame is determined to be the same target as the corresponding target in the previous frame, and the target in the current frame is assigned the same identity ID; otherwise, the target in the current frame is determined to be a new target, and a new identity ID is assigned to it.

4. The intelligent recognition method for climbing behavior based on target detection and attitude estimation according to claim 3, characterized in that, For a target in the current frame that is determined to be the same target as the target in the previous frame, the system adds its human target detection box, behavior category information, and human key point location information to the target trajectory sequence corresponding to the existing identity ID. The target trajectory sequence is used to support subsequent human motion feature extraction and multi-condition determination.

5. The intelligent recognition method for climbing behavior based on target detection and attitude estimation according to claim 1, characterized in that, The human motion characteristics include displacement change characteristics and posture change characteristics calculated based on the positions of key points on the human body. The displacement change characteristics include the displacement rate of the target in the vertical direction; the posture change characteristics include the swing amplitude of the upper or lower limbs and the duration of the legs being in a stationary state.

6. The intelligent recognition method for climbing behavior based on target detection and attitude estimation according to claim 1, characterized in that, The multi-condition determination logic is based on at least one of the following conditions: Based on the temporal statistical conditions of the behavior category labels in the target trajectory sequence, including: the duration of continuous occurrence of the climbing behavior category reaches a first threshold; The abnormal state conditions based on the single-model output confidence in the target trajectory sequence include: The duration of a high-confidence abnormal state output only by one of the target detection model or the human pose estimation model reaches a second threshold.

7. The intelligent recognition method for climbing behavior based on target detection and attitude estimation according to claim 1, characterized in that, Before extracting human motion features based on the target trajectory sequence, a filtering step is also included: verifying the number of visible human key points of the target corresponding to the target trajectory sequence in the video frame. If the number of visible points is lower than a preset threshold, the subsequent behavior determination of the target trajectory sequence is skipped.

8. The intelligent recognition method for climbing behavior based on target detection and attitude estimation according to claim 1, characterized in that, After determining that climbing behavior has occurred, the following steps are also included: Generate and save an early warning log, which includes at least: the identity ID of the abnormal target, the specific condition type that triggered the judgment, the coordinate information of the human key points of the target in the current frame, and the motion feature parameter values ​​on which the judgment is based; at the same time, trigger an early warning signal for the abnormal target.

9. The intelligent recognition method for climbing behavior based on target detection and attitude estimation according to claim 1, characterized in that, The target detection annotation data includes labels identifying climbing or normal walking behavior categories; the human posture estimation annotation data includes location information of key points on the human body.

10. A climbing behavior intelligent recognition system for target detection and attitude estimation, characterized in that, include: The acquisition and annotation module acquires the training image dataset, annotates the images in the dataset, and obtains object detection annotation data and human pose estimation annotation data. The data training module trains a target detection model based on the target detection annotation data, which is used to locate human targets in the image and output corresponding behavior category information; it also trains a human pose estimation model based on the human pose estimation annotation data, which is used to output the location information of human key points. The data input module synchronously inputs the video frame images to be identified into the trained target detection model and human pose estimation model, respectively obtaining human target detection boxes containing behavior category information and human key point location information; The target association module, based on the human target detection box and the coordinate information of human key points, performs cross-frame association on the same target in the video sequence to generate a target trajectory sequence with temporal identifiers; The behavior determination module extracts human motion features based on the target trajectory sequence and determines whether climbing behavior has occurred through preset multi-condition determination logic.

Images