A method, apparatus, electronic device, and storage medium for identifying climbing behavior.
By training an object detection model to identify bounding boxes for hands and heads in images, and combining position and duration to determine climbing behavior, the problem of misjudgment in climbing behavior recognition in existing technologies is solved, and higher recognition accuracy is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- AINNOVATION (HEFEI) TECH CO LTD
- Filing Date
- 2023-08-15
- Publication Date
- 2026-05-26
AI Technical Summary
In existing technologies, relying on human and physical facilities to prevent climbing behavior has limited effectiveness, and image recognition-based climbing behavior recognition is prone to misjudgment, making it difficult to accurately determine whether a person is climbing.
By training a target detection model, the rectangular bounding boxes of the target human hands and head in the image are identified. Based on the relative position of these boxes with the wall area and the duration, the climbing behavior is determined, such as moving from the second area of the wall area to the first area or moving in the opposite direction.
It improves the accuracy of climbing behavior recognition, can distinguish between climbing from top to bottom and climbing from bottom to top, reduces false judgments, and excludes passing human bodies by judging the duration.
Smart Images
Figure CN116824704B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for recognizing climbing behavior. Background Technology
[0002] As people's safety awareness continues to improve, security issues in some buildings often rely on guard patrols or the construction of walls. However, relying solely on manpower and physical facilities to prevent climbing is often ineffective. Currently, images can be collected and human figures can be identified to determine whether a person is within the walled area. However, this method is prone to misjudgments, such as when a person passes through the walled area. Improving the accuracy of human climbing behavior recognition is an urgent problem to be solved. Summary of the Invention
[0003] The purpose of some embodiments of this application is to provide a method, apparatus, electronic device, and storage medium for recognizing climbing behavior. Through the technical solutions of the embodiments of this application, an image to be recognized is acquired; based on the image to be recognized and a pre-trained target detection model, a target human body in the image to be recognized, and corresponding hand and head bounding boxes, are determined; based on the relative positions of the hand and head bounding boxes and the wall area, the climbing behavior of the target human body is determined, wherein the climbing behavior includes at least movement from a second area of the wall area to a first area or movement from a first area of the wall area to a second area. In any of the embodiments of this application, a target detection model is trained to identify the hand and head bounding boxes of a target human body in the image to be identified. The climbing behavior of the target human body is determined based on the relative positions of the hand and head bounding boxes and the wall area, as well as the duration of the hand and head bounding boxes within the wall area. For example, the target human body may move from the second area of the wall area to the first area or from the first area of the wall area to the second area. In this way, it is possible not only to determine whether the target human body is climbing down the wall or up the wall, but also to exclude passing human bodies by judging the duration, thereby improving the detection accuracy.
[0004] Firstly, some embodiments of this application provide a method for identifying climbing behavior, including:
[0005] Acquire the image to be recognized;
[0006] Based on the image to be identified and the pre-trained target detection model, the target human body in the image to be identified, as well as the hand rectangle and head rectangle corresponding to the target human body, are determined.
[0007] The climbing behavior of the target human body is determined based on the relative positions of the hand rectangle, the head rectangle, and the wall area, as well as the duration of the hand rectangle and the head rectangle within the wall area. The climbing behavior includes at least one of moving from the second area of the wall area to the first area or moving from the first area of the wall area to the second area.
[0008] Some embodiments of this application train a target detection model to identify the bounding boxes of the hands and head of a target human in an image to be identified. Based on the relative positions of the bounding boxes of the hands and head and the wall area, as well as the duration of the bounding boxes of the hands and head within the wall area, the climbing behavior of the target human is determined. For example, either the human is moving from the second area of the wall area to the first area or from the first area of the wall area to the second area. In this way, it is possible not only to determine whether the target human is climbing down the wall or up the wall, but also to exclude passing human beings by judging the duration, thereby improving the detection accuracy.
[0009] Optionally, the climbing behavior of the target human body is determined based on the relative positions of the hand rectangle, the head rectangle, and the wall area, and the duration of the hand rectangle and the head rectangle within the wall area. The climbing behavior includes at least one of moving from a second area of the wall area to a first area or moving from a first area of the wall area to a second area, including:
[0010] If the hand rectangle and the head rectangle are located in the first area of the wall area, and at least one hand rectangle is located above the head rectangle, and the first duration for at least one hand rectangle being located above the head rectangle is greater than a first preset value, then it is determined that the target human body moves from the first area of the wall area to the second area.
[0011] If the hand rectangle and the head rectangle are located in the second region of the wall area, and the second duration of the hand rectangle and the head rectangle being located in the second region of the wall area is greater than a second preset value, it is determined that the target human body moves from the second region of the wall area to the first region, wherein the wall area includes at least the first region and the second region, and the first region is located at the lower part of the second region.
[0012] Some embodiments of this application determine the climbing behavior of the target human body, such as climbing from top to bottom or from bottom to top, by judging the relative positions of the hand rectangle and head rectangle on the wall area, such as whether they are at the top or bottom of the wall area, and by judging the duration, thereby improving the accuracy of climbing behavior detection.
[0013] Optionally, after determining the target human body in the image to be identified, and the corresponding hand and head bounding boxes based on the image to be identified and a pre-trained target detection model, the method further includes:
[0014] The bounding box of the target human body in the image to be identified is tracked, and the target human body is numbered to obtain the numbering information corresponding to the target human body.
[0015] Based on the identification information of the target human body, the rectangular frames of the hand and the head are tracked.
[0016] Some embodiments of this application set numbering information for human body rectangles, which can effectively identify human bodies in images, number each human body, and facilitate human body tracking.
[0017] Optionally, the step of tracking the hand rectangle and the head rectangle based on the target human body's identification information includes:
[0018] Obtain the first center point of the hand rectangle;
[0019] Calculate the first distance between the first center point and the second center point of each of the human body rectangles;
[0020] Obtain the minimum distance from the first distance;
[0021] If the minimum distance is less than a third preset value, it is determined that the numbering information of the hand rectangle is the same as the numbering information of the human body rectangle corresponding to the minimum distance;
[0022] Calculate the overlapping area of the head rectangle and each of the human body rectangles;
[0023] If the area ratio of the overlapping region to the head rectangle is greater than a fourth preset value, then the numbering information of the head region is determined to be the same as the numbering information of the human body rectangle.
[0024] In some embodiments of this application, the numbering information of each hand rectangle and the numbering information of the head rectangle are determined by calculating the distance from the center point of the hand rectangle to the center point of each human body rectangle, thereby improving the accuracy of tracking.
[0025] Optionally, the method further includes:
[0026] If a target human body is detected climbing the fence area, an alarm message will be displayed.
[0027] Some embodiments of this application detect a target human climbing a fence area and display alarm information. The alarm information can be displayed on a terminal device or an audible alarm can be used to improve the timeliness of the alarm.
[0028] Optionally, the pre-trained target detection model is obtained in the following manner:
[0029] Obtain training sample data, and define the human body rectangle, hand rectangle, and head rectangle in the training sample data;
[0030] The neural network model is trained based on the training sample data to obtain the trained model;
[0031] If the loss function corresponding to the trained model is less than a preset value, then the trained model corresponding to the loss function is determined as the object detection model.
[0032] Some embodiments of this application train an object detection model using the YOLOv5 model, which has low computational requirements and low hardware resource requirements.
[0033] Secondly, some embodiments of this application provide a climbing behavior recognition device, including:
[0034] The acquisition module is used to acquire the image to be recognized;
[0035] The detection module is used to determine the target human body in the image to be identified, as well as the hand rectangle and head rectangle corresponding to the target human body, based on the image to be identified and the pre-trained target detection model.
[0036] The recognition module is used to determine the climbing behavior of the target human body based on the relative positions of the hand rectangle, the head rectangle, and the wall area, and the duration of the hand rectangle and the head rectangle within the wall area, wherein the climbing behavior includes at least one of moving from the second area of the wall area to the first area or moving from the first area of the wall area to the second area.
[0037] Some embodiments of this application train a target detection model to identify the bounding boxes of the hands and head of a target human in an image to be identified. Based on the relative positions of the bounding boxes of the hands and head and the wall area, as well as the duration of the bounding boxes of the hands and head within the wall area, the climbing behavior of the target human is determined. For example, either the human is moving from the second area of the wall area to the first area or from the first area of the wall area to the second area. In this way, it is possible not only to determine whether the target human is climbing down the wall or up the wall, but also to exclude passing human beings by judging the duration, thereby improving the detection accuracy.
[0038] Optionally, the identification module is used for:
[0039] If the hand rectangle and the head rectangle are located in the first area of the wall area, and at least one hand rectangle is located above the head rectangle, and the first duration for at least one hand rectangle being located above the head rectangle is greater than a first preset value, then it is determined that the target human body moves from the first area of the wall area to the second area.
[0040] If the hand rectangle and the head rectangle are located in the second region of the wall area, and the second duration of the hand rectangle and the head rectangle being located in the second region of the wall area is greater than a second preset value, it is determined that the target human body moves from the second region of the wall area to the first region, wherein the wall area includes at least the first region and the second region, and the first region is located at the lower part of the second region.
[0041] Some embodiments of this application determine the climbing behavior of the target human body, such as climbing from top to bottom or from bottom to top, by judging the relative positions of the hand rectangle and head rectangle on the wall area, such as whether they are at the top or bottom of the wall area, and by judging the duration, thereby improving the accuracy of climbing behavior detection.
[0042] Optionally, the detection module is further configured to:
[0043] The bounding box of the target human body in the image to be identified is tracked, and the target human body is numbered to obtain the numbering information corresponding to the target human body.
[0044] Based on the identification information of the target human body, the rectangular frames of the hand and the head are tracked.
[0045] Some embodiments of this application set numbering information for human body rectangles, which can effectively identify human bodies in images, number each human body, and facilitate human body tracking.
[0046] Optionally, the detection module is further configured to:
[0047] Obtain the first center point of the hand rectangle;
[0048] Calculate the first distance between the first center point and the second center point of each of the human body rectangles;
[0049] Obtain the minimum distance from the first distance;
[0050] If the minimum distance is less than a third preset value, it is determined that the numbering information of the hand rectangle is the same as the numbering information of the human body rectangle corresponding to the minimum distance;
[0051] Calculate the overlapping area of the head rectangle and each of the human body rectangles;
[0052] If the area ratio of the overlapping region to the head rectangle is greater than a fourth preset value, then the numbering information of the head region is determined to be the same as the numbering information of the human body rectangle.
[0053] In some embodiments of this application, the numbering information of each hand rectangle and the numbering information of the head rectangle are determined by calculating the distance from the center point of the hand rectangle to the center point of each human body rectangle, thereby improving the accuracy of tracking.
[0054] Optionally, the device further includes an alarm module, the alarm module being used for:
[0055] If a target human body is detected climbing the fence area, an alarm message will be displayed.
[0056] Some embodiments of this application detect a target human climbing a fence area and display alarm information. The alarm information can be displayed on a terminal device or an audible alarm can be used to improve the timeliness of the alarm.
[0057] Optionally, the apparatus further includes a model building module, the model building module being used for:
[0058] Obtain training sample data, and define the human body rectangle, hand rectangle, and head rectangle in the training sample data;
[0059] The neural network model is trained based on the training sample data to obtain the trained model;
[0060] If the loss function corresponding to the trained model is less than a preset value, then the trained model corresponding to the loss function is determined as the object detection model.
[0061] Some embodiments of this application train an object detection model using the YOLOv5 model, which has low computational requirements and low hardware resource requirements.
[0062] Thirdly, some embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can implement the climbing behavior recognition method as described in any embodiment of the first aspect.
[0063] Fourthly, some embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the climbing behavior recognition method as described in any embodiment of the first aspect.
[0064] Fifthly, some embodiments of this application provide a computer program product, the computer program product including a computer program, wherein when the computer program is executed by a processor, it can implement the climbing behavior recognition method as described in any embodiment of the first aspect. Attached Figure Description
[0065] To more clearly illustrate the technical solutions of some embodiments of this application, the accompanying drawings used in some embodiments of this application will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0066] Figure 1 A flowchart illustrating a method for recognizing climbing behavior provided in an embodiment of this application;
[0067] Figure 2 A schematic flowchart illustrating another method for identifying climbing behavior provided in an embodiment of this application;
[0068] Figure 3 A schematic diagram of the structure of a climbing behavior recognition device provided in an embodiment of this application;
[0069] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0070] The technical solutions of some embodiments of this application will now be described with reference to the accompanying drawings.
[0071] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0072] With the increasing awareness of safety, security issues in some construction sites often rely on guard patrols or the erection of walls. However, relying solely on manpower and physical facilities to prevent climbing is often ineffective. Currently, images can be captured and human figures identified to determine if a person is within a walled area. However, this method is prone to misjudgments, such as when a person passes through the walled area. Therefore, some embodiments of this application provide a method for identifying climbing behavior. This method includes acquiring an image to be identified; determining the target human body in the image, along with corresponding hand and head bounding boxes, based on the image and a pre-trained target detection model; and determining the relative positions of the hand and head bounding boxes to the walled area. The method determines the climbing behavior of a target human body, wherein the climbing behavior includes at least one of moving from the second area of the wall area to the first area or moving from the first area of the wall area to the second area. In this embodiment, the target detection model is trained to identify the hand and head bounding boxes of the target human body in the image to be identified, and the climbing behavior of the target human body is determined based on the relative position of the hand and head bounding boxes and the wall area, as well as the duration of the hand and head bounding boxes within the wall area. For example, moving from the second area of the wall area to the first area or moving from the first area of the wall area to the second area. In this way, it is possible not only to determine whether the target human body is climbing down the wall or up the wall, but also to exclude passing human bodies by judging the duration, thereby improving the detection accuracy.
[0073] like Figure 1 As shown, embodiments of this application provide a method for identifying climbing behavior, the method comprising:
[0074] S101. Obtain the image to be recognized;
[0075] Specifically, to monitor whether there is climbing behavior outside a building, a data acquisition device, such as a camera or video camera, is set up at a fixed location outside the building. The data acquisition device collects video data or photos of a preset fence area, and the terminal device receives the video data or photos sent by the data acquisition device, which is the image to be identified.
[0076] S102. Based on the image to be identified and the pre-trained target detection model, determine the target human body in the image to be identified, as well as the hand rectangle and head rectangle corresponding to the target human body.
[0077] Specifically, a target detection model is pre-trained on the terminal device. The YOLOv5 model is trained using sample data to obtain the target detection model. This target detection model is used to detect human body rectangles, hand rectangles, and head rectangles in an image. The hand rectangle includes at least one hand rectangle or two hand rectangles.
[0078] After acquiring the image to be recognized, the terminal device inputs the image into the pre-trained target detection model to identify the bounding boxes of the human hand and head in the image. During this process, the image to be recognized can contain one human body, or two or more human bodies; this is not specifically limited in this embodiment.
[0079] S103. Based on the relative positions of the hand rectangle, head rectangle, and the wall area, and the duration of the hand rectangle and head rectangle within the wall area, determine the climbing behavior of the target human body, wherein the climbing behavior includes at least one of moving from the second area of the wall area to the first area or moving from the first area of the wall area to the second area.
[0080] Specifically, after acquiring the hand and head bounding boxes of each human body in the image to be identified, the terminal device pre-divides the wall area into a first region and a second region, with the first region located below the second region. The terminal device determines the position of the hand and head bounding boxes relative to the first and second regions, and also determines the duration of the hand and head bounding boxes within a certain region to determine the climbing behavior of the target human body in the image to be identified. For example, the climbing behavior includes at least moving from the second region of the wall area to the first region or from the first region of the wall area to the second region, thereby determining whether the human body in the image to be identified is temporarily stopping or climbing the wall, thus improving the accuracy of climbing behavior recognition.
[0081] Some embodiments of this application train a target detection model to identify the bounding boxes of the hands and head of a target human in an image to be identified. Based on the relative positions of the bounding boxes of the hands and head and the wall area, as well as the duration of the bounding boxes of the hands and head within the wall area, the climbing behavior of the target human is determined. For example, either the human is moving from the second area of the wall area to the first area or from the first area of the wall area to the second area. In this way, it is possible not only to determine whether the target human is climbing down the wall or up the wall, but also to exclude passing human beings by judging the duration, thereby improving the detection accuracy.
[0082] Another embodiment of this application further supplements the description of the climbing behavior recognition method provided in the above embodiments.
[0083] Optionally, the climbing behavior of the target human body is determined based on the relative positions of the hand rectangle, the head rectangle, and the wall area, and the duration of the hand and head rectangles within the wall area. The climbing behavior includes at least one of movement from the second area of the wall area to the first area or from the first area of the wall area to the second area, including:
[0084] If the hand rectangle and the head rectangle are located in the first area of the wall area, and at least one hand rectangle is located above the head rectangle, and the first duration of at least one hand rectangle being located above the head rectangle is greater than a first preset value, then it is determined that the target human body moves from the first area of the wall area to the second area.
[0085] If the hand rectangle and head rectangle are located in the second region of the enclosure area, and the second duration of the hand rectangle and head rectangle being located in the second region of the enclosure area is greater than a second preset value, it is determined that the target human body moves from the second region of the enclosure area to the first region. The enclosure area includes at least the first region and the second region, and the first region is located at the lower part of the second region.
[0086] Specifically, when the terminal device judges the rectangular frames of the hand and head of the target human body, if the rectangular frames of the hand and head are located in the first area of the wall area, that is, in the lower area of the wall, and the duration of the rectangular frames of the hand and head in the lower area of the wall is greater than a first preset value, such as 2 seconds, and it is also necessary to determine that the position of the rectangular frame of the hand is higher than that of the rectangular frame of the head, then it can be determined that the target human body in the image to be recognized is climbing the wall from bottom to top.
[0087] When the terminal device judges the bounding boxes of the hand and head of the target human body, if the bounding boxes of the hand and head are located in the second area of the wall area, that is, in the upper area of the wall, and the duration of the bounding boxes of the hand and head in the lower area of the wall is greater than a second preset value, such as 2 seconds, then it can be determined that the target human body in the image to be recognized is climbing the wall from top to bottom.
[0088] Some embodiments of this application determine the climbing behavior of the target human body, such as climbing from top to bottom or from bottom to top, by judging the relative positions of the hand rectangle and head rectangle on the wall area, such as whether they are at the top or bottom of the wall area, and by judging the duration, thereby improving the accuracy of climbing behavior detection.
[0089] Optionally, after determining the target human body in the image to be identified, and the corresponding hand and head bounding boxes based on the image to be identified and a pre-trained target detection model, the method further includes:
[0090] The target human body bounding box in the image to be identified is tracked, and the target human body is numbered to obtain the number information corresponding to the target human body.
[0091] Based on the target human body's identification information, track the hand and head bounding boxes.
[0092] The number information refers to the number of the human body entering the image to be identified. For example, the first human body to enter the image is number 1, the second human body to enter is number 2, and so on.
[0093] Some embodiments of this application set numbering information for human body rectangles, which can effectively identify human bodies in images, number each human body, and facilitate human body tracking.
[0094] Optionally, based on the target human body's identification information, the hand bounding box and head bounding box are tracked, including:
[0095] Obtain the first center point of the hand rectangle;
[0096] Calculate the first distance between the first center point and the second center point of each human body rectangle;
[0097] Find the minimum distance in the first distance;
[0098] If the minimum distance is less than the third preset value, the numbering information of the hand rectangle is determined to be the same as the numbering information of the human body rectangle corresponding to the minimum distance;
[0099] Calculate the overlap area between the head rectangle and the rectangles of each body;
[0100] If the area ratio of the overlapping region to the head rectangle is greater than the fourth preset value, then the numbering information of the head region is determined to be the same as the numbering information of the human body rectangle.
[0101] Specifically, the terminal device judges each hand rectangle in the image to be recognized to determine which target human body each hand rectangle belongs to. This involves calculating the first center point of the hand rectangle, then calculating the distance between the first center point and the center point of each human body rectangle, selecting the minimum distance from multiple distances, and comparing the minimum distance with a third preset value. If the minimum distance is less than the third preset value, it means that the numbering information of the hand rectangle is the same as the numbering information of the target human body.
[0102] The terminal device calculates the overlapping area between the head rectangle and each human body rectangle. When the area ratio of the overlapping area to the head rectangle is greater than a fourth preset value, such as 0.8, the head rectangle is considered to belong to the corresponding tracked person, that is, the number information of the head rectangle is the same as the number information of the target human body.
[0103] In some embodiments of this application, the numbering information of each hand rectangle and the numbering information of the head rectangle are determined by calculating the distance from the center point of the hand rectangle to the center point of each human body rectangle, thereby improving the accuracy of tracking.
[0104] Optionally, the method further includes:
[0105] If a person is detected climbing the perimeter wall, an alarm will be displayed.
[0106] Some embodiments of this application detect a target human climbing a fence area and display alarm information. The alarm information can be displayed on a terminal device or an audible alarm can be used to improve the timeliness of the alarm.
[0107] Optionally, the pre-trained object detection model is obtained in the following way:
[0108] Obtain training sample data and analyze the bounding boxes for the human body, hand, and head in the training sample data.
[0109] The neural network model is trained based on the training sample data to obtain the trained model;
[0110] If the loss function corresponding to the trained model is less than the preset value, then the trained model corresponding to the loss function will be determined as the object detection model.
[0111] Some embodiments of this application train an object detection model using the YOLOv5 model, which has low computational requirements and low hardware resource requirements.
[0112] Figure 2 A schematic flowchart illustrating another method for identifying climbing behavior provided in this application embodiment, the method for identifying climbing behavior including:
[0113] Step 0: Get a camera image.
[0114] Step 1: The terminal device is equipped with a detection module, which includes a pre-trained target detection model. The pre-trained target detection model is used to detect human body rectangles, hand rectangles, and head rectangles.
[0115] The specific training process is as follows:
[0116] A training dataset of 5000 images is obtained, with annotations for three categories: human body bounding boxes, hand bounding boxes, and head bounding boxes. The YOLOv5 model is trained using this dataset, and a loss function is calculated. This loss function consists of at least three parts: bounding box loss, confidence loss, and classification loss. The overall loss is a weighted sum of the three losses, typically with the confidence loss having the highest weight, followed by the bounding box loss and classification loss.
[0117] The training image input resolution is 640*640, the batch size is 64, and the number of iterations (epoch) is 200. That is, the number of epochs is a hyperparameter that defines the number of times the learning algorithm works on the entire training dataset, i.e., how many times the dataset is trained.
[0118] During model inference, a confidence threshold of 0.5 and an NMS (Non-Maximum Suppression) threshold of 0.5 were used.
[0119] Step 2: The terminal device also includes a tracking module, which tracks the human body bounding box in the image to be recognized. Specifically, this includes:
[0120] First, target tracking is performed on the rectangle containing the person obtained in step 1 to obtain the ID information of each person.
[0121] Then, for the hand rectangle detected in step 1, calculate the minimum distance between the center point of the hand rectangle and the center points of all tracked human body rectangles. If the minimum distance is less than a certain threshold, the hand region is considered to belong to the corresponding tracked person, meaning the hand's identification information matches that of the person being tracked.
[0122] For the head bounding box detected in step 1, calculate its overlapping area with all tracked human bounding boxes. If the area ratio of the overlapping area to the head bounding box is greater than 0.8, then the head region is considered to belong to the corresponding tracked person, that is, the head's identification information matches the tracked person's identification information.
[0123] Step 3: Determine if the hand and head rectangles enter the fenced area. Based on the numbered rectangle information obtained in Step 2, determine if any rectangles have entered the fenced area. If so, proceed to Step 4; otherwise, proceed to Step 5.
[0124] Step 4: Determine if alarm data has been cached. If a numbered rectangle has entered the fenced area, check if the numbering information has been cached in that fenced area. If not, proceed to Step 5. If it has been cached, proceed to Step 6.
[0125] Step 5: Cache alarm data. Cache the current image data, as well as all currently identified bounding boxes.
[0126] Step 6: Determine the order of entry into the partition. If the human body rectangle information to which a certain number belongs appears first in the lower half of the fence (the first area) and then appears later in the upper half of the fence (the second area), then proceed to step 7; otherwise, proceed to step 8.
[0127] Step 7: Determine if the hand region of the abnormal number was located above the head region for a sustained period of time, such as 2 seconds. If yes, proceed to step 8; otherwise, proceed to step 0.
[0128] Step 8: Determine whether the hand and head rectangles belonging to the abnormal numbers have existed in all the second areas for a duration exceeding a certain time, such as 2 seconds. If yes, proceed to step 9; otherwise, proceed to step 0.
[0129] Step 9: Generate an alarm. Send out all cached alarm data for the alarm number where the anomaly exists.
[0130] It should be noted that each of the implementable methods in this embodiment can be implemented individually or in any combination without conflict. This application does not limit this.
[0131] Another embodiment of this application provides a climbing behavior recognition device for performing the climbing behavior recognition method provided in the above embodiments.
[0132] like Figure 3 The diagram shown is a structural schematic of a climbing behavior recognition device provided in an embodiment of this application. The climbing behavior recognition device includes, wherein:
[0133] The acquisition module 301 is used to acquire the image to be recognized;
[0134] The detection module 302 is used to determine the target human body in the image to be identified, as well as the hand rectangle and head rectangle corresponding to the target human body, based on the image to be identified and the pre-trained target detection model.
[0135] The recognition module 303 is used to determine the climbing behavior of the target human body based on the relative positions of the hand rectangle, the head rectangle and the wall area, and the duration of the hand rectangle and the head rectangle within the wall area. The climbing behavior includes at least one of moving from the second area of the wall area to the first area or moving from the first area of the wall area to the second area.
[0136] Regarding the apparatus in this embodiment, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0137] Some embodiments of this application train a target detection model to identify the bounding boxes of the hands and head of a target human in an image to be identified. Based on the relative positions of the bounding boxes of the hands and head and the wall area, as well as the duration of the bounding boxes of the hands and head within the wall area, the climbing behavior of the target human is determined. For example, either the human is moving from the second area of the wall area to the first area or from the first area of the wall area to the second area. In this way, it is possible not only to determine whether the target human is climbing down the wall or up the wall, but also to exclude passing human beings by judging the duration, thereby improving the detection accuracy.
[0138] Another embodiment of this application further illustrates the climbing behavior recognition device provided in the above embodiments.
[0139] Optionally, the identification module is used for:
[0140] If the hand rectangle and the head rectangle are located in the first area of the wall area, and at least one hand rectangle is located above the head rectangle, and the first duration of at least one hand rectangle being located above the head rectangle is greater than a first preset value, then it is determined that the target human body moves from the first area of the wall area to the second area.
[0141] If the hand rectangle and head rectangle are located in the second region of the enclosure area, and the second duration of the hand rectangle and head rectangle being located in the second region of the enclosure area is greater than a second preset value, it is determined that the target human body moves from the second region of the enclosure area to the first region. The enclosure area includes at least the first region and the second region, and the first region is located at the lower part of the second region.
[0142] Some embodiments of this application determine the climbing behavior of the target human body, such as climbing from top to bottom or from bottom to top, by judging the relative positions of the hand rectangle and head rectangle on the wall area, such as whether they are at the top or bottom of the wall area, and by judging the duration, thereby improving the accuracy of climbing behavior detection.
[0143] Optionally, the detection module is also used for:
[0144] The target human body bounding box in the image to be identified is tracked, and the target human body is numbered to obtain the number information corresponding to the target human body.
[0145] Based on the target human body's identification information, track the bounding boxes for the hand and head.
[0146] Some embodiments of this application set numbering information for human body rectangles, which can effectively identify human bodies in images, number each human body, and facilitate human body tracking.
[0147] Optionally, the detection module is also used for:
[0148] Obtain the first center point of the hand rectangle;
[0149] Calculate the first distance between the first center point and the second center point of each human body rectangle;
[0150] Find the minimum distance in the first distance;
[0151] If the minimum distance is less than the third preset value, the numbering information of the hand rectangle is determined to be the same as the numbering information of the human body rectangle corresponding to the minimum distance;
[0152] Calculate the overlap area between the head rectangle and the rectangles of each body;
[0153] If the area ratio of the overlapping region to the head rectangle is greater than the fourth preset value, then the numbering information of the head region is determined to be the same as the numbering information of the human body rectangle.
[0154] In some embodiments of this application, the numbering information of each hand rectangle and the numbering information of the head rectangle are determined by calculating the distance from the center point of the hand rectangle to the center point of each human body rectangle, thereby improving the accuracy of tracking.
[0155] Optionally, the device further includes an alarm module, which is used for:
[0156] If a person is detected climbing the perimeter wall, an alarm will be displayed.
[0157] Some embodiments of this application detect a target human climbing a fence area and display alarm information. The alarm information can be displayed on a terminal device or an audible alarm can be used to improve the timeliness of the alarm.
[0158] Optionally, the device further includes a model building module, which is used for:
[0159] Obtain training sample data and analyze the bounding boxes for the human body, hand, and head in the training sample data.
[0160] The neural network model is trained based on the training sample data to obtain the trained model;
[0161] If the loss function corresponding to the trained model is less than the preset value, then the trained model corresponding to the loss function will be determined as the object detection model.
[0162] Some embodiments of this application train an object detection model using the YOLOv5 model, which has low computational requirements and low hardware resource requirements.
[0163] Regarding the apparatus in this embodiment, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0164] It should be noted that each of the implementable methods in this embodiment can be implemented individually or in any combination without conflict. This application does not limit this.
[0165] This application also provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it can implement the operation of any of the methods corresponding to the climbing behavior recognition methods provided in the above embodiments.
[0166] This application also provides a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operation of any of the methods corresponding to the climbing behavior recognition methods provided in the above embodiments.
[0167] like Figure 4 As shown, some embodiments of this application provide an electronic device 400, which includes: a memory 410, a processor 420, and a computer program stored in the memory 410 and executable on the processor 420. When the processor 420 reads the program from the memory 410 via a bus 430 and executes the program, it can implement any of the methods included in the above-described climbing behavior recognition method.
[0168] Processor 420 can process digital signals and may include various computing architectures. For example, it may be a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements multiple instruction set combinations. In some examples, processor 420 may be a microprocessor.
[0169] Memory 410 can be used to store instructions executed by processor 420 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all of the functions of one or more modules described in the embodiments of this application. The processor 420 of this disclosure embodiment can be used to execute instructions in memory 410 to implement the methods shown above. Memory 410 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.
[0170] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0171] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0172] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. A method for identifying climbing behavior, characterized in that, The method includes: Acquire the image to be recognized; Based on the image to be identified and the pre-trained target detection model, the target human body in the image to be identified, as well as the hand rectangle and head rectangle corresponding to the target human body, are determined. The climbing behavior of the target human body is determined based on the relative positions of the hand rectangle, the head rectangle, and the wall area, as well as the duration of the hand rectangle and the head rectangle within the wall area. The climbing behavior includes at least one of moving from the second area of the wall area to the first area or moving from the first area of the wall area to the second area. The step of determining the climbing behavior of the target human body based on the relative positions of the hand rectangle, the head rectangle, and the wall area, and the duration of the hand rectangle and the head rectangle within the wall area, includes: if the hand rectangle and the head rectangle are located in a first area of the wall area, and at least one hand rectangle is located above the head rectangle, and the first duration of at least one hand rectangle being above the head rectangle is greater than a first preset value, then it is determined that the target human body is moving from the first area of the wall area to a second area; if the hand rectangle and the head rectangle are located in a second area of the wall area, and the second duration of the hand rectangle and the head rectangle being located in the second area of the wall area is greater than a second preset value, then it is determined that the target human body is moving from the second area of the wall area to the first area, wherein the wall area includes at least a first area and a second area, and the first area is located below the second area; The method further includes: when the target human body is detected to enter the wall area, caching the rectangular frame data of the target human body's hand and head, and determining the duration of the target human body's stay in the wall area and the order in which it enters the partition based on the cached data.
2. The method for identifying climbing behavior according to claim 1, characterized in that, After determining the target human body in the image to be identified, and the corresponding hand and head bounding boxes based on the image to be identified and a pre-trained target detection model, the method further includes: The bounding box of the target human body in the image to be identified is tracked, and the target human body is numbered to obtain the numbering information corresponding to the target human body. Based on the identification information of the target human body, the rectangular frames of the hand and the head are tracked.
3. The method for identifying climbing behavior according to claim 2, characterized in that, The step of tracking the hand rectangle and the head rectangle based on the target human body's identification information includes: Obtain the first center point of the hand rectangle; Calculate the first distance between the first center point and the second center point of each of the human body rectangles; Obtain the minimum distance from the first distance; If the minimum distance is less than a third preset value, it is determined that the numbering information of the hand rectangle is the same as the numbering information of the human body rectangle corresponding to the minimum distance; Calculate the overlapping area of the head rectangle and each of the human body rectangles; If the area ratio of the overlapping region to the head rectangle is greater than a fourth preset value, then the numbering information of the head region is determined to be the same as the numbering information of the human body rectangle.
4. The method for identifying climbing behavior according to claim 1, characterized in that, The method further includes: If a target human body is detected climbing the fence area, an alarm message will be displayed.
5. The method for identifying climbing behavior according to claim 1, characterized in that, The pre-trained target detection model is obtained in the following way: Obtain training sample data, and define the human body rectangle, hand rectangle, and head rectangle in the training sample data; The neural network model is trained based on the training sample data to obtain the trained model; If the loss function corresponding to the trained model is less than a preset value, then the trained model corresponding to the loss function is determined as the object detection model.
6. A climbing behavior recognition device, characterized in that, The device includes: The acquisition module is used to acquire the image to be recognized; The detection module is used to determine the target human body in the image to be identified, as well as the hand rectangle and head rectangle corresponding to the target human body, based on the image to be identified and the pre-trained target detection model. The recognition module is used to determine the climbing behavior of the target human body based on the relative positions of the hand rectangle, the head rectangle and the wall area, and the duration of the hand rectangle and the head rectangle within the wall area, wherein the climbing behavior includes at least one of moving from the second area of the wall area to the first area or moving from the first area of the wall area to the second area. The recognition module is configured to determine that the target human body is moving from the first area of the wall area to the second area if the hand rectangle and the head rectangle are located in the first area of the wall area, and at least one hand rectangle is located above the head rectangle, and the first duration of at least one hand rectangle being located above the head rectangle is greater than a first preset value; and to determine that the target human body is moving from the second area of the wall area to the first area if the hand rectangle and the head rectangle are located in the second area of the wall area, and the second duration of the hand rectangle and the head rectangle being located in the second area of the wall area is greater than a second preset value, wherein the wall area includes at least a first area and a second area, and the first area is located at the lower part of the second area; The detection module is further configured to cache the rectangular frame data of the target human body's hands and head when the target human body is detected to enter the wall area, and determine the duration of the target human body's stay in the wall area and the order in which it enters the partition based on the cached data.
7. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the climbing behavior recognition method according to any one of claims 1-5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, characterized in that, when the program is executed by a processor, it can implement the climbing behavior recognition method according to any one of claims 1-5.