A human skeleton-based three-dimensional area intrusion detection method and system

Abstract

The application belongs to the field of electronic information technology, and discloses a three-dimensional area intrusion detection method based on human skeleton, which comprises the following steps: step 1: when a human body enters a camera area, the human skeleton is recognized to obtain skeleton information, and a plurality of position points are marked in the skeleton information; step 2: when the human body enters an escalator, a first image is acquired, and the height relationship of two position points at different heights of a handrail and the human body is calculated; step 3: in the process of operation of the escalator, a second image is acquired, and a point on the handrail which maintains the same horizontal height with the human body is determined according to the height relationship and the two position points; and step 4: a plumb line is drawn on the point determined in step 3 on the handrail, and it is judged whether the human skeleton intersects with the plumb line; if the human skeleton intersects with the plumb line, an alarm is sent. The method of the application realizes accurate, rapid and intelligent alarm. Meanwhile, the application also provides a three-dimensional area intrusion detection system based on human skeleton.

Description

Technical Field

[0001] The present invention belongs to the field of electronic information technology, and more specifically, relates to a three-dimensional area intrusion detection method and system based on human skeleton. Background Art

[0002] Currently, there are two common methods for escalator area intrusion detection through image processing.

[0003] Two-dimensional mixing line method:

[0004] By identifying the position of the human body in the image and calculating whether the human movement trajectory intersects with the mixing line, it is possible to determine whether there is an intrusion in the two-dimensional space. This method requires relatively little computation and does not require human skeleton recognition. It is currently the most commonly used method.

[0005] However, this method is not suitable for judging in three-dimensional space with depth of field. As shown in the following image, the human arm does not actually cross the handrail area, but intersects with the trip wire. Because the trip wire has depth of field, a simple trip wire intersection method cannot be used to judge, otherwise it will cause many false positives.

[0006] In addition, when the human body does not move but only stretches the torso, there is no movement trajectory, and it is impossible to determine whether the human torso intersects with the mixing line.

[0007] Three-dimensional space algorithm:

[0008] By identifying the three-dimensional space of the human skeleton, calculating whether it intersects with the three-dimensional plane space, it is determined whether there is an invasion in the three-dimensional space.

[0009] As shown in the figure below, a two-dimensional image is input, and the three-dimensional spatial coordinates of the human skeleton are obtained through the AI ​​algorithm. Then the content of the entire image is mapped into three-dimensional space before an intrusion judgment can be made.

[0010] This intrusion judgment method is relatively accurate, but its AI algorithm has a very large computational load, which is not conducive to low-cost deployment.

[0011] Therefore, the substantive problem solved in this case is: how to improve the accuracy and intelligence of elevator intrusion while reducing the server computing power. Summary of the Invention

[0012] The main purpose of the present invention is to provide a three-dimensional area intrusion detection method based on human skeleton, which realizes accurate, rapid and intelligent alarm.

[0013] At the same time, the present invention also provides a three-dimensional area intrusion detection system based on human skeleton.

[0014] According to a first aspect of the present invention, a method for detecting three-dimensional intrusion in a human skeleton is provided. The method is applicable to the scenario of detecting intrusion in a sidebar with a regular height, and comprises the following steps:

[0015] Step 1: When a human body enters the camera area, the human skeleton is identified to obtain skeleton information, in which multiple position points are marked;

[0016] Step 2: When the human body moves to one side of the sidebar, a first image is acquired, and a height relationship between two positions at different heights of the sidebar and the human body is calculated;

[0017] Step 3: While the human body is moving along the siderail, a second image is acquired, and a point on the siderail that maintains the same horizontal height as the human body is determined based on the height relationship and the two position points;

[0018] Step 4: Draw a plumb line on the point determined in step 3 on the sidebar to determine whether the human skeleton intersects with the plumb line. If so, an alarm is issued.

[0019] The scenarios of intrusion detection with a sidebar of regular height in the present invention may include the following scenarios:

[0020] 1. Detection of climbing over handrails and other body parts intrusion on escalators as described below;

[0021] 2. Warning of people climbing over railings with a constant height, such as bridge railings and riverside railings;

[0022] 3. Human climbing warning for walls with constant height.

[0023] The following description is based on Scenario 1.

[0024] The above-mentioned three-dimensional region intrusion detection method based on human skeleton includes the following steps:

[0025] Step 1: When a human body enters the camera area, the human skeleton is identified to obtain skeleton information, in which multiple position points are marked;

[0026] Step 2: When a person enters the escalator, a first image is acquired, and the height relationship between the handrail and two positions at different heights of the person is calculated;

[0027] Step 3: During the operation of the escalator, a second image is acquired, and a point on the handrail that maintains the same horizontal height as the human body is determined based on the height relationship and the two position points;

[0028] Step 4: Draw a plumb line on the handrail at the point determined in step 3 to determine whether the human skeleton intersects with the plumb line. If so, an alarm is issued.

[0029] In the above-mentioned three-dimensional area intrusion detection method based on human skeleton, the position points in the skeleton information include: three or more points corresponding to the joints on the limbs, hip joints, shoulder joints, head and facial features.

[0030] In the above-mentioned three-dimensional area intrusion detection method based on human skeleton, the skeleton information is calculated by a human posture algorithm.

[0031] In the above-mentioned three-dimensional area intrusion detection method based on human skeleton, the upper entrance and the lower entrance of the escalator are both provided with sensors for sensing whether there is an object on the pedals of the escalator; when the sensor detects that there is an object at the upper entrance or the lower entrance of the escalator, step 2 is performed.

[0032] In the above-mentioned three-dimensional area intrusion detection method based on human skeleton, the step 2 is specifically as follows:

[0033] Step 21: When the human body enters the escalator, a first image is acquired;

[0034] Step 22: Draw a line and two position points representing the handrail from the first image;

[0035] Step 23: Draw horizontal lines along the two points and a vertical line between the two horizontal lines;

[0036] Step 24: Translate the vertical line obtained in step 23 so that the handrail line intersects the vertical line to obtain an intersection point, and determine the ratio of the distance from the intersection point to the upper end of the vertical line and the distance from the intersection point to the lower end of the vertical line; the ratio is the height relationship.

[0037] In the above-mentioned three-dimensional area intrusion detection method based on human skeleton, the step 3 is specifically as follows:

[0038] Step 31: Acquire a second image during the operation of the escalator;

[0039] Step 32: Draw a line and two position points representing the handrail from the second image;

[0040] Step 33: Draw horizontal lines along the two points and a vertical line between the two horizontal lines;

[0041] Step 34: Determine the point on the vertical line according to the height relationship, and draw a horizontal line along the point on the vertical line. The intersection of the horizontal line and the line representing the handrail is the point on the handrail that maintains the same horizontal height as the human body.

[0042] In addition, the present invention also provides a system for implementing the above method, comprising a camera and a server, wherein the camera's imaging area covers at least a portion of the sidebar;

[0043] The server includes the following modules:

[0044] Human posture recognition module: used to identify human skeleton;

[0045] Image analysis module: used to calculate the height relationship between the handrail and two points at different heights of the human body based on the image captured by the camera, determine the point on the handrail that maintains the same horizontal height as the human body, and determine whether the human skeleton intersects the plumb line;

[0046] Alarm module: used to issue an alarm when the human skeleton intersects with the plumb line.

[0047] One of the above technical solutions of the present invention has at least one of the following advantages or beneficial effects:

[0048] The method of the present invention uses the relative position relationship between the key joints of the human body and the handrail to reversely infer the handrail corresponding to the position of the person at any time, and marks the plumb line on the point on the handrail. It can be used to determine whether there is a risk of elevator intrusion and issue an alarm in time. At the same time, this method does not involve background AI calculations, can effectively reduce the data processing volume, improve the processing speed, and realize accurate, rapid and intelligent alarms.

[0049] At the same time, the present invention also provides an escalator three-dimensional area intrusion detection system based on human skeleton. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention will be further described below with reference to the accompanying drawings and embodiments;

[0051] Figure 1 is a schematic diagram of steps 2 and 3 of Example 1 of the present invention;

[0052] Figure 2 is a schematic diagram of the calibration of the plumb line in step 4 of Example 1 of the present invention;

[0053] Figure 3 This is an application scenario diagram of Example 1 of the present invention;

[0054] Figure 4 is a flow chart of embodiment 1 of the present invention;

[0055] Figure 5 This is a structural block diagram of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0056] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0057] Example 1

[0058] Reference Figures 1 to 4 As shown, a three-dimensional area intrusion detection method based on human skeleton includes the following steps:

[0059] Step 1: When a human body enters the camera area, the human skeleton is identified to obtain skeleton information, in which multiple position points are marked;

[0060] The camera area must cover the escalator. After a person enters the camera area, human skeleton recognition and tracking can be performed in advance, or human skeleton recognition can be performed at the same time as step 2.

[0061] The human skeleton recognition method of the present invention is a human posture algorithm.

[0062] The present invention is not limited to human posture algorithms, and various algorithms can give corresponding key positions, such as the positions of shoulder bones, hip bones, etc. The openpose and alphapose algorithms were used in the verification process.

[0063] The alphapose homepage is at https: / / github.com / MVIG-SJTU / AlphaPose, and the paper is at https: / / arxiv.org / abs / 1612.00137.

[0064] The Openpose homepage is at https: / / github.com / CMU-Perceptual-Computing-Lab / openpose, and the paper is at https: / / arxiv.org / abs / 1611.08050.

[0065] In actual applications, the most common situation for people on escalators is that their hands, feet, torso, etc. stick out of the elevator handrail. In this case, the elevator must be able to quickly identify and take emergency measures. Therefore, the location points recommended by the present invention include: joints on the limbs, hip joints, shoulder joints, head, and points corresponding to the five senses; in this way, in the judgment of step 4, it can be identified as comprehensively as possible whether the hands, feet or head are in a dangerous state.

[0066] However, the two position points generally used in steps 2 and 3 do not involve the joints on the arms, because the human body is not absolutely motionless, and the movement of the arms will affect the judgment of steps 2 and 3.

[0067] Preferably, the upper and lower entrances of the escalator are both equipped with sensors for sensing whether an object is located on the escalator steps; when the sensors detect an object at the upper or lower entrance of the escalator, step 2 is performed. The sensors may be infrared sensors, gravity sensors located on the steps, or the like.

[0068] Step 2: When a person enters the escalator, a first image is acquired, and the height relationship between the handrail and two positions at different heights of the person is calculated;

[0069] The hip and shoulder joints are recommended as the locations in this step and step 3; these two joints are connected to the limbs, have fixed positions, and are easy to identify.

[0070] This step specifically includes the following sub-steps:

[0071] Step 21: When the human body enters the escalator, a first image is acquired;

[0072] Step 22: Draw a line and two position points representing the handrail from the first image;

[0073] Since the features of the handrail are relatively obvious, the lines of the handrail can be easily identified based on existing image recognition technology. In some application environments, if the camera is fixed or does not rotate frequently, the position of the handrail can be pre-marked in all collected images, so that there is no need to re-identify and draw the handrail every time a new image is acquired.

[0074] Step 23: Draw horizontal lines along the two points and a vertical line between the two horizontal lines;

[0075] Step 24: Translate the vertical line obtained in step 23 so that the handrail line intersects the vertical line to obtain an intersection point, and determine the ratio of the vertical distance from the intersection point to the upper end of the vertical line and the vertical distance from the intersection point to the lower end of the vertical line; the ratio is the height relationship.

[0076] In some application scenarios, the vertical line in step 23 does not need to be translated. As mentioned above, in a scenario where the camera is fixed or does not rotate frequently, the position of the handrail at the entrance of the elevator is fixed, which is equivalent to the above-mentioned intersection being unchanged in the picture. In this case, the above-mentioned ratio can be obtained by directly drawing a horizontal line from a point pre-marked in the picture to intersect with the vertical line.

[0077] refer to Figure 1 , Figure 1 Another method for calibrating an elevator handrail is shown, that is, calibrating the two ends of the handrail and determining the line of the handrail according to the connecting line of the two ends of the handrail.

[0078] Step 3: During the operation of the escalator, a second image is acquired, and a point on the handrail that maintains the same horizontal height as the human body is determined based on the height relationship and the two position points;

[0079] Refer to step 2 and perform the reverse operation. Specifically, it includes the following sub-steps:

[0080] Step 31: Acquire a second image during the operation of the escalator;

[0081] Step 32: Draw a line and two position points representing the handrail from the second image;

[0082] Step 33: Draw horizontal lines along the two points and a vertical line between the two horizontal lines;

[0083] Step 34: Determine the point on the vertical line according to the height relationship, and draw a horizontal line along the point on the vertical line. The intersection of the horizontal line and the line representing the handrail is the point on the handrail that maintains the same horizontal height as the human body.

[0084] refer to Figure 1 The number 3 in FIG. 3 shows that the point on the handrail can be determined based on the height relationship between the point on the vertical line and the position point.

[0085] Step 4: Draw a plumb line on the handrail from the point determined in Step 3. Determine whether the human skeleton intersects the plumb line. If so, an alarm is issued. This alarm includes but is not limited to audible and visual alarms. After the alarm is issued, an emergency shutdown can be automatically implemented and maintenance personnel can be notified.

[0086] Figure 2 The plumb line is shown, which is Figure 2 The black line in .

[0087] Figure 3 A diagram showing a specific scenario in which the method of the present invention is applied to an escalator.

[0088] Through the above method, intrusion judgment and alarm can be carried out accurately and quickly.

[0089] The method of this embodiment can be implemented based on the following practical conditions:

[0090] 1. The images taken by the camera have a very long depth of field. Under normal circumstances, the warning line will intersect and block the human body.

[0091] 2. There is a high-side difference between the handrail and the ground, and the handrail has a baffle. The area below the baffle is not crossable, so only the 3D space above the baffle needs to be detected for intrusion.

[0092] 3. Regional invasions are all caused by humans. The human body’s limbs are connected and complete, and there will be no invasion of hands or feet separated from the torso.

[0093] This solution is particularly suitable for situations with a long depth of field. It does not require the drawing of background images and relies entirely on handrail and human skeleton recognition. By combining the 2D human skeleton and the warning line (the determination of the plumb line mentioned above), it realizes intrusion detection in 3D space, while avoiding the use of 3D algorithms with high computational complexity to achieve the same effect.

[0094] The present invention has the following advantages compared with the prior art based on the tasks implemented:

[0095] 1. Overcome the shortcomings of the 2D warning line method and realize intrusion detection in 3D space.

[0096] 2. Avoid using 3D skeleton algorithms that require excessive computation. Instead, adopt 2D skeleton recognition algorithms that require less computation, combined with simple algorithms from computer graphics, for low-cost deployment.

[0097] 3. By dynamically calculating the position of the human skeleton relative to the warning line, the intrusion situation can be judged more accurately.

[0098] 4. Based on the relative distance of the handrail height to the key bone points, it is possible to make judgments even when part of the human body is not captured.

[0099] 5. This program has no restrictions on height and age.

[0100] 6. Through the design of this system, it can meet current needs and has considerable economic value.

[0101] At the same time, this system can be expanded to other application scenarios, such as 3D spatial intrusion judgment in sensitive areas such as river embankments, bridge railings, and stage areas.

[0102] Example 2

[0103] refer to Figure 5 , a system for implementing the method described in Example 1, comprising a camera 1 and a server 2, wherein the camera's imaging area covers an escalator;

[0104] The server includes the following modules:

[0105] Human posture recognition module 21: used to recognize human skeleton;

[0106] Image analysis module 22: used to calculate the height relationship between two positions at different heights of the handrail and the human body based on the image obtained by the camera, determine the point on the handrail that maintains the same horizontal height as the human body, and determine whether the human skeleton intersects the plumb line;

[0107] Alarm module 23: used to issue an alarm and stop the elevator when the human skeleton intersects with the plumb line.

[0108] The working process of this system is:

[0109] The camera continuously acquires images and sends the images to the human posture recognition module and the image analysis module; the human posture recognition module identifies the human skeleton and sends the skeleton information to the image analysis module. When the human body enters the elevator at the initial stage, the image analysis module calculates the height relationship between the handrail and two position points at different heights of the human body; when the person is running on the escalator, the point on the handrail that maintains the same horizontal height as the human body is determined, and it is judged whether the human skeleton intersects with the plumb line. If there is an intersection, the calculation result is sent to the alarm module. If the human skeleton intersects with the plumb line, the alarm module issues an alarm and stops the elevator.

Claims

1. A three-dimensional area intrusion detection method based on human skeleton, characterized in that: The method is applicable to the scenario of intrusion detection of a sidebar with a regular height, wherein the sidebar is an escalator handrail; the method is specifically as follows: Step 1: When a human body enters the camera area, the human skeleton is identified to obtain skeleton information, in which multiple position points are marked; Step 2: When a person enters the escalator, a first image is acquired, and the height relationship between the handrail and two positions at different heights of the person is calculated; Step 3: During the operation of the escalator, a second image is acquired, and a point on the handrail that maintains the same horizontal height as the human body is determined based on the height relationship and the two position points; Step 4: Draw a plumb line on the handrail at the point determined in step 3, and determine whether the human skeleton intersects with the plumb line. If so, an alarm is issued; The step 2 is specifically as follows: Step 21: When the human body enters the escalator, a first image is acquired; Step 22: Draw a line and two position points representing the handrail from the first image; Step 23: Draw horizontal lines along the two points and a vertical line between the two horizontal lines; Step 24: Translate the vertical line obtained in step 23 so that the handrail line intersects the vertical line to obtain an intersection point, and determine the ratio of the vertical distance from the intersection point to the upper end of the vertical line and the vertical distance from the intersection point to the lower end of the vertical line; the ratio is the height relationship.

2. The three-dimensional area intrusion detection method based on human skeleton according to claim 1 is characterized in that: The position points in the skeleton information include: three or more points corresponding to the joints on the limbs, hip joints, shoulder joints, head and facial features.

3. The three-dimensional area intrusion detection method based on human skeleton according to claim 1 is characterized in that: The skeleton information is calculated by a human body posture algorithm.

4. The three-dimensional area intrusion detection method based on human skeleton according to claim 1 is characterized in that: The upper entrance and the lower entrance of the escalator are both provided with sensors for sensing whether there is an object on the steps of the escalator; when the sensors detect that there is an object at the upper entrance or the lower entrance of the escalator, step 2 is performed.

5. The three-dimensional area intrusion detection method based on human skeleton according to claim 1 is characterized in that: The step 3 is specifically as follows: Step 31: Acquire a second image during the operation of the escalator; Step 32: Draw a line and two position points representing the handrail from the second image; Step 33: Draw horizontal lines along the two points and a vertical line between the two horizontal lines; Step 34: Determine the point on the vertical line according to the height relationship, and draw a horizontal line along the point on the vertical line. The intersection of the horizontal line and the line representing the handrail is the point on the handrail that maintains the same horizontal height as the human body.

6. A system for implementing the method of claim 1, characterized in that: The system comprises a camera and a server, wherein the camera's imaging area covers at least a portion of the sidebar; The server includes the following modules: Human posture recognition module: used to identify human skeleton; Image analysis module: used to calculate the height relationship between the handrail and two points at different heights of the human body based on the image captured by the camera, determine the point on the siderail that maintains the same horizontal height as the human body, and determine whether the human skeleton intersects with the plumb line; Alarm module: used to issue an alarm when the human skeleton intersects with the plumb line.

Images