Human body out-of-boundary detection method, device and computer readable storage medium

Abstract

The application discloses a human body crossing boundary detection method and device and a computer readable storage medium. A protection boundary of a protection area is determined, a crossing boundary condition corresponding to the protection boundary is generated, a deployment position of a camera is calculated according to the protection boundary, a coverage range of a target camera shooting field of view is adjusted according to the deployment position of the camera, so that the protection boundary falls into the coverage range, a human body image shot by the target camera is read, human body key points of the human body image are extracted, and a crossing boundary situation of the human body in the protection area is detected according to the crossing boundary condition and the human body key points. Based on this, the crossing boundary condition of the human body entering the protection boundary can be flexibly set, the human body key points are extracted from the human body image shot by the camera, the crossing boundary situation of the human body in the protection area is judged by combining the crossing boundary condition and the human body key points, the crossing boundary situation of the human body can be accurately detected, and the misjudgment situation can be reduced, and the occurrence of false positives and false negatives can be reduced.

Description

Technical Field

[0001] The embodiments of the present invention relate to, but are not limited to, the field of human behavior detection technology, and in particular to a method, apparatus and computer-readable storage medium for detecting human boundary crossing. Background Technology

[0002] In various industries such as manufacturing, transportation, and security, there are often hazardous areas, such as machine operation stations and metallurgical furnaces. Personnel entering these areas may be injured, leading to safety accidents and potentially causing production disruptions. The consequences can be very serious.

[0003] Currently, in areas where it's feasible to install guardrails and other protective facilities, physical isolation can enhance safety. However, in areas where installing protective netting is not advisable, the primary methods used are disseminating regulations and using warning signs to remind people not to cross boundaries. However, people sometimes forget or ignore this information, leading to accidents. There are also methods using cameras to detect people entering protected areas, but these suffer from inaccurate detection, sometimes detecting people as if they've entered the protected area when they haven't, or vice versa, resulting in numerous false alarms. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] This invention provides a method, apparatus, and computer-readable storage medium for detecting human body crossing boundaries, which can solve the problem of inaccurate detection when human body crosses boundaries. It can automatically detect human body crossing boundaries and accurately detect human body crossing boundaries, reducing false detections and missed detections.

[0006] In a first aspect, embodiments of the present invention provide a method for detecting human body crossing boundaries, comprising: determining the protection boundary of a protected area; generating a boundary crossing condition corresponding to the protection boundary; calculating the deployment position of a camera based on the protection boundary; adjusting the coverage range of the target camera's field of view based on the camera's deployment position so that the protection boundary falls within the coverage range; reading human body images captured by the target camera and extracting key human body points from the human body images; and detecting the human body crossing the boundary of the protected area based on the boundary crossing condition and the key human body points.

[0007] Secondly, embodiments of the present invention provide a human body boundary crossing detection device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human body boundary crossing detection method described in the first aspect above.

[0008] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human body boundary detection method described in the first aspect above.

[0009] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer-executable program for causing a computer to perform the human body boundary crossing detection method described in the first aspect above.

[0010] This invention includes: determining the protection boundary of a protected area; generating boundary crossing conditions corresponding to the protection boundary; calculating the deployment position of a camera based on the protection boundary; adjusting the coverage area of ​​the target camera's field of view based on the camera's deployment position so that the protection boundary falls within the coverage area; reading human images captured by the target camera; extracting key human points from the human images; and detecting the human's boundary crossing within the protected area based on the boundary crossing conditions and the key human points. Based on this, this invention can flexibly set boundary crossing conditions for human entry into the protected area, extract key human points from the human images captured by the camera, and determine the human's boundary crossing within the protected area by combining the boundary crossing conditions and the key human points. Compared to existing technologies, this not only automates the monitoring of whether someone has entered or crossed the boundary within the protected area, reducing the workload of manual monitoring, but also accurately detects human boundary crossings, thereby reducing misjudgments and false alarms / missed alarms. As long as a human crosses the protection boundary, it can be accurately detected, avoiding false alarms caused by side-view or oblique camera views, where a person is judged to have crossed the boundary even if they have not entered the protected area, and false alarms where a person is judged to have entered the protected area but not as having entered. Furthermore, this invention allows for the determination of camera deployment locations based on protected boundaries. Accurately deployed multiple cameras can precisely reflect the defined boundary lines and the relative positions of adjacent human figures in the image, avoiding inaccuracies caused by inconsistent viewing angles and boundary lines. Adjusting the coverage area of ​​each camera's field of view ensures the protected boundary falls within its coverage area, thereby determining the camera's coverage zone and enabling designers to more accurately determine camera placement. For complex or large protected areas, the camera coverage and overlap can be visually observed, effectively reducing the workload of manually calculating coverage areas and minimizing errors in manually specifying locations. This reduces time, communication, and financial costs associated with rework, adding cameras, and structural components due to incorrect camera deployment.

[0011] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description, claims, and drawings. Attached Figure Description

[0012] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0013] Figure 1 This is a main flowchart of a method for detecting human body crossing boundaries according to an embodiment of the present invention;

[0014] Figure 2A This is a schematic diagram of a protected area being a closed polygon provided in one embodiment of the present invention;

[0015] Figure 2B This is a schematic diagram of a closed, concave protected area provided in one embodiment of the present invention;

[0016] Figure 2C This is a schematic diagram of a non-closed area provided in one embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of the camera coverage height and outer extension on the protection boundary provided in one embodiment of the present invention;

[0018] Figure 4 This is a diagram illustrating the detection effect of key human points according to an embodiment of the present invention;

[0019] Figure 5 This is a sub-flowchart of a method for detecting human body crossing boundaries according to an embodiment of the present invention;

[0020] Figure 6 This is a sub-flowchart of a method for detecting human body crossing boundaries according to an embodiment of the present invention;

[0021] Figure 7 This is a top view of a rectangular enclosed protective area provided in one embodiment of the present invention;

[0022] Figure 8 This is a sub-flowchart of a method for detecting human body crossing boundaries according to an embodiment of the present invention;

[0023] Figure 9 This is a top view schematic diagram of a long linear non-enclosed protective area provided in one embodiment of the present invention;

[0024] Figure 10This is a top view schematic diagram of adding a camera to a long, non-enclosed protective area according to an embodiment of the present invention;

[0025] Figure 11 This is a schematic diagram of the structure of a human body boundary crossing detection device provided in one embodiment of the present invention;

[0026] Figure 12 This is a schematic diagram of an electronic device structure provided in one embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0028] It should be understood that in the description of the embodiments of the present invention, "multiple" (or more than) means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. If "first," "second," etc., are used in the description, they are only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0029] In various industries such as manufacturing, transportation, and security, there are often hazardous areas, such as machine operation stations and metallurgical furnaces. Personnel entering these areas may be injured, leading to safety accidents and potentially causing production disruptions. The consequences can be very serious.

[0030] Currently, in areas where it's feasible to install guardrails and other protective facilities, physical isolation can enhance safety. However, in areas where installing protective netting is not advisable, the primary methods used are disseminating regulations and using warning signs to remind people not to cross boundaries. However, people sometimes forget or ignore this information, leading to accidents. There are also methods using cameras to detect people entering protected areas, but these suffer from inaccurate detection, sometimes detecting people as if they've entered the protected area when they haven't, or vice versa, resulting in numerous false alarms.

[0031] To address the problem of inaccurate detection of human body crossing boundaries in existing technologies, this invention provides a method, apparatus, and computer-readable storage medium for detecting human body crossing boundaries. First, a protection boundary is set according to the protected area, defining the inner and outer sides of the boundary as either protected or unprotected, and setting the boundary crossing conditions for a human body entering the protected boundary. Then, the deployment position of the camera is determined based on the protection boundary. Next, the video and images captured by the camera are analyzed to detect key human body points. Finally, the key human body points and the boundary crossing conditions are combined to determine whether the human body has crossed the boundary. Based on this, the problem of inaccurate detection of human body crossing boundaries is solved, enabling automatic and accurate detection of human body crossings, reducing false detections and missed detections. Based on this, the present invention can flexibly set the boundary-crossing conditions for human entry into the protected area. Key human points are extracted from the images captured by the camera, and the boundary-crossing conditions and key human points are combined to determine whether a person has crossed the protected area. Compared with existing technologies, this not only automates the monitoring of whether someone has entered or crossed the protected area, reducing the workload of manual monitoring, but also accurately detects human boundary crossings, thereby reducing misjudgments, false alarms, and missed alarms. As long as a person crosses the protected boundary, it can be accurately detected, avoiding false alarms caused by side-view or slanted camera views, where people are judged to have crossed the boundary even when they have not, and missed alarms where people are judged not to have entered the protected area even when they have actually entered. Furthermore, the present invention can determine the deployment position of the cameras based on the protected boundary. Accurately deployed multiple cameras can precisely reflect the defined boundary line and the relative position of adjacent human bodies in the image, avoiding the inaccurate reflection of the boundary line and the relative position of human bodies in the image due to inconsistent viewing angles and boundary lines. The coverage range of each camera's field of view is adjusted so that the protected boundary falls within the coverage range, thereby determining the camera's coverage area and allowing designers to more accurately determine the camera's position. For complex or large protected areas, the coverage of cameras and the overlap of each camera can be seen intuitively, which can effectively reduce the workload of manually calculating the coverage area. At the same time, it can reduce the error of manually specifying the location, thereby reducing the time, communication and financial costs caused by rework, additional cameras, structural components, etc. due to incorrect camera deployment.

[0032] like Figure 1 As shown, Figure 1 This is a flowchart of a method for detecting human body boundary crossing according to an embodiment of the present invention. The method for detecting human body boundary crossing includes, but is not limited to, the following steps:

[0033] Step 101: Determine the protection boundary of the protected area;

[0034] Step 102: Generate the boundary crossing conditions corresponding to the protected boundary;

[0035] Step 103: Calculate the deployment location of the cameras based on the protection boundary;

[0036] Step 104: Adjust the coverage of the target camera's field of view according to the camera's deployment location so that the protection boundary falls within the coverage area;

[0037] Step 105: Read the human body image captured by the camera and extract the human body key points from the human body image;

[0038] Step 106: Detect the boundary crossing situation of the human body in the protected area based on the boundary crossing conditions and key points of the human body.

[0039] It is understandable that protected areas include enclosed areas and open areas, such as... Figure 2A and Figure 2B As shown, a closed region can be a closed polygon or a closed concave shape, and the protection boundary conforms to the area that needs to be protected; while a non-closed region, such as... Figure 2C As shown, a straight line or a broken line can define one side of the protection boundary as the protected area, which is beneficial for boundary detection at entrances and exits. Therefore, the protection boundary can be set according to the actual shape of the site and the needs.

[0040] Understandably, boundary crossing conditions can include body part crossing conditions and time period crossing conditions. For body part crossing conditions, restrictions can be set for specific body parts to suit the actual protection scenario. For example, the condition can be set to allow hands to cross the boundary but prohibit feet from doing so. Alternatively, the default setting can be that all body parts cannot cross the boundary. Similarly, for time period crossing conditions, restrictions can be set for specific time periods to suit the actual protection scenario. For example, the condition can be set to allow crossing during the day but prohibit it at night.

[0041] Understandably, the deployment location of cameras can be determined by the endpoints of the protected boundary and the coverage area of ​​the camera's field of view. Existing technologies, which use single cameras positioned to the side or at an angle, can cause false alarms (e.g., people not entering the protected area are flagged as having crossed the boundary) and false alarms (e.g., people actually entering the protected area are flagged as not having entered). This invention, however, accurately deploys cameras at appropriate locations on the protected boundary, ensuring that the images captured by each camera precisely reflect the defined boundary line and the relative position of adjacent individuals. This avoids inaccurate representation of the boundary line and individual positions in the images due to inconsistencies between the viewing angle and the boundary line, thus overcoming the false alarms and false alarms related to people crossing the boundary in existing technologies.

[0042] Understandably, since each camera has its own field of view coverage, this coverage can be adjusted by modifying parameters such as the camera's field of view angle, mounting height, orientation, and optical axis. After deploying the cameras at each endpoint of the protection boundary, the coverage of the respective camera's field of view needs to be adjusted to ensure the entire protection boundary falls within its coverage area. It should be noted that the boundary line within the camera's coverage area is superimposed onto the camera's image. The alignment of the boundary lines between adjacent cameras is then observed. If there are discrepancies, the boundary lines are adjusted. This eliminates misalignment of boundary lines between adjacent camera images caused by camera installation errors.

[0043] Understandably, adjusting the coverage area of ​​a camera's field of view also includes adjusting the coverage height and extension of each camera. Coverage height refers to the height at which the camera captures a person within the protected boundary. The coverage height between any two adjacent cameras is generally set to be greater than the height of the person to ensure that the entire person on the boundary line is covered. Extension is the horizontal length extending from which the camera captures the person entering the protected boundary. This extension also extends to the non-protected side of the protected boundary. The camera's extension is generally set to about one meter to ensure that the camera can capture the entire process of the person entering the protected boundary, i.e., the entire process of the person crossing from the non-protected side to the protected side. It should be noted that, as... Figure 3 As shown, as long as the height of the intersection point of the field of view lines of any two adjacent cameras is greater than the minimum coverage height (generally the height of a human body), then the coverage height H between the field of view lines of any two adjacent cameras will meet the minimum coverage height requirement. Similarly, as long as the extension L of any camera is greater than a preset threshold, such as one meter, then the extension of each camera will meet the minimum extension requirement.

[0044] Understandably, by capturing images of the human body with a camera, deep learning methods can be used to extract key points from those images. For example, ... Figure 4 As shown, deep learning algorithms such as OpenPose can be used to detect and extract key human features from images captured by cameras. It should be noted that deep learning methods for human body recognition can adapt well to changes in environmental conditions such as lighting, exhibiting strong adaptability to variations in illumination and ground reflection.

[0045] Understandably, this invention allows for flexible setting of boundary-crossing conditions for human entry into the protected area. Key human points are extracted from images captured by a camera, and the boundary-crossing conditions and these key points are combined to determine if the human has crossed the protected area. In one embodiment, when a key human point satisfies the boundary-crossing condition, the human is determined to have crossed the boundary into the protected area. These key human points may include the head, torso, upper limbs, lower limbs, hands, feet, etc. For example, a rectangular protected boundary can be defined around equipment in a factory, with the boundary-crossing condition prohibiting all parts of the human body from entering at night. Therefore, if any key human point is determined to enter this rectangular protected boundary at night, the human is determined to have crossed the boundary into the protected area of ​​the factory equipment.

[0046] Understandably, when a person is determined to have crossed the boundary into the protected area, the equipment within the protected area can be controlled to slow down or stop operating, and warning devices can be controlled to output warning information, such as displaying warning messages, flashing warning lights, or continuously emitting alarm sounds. Simultaneously, images or videos of the person entering the protected area can be saved for future reference.

[0047] It is understood that the application scenarios of this invention can be applied in the industrial, security, and transportation fields to detect human encroachment. In the industrial field, it delineates dangerous areas to prevent personnel from entering; when someone enters, the equipment can be paused and a warning issued. In the security field, it delineates the entrances and exits of confidential units, issuing warnings and taking photos and videos when someone enters; or for the protection of residential areas, boundaries can be delineated, and warnings or images and videos can be saved when someone enters at night or other sensitive times. In the transportation field, it delineates protected areas near the tracks at subway and high-speed rail platforms, issuing warnings when someone enters. The method of this invention can also be applied to other scenarios requiring the detection of human entry.

[0048] Understandably, the entire process of human body boundary crossing detection can include the following: First, a protective boundary is set according to the protected area, defining the inner and outer sides of the boundary as either protected or unprotected, and setting the boundary crossing conditions for a human body entering the protected boundary. Then, the deployment locations and number of cameras are determined based on the protective boundary. Next, the video and images captured by the cameras are analyzed to detect key human body points. Then, combining the key human body points and the boundary crossing conditions, it is determined whether the human body has crossed the boundary. Finally, the machines or equipment within the protected area are controlled, such as stopping or slowing down the equipment, and relevant warning messages are set. Based on this, the problem of inaccurate detection of human body boundary crossings is solved; it can automatically detect human body boundary crossings and accurately detect boundary crossing situations, reducing false detections and missed detections.

[0049] Understandably, this invention determines whether a person has crossed the boundary of a protected area by combining boundary crossing conditions and key points of the human body. Compared to existing technologies, it not only automates the monitoring of whether someone has entered or crossed the boundary within the protected area, reducing the workload of manual monitoring, but also accurately detects boundary crossings, thereby reducing misjudgments, false alarms, and missed alarms. As long as a person crosses the protected boundary, it can be accurately detected, avoiding false alarms caused by side-view or slanted camera views, where a person is judged to have crossed the boundary even if they have not, and missed alarms where a person is judged not to have entered the protected area even if they have actually entered. Furthermore, this invention can determine the deployment location and number of cameras based on the protected boundary. At least two deployed cameras can accurately reflect the defined boundary line and the relative position of adjacent human bodies in the image, avoiding inaccurate reflection of the boundary line and the relative position of human bodies in the image due to inconsistent viewing angles and boundary lines. The coverage range of each camera's field of view is adjusted so that the protected boundary falls within the coverage range, thereby determining the camera's coverage area and allowing designers to more accurately determine the camera's location. For complex or large protected areas, the coverage of cameras and the overlap of each camera can be seen intuitively, which can effectively reduce the workload of manually calculating the coverage area. At the same time, it can reduce the error of manually specifying the location, thereby reducing the time, communication and financial costs caused by rework, additional cameras, structural components, etc. due to incorrect camera deployment.

[0050] like Figure 5 As shown, step 101 may include, but is not limited to, the following sub-steps:

[0051] Step 1011: Determine the orientation of the protection boundary based on the protected area;

[0052] Step 1012: Determine the protected side and non-protected side of the protection boundary based on the protected area;

[0053] The protected areas include both enclosed and unenclosed areas.

[0054] It is understandable that protected areas include enclosed areas and open areas, such as... Figure 2A and Figure 2B As shown, a closed region can be a closed polygon or a closed concave shape, and the protection boundary conforms to the area that needs to be protected; while a non-closed region, such as... Figure 2C As shown, a straight line or broken line can define one side of the protection boundary as the protected area, which is beneficial for boundary crossing detection at entrances and exits. Therefore, the orientation of the protection boundary can be set according to the actual shape of the site and the needs of the protected area. At the same time, the inner and outer sides of the protection boundary can be defined as the protected side and the non-protected side, respectively, based on the protected area.

[0055] like Figure 6 As shown, step 103 may include, but is not limited to, the following sub-steps:

[0056] Step 1031: Determine the endpoints of the protection boundary based on the protected area;

[0057] Step 1032: Determine the deployment location of the camera based on the endpoints of the protection boundary and the coverage of the camera's field of view.

[0058] Understandably, the deployment location of cameras can be determined based on the endpoints of the protected boundary and the coverage area of ​​the camera's field of view. For example, once the protected boundary is determined based on the protected area, cameras can be deployed at each endpoint of the boundary. If the field of view of the cameras deployed at the endpoints cannot completely cover the protected boundary between the endpoints, additional cameras can be added between the endpoints to ensure that the camera's field of view completely covers the protected boundary. Figure 7 As shown, taking a rectangular enclosed protected area as an example, the boundary of the rectangular enclosed protected area has four endpoints. Therefore, four cameras A, B, C, and D can be deployed at the four endpoints respectively. The field of view of cameras A, B, C, and D can cover the entire protected boundary. Figure 9 As shown, taking a long, linear, non-enclosed protected area as an example, camera A and camera B are deployed at opposite ends of the boundary. However, since the field of view of cameras A and B cannot completely cover the protected boundary between the endpoints, therefore, as Figure 10 As shown, camera C needs to be added between camera A and camera B so that the field of view of cameras A, B, and C can cover the entire protection boundary. Existing technologies using a single camera with a side-view or oblique view can cause false alarms (people not entering the protection area are judged as having crossed the boundary) and false alarms (people actually entering the protection area are judged as not having entered). This invention deploys cameras at each endpoint of the protection boundary, ensuring that the images captured by each camera accurately reflect the boundary line and the relative position of adjacent people. This avoids inaccurate image representation of the boundary line and people's relative positions due to inconsistent viewing angles and boundary lines, overcoming the false alarms and false alarms of people crossing the boundary in existing technologies. It should be noted that the number of cameras deployed can be determined by their deployment locations, for example, as... Figure 7 As shown in the rectangular protection area, if the endpoints of the rectangular protection area are determined as the camera deployment locations, then four cameras A, B, C, and D can be deployed to the four endpoints; for example... Figure 9 The long, linear, non-enclosed protected area shown includes two cameras, A and B, deployed at both ends, such as... Figure 10 As shown, one more camera C needs to be deployed between the endpoints to cover the entire protection boundary.

[0059] like Figure 8 As shown, step 104 may include, but is not limited to, the following sub-steps:

[0060] Step 1041: Adjust the coverage height between the fields of view of any two adjacent target cameras to be greater than the height of a human body;

[0061] Step 1042: Adjust the extension of the target camera's field of view to be greater than a preset threshold;

[0062] The coverage height is the height at which the human body is captured by the camera within the protection boundary, and the extension is the horizontal extension length of the human body as captured by the camera entering the protection boundary.

[0063] Understandably, to ensure the entire human body can be captured by the cameras within the protected boundary, the coverage height of each camera's field of view must meet certain conditions. This is because insufficient camera coverage height will prevent the human body from being fully captured within the protected boundary. Therefore, the coverage height between any two adjacent cameras needs to be greater than the height of the human body. Simultaneously, to ensure the entire process of the human body crossing the boundary from the unprotected side to the protected side is captured, the extension of each camera's field of view must meet certain conditions. Therefore, the extension of each camera's field of view needs to be greater than a preset threshold, typically set to one meter. It should be noted that... Figure 3 As shown, as long as the height of the intersection point of the field of view lines of any two adjacent cameras is greater than the minimum coverage height (generally the height of a person), then the coverage height H between the field of view lines of any two adjacent cameras will meet the minimum coverage height requirement. Similarly, as long as the extension L of any camera is greater than a preset threshold, such as one meter, then the extension of each camera will meet the minimum extension requirement. It should be noted that if the minimum coverage height of any camera does not reach the defined minimum coverage height, cameras can be added at or near the intersection of the coverage of two cameras to increase the coverage height.

[0064] Understandably, designers set information such as the camera's field of view, mounting height, position, optical axis, and extension to automatically calculate the camera's coverage area for designers to observe the effect. The following methods can be used to calculate the camera's coverage area:

[0065] Set the z-axis direction to be from bottom to top, the x-axis direction to be from left to right, and the y-axis direction to be from near to far.

[0066] The camera coordinates are C = [Cx, Cy, Cz]. The mounting height H is Cz. The external extension is Le. The camera's own field of view is defined by the horizontal field of view Fovh and the vertical field of view Fovv.

[0067] Generally, the camera maintains a consistent horizontal height and is not tilted. The camera's tilt angle can be adjusted forward and backward (vertically). The camera can also rotate horizontally. The counter-clockwise rotation around the x-axis is θx. The counter-clockwise rotation around the y-axis is θy. The counter-clockwise rotation around the z-axis is θz.

[0068] Calculate the rotation matrix R and translation matrix T based on the rotation angle and camera coordinates.

[0069] In the camera coordinate system, the plane perpendicular to the optical axis and at a distance of 1 is Pc. The four edges of the camera's viewpoint intersect this plane, forming four vertices A1, A2, A3, and A4 in a counterclockwise direction along the optical axis. The coordinates of the four vertices A1, A2, A3, and A4 in the camera coordinate system are respectively A1 = [-tan(Fovh), tan(Fovv), 1], A2 = [-tan(Fovh), -tan(Fovv), 1], A3 = [tan(Fovh), tan(Fovv), 1], and A4 = [tan(Fovh), tan(Fovv), 1].

[0070] Calculate the coordinates of the four points in the world coordinate system. Aw = RA + T.

[0071] Then, based on the coordinates Aw of the four points in the world coordinate system and the camera coordinates C, calculate the four view lines L1, L2, L3, and L4. Then calculate the intersection point Ph of the four lines and the horizontal plane.

[0072] Before calculating the intersection point, determine if the lines are horizontal to the horizontal plane. If they are horizontal, there is no intersection point.

[0073] In one implementation, after obtaining four horizontal intersection points Ph, these four points are connected to form four lines GL = [GL1, GL2, GL3, GL4], forming the camera's coverage area on the horizontal plane. Then, a circle Cy is generated with the point where the camera's z-axis position is 0 as the center and Le as the radius. It is then calculated whether the four lines GL and this circle Cy intersect. If they do, the minimum external extension requirement is not met, and the camera position and orientation need to be further adjusted.

[0074] In another implementation, after obtaining four horizontal intersection points Ph, the four points are connected to form four lines GL = [GL1, GL2, GL3, GL4], forming the camera's coverage area on the horizontal plane. Then, it is determined whether the point where the camera's z-axis position is 0 is within this coverage area. If not, the camera is adjusted to cover this point.

[0075] Designers observed the coverage effect and adjusted parameters to ensure that the extension requirements were met.

[0076] Continue setting up the other cameras and calculating the coverage area of ​​each camera.

[0077] Calculate the minimum height of overall coverage. The method for calculating the overall coverage height is as follows:

[0078] Obtain the coordinates and rotation angles θx, θy, and θz of two adjacent cameras C1 and C2, and calculate the four field-of-view edges L1, L2, L3, and L4. Divide the four edges into four groups according to their adjacency: G1 = [L1, L2], G2 = [L2, L3], G3 = [L3, L4], and G4 = [L4, L1]. Calculate the planes containing the two lines, designated as P1, P2, P3, and P4. This yields the four view edges of cameras C1 and C2. Calculate the intersection line of the adjacent planes of C1 and C2, and extract the portion within the intersection point range of this intersection line and the L1, L2, L3, and L4 points of this camera. Calculate the intersection point Pc of this portion of the intersection line and the vertical plane containing C1 and C2, and obtain the height Pch of this intersection point. This height represents the height of the overlapping area of ​​the adjacent cameras above the protection boundary line between them.

[0079] Calculate the intersection height Pch of each adjacent camera above the protection boundary line sequentially to obtain a list of intersection heights [Pch1, Pch2, ...]. Compare the heights in the list with the configured minimum coverage height Hmin. If the height of a certain Pch does not reach the defined minimum coverage height, a camera can be added at or near the coverage intersection of two cameras to increase the coverage height.

[0080] Based on this, by automatically calculating the camera coverage area, designers can more accurately determine the camera's position and angle, reducing workload, improving efficiency, and minimizing losses from rework and additional materials. By deploying cameras at boundary vertices, the image accurately reflects the defined boundary line and the relative position of adjacent human bodies, avoiding inaccuracies caused by inconsistent viewpoints and boundary lines. By judging the coordinates of key points on the human body, as well as boundary lines and internal / external settings, it determines whether a person has crossed the boundary. Compared to human body recognition methods, it can more accurately determine the boundary crossing status of various parts of the human body, reducing false alarms. Through the detection of coordinates of various parts of the human body, flexible definitions of boundary crossing standards can be achieved. For example, it supports recognizing feet crossing the boundary line as a boundary crossing, while not triggering an alarm for hands or heads crossing the boundary.

[0081] The method for detecting human body crossing boundaries provided by the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0082] like Figure 7 As shown, this illustrates the protection of a rectangular enclosed area. For example, the area surrounding equipment in a factory.

[0083] The corresponding steps for implementing boundary check are as follows:

[0084] a. Set the protection zone parameters. Set the coordinates of the four endpoints and connect the endpoints to form the protection zone. Define the inner and outer sides of the protection zone.

[0085] b. Define out-of-bounds conditions. For example, set it so that no part of the human body can cross the boundary.

[0086] c. Calculate the camera coverage area. Set the camera positions as endpoints. Set the minimum outer radius and minimum coverage height. Calculate the camera's field of view parameters, mounting height, and orientation angle on the ground. The results are as follows: Figure 7 The dashed line area of ​​camera A inside.

[0087] d. Calculate whether the coverage area meets the extension requirements. If not, issue a prompt. Continue to adjust the optical axis direction, camera parameters, position, etc., until the extension requirements are met.

[0088] e. Continue setting the information for the other cameras B, C, and D to obtain the coverage area of ​​each camera.

[0089] f. Observe whether the protected area has been fully covered. If it has been fully covered, calculate the coverage height between each camera and check whether it meets the minimum coverage height requirement.

[0090] The final positions and optical axis directions of each camera were determined.

[0091] g. Deploy cameras according to the camera's field of view angle, the obtained camera position, and the direction of the optical axis.

[0092] Observe the superimposed boundaries in the camera footage and adjust them to ensure that the boundaries in adjacent camera footage match.

[0093] h. Use deep learning algorithms such as OpenPose to detect images captured by cameras and extract key points of the human body.

[0094] i. Based on the definition of inside and outside the boundary and the setting of the boundary crossing conditions, extract the coordinates of key points of each person, compare them with the boundary coordinates, and determine whether they are inside the boundary. If they are inside the boundary, they are considered to have crossed the boundary.

[0095] j. If the boundary is determined to be crossed, the control equipment will stop operating, the control warning equipment will issue a warning message, and the images and videos of the boundary crossing will be stored.

[0096] like Figure 9 As shown, this involves protecting a non-enclosed area. For example, the area near the tracks at a high-speed rail station.

[0097] The corresponding steps for implementing boundary check are as follows:

[0098] a. Set area protection parameters. Set the coordinates of the three endpoints to form two connecting lines, which serve as boundaries. Define the inner and outer sides of the protected area.

[0099] b. Set boundary conditions. Set it so that hands can cross the boundary, but feet cannot.

[0100] c. Calculate the camera coverage area. Set the camera positions as endpoints. Set the minimum outer radius and minimum coverage height. Calculate the camera's field of view parameters, mounting height, and orientation angle on the ground. The results are as follows: Figure 9 The dashed lines mark the boundaries of camera A and camera B.

[0101] d. Calculate whether the coverage area meets the extension requirements. If not, issue a prompt. Continue to adjust the optical axis direction, camera parameters, position, etc., until the extension requirements are met.

[0102] e. Continue setting the information for other cameras to obtain the coverage area of ​​each camera.

[0103] f. Observe whether the protected area has been fully covered. If it has been fully covered, calculate the coverage height between each camera and check whether it meets the minimum coverage height requirement.

[0104] g. If the minimum coverage height does not meet the defined minimum coverage height, additional cameras can be added at or near the intersection of the coverage of the two cameras to increase the coverage height. For example... Figure 10 As shown, camera C is added between camera A and camera B.

[0105] The final positions and optical axis directions of each camera were determined.

[0106] h. Deploy cameras according to the camera's field of view angle, the obtained camera position, and the optical axis direction.

[0107] Observe the superimposed boundaries in the camera footage and adjust them to ensure that the boundaries in adjacent camera footage match.

[0108] i. Use deep learning algorithms such as OpenPose to detect images captured by cameras and extract key points of the human body.

[0109] j. Based on the definition of inside and outside the boundary and the setting of boundary crossing conditions, extract the coordinates of key points of each human body, compare them with the boundary coordinates, and determine whether key points of the human body other than the hands have crossed the boundary and entered the protected area. If they have entered the protected area, it is considered a boundary crossing.

[0110] k. If the boundary is determined to be crossed, the control alarm device will issue an alarm message and store the images and videos of the boundary crossing.

[0111] like Figure 11As shown in the figure, this embodiment of the invention also provides a detection device for human body crossing boundaries.

[0112] In one embodiment, the human body crossing detection device may include one or more processors and a memory. Figure 11 Let's take a processor and memory as an example. The processor and memory can be connected via a bus or other means. Figure 11 Taking a bus connection as an example, the human boundary detection device is externally connected to at least one camera. The connection method can be wireless or wired, and the human boundary detection device can control the operation of the camera.

[0113] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the human body boundary crossing detection method in the above embodiments of the present invention. The processor implements the human body boundary crossing detection method in the above embodiments of the present invention by running the non-transitory software program and the program stored in the memory.

[0114] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data required to execute the human body boundary crossing detection method described in the above embodiments of the present invention. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the human body boundary crossing detection device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0115] The non-transient software program required to implement the human body boundary crossing detection method in the above embodiments of the present invention, and the program itself, are stored in memory. When executed by one or more processors, the human body boundary crossing detection method in the above embodiments of the present invention is executed, for example, the method described above is executed. Figure 1 Steps 101 to 106 in the method are as follows. Figure 5 Steps 1011 to 1012 in the method are as follows. Figure 6 Steps 1031 to 1032 in the method are as follows. Figure 8Steps 1041 and 1042 of the method involve determining the protection boundary of the protected area, generating boundary crossing conditions corresponding to the protection boundary, calculating the deployment position of the camera based on the protection boundary, adjusting the coverage range of the target camera's field of view based on the camera's deployment position so that the protection boundary falls within the coverage range, reading the human body image captured by the target camera, extracting human body key points from the human body image, and detecting the human body's boundary crossing situation in the protected area based on the boundary crossing conditions and human body key points. Based on this, the present invention can flexibly set boundary crossing conditions for human bodies entering the protected area, extract human body key points from the human body image captured by the camera, and determine the human body's boundary crossing situation in the protected area by combining the boundary crossing conditions and human body key points. Compared with existing technologies, this not only achieves automated monitoring of whether someone has entered or crossed the boundary within the protected area, reducing the workload of manual monitoring, but also accurately detects human body boundary crossing situations, thereby reducing misjudgments and the occurrence of false alarms and missed alarms. Humans can be accurately detected as long as they cross the protection boundary, avoiding false alarms caused by side-view or slanted camera angles, where people are judged to have crossed the boundary even when they haven't, and false alarms where people are actually in the protection area but are judged not to have entered. Furthermore, this invention can determine the camera deployment position based on the protection boundary. Accurately deployed multiple cameras can precisely reflect the defined boundary line and the relative position of adjacent human bodies in the image, avoiding inaccurate representation of the boundary line and human body positions due to inconsistent viewing angles and boundary lines. Adjusting the coverage range of each camera's field of view ensures the protection boundary falls within the coverage area, thus determining the camera's coverage area and allowing designers to more accurately determine camera positions. For complex or large protection areas, the camera coverage and overlap can be visually observed, effectively reducing the workload of manually calculating coverage areas and minimizing errors in manually specifying positions. This reduces time, communication, and financial costs associated with rework, adding cameras, and structural components due to incorrect camera deployment.

[0116] like Figure 12 As shown, this embodiment of the invention also provides an electronic device.

[0117] In one embodiment, the electronic device includes: one or more processors and memory. Figure 12 Let's take a processor and memory as an example. The processor and memory can be connected via a bus or other means. Figure 12 Taking a bus connection as an example, the electronic device is externally connected to at least one camera, either wirelessly or wired, and can control the operation of the camera.

[0118] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the human body boundary crossing detection method in the above embodiments of the present invention. The processor implements the human body boundary crossing detection method in the above embodiments of the present invention by running the non-transitory software program and the program stored in the memory.

[0119] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data required to execute the human body boundary crossing detection method described in the above embodiments of the present invention. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the human body boundary crossing detection device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0120] The non-transient software program required to implement the human body boundary crossing detection method in the above embodiments of the present invention, and the program itself, are stored in memory. When executed by one or more processors, the human body boundary crossing detection method in the above embodiments of the present invention is executed, for example, the method described above is executed. Figure 1 Steps 101 to 106 in the method are as follows. Figure 5 Steps 1011 to 1012 in the method are as follows. Figure 6 Steps 1031 to 1032 in the method are as follows. Figure 8Steps 1041 and 1042 of the method involve determining the protection boundary of the protected area, generating boundary crossing conditions corresponding to the protection boundary, calculating the camera deployment position based on the protection boundary, adjusting the coverage area of ​​the target camera's field of view based on the camera's deployment position so that the protection boundary falls within the coverage area, reading the human body image captured by the target camera, extracting human body key points from the human body image, and detecting the human body's boundary crossing situation in the protected area based on the boundary crossing conditions and human body key points. Based on this, the present invention can flexibly set boundary crossing conditions for human bodies entering the protected area, extract human body key points from the human body image captured by the camera, and determine the human body's boundary crossing situation in the protected area by combining the boundary crossing conditions and human body key points. Compared with existing technologies, this not only achieves automated monitoring of whether someone has entered or crossed the boundary within the protected area, reducing the workload of manual monitoring, but also accurately detects human body boundary crossing situations, thereby reducing misjudgments and the occurrence of false alarms and missed alarms. Humans can be accurately detected as long as they cross the protection boundary, avoiding false alarms caused by side-view or slanted camera angles, where people are judged to have crossed the boundary even when they haven't, and false alarms where people are actually in the protection area but are judged not to have entered. Furthermore, this invention can determine the camera deployment position based on the protection boundary. Accurately deployed multiple cameras can precisely reflect the defined boundary line and the relative position of adjacent human bodies in the image, avoiding inaccurate representation of the boundary line and human body positions due to inconsistent viewing angles and boundary lines. Adjusting the coverage range of each camera's field of view ensures the protection boundary falls within the coverage area, thus determining the camera's coverage area and allowing designers to more accurately determine camera positions. For complex or large protection areas, the camera coverage and overlap can be visually observed, effectively reducing the workload of manually calculating coverage areas and minimizing errors in manually specifying positions. This reduces time, communication, and financial costs associated with rework, adding cameras, and structural components due to incorrect camera deployment.

[0121] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing a computer-executable program, which is executed by one or more control processors, for example, by... Figure 11 One or more processors can be executed to perform the human body boundary crossing detection method described in the above embodiments of the present invention, for example, to perform the above-described... Figure 1 Steps 101 to 106 in the method are as follows. Figure 5 Steps 1011 to 1012 in the method are as follows. Figure 6 Steps 1031 to 1032 in the method are as follows. Figure 8Steps 1041 and 1042 of the method involve determining the protection boundary of the protected area, generating boundary crossing conditions corresponding to the protection boundary, determining the deployment position of the camera based on the protection boundary, adjusting the coverage range of the target camera's field of view based on the camera's deployment position so that the protection boundary falls within the coverage range, reading the human body image captured by the target camera, extracting key human body points from the human body image, and detecting the human body's boundary crossing situation in the protected area based on the boundary crossing conditions and the key human body points. Based on this, the present invention can flexibly set boundary crossing conditions for human bodies entering the protected area, extract key human body points from the human body image captured by the camera, and determine the human body's boundary crossing situation in the protected area by combining the boundary crossing conditions and the key human body points. Compared with existing technologies, this not only achieves automated monitoring of whether someone has entered or crossed the boundary within the protected area, reducing the workload of manual monitoring, but also accurately detects human body boundary crossing situations, thereby reducing misjudgments and the occurrence of false alarms and missed alarms. Humans can be accurately detected as long as they cross the protection boundary, avoiding false alarms caused by side-view or slanted camera angles, where people are judged to have crossed the boundary even when they haven't, and false alarms where people are actually in the protection area but are judged not to have entered. Furthermore, this invention can determine the camera deployment position based on the protection boundary. Accurately deployed multiple cameras can precisely reflect the defined boundary line and the relative position of adjacent human bodies in the image, avoiding inaccurate representation of the boundary line and human body positions due to inconsistent viewing angles and boundary lines. Adjusting the coverage range of each camera's field of view ensures the protection boundary falls within the coverage area, thus determining the camera's coverage area and allowing designers to more accurately determine camera positions. For complex or large protection areas, the camera coverage and overlap can be visually observed, effectively reducing the workload of manually calculating coverage areas and minimizing errors in manually specifying positions. This reduces time, communication, and financial costs associated with rework, adding cameras, and structural components due to incorrect camera deployment.

[0122] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable programs, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable programs, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0123] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for detecting human body crossing boundaries, characterized in that, include: Determine the protection boundaries of the protected area; Generate the boundary crossing conditions corresponding to the protection boundary; Calculate the camera deployment location based on the protection boundary; Based on the deployment location of the cameras, adjust the coverage height between any two adjacent target cameras to be greater than the height of a human body, and adjust the outer extension of the target camera's field of view to be greater than a preset threshold, so that the protection boundary falls within the coverage area of ​​the target camera's field of view; wherein, the coverage height is the height of the human body captured by the camera at the protection boundary, and the outer extension is the horizontal extension length of the human body entering the protection boundary captured by the camera; if the minimum coverage height of the cameras does not reach the defined minimum coverage height, add a camera at or near the coverage intersection of any two adjacent cameras; Read the human body image captured by the target camera and extract the human body key points from the human body image; The system detects whether a human body has crossed the boundary of the protected area based on the boundary crossing conditions and the key points of the human body.

2. The method according to claim 1, characterized in that, The step of detecting whether a human body has crossed the boundary in the protected area based on the boundary crossing conditions and the key points of the human body includes: When it is determined that the key points of the human body meet the boundary crossing conditions, it is determined that the human body has crossed the boundary and entered the protected area.

3. The method according to claim 2, characterized in that, After determining that the key points of the human body meet the boundary crossing condition and that the human body has crossed the boundary into the protected area, the method further includes: Control the devices within the protected area to slow down or stop operating, and output warning messages; and / or, Save images or videos of a human body entering the protected area.

4. The method according to claim 1, characterized in that, The determination of the protection boundary of the protected area includes: The orientation of the protection boundary is determined based on the protected area; The protected side and non-protected side of the protection boundary are determined based on the protected area; The protected area includes both enclosed and unenclosed areas.

5. The method according to claim 1, characterized in that, The step of calculating the camera deployment location based on the protection boundary includes: The endpoints of the protection boundary are determined based on the protection area; The deployment location of the camera is determined based on the endpoints of the protection boundary and the coverage area of ​​the camera's field of view.

6. The method according to claim 5, characterized in that, After determining the deployment location of the camera based on the endpoint locations of the protection boundary and the coverage area of ​​the camera's field of view, the method further includes: The number of cameras deployed is determined based on their deployment locations.

7. The method according to any one of claims 1 to 6, characterized in that, The boundary crossing conditions include boundary crossing conditions for body parts and boundary crossing conditions for time periods.

8. A device for detecting human body crossing boundaries, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the human body boundary detection method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer-executable program for causing a computer to perform the human body boundary crossing detection method as described in any one of claims 1 to 7.

Images